A Comparison of High School Sports Injury Surveillance Data Reporting by Certified Athletic Trainers and Coaches

In the United States (US), high school sports participation has grown rapidly from an estimated 4.0 million participants in 1971–1972 to an estimated 7.0 million in 2006–2007.1 Although the health benefits of sports participation are undeniable,2 high school athletes participating in football, soccer, basketball, wrestling, volleyball, baseball, and softball sustained an estimated 1.4 million injuries during the 2005–2006 school year.3 The challenge is to reduce sports injury rates to their lowest possible level by applying preventive interventions developed through evidence-based science. Such efforts rely upon collecting accurate injury incidence, exposure, and risk and protective factor data.

Large-scale, national sports injury surveillance can be time consuming and costly. To capitalize on available resources, it is imperative to balance the time burden on data reporters with the need to collect high-quality data that reflect true injury rates and patterns in the population of interest. A number of authors have examined methodologic issues in sports injury surveillance, including optimal definitions for athletic exposure,4–7 injury,4–7 injury severity,4–6 and recurrent injuries.4,6,8 Additionally, researchers have discussed ideal study design,4 appropriate sample size and analysis methods,4,6,9–11 and potential sports injury causation models.12 However, it appears that few investigators have compared data quality from various types of reporters.13–17 

In large, national surveillance studies, it is neither economically nor practically feasible in terms of research staff time to maintain personal contact with thousands of reporters on a weekly or even monthly basis. Thus, data reporters must be capable of providing consistent and accurate information with a minimum of one-on-one interaction or oversight from study staff. The largest, most successful sports injury surveillance studies to date relied on certified athletic trainers (ATs) to report athletic exposure and injury information.3,18–20 These studies have demonstrated that ATs are capable of reporting exposure and injury-related information consistently and accurately over an extended period of time. Currently, the only national sports injury surveillance study capturing injury and athlete-exposure (AE) data for US high school athletes is the National High School Sports-Related Injury Surveillance Study.3 This surveillance system, implemented in 2005, has been very successful in collecting high-quality data from a large, nationally representative sample of US high school athletes using ATs as reporters. Unfortunately, athletes at many US high schools do not have access to an AT. Thus, although the use of medically trained reporters such as ATs appears to be optimal for national surveillance, there is a need to investigate the feasibility of depending on other data reporters, such as coaches and athletes, at schools that lack access to ATs.

Our objective was to examine the feasibility of calling on high school coaches and athletes as data reporters in a national, Internet-based sports injury surveillance study, using the same methods that have already proven successful in the National High School Sports-Related Injury Surveillance Study.3 Specifically, our aims were to (1) compare the proportion of enrolled ATs, coaches, and athletes who participated; (2) describe overall injury patterns reported by ATs, coaches, and athletes; and (3) correlate injury data received from ATs, coaches, and athletes.

This paper presents results of the On-line Surveillance of High School Sports Injuries, a 3-armed, prospective injury surveillance study of US athletes participating in 5 varsity sports: football, boys' and girls' soccer, and boys' and girls' basketball. The On-line Surveillance of High School Sports Injuries was a substudy of the National High School Sports-Related Injury Surveillance Study, the methods of which have been described previously.3 Briefly, the parent study is an ongoing, nationally representative surveillance study in which all reporting is done by ATs. Before the 2006–2007 school year, we invited US high schools with at least 1 National Athletic Trainers' Association–affiliated AT with a valid e-mail address to participate in the parent study or substudy (or both).

We intended to randomly select 20 schools interested in participating in this substudy, regardless of whether or not the school was in the parent study. The remaining methods will focus on this substudy. As a participation incentive, schools were offered a $900 honorarium, along with individualized injury reports and a copy of the summary report after the study concluded. Before the school was officially enrolled in the study, a participation agreement form signed by the principal, athletic director, AT, and a coach from each of the 5 sports of interest had to be returned.

Once the school was enrolled, we prepared and shipped individual AT and coach study packets to the school's AT, with instructions for the AT to distribute study packets to each participating coach. Each AT and coach study packet included a brief letter describing the study's purpose, a training packet containing detailed study information, instructions for accessing the RIO (Reporting Information Online; The Research Institute at Nationwide Children's Hospital, Columbus, OH) Web site, and a log-on identification (every coach and AT received a unique log-on identification). The coach's study packet also contained student-athlete and parental consent/assent forms, athlete log-on identifications, and instructions for athletes on accessing the RIO Web site and reporting. The coach was instructed to distribute the informed consent/assent forms to all of his or her varsity athletes, to encourage these athletes to return their signed forms, and to provide the athletes with their study log-on identifications upon return of the form. To maintain athlete confidentiality, these consent/assent forms were to be maintained at the school rather than returned to study staff.

At each participating school, an online data collection tool, High School RIO, was to be used concurrently but separately by the AT, a varsity coach from each of the 5 sports of interest, and varsity athletes from each of the 5 sports of interest to report exposure and injury information. Screenshots from RIO are shown in the Figure. Enrolled ATs and coaches received weekly e-mails throughout the study reminding them to report and asking them to remind their athletes to report. Participating ATs were asked to complete 45 weekly exposure reports, 1 for each week from July 31, 2006, through June 10, 2007. If ATs were also participating in the larger parent study, they were instructed to report for all sports included in the parent study and to report for all athletes (varsity, junior varsity, and freshmen). Coaches and athletes were asked to complete a weekly exposure report for each week their sport was in session, and they were instructed to report for their sport only. The ATs, coaches, and athletes were instructed to submit reports independently. Unfortunately, despite our efforts to garner athlete participation, only 1 athlete ever accessed RIO and reported. Thus, athlete-reported data were not included in the results.

Weekly exposure reports collected AE information. The exact exposure and injury definitions provided to reporters in exposure and injury reports are displayed in the Figure. One AE was defined as 1 athlete participating in 1 practice or competition. A reportable injury had to meet the following conditions: (1) occurred as a result of participation in an organized high school competition or practice, (2) required medical attention, and (3) resulted in restriction of the high school athlete's participation for 1 or more days beyond the day of injury. If a player sustained multiple injuries during the same injury event, the reporter was instructed to use his or her opinion to identify and provide information on only the most serious injury. For each cited injury, reporters were asked to complete a detailed injury report describing injured player demographics (eg, age, height, weight), the injury (eg, site, diagnosis, severity), and the event leading to injury (eg, mechanism, activity). Reporters were able to view all data submitted throughout the study and update reports as needed (eg, need for surgery, days until resuming play). This study was approved by the Institutional Review Board at The Research Institute at Nationwide Children's Hospital.

Data were analyzed using SAS (version 9.0; SAS Institute Inc, Cary, NC) and SPSS (version 15.0; SPSS Inc, Chicago, IL) software. Because ATs were considered the gold standard for injury surveillance reporting, coach compliance in submitting exposure and injury reports was measured by comparing the number of submissions from coaches with the number of submissions made by their ATs.

Theoretically, if all participating ATs and coaches were aware of all injuries, made similar decisions regarding whether or not an injury met the study's injury definition, and submitted a report for every injury they felt met the study's injury definition, then each injury would have 2 duplicate injury reports submitted, 1 from the AT and 1 from the coach. To compare the consistency of answers between injury reports submitted for the same injury, we attempted to link each coach-submitted injury report with the AT-submitted injury report for the same injury. Within each school, the following criteria were evaluated to link these reports: (1) the reports were for the same sport, (2) the reports indicated that the same body part was injured, and (3) the reports indicated that the injuries occurred less than 1 week apart. In some cases, injury reports meeting just 2 of these criteria could be linked if many of the other variable responses were similar.

Despite extensive recruitment and retention efforts by study staff (multiple e-mails and phone calls), initial interest by 215 ATs, a $900 participant honorarium, and random selection of 20 high schools, only 18 US high schools returned the signed participation agreement form to enroll. All 18 schools were concurrently participating in the parent study. Thus, a total of 18 ATs (1 per school) and 79 coaches (3–5 per school, depending on school-specific sport availability) enrolled.

All 18 ATs at participating schools logged onto RIO at least once to report. However, only 34 of the 79 coaches (43%) logged onto RIO to report. Participation differed by school and by sport. Seven schools had at least half of their enrolled coaches participate (23 of 32 coaches), 9 schools had fewer than half participate (11 of 41 coaches), and 2 schools had 0 participate (0 of 6 coaches). Coach participation was highest in boys' basketball (61% of enrolled coaches participated), followed by football (60%), girls' basketball (41%), boys' soccer (36%), and girls' soccer (13%).

Participating ATs were asked to submit 45 exposure reports (1 per week for each of the 45 study weeks). The ATs submitted an average of 43.5 exposure reports (range, 32–45; 96.7% of the expected 45; Table 1). Coaches were asked to submit an exposure report every week their sport was in session, which was determined from examining their ATs' exposure reports. On average, each sport was in session for 14.8 weeks (range, 9–22 weeks). The 34 participating coaches submitted an average of 5.4 exposure reports (range, 1–20; 36.5% of the expected 14.8).

Of the 34 participating coaches, 11 (32.4%) calculated AEs incorrectly at least once. Of the total 194 coach-submitted exposure reports, 48 erroneously reported the number of practices or competitions (or both) in the week instead of the sum of the number of athletes participating in each practice or competition that week.

Several coaches submitted injury reports for junior varsity athletes despite being instructed to submit injury reports only for varsity athletes. Overall, ATs submitted a total of 586 injury reports, and coaches submitted 74 (12.6% of the ATs' 586; Table 2). Restricting the comparison to varsity athletes only, ATs submitted 338 injury reports, and coaches submitted 55 (16.3% of the ATs' 338). As noted earlier, coaches only logged onto RIO to complete exposure reports for an average of 5.4 study weeks. To determine whether these missing injury reports were from the weeks that coaches did not report, we examined injury report submissions restricted to the weeks the coach submitted exposure reports. Coach compliance increased only slightly, with coaches submitting 74 (26.1%) of the total 284 AT-submitted injury reports and 55 (36.7%) of the 150 AT-submitted varsity injury reports. The remaining analyses are limited to varsity athlete injury reports.

Reported injury diagnoses were similar between ATs and coaches, with both reporting sprains and strains most frequently (47.9% and 47.3%, respectively), followed by contusions (11.8% and 10.9%, respectively) (Table 3). Body sites of injury varied, with ATs reporting the ankle (21.6%) and knee (17.2%) most frequently and coaches reporting the knee (24.1%) and hip, thigh, and upper leg (22.2%). With regard to time loss, ATs reported a greater proportion of injuries resulting in more than 21 days' time loss (16.6% and 8.0%, respectively) whereas coaches reported a greater proportion of injuries resulting in 1 to 2 days' time loss (30.0% and 17.2%, respectively). The ATs and coaches reported similar proportions of injuries requiring surgery (7.2% and 8.0%, respectively).

Because coaches submitted 74 total injury reports, the total possible was 74 linked AT-coach injury report pairs. However, only 38 (51.4%) of the coach-submitted injury reports could be linked with an AT-submitted injury report. Of these 38 pairs, 31 were matched on all 3 criteria (same sport, same body site injured, and injury dates less than 1 week apart), 4 were not matched on injury date, 2 were not matched on body site injured, and 1 was not matched on sport. When we examined these 38 linked pairs, we found that the questions with the highest correlations were sport, athlete's year in school, weather and field conditions, and playing surface, for which the AT and coach answered similarly in more than 94% of linked pairs (Table 4). The lowest correlations were type of practice and sport-specific activity, for which only 27.3% and 39.5% of all linked reports, respectively, contained the same answers. Answer consistency within linked injury report pairs with regard to basic injury-related questions was highest for body site of injury (94.7%) and need for surgery (83.8%) but was lower for diagnosis (63.2%) and time loss (55.3%).

The ATs and coaches had similar proportions of nonresponses to most questions. However, approximately 20% of injury reports submitted by ATs did not contain height or weight information (Table 5). In contrast, coaches exhibited relatively few nonresponses for height (1.4%) and weight (9.5%), and coaches had the highest number of nonresponses for need for surgery (9.5%).

The large and growing number of high school athletes1 coupled with their high injury incidence3 highlights the urgency of developing an effective national injury surveillance system. By monitoring injury rates and patterns over time, such a surveillance system provides the foundation for developing and evaluating preventive interventions. In this study, the first to assess the feasibility of relying on coaches as data reporters for a national sports injury surveillance system using a geographically dispersed sample (ie, representing 15 states across all 4 US Census geographic regions and including both small [enrollment of 1000 or less] and large [enrollment of more than 1000] schools), we found that coaches had very low participation rates. Therefore, whenever possible, ATs should be the primary data reporters in national surveillance.

Despite 100% participation among ATs in the 18 study schools, fewer than half of the enrolled coaches participated. Some coaches may have signed their school's participation agreement form without intending to participate, perhaps because they felt pressured to do so by their AT, other coaches, or school officials. On average, coaches who did participate submitted only one-third of all expected exposure reports. Some coaches may have misinterpreted study instructions and believed they were only required to submit exposure reports for the weeks in which they had an injury to report.

When coaches did submit exposure reports, they occasionally reported exposures incorrectly by recording the number of practices or competitions rather than athlete-practices or athlete-competitions. For example, a coach with a team of 50 athletes who all practiced 5 times in a week was more likely to incorrectly report 5 athlete-practices than to correctly report 250 athlete-practices. In contrast, all ATs reported AEs correctly throughout the study. Coaches also underreported injury incidence, submitting fewer than one-fifth of the number of injury reports submitted by ATs. Coaches may not have had the time or desire to submit an injury report for every injury coming to their attention, or they may not have been aware of all injuries occurring on their team, particularly if the injured athlete presented directly to the AT and missed little playing time.

Obtaining accurate AE and injury incidence data is a crucial prerequisite for calculating true injury rates, as indicated by the injury rate equation:

If a study's reporters underreported injury incidence, injury rates would be artificially low. If AEs were underreported, injury rates would be artificially high. Either way, inaccurate injury rates may lead researchers to underestimate or overestimate injury incidence, leading to potentially incorrect conclusions or inappropriate or ineffective preventive recommendations.

Sports injury incidence is sometimes used to estimate the number of sports injuries occurring regionally or nationally. Underreporting of injury incidence by data reporters would lead to inaccurate regional or national injury estimates. For example, using AT-reported data, the National High School Sports-Related Injury Surveillance Study estimated that approximately 1.4 million injuries were sustained in 9 US high school sports during the 2005–2006 school year.3 If coaches had been called on as data reporters in place of ATs and if a similar sampling scheme had been employed, then the 84% underreporting that we found would presumably result in a nationally estimated incidence of fewer than 300 000 injuries, a deficit of more than 1 million injuries. Additionally, undersubmission of valid injury reports decreases study generalizability and increases the likelihood of obtaining skewed injury patterns. For example, if coaches are less likely to submit an injury report for athletes who miss more than 21 days of play, then injury patterns based on coach-reported data would incorrectly suggest that athletic injuries are less severe than they actually are.

Surprisingly, 36 coach-submitted injury reports could not be linked to an AT-submitted injury report. One possibility is that the reported dates of injury occurrence were too different, which could happen if a coach submitted the report several weeks after the injury occurred. Because few coaches keep an injury log book, they may have had difficulty remembering the date. Discrepancies may also have occurred in choosing the most serious injury during events in which more than 1 injury occurred. For example, if an athlete sustained both a concussion and a mild ankle sprain at the same time, the coach may have reported the ankle sprain, whereas the AT may have reported the concussion. Also, it is possible that coaches submitted injury reports for events not meeting the injury definition, such as an athlete who initially appeared to be injured but returned the next day.

Numerous discrepancies were identified between linked AT and coach injury reports. Although we cannot ascertain for certain which reports were correct, we assume that ATs are more likely to report correct injury-related information, such as injured body site, need for surgery, and diagnosis, because they are medically trained. Previously, athletic therapists and medical doctors participating in a Canadian sports injury surveillance study were found to record similar diagnoses in more than 80% of reports.21 Conversely, coaches may have been more likely to report correct event-specific information, such as activity at time of injury, as they are more likely to be present when the injury occurs. Future researchers should examine whether ATs have difficulties reporting event-related information accurately, so that any problems can be addressed.

We made extensive efforts to obtain athletes' participation in this study. The athletic director, AT, and coach for each sport of interest signed a participation agreement form indicating that they were aware that athletes were supposed to participate. We also sent ATs and coaches weekly e-mails reminding them to ask their athletes to report and asking them to contact us if they had any questions. Despite these efforts, only 1 athlete logged onto RIO to report. Athlete nonparticipation was likely a combination of athletes not being made aware of the study, athletes being aware of the study but choosing not to participate, and athletes wanting to participate but being unable to do so. Study materials were supposed to flow from study staff to ATs, from ATs to coaches, and from coaches to athletes. Although it is likely that nearly all ATs passed study materials to coaches because all ATs participated, it is possible that some coaches did not pass study materials to athletes. However, almost half of all coaches participated and 1 athlete participated, so at least some coaches passed study materials onto their athletes. Thus, at least several athletes were aware of the study but either chose not to participate or were unable to report. Regardless of the reason, this lack of athlete participation demonstrates that athletes are not suitable reporters for large, national surveillance studies. Researchers cannot feasibly contact, elicit participation, disburse and collect informed consent or assent, and conduct regular, season-long follow-up among thousands of athletes dispersed across the country. Our data show that attempting to do so through mediators such as coaches is ineffective. Although athletes may be able to report reliably and consistently in small, localized studies, when researchers can initiate regular, individualized contact,16 such personalized attention has limited feasibility in a large, national study.

Although ATs play an important role in the high school setting,22 budget constraints have resulted in a decreased number of ATs employed by US high schools.23 Unfortunately, athletes who lack access to the preventive and rehabilitative care provided by ATs may be more likely to sustain injuries or may experience longer recovery times than athletes at schools with ATs.24 If injury rates and patterns differ between schools with and without ATs, then national high school sports injury surveillance cannot be completely representative until either all high schools have access to ATs or researchers develop methods to successfully collect high-quality data from schools without ATs. Although we strongly believe that ATs should be relied on as reporters whenever possible in high school sports injury surveillance, findings from this study suggest that relatively labor-intensive methodologic modifications, such as soliciting and compensating coaches directly, will likely be required in studies calling on coaches as reporters.

Like all studies, this study had limitations. Although we considered ATs to be the “gold standard” for injury surveillance reporting, we do not know whether ATs always reported correctly. However, previous internal validity analyses of the National High School Sports-Related Injury Surveillance Study indicated sensitivity and specificity of injury reporting by ATs was greater than 95% (R.D.C., E.E.Y., C.L.C., unpublished data, 2008). In this study, we only included schools with a National Athletic Trainers' Association–affiliated AT. Although this restriction may limit generalizability if coaches and athletes take a greater interest in injury prevention when their school lacks medical personnel, it was a necessary limitation to directly compare AT, coach, and athlete reporting. Additionally, we did not know the availability of computers among coaches and athletes. However, ATs at these schools were able to access computers regularly, suggesting that coaches and athletes likely could have had computer access if needed.

In conclusion, this study demonstrates the importance of depending on reliable data reporters in high school sports injury surveillance. Whenever possible, ATs should be the primary data reporters in large, national studies. In high schools without access to ATs, researchers must be willing to devote large amounts of time and resources to achieving high levels of participation and study compliance among other data reporters.

This study was funded by the National Athletic Trainers' Association Research and Education Foundation, grant 905GGP008. The content of this report is solely the responsibility of the authors and does not necessarily reflect the official view of the National Athletic Trainers' Association.