Since 1982, the National Collegiate Athletic Association (NCAA) has collaborated with athletic trainers (ATs) to create the largest ongoing collegiate sports injury database in the world. This report provides an operational update of the NCAA Injury Surveillance Program (NCAA ISP) during the academic years 2014–2015 through 2018–2019.
The NCAA ISP used a convenience sampling technique via a rolling recruitment model. The ATs at contributing institutions voluntarily submitted data into their respective electronic medical record systems; common data elements were pushed to and maintained by the Datalys Center. The ATs provided information about all team-related activities, even if no injury occurred during that activity, as well as detailed reports on each injury, including condition and circumstances.
The NCAA ISP has a long-standing role in supplying NCAA stakeholders with crucial injury surveillance data, playing a critical part in safeguarding student-athletes participating in collegiate sports.
Sports injury surveillance plays a crucial role in the development and continued improvement of injury prevention practices.1,2 The routine monitoring of injury incidence offered by surveillance has been particularly important within National Collegiate Athletic Association (NCAA) sports due to the large volume of student-athletes competing as part of the NCAA within collegiate institutions across the United States.3 The NCAA formally established an injury surveillance system in 1982, and this system has evolved through a series of adaptations to reach its current form as the NCAA Injury Surveillance Program (NCAA ISP).4,5 The NCAA ISP has had the exclusive capacity to capture a comprehensive association-wide scope of sports-related injuries among NCAA athletes and since its inception has captured sport-related injury and exposure information on nearly all NCAA-sponsored sports.6–14 As a result, it has been possible to appraise the burden of sport-related injuries in this population through examinations at various time points. In its current form, the NCAA ISP is uniquely positioned to comprehensively survey the landscape of sport-related injuries among collegiate student-athletes and identify emerging patterns in injury incidence, providing data to inform injury prevention practices.
It is important to acknowledge that surveillance methodology routinely evolves with time and retains the flexibility to align with evolutions in the area of study. It follows that the NCAA ISP has also adapted after technological advances in injury documentation and progress in sports medicine research. These adaptations have been previously described at routine intervals.4,5,15 Participation among NCAA membership institutions, the particular elements captured by the system, and the operations of the surveillance system continue to change with each academic year, and it is therefore important to recharacterize the methods of the surveillance system. Accordingly, we summarize the operational methods of the NCAA ISP for the 2014–2015 through 2018–2019 academic years.
SURVEILLANCE SYSTEM STRUCTURE
The NCAA ISP is an injury surveillance system aimed at providing a comprehensive appraisal of injuries among NCAA student-athletes. It is the exclusive association-wide injury surveillance system targeting the NCAA student-athlete population, and it captures both time loss (TL) and non–time loss (NTL) injuries. The scope and breadth of the surveillance system in its current form make it well-positioned to capture the epidemiology of sports-related injuries among NCAA student-athletes. The NCAA ISP is funded by the NCAA and maintained by the Datalys Center for Sports Injury Research and Prevention, an independent nonprofit research organization specializing in sports injury surveillance and epidemiology. The methods of the surveillance system have been reviewed and approved as an exempt study by the NCAA Research Review Board. Although methods of the ISP before the 2014–2015 academic year have been characterized previously,4,5 we focus on the operational methods from 2014–2015 through 2018–2019, which notably vary from previous years.
The academic year was defined as beginning July 1 and ending June 30, resultantly spanning 2 calendar years.
Sport participation was divided into phases (or segments) of the competitive season, per the definitions of the NCAA. Preseason included all formal team activities conducted before the first regular season competition; regular season included all formal team activities from the first regular season competition through the last regular season competition; and postseason included all formal team activities after the last regular season competition through the last postseason competition. Nontraditional season was used to refer to formal team activities during the legislated nontraditional season, whereas out of season referred to training time that fell outside of traditional and nontraditional seasons but within the academic year, excluding summer. Summer season encompassed training time falling between the completion of the spring term and start of fall term or the start of fall preseason practice—whichever came first. Reporting was only required for the championship season as defined by NCAA legislation (the segment of the playing season beginning with the first allowed date of practice and concluding with the NCAA championship). This included preseason, regular season, and postseason, although information from all seasons was captured by the system.
An exposure was defined as any team-sanctioned athletic activity in which student-athletes were participating and “exposed” to the risk of injury due to participation. These activities included junior varsity and varsity competition, junior varsity and varsity scheduled team practice, captain's practice, scrimmage, strength and conditioning, skill instruction, and walk-through (not collected until 2018–2019). Individual workouts were not included.
Number of Athletes
The number of student-athletes participating in an athletic activity was defined as the number of athletes who were at risk for injury due to their participation. A nonparticipant was defined as any athlete not participating in the team activity due to an injury or some other conflict (eg, class, physician appointment).
A reportableinjury was defined as an injury that (1) occurred as a result of participation in an organized intercollegiate practice or competition and (2) required attention from an athletic trainer (AT) or physician, regardless of time loss. Multiple injuries occurring from 1 injury event were able to be reported. A time loss (TL) injury was any injury evaluated or treated by an AT or physician in which an athlete returned the day after or beyond with respect to the date of injury. A NTL injury was any injury evaluated or treated by an AT or physician in which an athlete returned to participation on the date of injury.
During the 2014–2015 through 2018–2019 academic years, the NCAA ISP used a convenience sample of NCAA membership teams via a rolling recruitment model. Institutions and teams were recruited to participate in the NCAA ISP using multiple communication streams. Datalys Center staff recruited participants in person at professional conferences. In addition, communication efforts via the Datalys Center website, emails, and social media as well as word of mouth were used. Furthermore, certified electronic medical record (EMR) vendors also supported recruitment efforts. In 2018, the NCAA initiated a charge promoting participation in the ISP using educational and informational pushes to athletic directors, conference commissioners, and other leadership stakeholders. A participation summary is provided in Table 1. As described in subsequent sport-specific articles within this issue, this contributed to a marked increase in participation, resulting in approximately 30% of NCAA membership institutions participating in the ISP during the 2018–2019 academic year.
The ATs at participating institutions contributed data by entering information into their respective EMR systems or injury documentation applications. The NCAA ISP uses the common data element (CDE) strategy, allowing data to be pushed from various EMR systems and injury documentation applications. From 2014–2015 through 2018–2019, these systems included the Athletic Trainer System (ATS, Keffer Development Services), Presagia Sports (Presagia Corp), the Sports Injury Monitoring System (SIMS, FlanTech Computer Service), SportsWare (Computer Sports Medicine Inc), Vivature (Vivature Inc), and the Injury Surveillance Tool (IST, Datalys Center). The CDE export standard allows ATs to document injuries as part of their typical clinical practice instead of duplicating efforts to report injuries solely for the purposes of participation in the ISP. The ATs completed detailed reports on exposures (season, event type [eg, competition, practice], number of athletes) and injuries (condition [eg, site, diagnosis], circumstances [eg, activity, mechanism, playing surface]). A full listing of exposure and injury variables is provided in Table 2; concussion symptoms, which are captured separately as dichotomous variables for concussion observations only, are listed in Table 3. During academic years 2014–2015 through 2018–2019, response options were added to certain variables (eg, field location, activity at time of injury) to better accommodate sport-specific nuances. The ATs were able to view and update previously submitted information during the course of an academic year. Although not required, ATs had the ability to capture sports-related adverse health events beyond the scope of a reportable injury as defined above, such as illnesses and skin infections, that could not be directly associated with a team-sanctioned activity. Before contributing data, ATs received training materials regarding ISP participation either from Datalys Center staff or directly from their EMR vendor representatives. The ATs were not financially compensated for their data collection efforts; however, they could claim continuing education units each reporting period for their participation.
Given the CDE strategy used for data collection, integrating EMRs with the ISP is an essential component of the data collection process. The EMR software vendors contributing to the NCAA ISP underwent a certification process, modifying their systems and embedding secure data-transmission protocols that allowed the transfer of deidentified records to secure Datalys Center servers. Before export, data were stripped of identifying information, tagged with a unique 16-digit alphanumeric code, and encrypted. This process is Health Insurance Portability and Accountability Act (HIPAA) and Family Educational Rights and Privacy Act (FERPA) compliant. All incoming data were evaluated through a quality control (QC) process, and Datalys Center staff assisted ATs in resolving any concerns regarding invalid values.
The process of verification and review of incoming data was an essential component of the NCAA ISP, ensuring that data of the highest fidelity were retained in analysis datasets. Before data entered the research database, they were checked for accuracy and completeness through a process automated by a proprietary verification engine (VE). The VE identified partial or failed submissions due to errant or missing values. Datalys Center staff routinely conducted additional inspections and contacted ATs for assistance in reconciling any data quality issues. All fall sports data reported to the ISP were considered final as of February 15 in each academic year; data for all other sports were considered final as of June 30 in each academic year. Records submitted or modified beyond that date were not included or reflected in the research datasets.
Exposure data were considered valid if (1) they occurred between July 1 and June 30 (during the academic year) and (2) the number of athletes participating in the reported event was nonzero and nonmissing. Zero or missing values for the number of athletes remaining after the QC process were replaced with mean imputations estimated on the basis of all valid AE data captured from the same year, sport, division, and exposure event type. Beginning in 2016–2017, competition schedules provided by the NCAA for team sports and posted on school websites for individual sports were used to confirm accuracy in reporting of season (ie, preseason, regular season, postseason, or out of season) and event type (eg, competition, practice); both variables were duly updated as needed.
Injury events with multiple reported injuries were identified and evaluated for duplicate submissions. Records were retained if each reported injury had a separate specific injury diagnosis or different affected body parts (eg, ankle and knee injuries occurring in the same injury event); otherwise, duplicate injuries were removed. The VE contained a validation process to ensure agreement between a categorical injury outcome variable measuring days missed due to an injury and date of return to participation; ATs were asked to reconcile any disagreement during the QC process. As an addition to existing practices, from 2014–2015 through 2018–2019, if a discrepancy remained after the VE check and QC process, both injury outcome and date of return were set to missing because there was no valid method to reconcile the inconsistency.
Qualifying Criteria for Inclusion in Analysis Datasets
Qualification criteria details have been previously documented.15 To ensure data submitted by participating teams reflected an entire championship season, 2 criteria were used to determine a team's qualification on the basis of reported exposures: (1) a minimum of 8 weeks of exposure activity must have been reported and (2) at least 80% of regular season competitions must have been reported. In juxtaposition to previous years of the NCAA ISP,15 the reporting of at least 48 practice or competition exposures was specifically used to determine whether a team met the first criterion. Competition schedules for each team were used to determine whether the second criterion was met. Furthermore, unless a zero-injury season was verified by the reporting AT, at least 1 injury must have been reported during the championship season. On occasion, adaptations were made to accommodate nuances specific to the different EMR software vendors.
Underreporting is ubiquitous in sports injury surveillance. Specific to the NCAA ISP, this may be attributable to competing demands on ATs' time and the dynamic nature of the athletic training facility environment. It has been previously estimated that the ISP would capture approximately 88% of all TL medical care injury events.16 Under the assumption that underreporting does not vary by sport, year, school, or division, weights were subsequently further adjusted to correct for underreporting by scaling weighted counts up by a factor of 0.883−1.
The NCAA is a dynamic sporting microcosm involving elite student-athletes across various sports. The NCAA Injury Surveillance Program has been critical in expanding the understanding of injury incidence in this population.5,7,9–11,13,14 Given its history, scope, and continued improvements, the NCAA ISP most closely represents the complete spectrum of sports-related injuries occurring in this population. Particularly during the 2014–2015 through 2018–2019 academic years, participation in the ISP improved notably across most NCAA-sponsored sports. As a result, data currently captured within the surveillance system are more representative of the larger association-wide population than in previous years. Furthermore, during 2014–2015 through 2018–2019, new data elements were introduced to the system that have improved the comprehensiveness with which injury records are captured. Refinements in data collection and management practices have ensured that data of the highest possible fidelity are retained within analysis datasets, with minimal burden on the contributing ATs. These adjustments have been made with the objective of providing the most stable platform from which to identify emerging injury-related patterns among collegiate athletes and subsequently inform interventions oriented towards improving athlete health and safety. It is important to acknowledge the necessity of adapting surveillance methods to match the cadence of technological growth and advancements in sports medicine research. With that said, exposure ascertainment in sports injury surveillance has been previously discussed as a challenge.17 Whereas current methods used by the NCAA ISP are motivated towards ensuring valid data are collected as closely to real time as possible, it is important to note that exposure ascertainment still presents a notable burden on participating ATs and warrants further refinement. Also, as noted previously, participation in NCAA ISP markedly improved during the years 2014–2015 through 2018–2019. The constant juxtaposition of participation to the qualification process and weighting estimations described herein is relevant, and with continued improvements in participation, the qualification process and the weighting structure may require adjustments. Although it was considered premature to modify current qualification processes and weighting estimations, it is salient to continue monitoring participation after 2018–2019 to revisit these topics in the near future. Furthermore, the aforementioned definition of reportable injuries affords ATs an element of discretion in reporting. This flexibility may introduce reporting heterogeneity between ATs, particularly with regard to illnesses and infections, and improvements in definitional clarity (of reportable events) warrant consideration moving forward. Finally, the current practice of extracting CDEs from existing commercial software is understandably an efficient reporting framework; however, it is dependent on the implicit understanding that the aforementioned operational practices continue to match the pulse of technological adaptations in commercial injury-tracking software. Whereas significant strides have been made to ensure that the surveillance system accommodates a wide array of commercial EMR systems, further adaptations to operational procedures will be needed over time. Ultimately, continued improvements in surveillance methods will ensure that the NCAA ISP remains a valuable asset in informing health and safety initiatives for NCAA student-athletes.
The NCAA Injury Surveillance Program was funded by the NCAA. The Datalys Center is an independent nonprofit organization that manages the operations of the NCAA ISP. The content of this report is solely the responsibility of the authors and does not necessarily represent the official views of the funding organization. We thank the many ATs who have volunteered their time and efforts to submit data to the NCAA-ISP. Their efforts are greatly appreciated and have had a tremendously positive effect on the safety of collegiate student-athletes.
Authors Avinash Chandran, PhD, MS, and Sarah N. Morris, PhD, have contributed equally to manuscript preparation. The articles in this issue are published as accepted and have not been edited.