Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) Practices of Sports Medicine Professionals

The assessment and management of sport-related concussions should be a multifaceted approach that consists of a clinical examination, completion of a self-reported symptom checklist, postural assessment, and neurocognitive testing.1 Computerized neurocognitive testing has been deemed2–5 a more objective measure for determining the subtle cognitive changes associated with concussion. Recently, numerous concussion consensus statements and position papers1,6–9 have supported and emphasized the use of baseline preinjury and serial postinjury follow-up neurocognitive testing protocols.

Because of the difficulty in detecting the signs and symptoms that often accompany concussion, baseline neurocognitive testing has resulted in increased detection of postconcussion neurocognitive impairments.2,5,10,11 Moreover, baseline neurocognitive testing provides the most accurate representation of an athlete's preinjury cognitive status. The need for individual baseline examinations arises from individual differences in cognitive performance in the areas of attention, memory, concentration, information processing, and reaction time. Without this information, it is difficult to ascertain if a concussed athlete's postconcussion neurocognitive scores are the result of concussion or individual variability. In addition, baseline neurocognitive tests may be used as a tool to determine concussion resolution for return-to-play decisions.7 Although normative data may be valuable in clinical cases when baseline scores are not available for each athlete, collecting preconcussion and postconcussion neurocognitive data allows sports medicine professionals to track the cognitive recovery of each concussed athlete, rather than using a universal or “one-size-fits-all” approach to managing concussion.2,12,13 However, no authors have investigated the compliance or practice trends of sports medicine professionals using baseline neurocognitive testing at the high school and collegiate levels.

When baseline data are not available, neurocognitive scores can be compared with normative data. Normative data currently exist for sex, age, and educational level.14 However, other factors, such as history of concussion, race, and acculturation, are usually not included in these data. Therefore, it is important that clinicians conduct baseline tests for postconcussion comparisons.

Computerized neurocognitive testing provides accurate reaction time calculation, randomization of test trials, and automation of data collection and analysis. Given the large number of athletes participating within collegiate and high school athletic programs, computer-based neurocognitive screening measures may be beneficial. Despite the increased use of neurocognitive test batteries, little is known about the integration of baseline neurocognitive testing into concussion assessment and management. Therefore, the purpose of our study was to examine the practice trends of certified athletic trainers (ATs) using baseline neurocognitive testing at the high school and collegiate levels.

Approval for the study was granted by the university's institutional review board. Approximately 1209 American institutions (404 high schools, 805 universities and colleges) listed on the ImPACT Web site (http://www.impacttest.com) were contacted via e-mail regarding participation. The ImPACT neurocognitive test battery is a computer-based program for assessing neurocognitive function and concussion symptoms and is the most widely used computerized testing program in the sports setting. This neurocognitive test battery consists of 3 categories: demographics, concussion symptoms, and neurocognitive tests. Specifically, the software program consists of 6 modules that evaluate attentional processes, verbal recognition memory, visual working memory, visual processing speed, reaction time, numerical sequencing ability, and learning.15 

A 20-item survey was developed for the purpose of evaluating high school and collegiate institutions' neurocognitive testing practices and protocols. An expert panel of ATs and neuropsychologists reviewed the survey for content and face validity. The survey was then sent to the head AT at each institution. In the event that the head AT was not responsible for conducting ImPACT testing, he or she was asked to forward the e-mail to the individual who conducted ImPACT testing at that institution. A follow-up e-mail was sent to the head AT 3 weeks after the initial e-mail. By completing and returning the online survey, participants provided implied consent. The e-mail explained the study and gave an online link to the survey, which was hosted by SurveyMonkey.com (Menlo Park, CA). The survey took approximately 5 to 10 minutes to complete. All responses were returned to the survey Web site as anonymous data. Participants could withdraw at any time without penalty and were allowed to skip any questions they did not wish to answer.

Demographic information (eg, education level, years of experience as an AT, employment setting) was collected from all respondents. Participants were then asked (1) to specify the number of years they had been using ImPACT and their protocols and practices when using this tool, (2) if they were taking the time to administer baseline testing and to ensure the validity of baseline tests, (3) if normative data were used in the absence of a valid baseline, (4) to identify the sports in which athletes underwent baseline ImPACT testing, and (5) when they first readministered ImPACT after a concussion. Other items addressed subsequent retest protocols and methods used to make return-to-play decisions.

Respondents were also given 2 scenarios of reported symptoms and ImPACT scores and were asked about making return-to-play decisions. First, if an athlete was still reporting symptoms but ImPACT scores were back to baseline, participants were asked if they would allow the athlete to return to competition. Second, if an athlete was no longer reporting symptoms but had below-baseline scores on ImPACT, participants were asked if they would allow the athlete to return to competition. Finally, participants were asked if they had attended an ImPACT workshop and who was responsible for interpreting postconcussion ImPACT scores. Survey data were analyzed using descriptive statistics. All statistics were calculated using SPSS (version 15.0; SPSS Inc, Chicago, IL).

Of the 1209 institutions ATs contacted via e-mail, a total of 399 ATs (272 men, 127 women) completed the survey, for a response rate of 32.7%. Respondents reported an average of 13.5 (± 8.3) years of experience as an AT. More than one-third of participants had earned an MS degree (155/399 [38.8%]), followed by a BS degree (75/399 [18.8%]) and an MEd degree (41/399 [10.3%]) (Table 1). Over half of the ATs graduated from an accredited athletic training education program (209/399 [52.4%]). The most common employment setting was the high school (167/399 [41.9%]), followed by the university (162/399 [40.6%]) and the clinic (28/399 [7.0%]) (Table 2).

Respondents reported using ImPACT for 3.27 ± 2.25 years. Almost all participants reported administering baseline testing to their athletes (378/399 [94.7%]); however, only half examined whether or not the baseline tests were accurate (207/378 [54.8%]). A third of respondents who administered baseline tests (123/378 [32.5%]) readministered them every 2 years, with the majority of these retests taking place at the high school level (Table 3). When baseline data were not available, 81% of respondents compared the test scores with normative data. Most respondents administered baseline ImPACT tests to football players (334/378 [88.4%]), followed by women's soccer players (298/378 [78.8%]), men's soccer players (269/378 [71.2%]), men's basketball players (259/378 [68.5%]), and women's basketball players (251/378 [66.4%]) (Table 4).

Various methods were used by ATs to assess concussions. All participants completed a clinical examination (100%); this step was followed by a computerized neuropsychological test (349/399 [87.5%]) and a physician recommendation (342/399 [85.7%]) (Table 5). More than half of the respondents (215/399 [53.9%]) administered the first postconcussion test 1 to 2 days after the injury (Table 6). In addition, one-third of respondents (120/399 [30.1%]) reported administering the second postconcussion test after the athlete was symptom free (Table 7).

When presented with a scenario on return-to-play decisions, 95.5% (381/399) of ATs would not return an athlete to competition despite a return to baseline performance on ImPACT if the athlete was still experiencing concussion symptoms. When asked if they would return an athlete who is symptom free but who scores below ImPACT baseline scores, 86.5% (345/399) responded no, 9.8% (39/399) responded yes, and 3.8% (15/399) indicated that it depended on the importance of the competition. Additional results indicated that both ATs and a physician interpreted the ImPACT results 27.8% (111/399) of the time, followed by interpretation by an AT alone (72/399 [18.1%]), a physician alone (43/399 [10.8%]), and then a neuropsychologist alone (27/399 [6.8%]) (Table 8). Finally, fewer than half of the participants had attended an ImPACT training workshop (168/399 [42.1%]), with only 26.4% (19/72) of ATs who examined ImPACT data reporting attendance at an ImPACT workshop.

We investigated the computerized neurocognitive testing practices of ATs. Overall, the majority of ATs administered baseline neurocognitive testing to their athletes; however, only half reported verifying the validity of these results. Our findings have significant implications, because if baseline scores are invalid as a result of poor motivational efforts or misinterpretation of instructions, sports medicine professionals cannot accurately interpret neurocognitive status after a concussion. Comparing postconcussion neurocognitive test scores with invalid baseline scores could predispose an athlete to being prematurely cleared for returning to competition, which could in turn potentially place the athlete at risk for catastrophic consequences.16 

Ensuring the validity of baseline neurocognitive testing is recommended in user and clinical interpretation manuals.17 Specifically, the ImPACT clinical interpretation manual17 provides suggestions for ensuring validity on baseline test administrations, yet our results indicate that these instructions are often ignored. A baseline test is invalid if the impulse control composite score is greater than 30 (indicating that the test participant was not paying attention, did not understand the directions, or purposefully “sabotaged” performance to ensure a low baseline score), the processing speed composite score is less than 25, reaction time scores are greater than 0.80 (in an athlete with no history of concussion or learning disabilities), the verbal memory composite score is below 70, and the visual memory composite score is below 60.17 Any athlete who scores below these cutoff values should be retested at a later date.

Nearly all of the ATs surveyed indicated that they conduct baseline testing; however, just over two-thirds of ATs conducted baseline tests with men's and women's soccer and basketball players. Concussions have been reported18–21 to constitute 2% to 11% of all soccer injuries. Research by Barnes et al18 indicated that male and female soccer players are at a 50% and 22% risk, respectively, of sustaining a concussion within a 10-year period. Additionally, Boden et al19 indicated that 17 (59%) collegiate men and 12 (41%) collegiate women were diagnosed with concussions over 2 soccer seasons. Although concussion rates for basketball players are slightly lower than those for football and ice hockey players, concussions in basketball players still accounted for 4.7% and 3.2% of all injuries for collegiate females and males, respectively.22 In addition, high school basketball concussions accounted for 6.2% of all injuries in females and 5.7% of all injuries in males.21 Thus, concussions in basketball and soccer players are relatively comparable with those in football (6.0%) and hockey (7.9) players.22 Although normative data are available for computerized neurocognitive test batteries, baseline measures still provide the most reliable and accurate comparisons for postconcussion measures. Therefore, ATs should administer baseline neurocognitive tests to all collision-sport and contact-sport athletes to ensure accurate management of sport-related concussions.

Our results revealed inconsistent use of postconcussion protocols. Only half of the respondents administered the first postconcussion test within 1 to 2 days postinjury. A total of 15% administered the first posttest either when the athlete was symptom free or within the first 24 hours. Researchers1 have suggested 2 common retest protocols. A fixed time protocol (eg, 2 days postconcussion and 1 week postconcussion) can be implemented until the athlete is back to baseline. This type of protocol is effective in tracking improvement (ie, recovery of cognitive function) and is very popular in research studies.23,24 Another recommendation for administering postconcussion neurocognitive testing is to test only when athletes are asymptomatic.1 This method eliminates practice effects and decreases cost and time due to multiple test administrations.

Current consensus statements and position papers1,6,7,9 recommend that athletes should not return to play until they are asymptomatic and neurocognitive scores are back to baseline. It appears that these guidelines and recommendations are being followed, as we found that 86% of ATs would not return even an asymptomatic athlete to competition if he or she was still clinically impaired on ImPACT, whereas 95% would not return an athlete to competition if he or she was still symptomatic but ImPACT scores had returned to baseline values. Returning an athlete to play too soon has been shown16 to increase the risk of cumulative neurocognitive impairments and potential catastrophic injury associated with second-impact syndrome. Continued education and awareness of potential problems associated with premature return to play are essential for the health and welfare of athletes.

The qualifications required for the interpretation of computerized neurocognitive tests have received little attention in the literature. The current computerized neurocognitive testing batteries available have been made “user friendly” with respect to interpreting postconcussion scores. However, many of these tests have been based on paper-and-pencil versions that require training and years of experience. We found that the majority of ATs are interpreting ImPACT results without attending a neuropsychological testing workshop. This workshop is not a requirement, as interpretation guidelines and recommendations are documented in the ImPACT user manual. Yet, considering the high rate of ATs who did not double check for baseline validity in the current study (which is also recommended in the user manual), it is possible that postinjury ImPACT data are being interpreted incorrectly.

We focused on only 1 subset (ie, ImPACT users) of neurocognitive testing in concussion management. As Ferrara et al25 and Notebaert and Guskiewicz26 reported, relatively few ATs (15% and 18%, respectively) used neurocognitive testing to manage concussion. Notebaert and Guskiewicz26 also suggested that these low numbers may be the result of limited accessibility to computerized equipment and neuropsychologists for consultation, inadequate resources and funding, lack of experience and knowledge of neuropsychological testing, and positional time constraints. More specifically, Notebaert and Guskiewicz26 reported that ATs with more experience in the field used neurocognitive testing more often than did those with less experience, and ATs working at colleges and universities used it more often than did those working at high schools. Although we did not examine overall neurocognitive testing usage patterns among ATs, issues of time, access, knowledge, and experience might play roles in the implementation of baseline neurocognitive testing. Hence, future researchers should explore the barriers to neurocognitive testing and strategies for mitigating them among ATs. For example, investigators might compare computerized neurocognitive testing implementation practices of ATs who complete a training workshop with those who receive no training.

This study is not without certain limitations inherent to survey research. Our response rate of 33% was low. Such a low rate might have provided an inaccurate representation, as several schools could receive medical coverage from 1 nonrespondent (eg, a sports medicine clinic responsible for 10 high schools). Nonetheless, the response rate for our study was similar to that of previously published concussion management surveys of ATs (34% in Notebaert and Guskiewicz26). In addition, not all institutions included in the study offered all the listed sports. Consequently, the proportions of specific sports that were baseline tested may not accurately reflect actual ImPACT use. Another limitation to this study was the use of only those institutions listed on the ImPACT Web site. We contacted the developers of HeadMinder (New York, NY), Automated Neuropsychological Assessment Metrics (Defense and Veterans Brain Injury Center, Washington, DC), and CogState Sport (Melbourne, Australia) and invited them to participate in our study, but they were unable to provide access to a list of institutions currently using their computerized neurocognitive software. Given the similarities of these computerized neuropsychological testing programs, we anticipate that our findings have implications for users of all 4 testing programs. Developers of all computerized neuropsychological testing programs are urged to emphasize to users the importance of validating baseline tests before making return-to-play decisions based on comparisons with postconcussion scores.

A variable that may have influenced the reported use of baseline testing was the employment setting of the health professional. A total of 7% of respondents were employed in a clinical setting and, therefore, they may not have had the opportunity to baseline test athletes or to administer a retest protocol. These professionals (eg, ATs who were also clinical neuropsychologists or neurologists) are often involved only in postconcussion management through referrals from colleges and high schools and typically are not present during preseason baseline ImPACT testing.

As neurocognitive testing increases in popularity in the sports medicine field, it is important for practitioners to take the time to use this tool properly. In addition, practitioners could benefit from reviewing pertinent material (eg, user manuals and relevant publications) on neurocognitive testing administration and interpretation. This information will not only help them interpret and understand the scores but will also place them in a position to educate and help the concussed athlete understand the meaning of the scores. Such knowledge could also enhance communication and adherence to further clinical recommendations made by medical professionals. Future researchers should focus on expanding and improving educational efforts for practitioners using neurocognitive testing as well as other tools (eg, symptom checklists and postural assessments) in the management of sport-related concussion.