The Value of Various Assessment Techniques in Detecting the Effects of Concussion on Cognition, Symptoms, and Postural Control

A common question among certified athletic trainers and other sports medicine professionals is “Which assessment tools should be incorporated into a concussion assessment plan?” Although a plethora of articles related to concussion assessment have been published this decade, Broglio and Puetz1 provide one of only 2 meta-analyses to date in the area of sport-related concussion that synthesize this vast body of knowledge. The main findings show large negative effects for neurocognitive tests, postural stability testing, and self-report symptom scales at the initial assessment (1.59 ± 1.74 days; range, 0.003–7 days) after injury, providing quantitative data to support the multifaceted approach that has been advocated by several consensus and position statements.2–4 

The authors reported both the Z statistic and effect sizes, which may be useful for clinicians interpreting these data. The Z statistic reflects the probability of a true or real finding, as opposed to one occurring by chance, and is used to derive the P value, which indicates whether the finding is statistically significant. The Z score is a standardized variate taken from the normal distribution. That is, 68% of the normal distribution falls between −1 Z and +1 Z, and 95% of the normal distribution falls between −2 Z and +2 Z. All the Z scores reported in this meta-analysis were greater than 1.0, indicating that the differences in cognition, postural stability, and reported symptoms were the result of true differences, not chance. Effect sizes are useful in understanding the magnitude of deficits reported after concussion. The effect sizes noted in this paper were calculated by subtracting the baseline score from the follow-up score and dividing by the SD of the baseline score.1 Effect sizes can be interpreted as small (<0.20), moderate (0.21–0.79), or large (>0.80). The larger the effect size, the more that particular domain was affected after a sport-related concussion.

According to the findings of this meta-analysis, clinicians may gain the greatest amount of information regarding postconcussion deficits by using the Standardized Assessment of Concussion (SAC), a measure of postural control (such as the Balance Error Scoring System [BESS]), and a self-report symptom scale during the initial assessment on the sideline or within the first 48 hours postinjury. Additionally, the continued use of a more complex neurocognitive battery (pencil and paper or computerized), postural control measure, and self-report symptom scale may provide more information during subsequent postinjury follow-up assessments and aid in making return-to-play decisions.

Not surprisingly, self-report symptoms demonstrated the largest effects at both the immediate and follow-up assessment points, greater than the effects on neurocognitive tests in the studies that assessed both measures. Also, the number of assessments performed during the 14-day follow-up window was related to the effect size. As the number of assessments increased, the number of self-report symptoms decreased, thus demonstrating recovery over time. Although the number of administrations of certain assessment tools may be a concern with respect to learning effects, this is not likely an issue with the use of a symptom scale or checklist that measures an athlete's self-report status. These findings, combined with the recommendations2 that no athlete begin a return-to-play progression until he or she is asymptomatic, should reiterate the need for all clinicians to use some tool to assess self-report symptoms on all athletes with suspected concussions. The tool can range from a basic symptom checklist, in which the athlete is asked about symptoms he or she is experiencing and the clinician records a simple yes or no, to graded symptom scales, which allow the number of symptoms, their severity, and their duration to be quantified.2 In fact, some computerized neurocognitive tests and the Sport Concussion Assessment Tool4 have a graded symptom scale built into the tool. However, clinicians should not use these tools as the only means of determining recovery or in return-to-play decisions.

Of the neurocognitive tests included in the meta-analysis, the test with the largest negative effect at the immediate postinjury assessment was the SAC. This finding supports the work of other authors investigating the SAC who showed that it was most sensitive in detecting cognitive deficits in the first 48 hours after concussion; sensitivity diminished after that time.5 The SAC is often referred to as a mental status test because it is most useful on the sideline and in the immediate postinjury phase. With respect to more complex neurocognitive measures, the largest negative effect on cognition reported at the follow-up assessment was identified using pencil-and-paper batteries. These results are similar to those reported by Belanger and Vanderploeg,6 who reported overall mild to moderate negative effects (d  =  0.49) in cognitive function immediately after concussion and little effect, indicating full neurocognitive recovery, by 7 to 10 days postinjury. Additionally, comparable effect sizes for computerized and paper-and-pencil neurocognitive batteries were noted.6 Although a lesser effect was found in both of these meta-analyses with computerized neurocognitive programs, such programs may be more clinician friendly in terms of use by athletic trainers.

Although only 6 of the included studies reported on postural control, large negative effects were noted at both the immediate and follow-up assessment points, demonstrating the need for assessment of postural control as part of a concussion protocol. Of the included studies, 3 groups used a computerized force platform, whereas 2 groups used a clinical measure (the BESS), and 1 group assessed postural control with both types of measures. Because of the limited number of investigators using each method, the authors were not able to directly compare the force platform results with those on the BESS. However, there are several clinical advantages to using the BESS as a measure of postural control. It is cost effective, requiring only a stopwatch and a foam pad, and it takes only 5 minutes to administer. In addition, the foam pad can be transported easily, allowing for sideline assessment of postural control after concussion. The conclusions drawn by the authors regarding the use of a multifaceted concussion assessment plan are further supported by the work of others,5,7,8 who demonstrated increased sensitivity when a battery of tests was compared with the use of each individual tool. A simple battery of the SAC, BESS, and a graded symptom scale was 94% sensitive at the time of injury, compared with the SAC (80%), BESS (34%), or graded symptom scale (89%) alone.5 Similarly, adding self-report symptoms to the ImPACT (Pittsburgh, PA) computerized neurocognitive battery increased the sensitivity from 64% to 83%.8 Finally, Broglio et al7 demonstrated higher sensitivities for ImPACT, the Headminder Concussion Resolution Index (New York, NY), and a pencil-and-paper battery when self-report symptoms and postural control were added to each battery.

Although the authors of this meta-analysis concluded that their findings support the use of a multifaceted assessment battery, surveys of clinical practice show that athletic trainers are often not using these tools.9,10 In the second of these surveys, 77% of athletic trainers reported using a symptom checklist, up from the 33% found during the first practice pattern survey. Use of both the SAC and the BESS increased among athletic trainers, as reported in 2004, from 11% to 43% and 5% to 15%, respectively. Lastly, when participants were asked about the use of neurocognitive testing, similar response rates were noted on both surveys (15% and 16%). In general, a greater percentage of athletic trainers surveyed in 2004 used these objective concussion assessment tools than in the earlier study.10 Interestingly, a decrease in the use of neurocognitive tests was noted at both the collegiate and high school levels, possibly indicating barriers to implementing this type of testing in those settings.

A multifaceted approach to concussion assessment has been advocated via consensus statements2,3 and has now been substantiated with research,1,5,7,8 but each clinician must also evaluate the practicality and feasibility of each component before developing his or her own concussion plan. When developing an assessment plan for a particular setting, factors that must be considered include cost, time required to test, equipment, personnel, and administrative support. Clinicians will need to take the information presented in the meta-analysis and use their own clinical experience to determine how best to use these data to practice in an evidence-based manner.

Several limitations of the meta-analysis and the current body of literature should be addressed to allow clinicians to appreciate the value in the findings and to judge their relevance to clinical practice. First, in the meta-analysis, neurocognitive assessments were broadly grouped as either pencil-and-paper or computer assessments. This classification does not allow analysis of specific neurocognitive domains, such as reaction time or verbal memory. Also lacking is information on athletes and high-risk concussion activities (eg, military service) other than that associated with male, collegiate Americans participating in football. The authors of the meta-analysis reported that individuals were predominately male (92.9%) American football athletes (72%), with a mean age of 19.0 ± 0.40 years. Therefore, these findings may not be generalizable to female athletes, adolescent athletes, or individuals participating in other sports or activities. Clinicians should use caution when reviewing the evidence-based concussion information for young athletes, especially time components for return to play and normative values for objective tests.

The current body of literature is also incomplete with respect to studies investigating sideline concussion assessment tools and factors that may influence performance, such as environmental demands, sport-specific demands (eg, testing an ice hockey athlete with skates on), and learning effects with repeat administration. There is also little evidence regarding the long-term prognosis for postconcussion syndrome, athletes with a return-to-play window that is longer than 14 days, and athletes with a history of multiple concussions. Although great strides have been made in improving our knowledge of concussion assessment and recovery, future researchers should begin to investigate these unanswered or less-studied areas to aid clinicians in making evidence-based decisions in the management of concussions for all athletes.