Broglio SP, Puetz TW. The effect of sport concussion on neurocognitive function, self-report symptoms, and postural control: a meta-analysis. Sports Med. 2008;38(1):53–67.
How effective are various concussion assessment techniques in detecting the effects of concussion on cognition, balance, and symptoms in athletes?
Studies published between January 1970 and June 2006 were identified from the PubMed and PsycINFO databases. Search terms included concussion, mild traumatic brain injury, sport, athlete, football, soccer, hockey, boxing, cognition, cognitive impairment, symptoms, balance, and postural control. The authors also handsearched the reference list of retrieved articles and sought the opinions of experts in the field for additional studies.
Studies were included if they were published in English; described a sample of athletes concussed during athletic participation; reported outcome measures of neurocognitive function, postural stability, or self-report symptoms; compared the postconcussion assessments with preseason (healthy) baseline scores or a control group; completed at least 1 postinjury assessment within the first 14 days after the concussion (to reflect neurometabolic recovery); and provided enough information for the authors to calculate effect sizes (means and SDs at baseline and postinjury time points). Selected studies were grouped according to their outcome measure (neurocognitive function, symptoms, or postural control) at initial and follow-up (if applicable) time points. Excluded articles included review articles, abstracts, case studies, editorials, articles without baseline data, and articles with data extending beyond the 14-day postinjury time frame.
From each study, the following information was extracted by one author and checked by the second author: participant demographics (sport, injury severity, incidence of loss of consciousness, and postconcussion assessment times), sample sizes, and baseline and postconcussion means and SDs for all groups. All effect sizes (the Hedge g) were computed so that decreases in neurocognitive function and postural control or increases in symptom reports resulted in negative effect sizes, demonstrating deficits in these domains after concussion. The authors also extracted the following moderators: study design (with or without control group), type of neurocognitive technique (Standardized Assessment of Concussion, computerized test, or pencil-and-paper test), postconcussion assessment time, and number of postconcussion assessments.
The search identified 3364 possible abstracts, which were then screened by the authors, with 89 articles being further reviewed for relevancy. Fifty articles were excluded because of insufficient data to calculate effect sizes, lack of a baseline assessment or control group, or because the data had been published in more than one study. The remaining 39 studies met all of the inclusion criteria and were used in the meta-analysis; 34 reported neurocognitive outcome measures, 14 provided self-report symptom outcomes, and 6 presented postural control as the dependent variable. The analyzed studies included 4145 total participants (concussed and control) with a mean age of 19.0 ± 0.4 years. The quality of each included study was also evaluated by each of the 2 authors independently using a previously published 15-item scale; the results demonstrated excellent agreement between the raters (intraclass correlation coefficient = 0.91, 95% confidence interval [CI] = 0.83, 0.95). The quality appraisal addressed randomization, sample selection, outcome measures, and statistical analysis, among other methodologic considerations. Quality scores of the included studies ranged from 5.25 to 9.00 (scored from 0–15).
The initial assessment demonstrated a deficit in neurocognitive function (Z = 7.73, P < .001, g = −0.81 [95% CI = −1.01, −0.60]), increase in self-report symptoms (Z = 2.13, P = .03, g = −3.31 [95% CI = −6.35, −0.27]), and a nonsignificant decrease in postural control (Z = 1.29, P = .19, g = −2.56 [95% CI = −6.44, 1.32]).
For the follow-up assessment analyses, a decrease in cognitive function (Z = 2.59, P = .001, g = −26 [95% CI = −0.46, −0.06]), an increase in self-report symptoms (Z = 2.17, P = .03, g = −1.09 [95% CI = −2.07, −0.11]), and a nonsignificant decrease in postural control (Z = 1.59, P = 0.11, g = −1.16 [95% CI = −2.59, 0.27]) were found.
Neurocognitive and symptom outcomes variables were reported in 10 studies, and the authors were able to compare changes from baseline in these measures during the initial assessment time point. A difference in effect sizes was noted (QB(1) = 5.28, P = .02), with the increases in self-report symptoms being greater than the associated deficits in neurocognitive function.
Sport-related concussion had a large negative effect on cognitive function during the initial assessment and a small negative effect during the first 14 days postinjury. The largest neurocognitive effects were found with the Standardized Assessment of Concussion during the immediate assessment and with pencil-and-paper neurocognitive tests at the follow-up assessment. Large negative effects were noted at both assessment points for postural control measures. Self-report symptoms demonstrated the greatest changes of all outcomes variables, with large negative effects noted both immediately after concussion and during the follow-up assessment. These findings reiterate the recommendations made to include neurocognitive measures, postural control tests, and symptom reports into a multifaceted concussion battery to best assess these injuries.
A common question among certified athletic trainers and other sports medicine professionals is “Which assessment tools should be incorporated into a concussion assessment plan?” Although a plethora of articles related to concussion assessment have been published this decade, Broglio and Puetz1 provide one of only 2 meta-analyses to date in the area of sport-related concussion that synthesize this vast body of knowledge. The main findings show large negative effects for neurocognitive tests, postural stability testing, and self-report symptom scales at the initial assessment (1.59 ± 1.74 days; range, 0.003–7 days) after injury, providing quantitative data to support the multifaceted approach that has been advocated by several consensus and position statements.2–4
The authors reported both the Z statistic and effect sizes, which may be useful for clinicians interpreting these data. The Z statistic reflects the probability of a true or real finding, as opposed to one occurring by chance, and is used to derive the P value, which indicates whether the finding is statistically significant. The Z score is a standardized variate taken from the normal distribution. That is, 68% of the normal distribution falls between −1 Z and +1 Z, and 95% of the normal distribution falls between −2 Z and +2 Z. All the Z scores reported in this meta-analysis were greater than 1.0, indicating that the differences in cognition, postural stability, and reported symptoms were the result of true differences, not chance. Effect sizes are useful in understanding the magnitude of deficits reported after concussion. The effect sizes noted in this paper were calculated by subtracting the baseline score from the follow-up score and dividing by the SD of the baseline score.1 Effect sizes can be interpreted as small (<0.20), moderate (0.21–0.79), or large (>0.80). The larger the effect size, the more that particular domain was affected after a sport-related concussion.
According to the findings of this meta-analysis, clinicians may gain the greatest amount of information regarding postconcussion deficits by using the Standardized Assessment of Concussion (SAC), a measure of postural control (such as the Balance Error Scoring System [BESS]), and a self-report symptom scale during the initial assessment on the sideline or within the first 48 hours postinjury. Additionally, the continued use of a more complex neurocognitive battery (pencil and paper or computerized), postural control measure, and self-report symptom scale may provide more information during subsequent postinjury follow-up assessments and aid in making return-to-play decisions.
Not surprisingly, self-report symptoms demonstrated the largest effects at both the immediate and follow-up assessment points, greater than the effects on neurocognitive tests in the studies that assessed both measures. Also, the number of assessments performed during the 14-day follow-up window was related to the effect size. As the number of assessments increased, the number of self-report symptoms decreased, thus demonstrating recovery over time. Although the number of administrations of certain assessment tools may be a concern with respect to learning effects, this is not likely an issue with the use of a symptom scale or checklist that measures an athlete's self-report status. These findings, combined with the recommendations2 that no athlete begin a return-to-play progression until he or she is asymptomatic, should reiterate the need for all clinicians to use some tool to assess self-report symptoms on all athletes with suspected concussions. The tool can range from a basic symptom checklist, in which the athlete is asked about symptoms he or she is experiencing and the clinician records a simple yes or no, to graded symptom scales, which allow the number of symptoms, their severity, and their duration to be quantified.2 In fact, some computerized neurocognitive tests and the Sport Concussion Assessment Tool4 have a graded symptom scale built into the tool. However, clinicians should not use these tools as the only means of determining recovery or in return-to-play decisions.
Although only 6 of the included studies reported on postural control, large negative effects were noted at both the immediate and follow-up assessment points, demonstrating the need for assessment of postural control as part of a concussion protocol. Of the included studies, 3 groups used a computerized force platform, whereas 2 groups used a clinical measure (the BESS), and 1 group assessed postural control with both types of measures. Because of the limited number of investigators using each method, the authors were not able to directly compare the force platform results with those on the BESS. However, there are several clinical advantages to using the BESS as a measure of postural control. It is cost effective, requiring only a stopwatch and a foam pad, and it takes only 5 minutes to administer. In addition, the foam pad can be transported easily, allowing for sideline assessment of postural control after concussion. The conclusions drawn by the authors regarding the use of a multifaceted concussion assessment plan are further supported by the work of others,5,7,8 who demonstrated increased sensitivity when a battery of tests was compared with the use of each individual tool. A simple battery of the SAC, BESS, and a graded symptom scale was 94% sensitive at the time of injury, compared with the SAC (80%), BESS (34%), or graded symptom scale (89%) alone.5 Similarly, adding self-report symptoms to the ImPACT (Pittsburgh, PA) computerized neurocognitive battery increased the sensitivity from 64% to 83%.8 Finally, Broglio et al7 demonstrated higher sensitivities for ImPACT, the Headminder Concussion Resolution Index (New York, NY), and a pencil-and-paper battery when self-report symptoms and postural control were added to each battery.
Although the authors of this meta-analysis concluded that their findings support the use of a multifaceted assessment battery, surveys of clinical practice show that athletic trainers are often not using these tools.9,10 In the second of these surveys, 77% of athletic trainers reported using a symptom checklist, up from the 33% found during the first practice pattern survey. Use of both the SAC and the BESS increased among athletic trainers, as reported in 2004, from 11% to 43% and 5% to 15%, respectively. Lastly, when participants were asked about the use of neurocognitive testing, similar response rates were noted on both surveys (15% and 16%). In general, a greater percentage of athletic trainers surveyed in 2004 used these objective concussion assessment tools than in the earlier study.10 Interestingly, a decrease in the use of neurocognitive tests was noted at both the collegiate and high school levels, possibly indicating barriers to implementing this type of testing in those settings.
A multifaceted approach to concussion assessment has been advocated via consensus statements2,3 and has now been substantiated with research,1,5,7,8 but each clinician must also evaluate the practicality and feasibility of each component before developing his or her own concussion plan. When developing an assessment plan for a particular setting, factors that must be considered include cost, time required to test, equipment, personnel, and administrative support. Clinicians will need to take the information presented in the meta-analysis and use their own clinical experience to determine how best to use these data to practice in an evidence-based manner.
Several limitations of the meta-analysis and the current body of literature should be addressed to allow clinicians to appreciate the value in the findings and to judge their relevance to clinical practice. First, in the meta-analysis, neurocognitive assessments were broadly grouped as either pencil-and-paper or computer assessments. This classification does not allow analysis of specific neurocognitive domains, such as reaction time or verbal memory. Also lacking is information on athletes and high-risk concussion activities (eg, military service) other than that associated with male, collegiate Americans participating in football. The authors of the meta-analysis reported that individuals were predominately male (92.9%) American football athletes (72%), with a mean age of 19.0 ± 0.40 years. Therefore, these findings may not be generalizable to female athletes, adolescent athletes, or individuals participating in other sports or activities. Clinicians should use caution when reviewing the evidence-based concussion information for young athletes, especially time components for return to play and normative values for objective tests.
The current body of literature is also incomplete with respect to studies investigating sideline concussion assessment tools and factors that may influence performance, such as environmental demands, sport-specific demands (eg, testing an ice hockey athlete with skates on), and learning effects with repeat administration. There is also little evidence regarding the long-term prognosis for postconcussion syndrome, athletes with a return-to-play window that is longer than 14 days, and athletes with a history of multiple concussions. Although great strides have been made in improving our knowledge of concussion assessment and recovery, future researchers should begin to investigate these unanswered or less-studied areas to aid clinicians in making evidence-based decisions in the management of concussions for all athletes.
Tamara C. Valovich McLeod, PhD, ATC, provided conception and design; analysis and interpretation of the data; and drafting, critical revision, and final approval of the manuscript.