Context: 

Meta-analyses examining construct-specific cognitive impairment concurrently with self-reported symptoms postconcussion are sparse.

Objective: 

To review the literature on the effects of concussion on construct-specific neurocognitive declines and to compare them with self-reported symptoms before 1 week and between 1 and 3 weeks postconcussion.

Data Sources: 

Relevant studies in PubMed, CINAHL, and PsycINFO published from January 1, 1999 through November 30, 2015.

Study Selection: 

Studies were included if participants completed the Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) before and after concussion and if test performance and Postconcussion Symptom Scale (PCSS) scores were reported at both times.

Data Extraction: 

After reviewing the full texts, we extracted data from 17 studies consisting of 29 independent samples; therefore, this meta-analysis consisted of 1777 unique participants.

Data Synthesis: 

The Hedges g effect size (ES) was estimated. A random-effects or fixed-effects model was used based on heterogeneity findings. When heterogeneity was present, we used meta-regression to assess unexplained between-studies variance. Within the first week of injury, the ESs were small to moderate for cognitive declines, ranging from −0.43 (95% confidence interval [CI] = −0.52, −0.35) to −0.67 (95% CI = −0.77, −0.58), and large for the PCSS score (Hedges g = −0.81; 95% CI = −0.91, −0.71). After 1 week, the ESs for cognitive declines (Hedges g range = −0.25 [95% CI = −0.35, −0.15] to −0.37 [95% CI = −0.55, −0.19]) and PCSS score (Hedges g = −0.38; 95% CI = −0.53, −0.22) were also small. Within 2 weeks of injury, PCSS score and time since injury weakly moderated the cognitive ES.

Conclusions: 

When a neurocognitive test was administered within 1 week of injury, the ES was larger for self-reported symptoms than for ImPACT scores generated at the same session. After 1 week of injury, the ESs for ImPACT and PCSS scores were comparable. If the athlete reports symptoms within 1 week of injury, administering a cognitive test does not appear to offer additional information to the clinician. However, if the athlete does not report symptoms postconcussion, cognitive testing may inform the clinical management of the injury.

Key Points
  • Construct-specific cognitive declines were less than documented self-reported symptoms within 1 week postconcussion and were comparable with self-reported symptoms after 1 week postconcussion.

  • Whereas Immediate Post-Concussion Assessment and Cognitive Testing provides overlapping information when the athlete has symptoms, it may add to the clinical management of injury as concussion-related symptoms diminish.

Over the past 2 decades, computerized neurocognitive tests (CNTs) have gained popularity for assessing concussion and are considered an integral part of concussion management.1  Researchers2,3  have suggested that CNTs improve the sensitivity of the concussion-assessment battery compared with self-reported symptoms alone. For example, Van Kampen et al3  reported that using Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT; ImPACT Applications, Inc, Pittsburgh, PA) resulted in a 19% increase in the sensitivity of the concussion-assessment battery. In addition, CNT is an objective measure of cognitive functioning that offsets the potential for suppressed symptom reports during the clinical evaluation.4 

Despite the growing popularity of CNTs, many investigators5,6  have remained skeptical about its utility in concussion assessment. This skepticism has resulted, in part, from multiple indices that often yield contradictory results among and between these indices and other evaluative measures (eg, clinical interview, self-reported symptoms).5  Whereas authors of meta-analyses79  have quantified the cognitive effects of concussion, most have provided a single effect size (ES) for different cognitive constructs across multiple test batteries.7,9  Given the heterogeneous nature of concussions, CNTs that were designed to evaluate cognitive declines postconcussion must address construct-specific changes (eg, speed, memory).8,9  In clinical practice, medical professionals typically select and administer 1 CNT to evaluate the neurocognitive effects of concussion. Therefore, meta-analyses that provide an aggregated ES across multiple CNTs may not reflect the ability of 1 specific neurocognitive test to document construct-specific cognitive declines postconcussion.

Although the authors of some original investigations and position statements1,4,10  have suggested that the clinical recovery from concussion is achieved within 1 week of injury, other researchers11,12  have documented lingering cognitive effects after the first week of injury. The utility of CNTs in the clinical management of concussion depends, in part, on their ability to quantify the possible effects of concussion, particularly after the expected time for resolution of self-reported symptoms (approximately 1 week). Therefore, examining the utility of CNTs in documenting the effects of concussion before and after 1 week is warranted. Whereas Kontos et al8  reported construct-specific cognitive declines (Hedges g range = −0.18 to 0.27) within 1 week of injury, they did not address neurocognitive performance beyond the first week postconcussion and did not directly compare the ES for cognitive declines against that for self-reported symptoms.

The current standard of care for concussion management precludes athletes from returning to sport activities until they are symptom free both at rest and during exertion.1,10  During symptom recovery, administering CNTs to track cognitive declines may be redundant. Conversely, when an athlete's symptom reports are improving, documenting cognitive declines may add to the sensitivity of the concussion-assessment battery.13  However, CNT scores must be interpreted in light of the high rates of false-positives and false-negatives documented for CNT batteries.6  The percentages of symptom-free athletes presenting with cognitive declines (39%–53.8%) postconcussion are comparable with the false-positive rate of declines (29.6%–42.7%) observed in healthy participants.6  From a clinical perspective, the benefit of a CNT depends on its ability to document reliable cognitive declines beyond symptom resolution.6,13  Yet a gap exists in the literature regarding the evaluation of athletes with concussion who no longer report concussion-related symptoms.

Investigators79  have suggested that multiple factors, such as sex, time since injury, and concussion history, moderate the observed effects of concussion. However, it is unclear if these factors explain between-studies variances in cognitive declines and self-reported symptoms postconcussion.

Given the lack of meta-analyses in which construct-specific cognitive impairment reported concurrently with self-reported symptoms has been examined, we conducted this review. Therefore, the primary purpose of our study was to review the literature and examine the ES of concussion on individual cognitive indices compared with the ES of self-reported symptoms at 2 time points: within the first week postinjury and between 1 and 3 weeks postinjury. A secondary purpose of this review was to examine whether between-studies variances in ES (ie, heterogeneity) were explained by baseline symptom scores, postconcussion symptom score, percentage of females, percentage of participants with concussion history, or time since concussion.

To allow for concurrent examination of construct-specific cognitive declines and self-reported symptoms, we chose a CNT that concurrently describes self-reported symptoms and multiple aspects of cognitive functioning, including verbal memory, visual memory, visual motor processing speed, and reaction time.14  The ImPACT, one of the most comprehensive and widely used cognitive tests,1517  has greater sensitivity in documenting cognitive declines postconcussion than other CNTs.2,6 

As part of a larger review of the validity and utility of ImPACT, we completed an initial electronic search of published studies in PubMed, CINAHL, and PsycINFO. The search focused on publications from January 1999 through November 2014. The search was updated in November 2015. For both searches, we used the search terms ImPACT OR immediate post-concussion assessment and cognitive test OR impact testing OR neurocognitive testing OR neurocognitive OR neuropsychological testing OR neuropsychological and concussion OR mTBI OR mild traumatic brain injury OR post concussive syndrome OR mild head injury OR closed head injury. The search filters of English-language publications and studies that included human participants were applied. Review articles, abstracts, case studies, editorials, and grey literature were excluded from the analysis. We also performed a hand search of the reference lists of included studies.

The inclusion criteria were (1) participants completed ImPACT within or after 1 week of concussion and the scores were compared with their own baseline scores and (2) the Postconcussion Symptom Scale (PCSS) scores at baseline and postconcussion were provided, which allowed calculation of the ESs. Given that concussion continues to be a clinical diagnosis based on a provider's opinion, supported by a number of multifaceted evaluative tools,10  we did not exclude studies that failed to supply an operational definition of concussion. Studies were excluded if (1) ImPACT was completed by healthy participants (ie, no concussion), such as examinations of the psychometric properties of ImPACT reviewed elsewhere18,19 ; (2) ImPACT subscales or a version of the test that is no longer available (ie, version 1) was used; (3) the total symptom score was not reported or symptoms were reported as clusters (eg, somatic, sleep) rather than as a total score; or (4) postconcussion ImPACT scores were compared with the scores of healthy control participants rather than the individual's own baseline scores.

Data Extraction and Coding

Two raters (K.S., D.P.) completed a thorough review of the titles and abstracts of the studies. They also independently reviewed the full text of each study and extracted the data using an electronic spreadsheet. Disagreement between the raters regarding the retrieved data was resolved by consensus. The sample size, demographic characteristics of participants, time between concussion and postconcussion ImPACT testing, ImPACT composite scores, and PCSS scores were retrieved in addition to the scores for applicable time points (baseline, within 1 week of concussion, and between 1 and 3 weeks after concussion). If ImPACT was completed more than once within the periods specified for this study (within 1 week and between 1 and 3 weeks), the scores of the first testing session within a given interval were chosen for this meta-analysis. For studies in which the authors reported ImPACT scores for more than 1 sample, we assumed that these samples were independent and reported them separately.

Assessment of Reporting and Methodologic Quality

We used the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) statement to assess the reporting quality of the reviewed studies.20  The STROBE statement evaluates 22 aspects related to the reporting of observational studies. Each aspect was assigned a numeric value of 1 if it was explicitly described and presented in a study and 0 if it was inadequately described or absent in a study. As such, a total score out of 22 was calculated to reflect the overall reporting quality of each study; a higher score indicated better reporting quality. We did not use reporting-quality scores as moderators in the pooled analysis because of the potential to confound the findings.

Meta-Analysis Process

Assessing Heterogeneity.

Heterogeneity refers to the extent of variability among studies. Statistical tests are used to quantify the degree of heterogeneity among studies. In this meta-analysis, we assessed heterogeneity separately for studies included in both analyses using the Q statistic as a test of the null hypothesis of homogeneity. The I2 index was used to estimate the degree of heterogeneity across studies when the null hypothesis was rejected at an α level of .10.21,22  Higgins and Thompson22  described I2 values for interpreting magnitude as percentages of 25% (I2 = 25), 50% (I2 = 50), and 75% (I2 = 75), meaning low, medium, and high heterogeneity, respectively. If heterogeneity was detected at an α level of .10, ESs were estimated using a random-effects model.

Effect-Size Calculation and Interpretation.

The ES provides a quantitative description of the size of an effect beyond the level of statistical significance.23  Effect sizes provide interpretable data that are independent of units of measurement and the influence of sample size.23  The ESs of changes for all ImPACT scores and PCSS scores were estimated separately for each study at testing time points within 1 week and between 1 and 3 weeks postinjury. Overall ESs of change for studies grouped by postinjury outcome (within 1 week and between 1 and 3 weeks) were estimated as standardized mean difference (Hedges g): g = (Mfu–Mb)/Spooled, where Mfu indicates the mean score at follow-up postinjury, Mb indicates the mean score at preinjury baseline, and Spooled indicates the estimated pooled variance across 2 measures24  with adjustment for preinjury to postinjury correlation in each outcome score.25  We wanted the sign of the ES to be consistent for all 4 composite scores (ie, a negative ES indicates a decline postconcussion for any of the 4 scores); therefore, the ES for reaction time was calculated as (Mb–Mfu)/Spooled. Overall weighted ESs with 95% confidence intervals (CIs) and across both analyses (ie, within 1 week and between 1 and 3 weeks) were estimated using a random-effects model when heterogeneity was detected or with a fixed-effects model when heterogeneity was not detected. In both cases, the ESs were tested against the null hypothesis of no effect at an α level of .05. The ESs were interpreted as suggested by Cohen26: trivial (Hedges g < 0.19), small (Hedges g = 0.20–0.49), moderate (Hedges g = 0.50–0.79), and large (Hedges g > 0.8).

Meta-Regression for Between-Study Variance.

Meta-regression is used to explore possible factors contributing to significant between-studies variance (ie, heterogeneity). For this meta-analysis, a random-effects model was used to test the covariate effect of time of follow-up and baseline and follow-up PCSS scores on ImPACT ESs. Furthermore, the percentage of female participants and the percentage of participants with a history of concussion were tested as covariates on ImPACT ESs. For the PCSS score ES, follow-up time, percentage of females, and percentage of participants with previous concussions were tested as covariates. Meta-regression results were reported as β coefficient estimates with 95% CIs, z scores, and P values for each covariate and omnibus test of model difference at an α level of .05.27  A covariate effect was considered different when the z score (β/standard error) was different (P < .05). The I2 was determined to describe the total variance between studies that was explained by the meta-regression model. For the meta-regression of 1 to 3 weeks, we conducted a post hoc analysis to determine the power based on the number of studies included. Meta-analytic procedures were conducted using Comprehensive Meta-Analysis (version 3.3; Biostat, Englewood, NJ) and SPSS (version 23.0; IBM Corp, Armonk, NY).

Assessment of Publication Bias.

Given that studies documenting positive findings are more likely to be published than studies with negative findings, pooled results in meta-analyses can be subject to publication bias. In this meta-analysis, we assessed publication bias using visual inspection of funnel plots of the ESs (Hedges g) versus standard errors for studies grouped by time of outcome measurement.28  For outcomes with funnel plots indicating asymmetry as potential evidence of publication bias, we used the Egger regression intercept test (β coefficient, t value, P value). Publication bias was assessed against a 1-tailed Egger regression intercept test with a critical value of P < .05. Statistical evidence of publication bias was further investigated with the trim-and-fill method of Duval and Tweedie29  to estimate the number of missing studies and provide an adjusted ES.

Search Yield

The search yielded 5943 studies. The raters examined the study abstract and identified 26 studies for full-text review. After reviewing the 26 studies and reapplying the same inclusion criteria, we excluded 9 studies. Reasons for exclusion were the absence of the total PCSS score (n = 5),3034  use of ImPACT version 1 (n = 2),35,36  absence of baseline data (n = 1),37  and lack of details regarding testing time points (n = 1).13  As such, 17 studies consisting of 29 independent samples were included (Figure).

Figure. 

Study-selection process.

Figure. 

Study-selection process.

Characteristics of Included Studies and Reporting Quality

A total of 1777 patients (1250 males, 527 females) with concussion were included in this meta-analysis. Individual studies included participants from 13 to 33 years old representing clinical populations that routinely complete ImPACT during the clinical management of concussion. Participants in the reviewed studies included middle and high school–aged children, college-aged adults, and professional athletes. The sample sizes in the reviewed studies varied from 15 to 222 participants. Twenty-nine independent samples were included: 26 samples were included in the within-1-week analysis, and 10 samples were included in the 1- to 3-week analysis. The average (standard deviation) time points considered for the within-1-week analysis and the 1- to 3-week analysis were 2.2 (1.4) days and 9.7 (2.6) days, respectively. In 7 samples, ImPACT was serially administered at both time points; therefore, the designated ImPACT scores were considered for the appropriate analysis (Table 1). The reporting quality was generally moderate to high, and STROBE scores ranged from 17 to 21 (Table 1).

Table 1. 

Characteristics of Included Studies

Characteristics of Included Studies
Characteristics of Included Studies

Heterogeneity

The within-1-week analysis demonstrated heterogeneity for all outcomes (P ≤ .007), with low heterogeneity for processing speed (I2 = 45.37) and medium heterogeneity for all other ImPACT and PCSS scores (I2 = 73.42), further indicating the level of unexplained variance across studies (Table 2). For the 1- to 3-week analyses, we observed heterogeneity for verbal memory, reaction time, and PCSS score, with the unexplained between-studies variance ranging from 54.53% (PCSS) to 73.97% (verbal memory; Table 3).

Table 2. 

Effect Sizes of Change (Hedges g) and Heterogeneity Estimations in Immediate Post-Concussion Assessment and Cognitive Testinga and Postconcussion Symptom Scale Scores for Independent Samples Within 1 Week Postconcussion (kb = 26)

Effect Sizes of Change (Hedges g) and Heterogeneity Estimations in Immediate Post-Concussion Assessment and Cognitive Testinga and Postconcussion Symptom Scale Scores for Independent Samples Within 1 Week Postconcussion (kb = 26)
Effect Sizes of Change (Hedges g) and Heterogeneity Estimations in Immediate Post-Concussion Assessment and Cognitive Testinga and Postconcussion Symptom Scale Scores for Independent Samples Within 1 Week Postconcussion (kb = 26)
Table 3. 

Effect Sizes of Change (Hedges g) and Heterogeneity Estimations in Immediate Postconcussion Assessment and Cognitive Testinga and Postconcussion Symptom Scale Scores for Independent Samples Between 1 and 3 Weeks Postconcussion (kb = 10)

Effect Sizes of Change (Hedges g) and Heterogeneity Estimations in Immediate Postconcussion Assessment and Cognitive Testinga and Postconcussion Symptom Scale Scores for Independent Samples Between 1 and 3 Weeks Postconcussion (kb = 10)
Effect Sizes of Change (Hedges g) and Heterogeneity Estimations in Immediate Postconcussion Assessment and Cognitive Testinga and Postconcussion Symptom Scale Scores for Independent Samples Between 1 and 3 Weeks Postconcussion (kb = 10)

Effect Size

All ImPACT and PCSS scores demonstrated ESs of change, which were adjusted for time from injury to follow-up correlation, that were different for studies when the postinjury follow-up measurement was within 1 week (Table 2; Appendix 1A through E). The greatest change was observed in PCSS score (Hedges g = −0.81), and the smallest change was seen in processing speed (Hedges g = −0.43). Comparing the ES CIs, the overall ES of change was greater for the PCSS (Hedges g = −0.81, 95% CI = −0.91, −0.71) than for reaction time (Hedges g = −0.58, 95% CI = −0.68, −0.47) and processing speed (Hedges g = −0.43, 95% CI = −0.52, −0.35). Similarly, the overall ES of change was greater for visual memory (Hedges g = −0.64, 95% CI = −0.74, −0.54) and verbal memory (Hedges g = −0.67, 95% CI = −0.77, −0.58) than for processing speed.

A small but different ES was observed for all ImPACT and PCSS scores measured between 1 and 3 weeks postconcussion, with the greatest effect in the PCSS (Hedges g = −0.38) and the smallest effect in visual memory (Hedges g = −0.25; Table 3; Appendix 2A through E).

Meta-Regression of Unexplained Between-Studies Variance

Across the 26 samples with outcomes measured within 1 week, meta-regression identified time since concussion as a univariate moderator for all ImPACT scores. As time since concussion increased over the 7 days, a smaller decline in cognitive scores was observed. A larger postinjury PCSS score was associated with a larger cognitive decline ES for verbal and visual memory (Table 4). Time postinjury and postinjury PCSS score influenced processing speed ES, but only time postinjury was different from both moderators in the model. The greatest amount of between-studies variance was explained by both time postinjury and follow-up PCSS score for verbal memory (81%). Time postinjury was a moderator for reaction time, which explained the 5% of the between-studies variance (Table 4). Preinjury PCSS score, percentage of females, and concussion history were not associated with ES within 1 week postconcussion. Time postinjury did not explain the degree of between-studies variance in PCSS scores.

Table 4. 

Meta-Regression Results for Time to Measurement and Follow-up Postconcussion Symptom Scale Scores as Moderators of 1-Week Effect Size of Change in Immediate Postconcussion Assessment and Cognitive Testinga Scores (kb = 26)

Meta-Regression Results for Time to Measurement and Follow-up Postconcussion Symptom Scale Scores as Moderators of 1-Week Effect Size of Change in Immediate Postconcussion Assessment and Cognitive Testinga Scores (kb = 26)
Meta-Regression Results for Time to Measurement and Follow-up Postconcussion Symptom Scale Scores as Moderators of 1-Week Effect Size of Change in Immediate Postconcussion Assessment and Cognitive Testinga Scores (kb = 26)

Meta-regression results indicated that preinjury PCSS score moderated the reaction time ES in studies with outcomes measured between 1 and 3 weeks postconcussion (β = 0.09, z = 2.06, P = .04), whereby greater preinjury PCSS score was associated with a smaller decline in postinjury reaction time. A model using the preinjury PCSS score explained 37% of the between-studies variance in reaction time change preinjury to postinjury. Time postinjury was a moderator of PCSS score ES (β = 0.06, z = 3.78, P < .001), with the model explaining 100% of the between-studies variance; the PCSS score ES from preinjury to postinjury (between 1 and 3 weeks) decreased as the postinjury measurement time increased. No tested covariates moderated the ES for verbal memory. Percentage of females and concussion history were not moderating influences on the ES of change in any outcome.

For the meta-regression of 1 to 3 weeks postinjury, the post hoc analysis showed that the power for the time regression coefficient was 24% for verbal memory and 6% for reaction time.

Publication Bias

For the samples with outcomes measured within 1 week postconcussion, funnel-plot inspection supplied evidence for possible publication bias for all ImPACT and PCSS scores. The Egger regression test indicated publication bias for reaction time (β = 1.83, t24 = 1.85, P = .04) and PCSS score (β = 3.31, t24 = 3.60, P = .001; Table 5). For ImPACT reaction time, the trim-and-fill method suggested that 6 studies were missing and the point estimate for Hedges g, adjusted for publication bias, was −0.67 (95% CI = −0.77, −0.56). For PCSS score, the trim-and-fill method suggested that 10 studies were missing and the point estimate for Hedges g, adjusted for publication bias, was −0.98 (95% CI = −1.09, −0.87).

Table 5. 

Egger Regression Intercept Test for Potential Publication Bias for Immediate Postconcussion Assessment and Cognitive Testinga and Postconcussion Symptom Scale Scores Measured Within 1 Week Postconcussion

Egger Regression Intercept Test for Potential Publication Bias for Immediate Postconcussion Assessment and Cognitive Testinga and Postconcussion Symptom Scale Scores Measured Within 1 Week Postconcussion
Egger Regression Intercept Test for Potential Publication Bias for Immediate Postconcussion Assessment and Cognitive Testinga and Postconcussion Symptom Scale Scores Measured Within 1 Week Postconcussion

For the samples with outcomes measured between 1 and 3 weeks after concussion, funnel-plot inspection showed potential asymmetry for ImPACT reaction time. The Egger regression intercept test for potential publication bias was not different for reaction time (β = −1.35, t24= 0.84, P = .21).

Effect Sizes

Within 1 week postinjury, concussion had a small to moderate effect on construct-specific measures of speed and memory as measured by ImPACT. During the same assessment period, the ES of self-reported symptoms was larger than for cognitive declines. After 1 week postinjury, the PCSS score ES was comparable with that of cognitive declines observed for the 4 cognitive composite scores. Inspection of construct-specific cognitive declines revealed that whereas verbal and visual memory scores appeared to have larger ESs within 1 week of concussion than did speed scores (processing speed and reaction time), only processing speed was different from verbal and visual memory (ie, CIs did not overlap). These findings were comparable with those reported by Kontos et al,8  who demonstrated a slightly greater ES for visual memory (Hedges g = −0.25) than for processing speed (Hedges g = −0.18). After 1 week, the ESs for speed and memory were comparable.

When assessments were administered within 1 week postinjury, the differences in the PCSS score ES from the processing speed and reaction time ESs were significant (ie, CIs did not overlap), even after adjusting for possible publication bias for PCSS score and reaction time. These findings are similar to those reported by Kontos et al,8  who observed a greater ES for self-reported symptoms within 1 week postinjury (Hedges g = −0.23) than the ES for reaction time (Hedges g = −0.11) and processing speed (Hedges g = −0.18). A minimal overlap existed between the upper limit of the 95% CI for the PCSS score (−0.71) and the lower limits for verbal memory (−0.77) and visual memory (−0.74).

The cognitive declines reported within 1 week postconcussion were likely influenced by the inclusion of participants with symptoms. This assumption is further supported by the results of the meta-regression, in which a large proportion of between-studies variance for verbal and visual memory was explained by postinjury PCSS scores. These findings suggest that athletes who are acutely symptomatic demonstrate greater memory declines than those who are less symptomatic or symptom free as they recover from injury. The work of Nelson et al6  further supports these findings: when symptomatic and asymptomatic patients were considered together, the sensitivity of ImPACT scores to the acute effects of concussion within the first 24 hours ranged from 24.4% to 39.5%. However, the sensitivity of ImPACT diminished to 5.2% when only symptom-free athletes were considered during the same period.6 

The trend toward a greater ES for PCSS score changes (Hedges g = −0.81) within 1 week of concussion than for cognitive declines (Hedges g = −0.43 to −0.67) was consistent with previous meta-analyses.7,9  When initial postconcussion testing was conducted within 7 days postconcussion, Broglio and Puetz7  reported a greater ES for self-reported symptoms (Hedges g = −3.31) followed by cognitive declines (Hedges g = −0.70). Similarly, Dougan et al9  reported larger ESs for self-reported symptoms (Hedges g = −0.66) than for cognitive declines (Hedges g = −0.40). The range of within-1-week cognitive declines in ImPACT scores that we reported (−0.43 to −0.67) was larger than the overall ES that Kontos et al8  reported for ImPACT score (Hedges g = −0.19; P < .05). These differences may be attributed to differences in the reviewed studies or the ES computation method. In addition, we calculated the ES only when postinjury scores were compared with baseline scores; Kontos et al8  reported ESs in mixed studies of control group and baseline comparisons.

After 1 week postinjury, the ES for the PCSS score was comparable with the ESs of ImPACT cognitive scores, as indicated by the overlap between the CIs for all of the ESs. The range of ESs for observed cognitive declines in ImPACT scores (−0.25 to −0.37) was slightly higher than the aggregated ES (Hedges g = −0.19) for CNTs that Broglio and Puetz7  reported within the first 14 days postconcussion.

Meta-Regression and Unexplained Between-Studies Variance

For the meta-regression of the analysis within 1 week of concussion, the emergence of time since injury as a univariate moderator to explain the between-studies variance for all the ImPACT composite scores is consistent with previous works.6,7,9  For instance, Dougan et al9  reported that the cognitive score ES decreased from −0.76 within 24 hours postconcussion to −0.44 after the first day postconcussion. However, within 1 week postconcussion, time did not moderate the PCSS score. These findings may be explained by the biopsychosocial nature of symptoms in which the perceived severity of symptoms is affected by many factors and is not perfectly related to the severity of injury or time since injury.53,54 

For the meta-regression of the analysis for 1 to 3 weeks after concussion, the emergence of time since concussion as a factor to explain between-studies variance in PCSS scores but not ImPACT composite scores was opposite to the trend observed in the within-1-week analysis. This difference may be explained by the fact that symptoms are usually more delineated after the acute period of concussion, leading to different symptom trajectories that may not be reflected in the total PCSS score.55  In addition, these findings could be explained by the small number of studies included in the analysis, which may have underpowered the analysis of cognitive scores. For example, the post hoc analysis revealed that the power for the time regression coefficient was 24% and 6% for verbal memory and reaction time, respectively. As such, the association between a greater preinjury PCSS score and a smaller ES for reaction time beyond the first week was unexpected and may be a spurious finding (ie, false positive). An alternative explanation could be that greater preinjury symptoms are associated with worse baseline reaction times, which may have led to smaller differences between baseline and postinjury scores (ie, smaller ES). Given that no researchers have examined baseline associations between symptoms and ImPACT scores, this explanation remains speculative.

The meta-regression results did not support sex or concussion history as moderating factors for concussion effects on speed and memory at the 2 time points. The lack of association between ES and multiple previous concussions was expected because ImPACT was not designed to detect subtle cumulative declines in cognitive function attributed to multiple concussions. These findings may also be attributed to the small number of studies and the lack of details regarding concussion history, which warrant a cautious interpretation of these results. The lack of sex effects in this review was comparable with that of previous investigators3941  who did not find differences in ESs as a function of sex for verbal memory, processing speed, and reaction time postconcussion. However, the findings of visual memory differed from those reported by Covassin et al,3941  who observed that female participants experienced a greater decline in visual memory than did male participants. Moreover, the lack of sex effects differs from the report of Dougan et al,9  who noted sex as a moderator for neurocognitive outcomes in the first 10 days postconcussion. The difference could be explained by the more stringent criteria in our meta-analysis; we compared participants' postinjury and baseline performances on 1 specific CNT (ie, ImPACT), whereas Dougan et al9  examined an aggregated neurocognitive ES across multiple paper-and-pencil and computerized tests. Our stringent criteria may have limited the power needed to document possible sex effects because of the small number of studies in our review (n = 17) compared with the 78 studies in Dougan et al.9 

Limitations

Our review had some limitations in search strategy and the meta-analytic process. We included only participants whose postconcussion ImPACT scores were compared with their own baseline scores. Therefore, the cognitive effects of concussion could be different when compared with a control group rather than the individual's own baseline. We examined only ImPACT and PCSS scores, and these findings cannot be generalized to other cognitive tests not included in this review. The authors of 3 studies4042  included in this review did not report an operational definition of concussion, and the definition of concussion in the remaining 16 studies varied, which may have influenced the observed ESs that we reported.

The results for concussion history and sex as possible moderators for the effects of concussion should be interpreted with caution. Concussion history was self reported and, therefore, may have been subject to recall bias. In addition, the findings about a sex effect may be limited due to the predominance of male participants (70%) in the included studies (Table 1). Whereas the included studies addressed postinjury performance relative to the individual's own baseline, the authors of most studies did not examine whether the observed cognitive decline exceeded the expected change attributed to measurement error. Without this analysis, investigators would not be able to confirm if postinjury cognitive decline could be ascribed to concussion or was a false positive attributed to the expected measurement error.18  Although multiple forms of ImPACT were developed to overcome a possible practice effect, emerging evidence56  has indicated that some forms are not equivalent. Nonequivalence among some forms may have contributed to between-studies variance, but the effect of the forms could not be quantified because most authors did not specify the ImPACT form used.

In most previous meta-analyses, researchers have employed different methods to calculate the ES for within-participant comparisons and have not accounted for the relationship between the baseline and postinjury scores. We estimated correlations for preinjury to postinjury outcome scores for each study as recommended by Morris and DeShon,25  who suggested that sampling variance may be biased by the assumption that the time effect is the same across all studies. This adjustment for repeated measures is assumed to allow for previously published values that may have been obtained from independent groups.

The precision of the ES estimates across the 5 ImPACT outcome measures should be interpreted with caution. Whereas the ESs in this meta-analysis were estimated independently for each outcome, the stochastic dependence between separate outcomes taken from the same sample (ie, convergence) may bias the statistical inference.57  Van den Noortgate et al58  reported that this inter-item dependence may result in underestimated standard errors, narrow CIs, and an inflated type I error rate. Statistical consideration for this potential source of bias in ES estimation requires consideration for the inter-item covariance across the studies. However, the authors of the reviewed studies did not report these correlations, and the ImPACT user's manual59  described these relationships only for the first version of the test, which was excluded from this review. Therefore, this meta-analysis did not account for possible dependence among the observed ESs.

With this meta-analysis, we are the first to directly compare construct-specific cognitive declines and self-reported symptoms postconcussion. We conclude that construct-specific cognitive declines were less than documented self-reported symptoms within 1 week of injury and were comparable with self-reported symptoms after the first week of injury. These findings highlight the importance of using ImPACT on a case-by-case basis as part of a multifaceted postconcussion assessment and suggest that, when the athlete has symptoms, ImPACT provides overlapping information. As concussion-related symptoms diminish, however, ImPACT appears to inform the clinical management of injury. From a clinical perspective, clinicians using ImPACT in patients with symptoms should place a greater emphasis on the extent of cognitive declines as indicated by the number of composite scores with declines greater than what is attributable to measurement error (ie, reliable change index). When a clinician suspects that an athlete is underreporting symptoms, ImPACT as part of a multifaceted performance assessment may help to quantify the possible effects of concussion. Given that concussion results in a myriad of signs and symptoms leading to multiple clinical trajectories of recovery,55,60  future researchers should examine if construct-specific cognitive declines have a prognostic utility in informing recovery trajectories.

1
McCrory
P,
Meeuwisse
WH,
Aubry
M,
et al.
Consensus statement on concussion in sport: the 4th International Conference on Concussion in Sport held in Zurich, November 2012
.
Br J Sports Med
.
2013
;
47
(
5
):
250
258
.
2
Broglio
SP,
Macciocchi
SN,
Ferrara
MS.
Sensitivity of the concussion assessment battery
.
Neurosurgery
.
2007
;
60
(
6
):
1050
1057
.
3
Van Kampen
DA,
Lovell
MR,
Pardini
JE,
Collins
MW,
Fu
FH.
The “value added” of neurocognitive testing after sports-related concussion
.
Am J Sports Med
.
2006
;
34
(
10
):
1630
1635
.
4
McCrea
M,
Guskiewicz
KM,
Marshall
SW,
et al.
Acute effects and recovery time following concussion in collegiate football players: the NCAA Concussion Study
.
JAMA
.
2003
;
290
(
19
):
2556
2563
.
5
Randolph
C,
McCrea
M,
Barr
WB.
Is neuropsychological testing useful in the management of sport-related concussion?
J Athl Train
.
2005
;
40
(
3
):
139
152
.
6
Nelson
LD,
LaRoche
AA,
Pfaller
AY,
et al.
Prospective, head-to-head study of three Computerized Neurocognitive Assessment Tools (CNTs): reliability and validity for the assessment of sport-related concussion
.
J Int Neuropsychol Soc
.
2016
;
22
(
1
):
24
37
.
7
Broglio
SP,
Puetz
TW.
The effect of sport concussion on neurocognitive function, self-report symptoms and postural control: a meta-analysis
.
Sports Med
.
2008
;
38
(
1
):
53
67
.
8
Kontos
AP,
Braithwaite
R,
Dakan
S,
Elbin
RJ.
Computerized neurocognitive testing within 1 week of sport-related concussion: meta-analytic review and analysis of moderating factors
.
J Int Neuropsychol Soc
.
2014
;
20
(
3
):
324
332
.
9
Dougan
BK,
Horswill
MS,
Geffen
GM.
Athletes' age, sex, and years of education moderate the acute neuropsychological impact of sports-related concussion: a meta-analysis
.
J Int Neuropsychol Soc
.
2014
;
20
(
1
):
64
80
.
10
Broglio
SP,
Cantu
RC,
Gioia
GA,
et al.
National Athletic Trainers' Association position statement: management of sport concussion
.
J Athl Train
.
2014
;
49
(
2
):
245
265
.
11
Henry
LC,
Elbin
RJ,
Collins
MW,
Marchetti
G,
Kontos
AP.
Examining recovery trajectories after sport-related concussion with a multimodal clinical assessment approach
.
Neurosurgery
.
2016
;
78
(
2
):
232
241
.
12
Prichep
LS,
McCrea
M,
Barr
W,
Powell
M,
Chabot
RJ.
Time course of clinical and electrophysiological recovery after sport-related concussion
.
J Head Trauma Rehabil
.
2013
;
28
(
4
):
266
273
.
13
Broglio
SP,
Macciocchi
SN,
Ferrara
MS.
Neurocognitive performance of concussed athletes when symptom free
.
J Athl Train
.
2007
;
42
(
4
):
504
508
.
14
Maroon
JC,
Lovell
MR,
Norwig
J,
Podell
K,
Powell
JW,
Hartl
R.
Cerebral concussion in athletes: evaluation and neuropsychological testing
.
Neurosurgery
.
2000
;
47
(
3
):
659
669
.
15
Cole
WR,
Arrieux
JP,
Schwab
K,
Ivins
BJ,
Qashu
FM,
Lewis
SC.
Test-retest reliability of four computerized neurocognitive assessment tools in an active duty military population
.
Arch Clin Neuropsychol
.
2013
;
28
(
7
):
732
742
.
16
Meehan
WP
III,
d'Hemecourt
P,
Collins
CL,
Taylor
AM,
Constock
RD.
Computerized neurocognitive testing for the management of sport-related concussions
.
Pediatrics
.
2012
;
129
(
1
):
38
44
.
17
Covassin
T,
Elbin
R
III,
Stiller-Ostrowski
JL.
Current sport-related concussion teaching and clinical practices of sports medicine professionals
.
J Athl Train
.
2009
;
44
(
4
):
400
404
.
18
Alsalaheen
B,
Stockdale
K,
Pechumer
D,
Broglio
SP.
Measurement error in the Immediate Postconcussion Assessment and Cognitive Testing (ImPACT): systematic review
.
J Head Trauma Rehabil
.
2016
;
31
(
4
):
242
251
.
19
Alsalaheen
B,
Stockdale
K,
Pechumer
D,
Broglio
SP.
Validity of the Immediate Post Concussion Assessment and Cognitive Testing (ImPACT)
.
Sports Med
.
2016
;
46
(
10
):
1487
1501
.
20
von Elm
E,
Altman
DG,
Egger
M,
et al.
The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies
.
Ann Intern Med
.
2007
;
147
(
8
):
573
577
.
21
Huedo-Medina
TB,
Sanchez-Meca
J,
Marin-Martinez
F,
Botella
J.
Assessing heterogeneity in meta-analysis: Q statistic or I2 index?
Psychol Methods
.
2006
;
11
(
2
):
193
206
.
22
Higgins
JP,
Thompson
SG.
Quantifying heterogeneity in a meta-analysis
.
Stat Med
.
2002
;
21
(
11
):
1539
1558
.
23
Fritz
CO,
Morris
PE,
Richler
JJ.
Effect size estimates: current use, calculations, and interpretation
.
J Exp Psychol Gen
.
2012
;
141
(
1
):
2
18
.
24
Hedges
L,
Olkin
I.
Statistical Methods for Meta-Analysis
.
Orlando, FL
:
Academic Press, Inc;
1985
:
76
85
.
25
Morris
SB,
DeShon
RP.
Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs
.
Psychol Methods
.
2002
;
7
(
1
):
105
125
.
26
Cohen
J.
Statistical Power Analysis for the Behavioral Sciences. 2nd ed
.
New York, NY
:
Routledge Academic;
1988
:
25
.
27
Borenstein
M,
Hedges
LV,
Higgins
JPT,
Rothstein
HR.
Introduction to Meta-Analysis
.
Hoboken, NJ
:
John Wiley & Sons, Ltd;
2009
:
187
204
.
28
Egger
M,
Davey Smith
G,
Schneider
M,
Minder
C.
Bias in meta-analysis detected by a simple, graphical test
.
BMJ
.
1997
;
315
(
7109
):
629
634
.
29
Duval
S,
Tweedie
R.
Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis
.
Biometrics
.
2000
;
56
(
2
):
455
463
.
30
Covassin
T,
Crutcher
B,
Wallace
J.
Does a 20 minute cognitive task increase concussion symptoms in concussed athletes?
Brain Inj
.
2013
;
27
(
13–14
):
1589
1594
.
31
Covassin
T,
Moran
R,
Wilhelm
K.
Concussion symptoms and neurocognitive performance of high school and college athletes who incur multiple concussions
.
Am J Sports Med
.
2013
;
41
(
12
):
2885
2889
.
32
Echlin
PS,
Skopelja
EN,
Worsley
R,
et al.
A prospective study of physician-observed concussion during a varsity university ice hockey season: incidence and neuropsychological changes. Part 2 of 4
.
Neurosurg Focus
.
2012
;
33(6):E2:1–11.
33
Kontos
AP,
Elbin
RJ,
Lau
B,
et al.
Posttraumatic migraine as a predictor of recovery and cognitive impairment after sport-related concussion
.
Am J Sports Med
.
2013
;
41
(
7
):
1497
1504
.
34
Sosnoff
JJ,
Broglio
SP,
Ferrara
MS.
Cognitive and motor function are associated following mild traumatic brain injury
.
Exp Brain Res
.
2008
;
187
(
4
):
563
571
.
35
Lovell
MR,
Collins
MW,
Iverson
GL,
et al.
Recovery from mild concussion in high school athletes
.
J Neurosurg
.
2003
;
98
(
2
):
296
301
.
36
Lovell
MR,
Collins
MW,
Iverson
GL,
Johnston
KM,
Bradley
JP.
Grade 1 or “ding” concussions in high school athletes
.
Am J Sports Med
.
2004
;
32
(
1
):
47
54
.
37
Ponsford
J,
Cameron
P,
Fitzgerald
M,
Grant
M,
Mikocka-Walus
A.
Long-term outcomes after uncomplicated mild traumatic brain injury: a comparison with trauma controls
.
J Neurotrauma
.
2011
;
28
(
6
):
937
946
.
38
Collins
M,
Lovell
MR,
Iverson
GL,
Ide
T,
Maroon
J.
Examining concussion rates and return to play in high school football players wearing newer helmet technology: a three-year prospective cohort study
.
Neurosurgery
.
2006
;
58
(
2
):
275
286
.
39
Covassin
T,
Elbin
RJ,
Harris
W,
Parker
T,
Kontos
A.
The role of age and sex in symptoms, neurocognitive performance, and postural stability in athletes after concussion
.
Am J Sports Med
.
2012
;
40
(
6
):
1303
1312
.
40
Covassin
T,
Elbin
RJ,
Bleecker
A,
Lipchik
A,
Kontos
AP.
Are there differences in neurocognitive function and symptoms between male and female soccer players after concussions?
Am J Sports Med
.
2013
;
41
(
12
):
2890
2895
.
41
Covassin
T,
Schatz
P,
Swanik
CB.
Sex differences in neuropsychological function and post-concussion symptoms of concussed collegiate athletes
.
Neurosurgery
.
2007
;
61
(
2
):
345
350
.
42
Covassin
T,
Stearne
D,
Elbin
R.
Concussion history and postconcussion neurocognitive performance and symptoms in collegiate athletes
.
J Athl Train
.
2008
;
43
(
2
):
119
124
.
43
Iverson
GL,
Brooks
BL,
Collins
MW,
Lovell
MR.
Tracking neuropsychological recovery following concussion in sport
.
Brain Inj
.
2006
;
20
(
3
):
245
252
.
44
Kontos
AP,
Elbin
RJ
III,
Covassin
T,
Larson
E.
Exploring differences in computerized neurocognitive concussion testing between African American and White athletes
.
Arch Clin Neuropsychol
.
2010
;
25
(
8
):
734
744
.
45
Lovell
MR,
Solomon
GS.
Neurocognitive test performance and symptom reporting in cheerleaders with concussions
.
J Pediatr
.
2013
;
163
(
4
):
1192.e1
1195.e1
.
46
McClincy
MP,
Lovell
MR,
Pardini
J,
Collins
MW,
Spore
MK.
Recovery from sports concussion in high school and collegiate athletes
.
Brain Inj
.
2006
;
20
(
1
):
33
39
.
47
McGrath
N,
Dinn
WM,
Collins
MW,
Lovell
MR,
Elbin
RJ,
Kontos
AP.
Post-exertion neurocognitive test failure among student-athletes following concussion
.
Brain Inj
.
2013
;
27
(
1
):
103
113
.
48
Pellman
EJ,
Lovell
MR,
Viano
DC,
Casson
IR.
Concussion in professional football: recovery of NFL and high school athletes assessed by computerized neuropsychological testing. Part 12
.
Neurosurgery
.
2006
;
58
(
2
):
263
274
.
49
Zuckerman
SL,
Solomon
GS,
Forbes
JA,
Haase
RF,
Sills
AK,
Lovell
MR.
Response to acute concussive injury in soccer players: is gender a modifying factor?
J Neurosurg Pediatr
.
2012
;
10
(
6
):
504
510
.
50
Mihalik
JP,
McCaffrey
MA,
Rivera
EM,
et al.
Effectiveness of mouthguards in reducing neurocognitive deficits following sports-related cerebral concussion
.
Dent Traumatol
.
2007
;
23
(
1
):
14
20
.
51
Iverson
GL,
Lovell
MR,
Collins
MW.
Interpreting change on ImPACT following sport concussion
.
Clin Neuropsychol
.
2003
;
17
(
4
):
460
467
.
52
Mihalik
JP,
Stump
JE,
Collins
MW,
Lovell
MR,
Field
M,
Maroon
JC.
Posttraumatic migraine characteristics in athletes following sports-related concussion
.
J Neurosurg
.
2005
;
102
(
5
):
850
855
.
53
Yuen
KM,
Tsai
YH,
Lin
WC,
Yang
CC,
Huang
SJ.
Retrospectively evaluated preinjury personality traits influence postconcussion symptoms
.
Appl Neuropsychol Adult
.
2016
;
23
(
5
):
322
332
.
54
Vargas
G,
Rabinowitz
A,
Meyer
J,
Arnett
PA.
Predictors and prevalence of postconcussion depression symptoms in collegiate athletes
.
J Athl Train
.
2015
;
50
(
3
):
250
255
.
55
Collins
MW,
Kontos
AP,
Reynolds
E,
Murawski
CD,
Fu
FH.
A comprehensive, targeted approach to the clinical care of athletes following sport-related concussion
.
Knee Surg Sports Traumatol Arthrosc
.
2014
;
22
(
2
):
235
246
.
56
Resch
JE,
Macciocchi
S,
Ferrara
MS.
Preliminary evidence of equivalence of alternate forms of the ImPACT
.
Clin Neuropsychol
.
2013
;
27
(
8
):
1265
1280
.
57
Gleser
LJ,
Olkin
I.
Stochastically dependent effect sizes
.
In
:
Cooper
H,
Hedges
LV,
Valentine
J,
eds
.
The Handbook of Research Synthesis and Meta-Analysis. 2nd ed
.
New York, NY
:
Russell Sage Foundation;
2009
:
357
376
.
58
Van den Noortgate
W,
Lopez-Lopez
JA,
Marin-Martinez
F,
Sanchez-Meca
J.
Meta-analysis of multiple outcomes: a multilevel approach
.
Behav Res Methods
.
2015
;
47
(
4
):
1274
1294
.
59
Lovell
MR,
Russo
L,
Maroon
J,
et al
for ImPACT Team
.
ImPACT Version 2.0 Clinical User's Manual
.
Hilton Head, SC: ImPACT; 2003:18.
.
60
Ellis
MJ,
Leddy
JJ,
Willer
B.
Physiological, vestibulo-ocular and cervicogenic post-concussion disorders: an evidence-based classification system with directions for treatment
.
Brain Inj
.
2015
;
29
(
2
):
238
248
.
Appendix 1. 

Effect sizes for A, verbal memory, B, visual memory, C, processing speed, D, reaction time, and, E, Postconcussion Symptom Scale score within 1 week postconcussion. Continued on next page.

Appendix 1. 

Effect sizes for A, verbal memory, B, visual memory, C, processing speed, D, reaction time, and, E, Postconcussion Symptom Scale score within 1 week postconcussion. Continued on next page.

Appendix 1. 

Effect sizes for A, verbal memory, B, visual memory, C, processing speed, D, reaction time, and, E, Postconcussion Symptom Scale score within 1 week postconcussion. Continued on next page.

Appendix 1. 

Effect sizes for A, verbal memory, B, visual memory, C, processing speed, D, reaction time, and, E, Postconcussion Symptom Scale score within 1 week postconcussion. Continued on next page.

Appendix 1. 

Continued from previous page.

Appendix 1. 

Continued from previous page.

Appendix 2. 

Effect sizes for A, verbal memory, B, visual memory, C, processing speed, D, reaction time, and, E, Postconcussion Symptom Scale score between 1 and 3 weeks postconcussion.

Appendix 2. 

Effect sizes for A, verbal memory, B, visual memory, C, processing speed, D, reaction time, and, E, Postconcussion Symptom Scale score between 1 and 3 weeks postconcussion.