Context

Preseason testing can be time intensive and cost prohibitive. Therefore, using normative data for postconcussion interpretation in lieu of preseason testing is desirable.

Objective

To establish the recovery trajectory for clinical reaction time (RTclin) and assess the usefulness of changes from baseline (comparison of postconcussion scores with individual baseline scores) and norm-based cutoff scores (comparison of postconcussion scores with a normative mean) for identifying impairments postconcussion.

Design

Case-control study.

Setting

Multisite clinical setting.

Patients or Other Participants

An overlapping sample of 99 participants (age = 19.0 ± 1.1 years) evaluated within 6 hours postconcussion, 176 participants (age = 18.9 ± 1.1 years) evaluated at 24 to 48 hours postconcussion, and 214 participants (age = 18.9 ± 1.1 years) evaluated once they were cleared to begin a return-to-play progression were included. Participants with concussion were compared with 942 control participants (age = 19.0 ± 1.0 years) who did not sustain a concussion during the study period but completed preseason baseline testing at 2 points separated by 1 year (years 1 and 2).

Main Outcome Measure(s)

At each time point, follow-up RTclin (ie, postconcussion or year 2) was compared with the individual year 1 preseason baseline RTclin and normative baseline data (ie, sex and sport specific). Receiver operating characteristic curves were calculated to compare the sensitivity and specificity of RTclin change from baseline and norm-based cutoff scores.

Results

Clinical reaction time performance declined within 6 hours (18 milliseconds, 9.2% slower than baseline). The decline persisted at 24 to 48 hours (15 milliseconds, 7.6% slower than baseline), but performance recovered by the time of return-to-play initiation. Within 6 hours, a change from baseline of 16 milliseconds maximized combined sensitivity (52%) and specificity (79%, area under the curve [AUC] = 0.702), whereas a norm-based cutoff score of 19 milliseconds maximized combined sensitivity (46%) and specificity (86%, AUC = 0.700). At 24 to 48 hours, a change from baseline of 2 milliseconds maximized combined sensitivity (64%) and specificity (61%, AUC = 0.666), whereas a norm-based cutoff score of 0 milliseconds maximized combined sensitivity (63%) and specificity (62%, AUC = 0.647).

Conclusions

Norm-based cutoff scores can be used for interpreting RTclin scores postconcussion in collegiate athletes when individual baseline data are not available, although low sensitivity and specificity limit the use of RTclin as a stand-alone test.

Key Points
  • Clinical reaction time performance declined within 6 hours postconcussion (18 milliseconds, 9.2% slower than baseline). The decline persisted at 24 to 48 hours postconcussion (15 milliseconds, 7.6% slower than baseline), but performance recovered by the time of return-to-play initiation.

  • Health care providers can use normative data in lieu of individual baseline measures to interpret clinical reaction time scores postconcussion.

Patients with concussions are best managed using a multimodal and multifaceted assessment battery.14  According to the consensus statement on concussion in sport from the 5th International Conference on Concussion in Sport,1  baseline testing may be useful but is not necessary for interpreting postconcussion scores. Baseline scores are thought to account for patients' preinjury differences, thereby providing a valid comparison for postconcussion outcomes.5  Although routinely performed at the collegiate level, baseline testing can be time intensive and cost prohibitive.5  The need for baseline testing varies across assessment tools. For example, Schmidt et al5  compared baseline scores with normative data for interpreting Automated Neuropsychological Assessment Metrics composite scores, Sensory Organization Test total scores, and graded symptom checklist scores postconcussion and concluded that clinicians may consider using normative data in lieu of individualized baseline measures. Broglio et al,6  on the other hand, compared the use of same-season baseline scores of concussed athletes with the use of previous-season baseline scores of concussed athletes and the use of baseline scores of nonconcussed control individuals (ie, normative data) for interpreting Standardized Assessment of Concussion total scores, Sport Concussion Assessment Tool (SCAT) symptom and symptom severity scores, Balance Error Scoring System total scores, Brief Symptom Inventory-18 subscores, and Immediate Post-Concussion Assessment and Cognitive Testing composite scores and concluded that annual baseline testing optimized assessment processes. Lempke et al,7  in a recent meta-analysis, demonstrated robust reaction time (RT) deficits in the short term postconcussion but negligible differences when comparing between-participants and within-participant effects, suggesting that preinjury baseline scores may not add clinical value in determining postconcussion RT impairment. However, the analyses were not specific to functional RT tests (eg, clinical RT [RTclin]).

Impaired RT is common postconcussion,7  and nearly 20% of athletic trainers included RT testing in concussion diagnosis.8  The RTclin assessment was developed as a simple clinical method for measuring RT that is cost effective and requires minimal equipment (ie, a rigid rod and weighted rubber disk).9  Its test-retest reliability compares favorably with that of computerized RT assessments (RTclin intraclass correlation coefficient [ICC] = 0.645, computerized RT assessment ICC = 0.512),10  and acutely, its sensitivity and specificity were similar to those of other commonly used concussion-assessment tools (sensitivity = 75%, specificity = 68%, critical change value = 0 milliseconds), although the recovery trajectory of RTclin remains unknown.11  Lempke et al7  reported that RT deficits persisted in the intermediate-term period (21–59 days postconcussion) yet resolved and improved by the long-term period (80–365 days postconcussion); however, the deficits were not specific to RTclin performance. Furthermore, the need for baseline RTclin testing has not been examined. However, normative data have been reported for collegiate student-athletes.12  Therefore, the purpose of our study was 2-fold: to establish the recovery trajectory for RTclin and compare changes from baseline (comparison of postconcussion scores with individual baseline scores) and norm-based cutoff scores (comparison of postconcussion scores with a normative mean) to identify impairments postconcussion. We hypothesized that RTclin would return to baseline by the time the athletes began their return-to-play (RTP) progression, and based on the work of Eckner et al,11  who suggested that a 0-millisecond critical change value maximized sensitivity and specificity, we hypothesized that baseline comparisons would better differentiate concussed from control participants.

Participants

Volunteers were recruited through the National Collegiate Athletic Association–Department of Defense Grand Alliance Concussion Assessment, Research and Education (CARE) Consortium; details of the structure and methods of this consortium have been previously described.13  Because RTclin is an optional assessment, it is used at only a subset of CARE sites. A total of 2579 participants completed preseason baseline RTclin testing at the time of enrollment (year 1) and 1131 participants completed preseason baseline RTclin testing 1 year later (year 2). Participants who sustained a concussion after preseason baseline testing (n = 356) were evaluated within 6 hours postconcussion (n = 106 concussions), at 24 to 48 hours postconcussion (n = 197 concussions), and at the time they were cleared to begin an RTP progression (n = 251 concussions). The concussion group sustained a clinician-diagnosed concussion according to the consensus definition obtained through evidence-based guidelines and adopted by the National Collegiate Athletic Association–Department of Defense Grand Alliance CARE Consortium.13,14  If a participant sustained multiple concussions during the study period, only data from the first concussion were included in the analyses (ie, 3 were excluded within 6 hours postconcussion, 7 were excluded at 24 to 48 hours postconcussion, and 23 were excluded at the time of RTP initiation). Control participants (n = 942) underwent preseason baseline testing at 2 points 1 year apart (years 1 and 2) but did not sustain a concussion during the study period. Participants provided written informed consent, and the local institutional review board at each of the 2 performance sites, as well as the US Army Medical Research and Materiel Command Human Research Protection Office, reviewed and approved all study procedures.

The RTclin Procedures

The protocol for RTclin testing has been described in detail.911,1519  Briefly, an 80-cm rigid rod wrapped in friction tape and attached to a weighted rubber disk was dropped at random intervals between 2 and 5 seconds. Participants sat with their dominant forearm resting on a table and their hand positioned over the edge. The weighted rubber disk was aligned with the top of the individual's open hand. The examiner released the rod, and the participant caught it as quickly as possible. The rod was marked in 0.5-cm increments, and for each trial, RTclin was calculated from the distance the rod fell: \(\def\upalpha{\unicode[Times]{x3B1}}\)\(\def\upbeta{\unicode[Times]{x3B2}}\)\(\def\upgamma{\unicode[Times]{x3B3}}\)\(\def\updelta{\unicode[Times]{x3B4}}\)\(\def\upvarepsilon{\unicode[Times]{x3B5}}\)\(\def\upzeta{\unicode[Times]{x3B6}}\)\(\def\upeta{\unicode[Times]{x3B7}}\)\(\def\uptheta{\unicode[Times]{x3B8}}\)\(\def\upiota{\unicode[Times]{x3B9}}\)\(\def\upkappa{\unicode[Times]{x3BA}}\)\(\def\uplambda{\unicode[Times]{x3BB}}\)\(\def\upmu{\unicode[Times]{x3BC}}\)\(\def\upnu{\unicode[Times]{x3BD}}\)\(\def\upxi{\unicode[Times]{x3BE}}\)\(\def\upomicron{\unicode[Times]{x3BF}}\)\(\def\uppi{\unicode[Times]{x3C0}}\)\(\def\uprho{\unicode[Times]{x3C1}}\)\(\def\upsigma{\unicode[Times]{x3C3}}\)\(\def\uptau{\unicode[Times]{x3C4}}\)\(\def\upupsilon{\unicode[Times]{x3C5}}\)\(\def\upphi{\unicode[Times]{x3C6}}\)\(\def\upchi{\unicode[Times]{x3C7}}\)\(\def\uppsy{\unicode[Times]{x3C8}}\)\(\def\upomega{\unicode[Times]{x3C9}}\)\(\def\bialpha{\boldsymbol{\alpha}}\)\(\def\bibeta{\boldsymbol{\beta}}\)\(\def\bigamma{\boldsymbol{\gamma}}\)\(\def\bidelta{\boldsymbol{\delta}}\)\(\def\bivarepsilon{\boldsymbol{\varepsilon}}\)\(\def\bizeta{\boldsymbol{\zeta}}\)\(\def\bieta{\boldsymbol{\eta}}\)\(\def\bitheta{\boldsymbol{\theta}}\)\(\def\biiota{\boldsymbol{\iota}}\)\(\def\bikappa{\boldsymbol{\kappa}}\)\(\def\bilambda{\boldsymbol{\lambda}}\)\(\def\bimu{\boldsymbol{\mu}}\)\(\def\binu{\boldsymbol{\nu}}\)\(\def\bixi{\boldsymbol{\xi}}\)\(\def\biomicron{\boldsymbol{\micron}}\)\(\def\bipi{\boldsymbol{\pi}}\)\(\def\birho{\boldsymbol{\rho}}\)\(\def\bisigma{\boldsymbol{\sigma}}\)\(\def\bitau{\boldsymbol{\tau}}\)\(\def\biupsilon{\boldsymbol{\upsilon}}\)\(\def\biphi{\boldsymbol{\phi}}\)\(\def\bichi{\boldsymbol{\chi}}\)\(\def\bipsy{\boldsymbol{\psy}}\)\(\def\biomega{\boldsymbol{\omega}}\)\(\def\bupalpha{\bf{\alpha}}\)\(\def\bupbeta{\bf{\beta}}\)\(\def\bupgamma{\bf{\gamma}}\)\(\def\bupdelta{\bf{\delta}}\)\(\def\bupvarepsilon{\bf{\varepsilon}}\)\(\def\bupzeta{\bf{\zeta}}\)\(\def\bupeta{\bf{\eta}}\)\(\def\buptheta{\bf{\theta}}\)\(\def\bupiota{\bf{\iota}}\)\(\def\bupkappa{\bf{\kappa}}\)\(\def\buplambda{\bf{\lambda}}\)\(\def\bupmu{\bf{\mu}}\)\(\def\bupnu{\bf{\nu}}\)\(\def\bupxi{\bf{\xi}}\)\(\def\bupomicron{\bf{\micron}}\)\(\def\buppi{\bf{\pi}}\)\(\def\buprho{\bf{\rho}}\)\(\def\bupsigma{\bf{\sigma}}\)\(\def\buptau{\bf{\tau}}\)\(\def\bupupsilon{\bf{\upsilon}}\)\(\def\bupphi{\bf{\phi}}\)\(\def\bupchi{\bf{\chi}}\)\(\def\buppsy{\bf{\psy}}\)\(\def\bupomega{\bf{\omega}}\)\(\def\bGamma{\bf{\Gamma}}\)\(\def\bDelta{\bf{\Delta}}\)\(\def\bTheta{\bf{\Theta}}\)\(\def\bLambda{\bf{\Lambda}}\)\(\def\bXi{\bf{\Xi}}\)\(\def\bPi{\bf{\Pi}}\)\(\def\bSigma{\bf{\Sigma}}\)\(\def\bPhi{\bf{\Phi}}\)\(\def\bPsi{\bf{\Psi}}\)\(\def\bOmega{\bf{\Omega}}\)\(t = \sqrt {\left( {2 \times d \div g} \right)} \)⁠, where d was the distance the rod fell and g was the acceleration due to gravity. Participants completed 2 practice trials followed by 8 data-acquisition trials, and the mean RTclin was used for analyses, consistent with current clinical practice recommendations.

Caccese et al12  identified performance factors that might affect RTclin. These included sex; race; ethnicity; dominant hand; sport type; number of previous concussions; presence of anxiety, learning disability, attention deficit/hyperactivity disorder, depression, or migraine headache; self-reported sleep the night before the test; mass; height; age; total number of SCAT-3 symptoms; and total SCAT-3 symptom burden. In this model, sex, race, sport type, and height were the only predictors of RTclin, although for both race and height, the effect sizes were small.12  Therefore, we concluded that, when adjusting for sex and sport type, the following normative data may be considered (mean ± SD): female, noncontact = 211.5 ± 25.8 milliseconds, limited contact = 212.1 ± 24.3 milliseconds, contact = 203.7 ± 21.5 milliseconds; male, noncontact = 199.4 ± 26.7 milliseconds, limited contact = 196.3 ± 23.9 milliseconds, contact = 195.0 ± 23.8 milliseconds.12  We used these normative data as referents for our analyses.

Statistical Analyses

Repeated-measures analyses of variance were used to compare year 1 preseason baseline RTclin and follow-up RTclin (ie, postconcussion or year 2) tests in concussed versus control student-athletes. In addition, repeated-measures analyses of variance were conducted to compare normative baseline data (ie, sex and sport contact type) and follow-up RTclin testing in concussed versus control student-athletes, as well as within-participant SDs between groups across time. When appropriate, we calculated post hoc independent-samples t tests to compare groups at specific time points. Bonferroni corrections were applied to post hoc t tests to correct for multiple comparisons (P < .025). The effect sizes at baseline and follow-up testing were described using the Cohen d for independent-samples t tests (Cohen d = [M2M1] / SDpooled, where M indicates the mean and 0.2 represents a small effect size; 0.5, a medium effect size; and 0.8, a large effect size20 ).

Receiver operating characteristic (ROC) curves were created to compare the sensitivity and specificity of RTclin change from baseline and norm-based cutoff scores. Sensitivity and specificity values were summed at each cutoff value, with the highest summed score interpreted as having the greatest combined sensitivity and specificity. The 95% minimal detectable change (MDC) was calculated based on the RTclin scores of control participants using the standard formula: MDC = 1.96 × SEM × √2. Sensitivity and specificity based on the 95% MDC are also presented. We conducted all analyses using JMP (version 14.0; SAS Institute Inc).

We observed a decline in RTclin performance within 6 hours postconcussion (18 milliseconds; 9.2% slower than that of baseline) that persisted at 24 to 48 hours postconcussion (15 milliseconds; 7.6% slower than that of baseline) but recovered by the time of RTP initiation (Figure 1). Among control participants, test-retest reliability between year 1 and year 2 baseline testing demonstrated an ICC (2,k) of 0.532 (95% CI = 0.464, 0.592) and MDC of 45 milliseconds.

Figure 1

Recovery curve for the concussion group. Error bars represent the standard error. Baseline refers to the athletes' individual baselines. Concussion individual baseline = 197 ± 2 milliseconds, within 6 hours = 215 ± 3 milliseconds, at 24 to 48 hours = 212 ± 2 milliseconds, at return-to-play initiation = 196 ± 2 milliseconds. a Indicates difference (P < .025).

Figure 1

Recovery curve for the concussion group. Error bars represent the standard error. Baseline refers to the athletes' individual baselines. Concussion individual baseline = 197 ± 2 milliseconds, within 6 hours = 215 ± 3 milliseconds, at 24 to 48 hours = 212 ± 2 milliseconds, at return-to-play initiation = 196 ± 2 milliseconds. a Indicates difference (P < .025).

Close modal

Within 6 Hours Postconcussion

A total of 99 participants with preseason baseline scores were evaluated within 6 hours of sustaining a concussion (Table 1). As noted, 942 uninjured control participants completed preseason baseline testing at years 1 and 2 (Table 1). Relative to each person's year 1 preseason baseline results, we observed a time-by-group interaction (F1,1039 = 62.088, P < .001; Figure 2A) with an RTclin that was 18 milliseconds slower in the concussion group and 5 milliseconds faster in the control group. Post hoc independent-samples t tests revealed that the 2 groups had similar baseline RTclin performance (concussion = 197 ± 23 milliseconds, control = 199 ± 24 milliseconds; t1040 = −1.267, P = .21, Cohen d = 0.14) but RTclin was slower in the concussion group postconcussion (concussion = 215 ± 34 milliseconds, control = 194 ± 25 milliseconds; t1040 = 7.427, P < .001, Cohen d = 0.68). We observed no time-by-group interaction in the within-participant SD (F1,1039 = 0.126, P = .72). Relative to normative predicted baselines, we noted a time-by-group interaction (F1,1039 = 69.482, P < .001; Figure 2B) with an RT that was 15 milliseconds slower in the concussion group and 8 milliseconds faster in the control group. Post hoc independent-samples t tests revealed a faster normative predicted baseline RTclin in the concussion group (concussion = 200 ± 6 milliseconds, control = 202 ± 7 milliseconds; t1040 = −2.430, P = .02, Cohen d = 0.28) and the same postconcussion difference described earlier.

Table 1

Group Characteristics

Group Characteristics
Group Characteristics
Figure 2

Change in mean clinical reaction time in concussion and control groups at <6 hours postconcussion from A, baseline testing, and B, normative data. Error bars represent standard error. The group-by-time interaction was different for the changes from baseline and normative data. C, Receiver operating characteristic curves based on comparisons of postconcussion−baseline testing and postconcussion−normative data. Concussion individual baseline = 197 ± 2 milliseconds, normative = 200 ± 1 milliseconds, postconcussion = 215 ± 3 milliseconds. Control group year 1 baseline = 199 ± 1 milliseconds, year 2 baseline = 194 ± 1 milliseconds, normative = 202 ± 0 milliseconds. Abbreviation: AUC, area under the curve. a Indicates difference (P < .025).

Figure 2

Change in mean clinical reaction time in concussion and control groups at <6 hours postconcussion from A, baseline testing, and B, normative data. Error bars represent standard error. The group-by-time interaction was different for the changes from baseline and normative data. C, Receiver operating characteristic curves based on comparisons of postconcussion−baseline testing and postconcussion−normative data. Concussion individual baseline = 197 ± 2 milliseconds, normative = 200 ± 1 milliseconds, postconcussion = 215 ± 3 milliseconds. Control group year 1 baseline = 199 ± 1 milliseconds, year 2 baseline = 194 ± 1 milliseconds, normative = 202 ± 0 milliseconds. Abbreviation: AUC, area under the curve. a Indicates difference (P < .025).

Close modal

The ROC curve analysis using change from baseline demonstrated that a cutoff value of 16 milliseconds (ie, follow-up RTclin ≥16 milliseconds slower than baseline is considered abnormal) maximized combined sensitivity and specificity, with a summed sensitivity and specificity value of 1.3 corresponding to 52% sensitivity and 79% specificity (area under the curve [AUC] = 0.702, P < .001; Figure 2C). A change from baseline that exceeded the 95% MDC resulted in 16% sensitivity and 96% specificity. Results of the ROC curve analysis using norm-based cutoff scores demonstrated that a cutoff value of 19 milliseconds maximized combined sensitivity and specificity, with a summed sensitivity and specificity value of 1.3 corresponding to 46% sensitivity and 86% specificity (AUC = 0.700, P < .001; Figure 2C). A norm-based cutoff score that exceeded the 95% MDC resulted in 16% sensitivity and 98% specificity. Cutoff values that maximize combined sensitivity and specificity may not be the most important clinical metrics, so we present other change scores favoring different sensitivity and specificity trade-offs at each time point in Tables 2 and 3 and Supplemental Table 1 (available online at http://dx.doi.org/10.4085/1062-6050-457-20.S1) for changes from both baseline and norm-based cutoff scores.

Table 2

Baseline Change Scores for Various Sensitivities and Associated Specificitiesa

Baseline Change Scores for Various Sensitivities and Associated Specificitiesa
Baseline Change Scores for Various Sensitivities and Associated Specificitiesa
Table 3

Normative Change Scores for Various Sensitivities and Associated Specificitiesa

Normative Change Scores for Various Sensitivities and Associated Specificitiesa
Normative Change Scores for Various Sensitivities and Associated Specificitiesa

Within 24 to 48 Hours Postconcussion

A total of 176 participants with preseason baseline scores were evaluated within 24 to 48 hours of sustaining a concussion (Table 1). The same 942 control participants were asssessed. Relative to each participant's year 1 preseason baseline results, we observed a time-by-group interaction (F1,1116 = 73.319, P < .001; Figure 3A) with an RTclin that was 15 milliseconds slower in the concussion group and 6 milliseconds faster in the control group. Post hoc independent-samples t tests revealed that the 2 groups had similar baseline RTclin performance (concussion = 197 ± 24 milliseconds, control = 200 ± 24 milliseconds; t1117 = −1.591, P = .11, Cohen d = 0.13) but RTclin was slower in the concussion group postconcussion (concussion = 212 ± 36 milliseconds, control = 194 ± 25 milliseconds; t1117 = 7.724, P < .001, Cohen d = 0.55). We found no time-by-group interaction in the within-participant SD (F1,1116 = 2.485, P = .12). Relative to normative predicted baselines, we demonstrated a time-by-group interaction (F1,1116 = 67.580, P < .001; Figure 3B) with an RTclin that was 10 milliseconds slower in the concussion group and 8 milliseconds faster in the control group. Post hoc independent-samples t tests showed no differences between the groups' normative data (concussion = 202 ± 6 milliseconds, control = 202 ± 7 milliseconds; t1117 = −0.991, P = .32, Cohen d = 0.07) and the same postconcussion difference described earlier.

Figure 3

Changes in mean clinical reaction time of concussion and control groups at 24 to 48 hours postconcussion from A, baseline testing, and B, normative data. Error bars represent standard error. The group-by-time interaction was different for the changes from baseline and from normative data. C, Receiver operating characteristic curves based on comparisons of 24 to 48 hours−baseline testing and 24 to 48 hours−normative data. Concussion individual baseline = 197 ± 2 milliseconds, normative = 202 ± 1 milliseconds, postconcussion = 212 ± 2 milliseconds. Control group year 1 baseline = 200 ± 1 milliseconds, year 2 baseline = 194 ± 1 milliseconds, normative = 202 ± 0 milliseconds. Abbreviation: AUC, area under the curve. a Indicates difference (P < .025).

Figure 3

Changes in mean clinical reaction time of concussion and control groups at 24 to 48 hours postconcussion from A, baseline testing, and B, normative data. Error bars represent standard error. The group-by-time interaction was different for the changes from baseline and from normative data. C, Receiver operating characteristic curves based on comparisons of 24 to 48 hours−baseline testing and 24 to 48 hours−normative data. Concussion individual baseline = 197 ± 2 milliseconds, normative = 202 ± 1 milliseconds, postconcussion = 212 ± 2 milliseconds. Control group year 1 baseline = 200 ± 1 milliseconds, year 2 baseline = 194 ± 1 milliseconds, normative = 202 ± 0 milliseconds. Abbreviation: AUC, area under the curve. a Indicates difference (P < .025).

Close modal

The ROC curve analysis using change from baseline indicated that a cutoff value of 2 milliseconds maximized combined sensitivity and specificity, with a summed sensitivity and specificity value of 1.2 corresponding to 64% sensitivity and 61% specificity (AUC = 0.666, P < .001; Figure 3C). A change from baseline that exceeded the 95% MDC resulted in 18% sensitivity and 96% specificity. The ROC curve analysis using norm-based cutoff scores demonstrated that a cutoff value of 0 milliseconds maximized combined sensitivity and specificity, with a summed sensitivity and specificity value of 1.2 corresponding to 63% sensitivity and 62% specificity (AUC = 0.647, P < .001; Figure 3C). A norm-based cutoff score that exceeded the 95% MDC resulted in 13% sensitivity and 98% specificity. Other change scores favoring different sensitivity and specificity trade-offs at each time point are presented in Tables 2 and 3 and Supplemental Table 2 for changes from both baseline and norm-based cutoff scores.

At RTP Initiation

A total of 214 participants with preseason baseline scores were evaluated at RTP initiation (Table 1). The same 942 control participants were assessed. Relative to each participant's year 1 preseason baseline results, we observed no time-by-group interaction (F1,1154 = 3.541, P = .06; Figure 4A). No time-by-group interaction was present in the within-participants SD (F1,1154 = 0.029, P = .87). Relative to normative predicted baselines, we found no time-by-group interaction (F1,1154 = 2.221, P = .14; Figure 4B). The ROC curve analysis using change from baseline reflected no difference (AUC = 0.526, P = .24; Figure 4C). The ROC curve analysis using norm-based cutoff scores indicated no difference (AUC = 0.517, P = .44; Figure 4C). Other change scores favoring different sensitivity and specificity trade-offs at each time point are presented in Tables 2 and 3 and Supplemental Table 3 for changes from both baseline and norm-based cutoff scores.

Figure 4

Changes in mean clinical reaction time in concussion and control groups at return-to-play (RTP) initiation from A, baseline testing, and B, normative data. Error bars represent standard error. The group-by-time interaction was not different for the changes from baseline or from normative data. C, Receiver operating characteristic curves based on comparisons of RTP initiation−baseline testing and RTP initiation−normative data. Concussion individual baseline = 198 ± 2 milliseconds, normative = 201 ± 0 milliseconds, postconcussion = 196 ± 2 milliseconds. Control group year 1 baseline = 200 ± 1 milliseconds, year 2 baseline = 194 ± 1 milliseconds, normative = 202 ± 0 milliseconds. Abbreviation: AUC, area under the curve.

Figure 4

Changes in mean clinical reaction time in concussion and control groups at return-to-play (RTP) initiation from A, baseline testing, and B, normative data. Error bars represent standard error. The group-by-time interaction was not different for the changes from baseline or from normative data. C, Receiver operating characteristic curves based on comparisons of RTP initiation−baseline testing and RTP initiation−normative data. Concussion individual baseline = 198 ± 2 milliseconds, normative = 201 ± 0 milliseconds, postconcussion = 196 ± 2 milliseconds. Control group year 1 baseline = 200 ± 1 milliseconds, year 2 baseline = 194 ± 1 milliseconds, normative = 202 ± 0 milliseconds. Abbreviation: AUC, area under the curve.

Close modal

Site Differences in RTclin

Participants were recruited from 2 sites. At baseline, the participants at site 1 had a faster RTclin than did participants at site 2 (mean difference [95% CI] = −7.1 milliseconds [−10.1, −4.1 milliseconds]). However, change scores from baseline were not different between sites 1 and 2 for the concussed group (mean difference [95% CI] within 6 hours postconcussion = −6.7 milliseconds [−21.1, 7.6 milliseconds], at 24–48 hours postconcussion = −0.7 milliseconds [−12.0, 10.6 milliseconds], or at RTP initiation = −3.0 milliseconds [−10.8, 4.8 milliseconds]).

Our aims for this study were to establish the recovery trajectory for RTclin and determine if baseline testing is needed to interpret postconcussion RTclin scores. We hypothesized that RTclin would return to baseline by the time the athletes began their RTP progression. This hypothesis was supported; our findings suggested that RTclin scores slow (ie, worsen) postconcussion and then gradually return to baseline by RTP initiation (Figure 1). Based on previous work11  that indicated a 0-millisecond critical change value maximized sensitivity and specificity, we proposed that baseline comparisons would better differentiate participants with concussion from control participants. However, this hypothesis was not supported. Changes from both baseline and norm-based cutoff scores differentiated the concussed group from the control group (eg, within 6 hours postconcussion, baseline AUC = 0.702 and normative AUC = 0.700), and sensitivity and specificity were similar between methods (eg, comparison of within 6 hours postconcussion and baseline: sensitivity = 52% and specificity = 79%; comparison with normative values: sensitivity = 46% and specificity = 86%). These results suggest that health care providers without adequate resources and the time to perform baseline assessments can use normative data in lieu of individual baseline measures for interpreting RTclin scores postconcussion.

Sensitivity and specificity were similar using changes from both baseline and norm-based cutoff scores, but despite small to medium effect sizes, combined sensitivity and specificity was low. These findings indicate that comparing RTclin scores with normative values is appropriate for identifying RT impairments postconcussion but sensitivity and specificity values are inadequate for clinical use in isolation. Our critical value change scores 24 to 48 hours postconcussion were similar to those of Eckner et al,11  who reported that a change score of 0 milliseconds maximized combined sensitivity (75%) and specificity (68%) within 48 hours postconcussion, although combined sensitivity and specificity was higher in their cohort of 28 athletes with concussion. In our study, a change from an individualized baseline value of 2 milliseconds maximized the combined sensitivity (64%) and specificity (61%) and a norm-based cutoff score of 0 milliseconds maximized the combined sensitivity (63%) and specificity (62%) at 24 to 48 hours postconcussion. When interpreting change scores using norm-based cutoff scores, clinicians can subtract a patient's sex- and sport-adjusted mean normative RTclin score from the postconcussion mean RTclin. For example, a men's football player with a 210-millisecond postconcussion RTclin would have a change score of 15 milliseconds from his normative RTclin. At 24 to 48 hours, this would exceed the norm-based cutoff score of 0 milliseconds. The RTclin is slower among concussed than nonconcussed athletes, but as with other common concussion-assessment tools, when RTclin is used in isolation, sensitivity and specificity are low, thereby falling short of the standards for clinical utility.6 

In this study, we observed a decline in RTclin performance within 6 hours postconcussion (18 milliseconds, 9.2% slower than that of the individual baseline) that persisted at 24 to 48 hours postconcussion (15 milliseconds, 7.6% slower than that of the individual baseline) but recovered by RTP initiation (Figure 1). These results are similar to those of Eckner et al,11  who noted a decline in RTclin performance within 48 hours postconcussion (8.4% slower than that of the individual baseline) among 28 athletes with concussion. Similar to the findings of Eckner et al,11  our participants' follow-up RTclin performance also showed a trend toward improvement in the control group, suggesting a potential learning effect. Therefore, the contrasts between RTclin decline observed in concussed student-athletes and the RTclin improvement in control student-athletes were different within 6 hours postconcussion and at 24 to 48 hours postconcussion but not at RTP initiation. The RTclin was slower within 48 hours postconcussion but recovered by RTP initiation.

Optimal neuromotor control during sports participation is critical for maximizing performance and preventing injuries. The RT may be an important component of a multifaceted examination for evaluating neuromotor recovery postconcussion8  to prevent athletes from returning to play prematurely.7  Postconcussion RT deficits are robust in the first 3 days postconcussion7  and have been reported to persist for up to 60 days.7  Contrary to previous work, our findings suggest that RTclin scores aligned well with symptom recovery (ie, RTclin scores returned to baseline by the initiation of the RTP progression). Conflicting findings are not surprising because population, methodologic, and RT assessment characteristics all contribute to RT outcomes.7  Despite variations in testing conditions, RT deficits are apparent postconcussion, and clinical assessments of RT (eg, RTclin) may be essential tools in evaluating neuromotor recovery. Importantly, RTclin should be considered 1 component of a multifaceted concussion-assessment battery and not a stand-alone diagnostic test.

Our study had limitations. First, this is the first study to compare changes from baseline and norm-based cutoff scores in interpreting RTclin scores postconcussion. However, the participants in this research provided normative data in previous work. Therefore, the sensitivity and specificity based on normative data may be higher than expected with an independent cohort. Second, not all athletes were assessed at all time points. The recovery curve presented in Figure 1 includes any concussed athlete tested at a minimum of 1 postconcussion time point and may not represent the true recovery curve for all athletes across time. Third, the concussion and control groups were not assessed at the same time points. The control group comprised participants who had undergone preseason baseline assessments 1 year apart. Given that the stability of any test is expected to decrease over longer retest intervals, this discrepancy may have decreased the sensitivity and specificity trade-offs identified in this study and may account for the greater combined sensitivity and specificity in previous work.11  Fourth, the data presented herein reflect only collegiate student-athletes and may not be generalizable to other cohorts. Fifth, the examiners were not blinded to the student-athletes' concussion status during follow-up testing, although they were probably not aware of the participants' baseline performance. Sixth, participants at site 1 had a faster baseline RTclin than those at site 2 had. This was an unavoidable limitation and may have been related to institutional differences (eg, test administrators or athlete performance).

Our results suggest that normative data can be used in lieu of individual baseline measures for interpreting RTclin scores postconcussion in collegiate athletes. For individual baseline measures, change scores of 16 milliseconds and 2 milliseconds used as cutoffs within 6 hours and at 24 to 48 hours, respectively, postconcussion maximized combined sensitivity and specificity. With normative data, combined sensitivity and specificity was maximized using change scores of 19 milliseconds and 0 milliseconds as cutoffs within 6 hours and at 24 to 48 hours, respectively, postconcussion. These combined sensitivity (46%–64%) and specificity (61%–86%) values were similar to those of other common concussion-assessment tools, although RTclin may have even greater utility when used as part of a concussion-assessment battery.

This study was supported in part by the Grand Alliance CARE Consortium, funded by the National Collegiate Athletic Association and the US Department of Defense. The US Army Medical Research Acquisition Activity, Fort Detrick, Maryland, is the awarding and administering acquisition office. This study was supported by the Office of the Assistant Secretary of Defense for Health Affairs through the Psychological Health and Traumatic Brain Injury Program under award W81XWH-14-2-0151. Opinions, interpretations, conclusions and recommendations are those of the authors and are not necessarily endorsed by the US Department of Defense (Defense Health Program funds).

1. 
McCrory
P,
Meeuwisse
W,
Dvorak
J,
et al
Consensus statement on concussion in sport—the 5th International Conference on Concussion in Sport held in Berlin, October 2016
.
Br J Sports Med
.
2017
;
51
(11)
:
838
847
.
2. 
Lumba-Brown
A,
Yeates
KO,
Sarmiento
K,
et al
Centers for Disease Control and Prevention guideline on the diagnosis and management of mild traumatic brain injury among children
.
JAMA Pediatr
.
2018
;
172
(11)
:
e182853
.
3. 
VA/DoD clinical practice guidelines: management of concussion-mild traumatic brain injury
.
US Department of Veterans Affairs website
.
Published
2016
.
Accessed December 16, 2020.
4. 
Harmon
KG,
Clugston
JR,
Dec
K,
et al
American Medical Society for Sports Medicine position statement on concussion in sport
.
Br J Sports Med
.
2019
;
53
(4)
:
213
225
.
5. 
Schmidt
JD,
Register-Mihalik
JK,
Mihalik
JP,
Kerr
ZY,
Guskiewicz
KM.
Identifying impairments after concussion: normative data versus individualized baselines
.
Med Sci Sports Exerc
.
2012
;
44
(9)
:
1621
1628
.
6. 
Broglio
SP,
Harezlak
J,
Katz
B,
et al
Acute sport concussion assessment optimization: a prospective assessment from the CARE consortium
.
Sports Med
.
2019
;
49
(12)
:
1977
1987
.
7. 
Lempke
LB,
Howell
DR,
Eckner
JT,
Lynall
RC.
Examination of reaction time deficits following concussion: a systematic review and meta-analysis
.
Sports Med
.
2020
;
50
(7)
:
1341
1359
.
8. 
Lempke
LB,
Schmidt
JD,
Lynall
RC.
Athletic trainers' concussion-assessment and concussion-management practices: an update
.
J Athl Train
.
2020
;
55
(1)
:
17
26
.
9. 
Eckner
JT,
Whitacre
RD,
Kirsch
NL,
Richardson
JK.
Evaluating a clinical measure of reaction time: an observational study
.
Percept Mot Skills
.
2009
;
108
(3)
:
717
720
.
10. 
Eckner
JT,
Kutcher
JS,
Richardson
JK.
Between-seasons test-retest reliability of clinically measured reaction time in National Collegiate Athletic Association Division I athletes
.
J Athl Train
.
2011
;
46
(4)
:
409
414
.
11. 
Eckner
JT,
Kutcher
JS,
Broglio
SP,
Richardson
JK.
Effect of sport-related concussion on clinically measured simple reaction time
.
Br J Sports Med
.
2014
;
48
(2)
:
112
118
.
12. 
Caccese
JB,
Eckner
JT,
Franco-MacKendrick
L,
et al
Clinical reaction-time performance factors in healthy collegiate athletes
.
J Athl Train
.
2020
;
55
(6)
:
601
607
.
13. 
Broglio
SP,
McCrea
M,
McAllister
T,
et al
A national study on the effects of concussion in collegiate athletes and US military service academy members: the NCAA–DoD Concussion Assessment, Research and Education (CARE) consortium structure and methods
.
Sports Med
.
2017
;
47
(7)
:
1437
1451
.
14. 
Carney
N,
Ghajar
J,
Jagoda
A,
et al
Concussion guidelines step 1: systematic review of prevalent indicators
.
Neurosurgery
.
2014
;
75
(suppl 1)
:
S3
S15
.
15. 
Eckner
JT,
Lipps
DB,
Kim
H,
Richardson
JK,
Ashton-Miller
JA.
Can a clinical test of reaction time predict a functional head-protective response?
Med Sci Sports Exerc
.
2011
;
43
(3)
:
382
387
.
16. 
Eckner
JT,
Kutcher
JS,
Richardson
JK.
Effect of concussion on clinically measured reaction time in 9 NCAA Division I collegiate athletes: a preliminary study
.
PM R
.
2011
;
3
(3)
:
212
218
.
17. 
Eckner
JT,
Chandran
S,
Richardson
JK.
Investigating the role of feedback and motivation in clinical reaction time assessment
.
PM R
.
2011
;
3
(12)
:
1092
1097
.
18. 
Eckner
JT,
Kutcher
JS,
Richardson
JK.
Pilot evaluation of a novel clinical test of reaction time in National Collegiate Athletic Association Division I football players
.
J Athl Train
.
2010
;
45
(4)
:
327
332
.
19. 
Eckner
JT,
Richardson
JK,
Kim
H,
Joshi
MS,
Oh
YK,
Ashton-Miller
JA.
Reliability and criterion validity of a novel clinical test of simple and complex reaction time in athletes
.
Percept Mot Skills
.
2015
;
120
(3)
:
841
859
.
20. 
Cohen
J.
Statistical Power Analysis for the Behavioral Sciences
.
Elsevier Science
;
2013
:
40
.

Supplementary data