Evidence suggests that concussion prolongs reaction time (RT). We have developed a simple, reliable clinical tool for measuring reaction time that may be of value in the assessment of concussion in athletes.
To compare baseline values of clinical RT (RTclin) obtained using the new clinical reaction time apparatus with computerized RT (RTcomp) obtained using a validated computerized neuropsychological test battery.
Data were collected during a National Collegiate Athletic Association Division I collegiate football team's preparticipation physical examination session.
Ninety-four Division I collegiate football players.
The RTclin was measured using a 1.3-m measuring stick embedded in a weighted rubber disk that was released and caught as quickly as possible. The RTcomp was measured using the simple RT component of CogState Sport.
For the 68 athletes whose CogState Sport tests passed the program's integrity check, RTclin and RTcomp were correlated (r = 0.445, P < .001). Overall, mean RTclin was shorter and less variable than mean RTcomp (203 ± 20 milliseconds versus 268 ± 44 milliseconds; P < .001). When RTclin and RTcomp were compared between those athletes with (n = 68) and those without (n = 26) valid CogState Sport test sessions, mean RTclin was similar (202 ± 19 milliseconds versus 207 ± 23 milliseconds; P = .390), but mean RTcomp was different (258 ± 35 milliseconds versus 290 ± 55 milliseconds; P = .009).
The RTclin was positively correlated with RTcomp and yielded more consistent reaction time values during baseline testing. Given that RTclin is easy to measure using simple, inexpensive equipment, further prospective study is warranted to determine its clinical utility in the assessment of concussion in athletes.
In this sample of male collegiate football players, using a simple and inexpensive device, clinical reaction time was a valid measure of baseline reaction time.
Clinical reaction time on the device was both positively correlated with and more consistent than reaction time obtained via computerized testing.
Clinical reaction time assessment may become a valuable new tool in the multifaceted approach to diagnosis and management of concussion in athletes.
The use of neuropsychological testing for the assessment of sport concussion has rapidly grown in the United States since its introduction to the sports medicine community by Barth et al1 in the 1980s. Over the past decade, computerized neuropsychological test batteries have become increasingly popular. Computerized batteries offer many benefits over traditional paper-and- pencil neuropsychological tests. These include standardization of stimulus presentation; shorter administration times; the ability to rapidly and accurately analyze and store data, facilitating comparison with prior test performance; the existence of multiple equivalent forms of the test to minimize practice effects; and the ability to simultaneously assess multiple athletes during a baseline testing session independent of a trained neuropsychologist.2,3 Another advantage of computerized neuropsychological test batteries such as ImPACT (Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT Applications Inc, Pittsburgh, PA),4 CogState Sport (CogState Ltd, Melbourne, Australia),5 and ANAM (Automated Neuropsychological Assessment Metrics; Defense and Veterans Brain Injury Center, Washington, DC)6 is that they can accurately measure reaction time to the millisecond level. This gives computerized testing a considerable advantage over more traditional test methods in detecting the subtle impairments in central processing speed that are typically present after a concussion.
A considerable body of literature7–13 supports the paradigm of prolonged reaction time immediately after sport-related concussion, followed by gradual improvement with an eventual return to baseline. In addition to absolute prolongation of reaction time, response variability has been shown to increase after concussion.10 Prolongation of reaction time after concussion parallels the persistence of postconcussive symptoms.12,13 Furthermore, prolonged reaction time may persist even after full resolution of self-reported symptoms and return to sport based on clinical evaluation.11 These findings suggest that a reaction time measure adds sensitivity to the clinical assessment of concussion compared with self-reported symptoms and a general physical examination alone.
Despite the advantages of computerized neuropsychological testing, it has a limited role in the initial diagnosis of concussion, which must occur on the playing field at the time of injury. Concussion remains a clinical diagnosis that is made by the athletic trainer or team physician based on the athlete's presenting signs and symptoms. Furthermore, the total cost associated with computerized neuropsychological testing, which has been estimated at $669 to $677 per athlete,14 including the need for a computer platform on which to administer the test, restricts its accessibility. This limitation is most marked for high-school aged and younger athletes, who have fewer available sport-related health care resources than collegiate and professional athletes, and is problematic given that 65% of sport-related concussions occur in persons aged 5 to 18 years.15 Additional concern can arise when the postinjury neuropsychological test results are compared with the preseason baseline test. An athlete whose motivation is lacking during baseline testing may not put forth his or her best effort, resulting in poor baseline test performance. After injury, however, the athlete is likely to be highly motivated to perform well on neuropsychological tests. Thus, return-to-play decision making may be based on a flawed comparison.16,17
A simple, inexpensive clinical measure of reaction time that can be used on the sideline or in the training room at the time of injury would be a valuable addition to the sports medicine practitioner's “diagnostic toolbox” for sport-related concussion. Furthermore, such a tool might be valuable for tracking recovery from concussion in young athletes who do not have access to formal computerized neuropsychological testing, and an intrinsically motivating test would reduce baseline inconsistencies in test results due to poor motivation. In order to give clinicians access to such a test of reaction time, we developed the clinical reaction time apparatus, which involves standardizing a simple experiment commonly performed in high school physics classrooms.18 The apparatus is a thin, rigid cylinder to which a weighted disk is attached. The weighted disk ensures verticality and consistency of hand position. The apparatus is released by the examiner and then caught as quickly as possible by the athlete being evaluated. The distance the apparatus falls before being arrested is measured and converted into clinical reaction time (RTclin) using the formula for a free body falling under the influence of gravity. Pilot work in a population of healthy adults has shown RTclin to be a reliable and valid measure of reaction time.19,20
The purpose of our study was to compare, in a population of collegiate athletes, baseline RTclin with computerized reaction time (RTcomp) obtained using an established computerized neuropsychological test battery. Our primary hypothesis was that RTclin and RTcomp would be positively correlated in those athletes who put forth a valid effort on their baseline CogState Sport tests, as measured by passing the program's internal integrity check (see “Methods” section). Failure of the program's integrity check process may be a marker for poor or suspect motivation during the test session. Because reaction time performance is affected by motivation, a secondary hypothesis was that different relationships would be observed between RTclin and RTcomp in those who passed versus those who failed the CogState Sport integrity check.
We recruited 94 National Collegiate Athletic Association Division I collegiate football players at a single university during a preparticipation physical examination session. Before testing, all athletes provided informed written consent, and this research was approved by the institutional review board at our institution. All members of the university's football squad who were at least 18 years of age on the day of the preparticipation examination were eligible for inclusion. Exclusion criteria included self-report of an unresolved concussion or any active injury affecting the right hand or arm that might affect the athlete's ability to participate in RTclin testing. We recorded each athlete's age and dominant hand at the time of testing.
All athletes completed a single baseline CogState Sport computerized neuropsychological test (version 5.6.4) session. Testing was conducted in groups of 8 athletes at a time on separate personal computers in a computer laboratory and was supervised by physicians familiar with the CogState Sport program. Each athlete wore headphones during the test battery to minimize outside distractions. The CogState Sport computerized cognitive assessment is based on the presentation of playing cards on a computer monitor and includes detection (simple reaction time), identification (choice reaction time), 1-card learning (a continuous visual recognition learning task), and 1-back tests designed to measure psychomotor function, visual attention, continuous visual recognition learning, visual recognition memory, working memory, and attention.21 The simple reaction time task involves the presentation of a single playing card face down in the middle of the computer screen. The athlete must press the k key as quickly as possible whenever the card turns face up. At least 35 trials are completed, with additional trials added if the athlete presses the k key before the card turns face up. When such an anticipatory button press occurs, a brief warning is sounded before the next trial to indicate the error. The program also includes a self-reported survey of clinical symptoms and concussion history.
The CogState Sport program undergoes several internal integrity checks (D. Darby, MBBS, PhD, FRACP, CogState Ltd, written communication, September 2009):
(1) Simple reaction time accuracy (ie, nonanticipatory response rate) is 90% or greater.
(2) Choice reaction time accuracy (ie, correctly indicating whether the card presented is red) is 80% or greater.
(3) One-card learning accuracy (ie, correctly indicating whether the card presented has previously appeared during the task) is 53% or greater.
(4) Mean simple reaction time is faster than mean choice reaction time.
(5) Mean simple reaction time is faster than mean 1-back reaction time (ie, the reaction time to decide whether the card presented is identical to the immediately previous card).
The software then reports whether the athlete's performance passes or fails the integrity check process. The raw simple reaction time data for all nonanticipatory trials were extracted from each athlete's CogState Sport test session for analysis. In addition, we noted each athlete's concussion history, as self-reported in CogState Sport.
Each athlete participated in simple RTclin testing using the clinical reaction time apparatus, administered by 1 of 3 examiners. These included 2 of the study investigators (J.T.E. and J.K.R.) and 1 member of the university's athletic training staff. The apparatus is a measuring stick, 1.3 m long, coated in high-friction tape and marked in 0.5-cm increments, that is embedded in a weighted rubber disk. The athlete sits with the right forearm resting comfortably on a horizontal desk surface, such that the proximal edge of the hypothenar eminence is positioned at the edge of the desk surface. The examiner suspends the apparatus vertically, with the weighted disk positioned inside the athlete's open hand, such that the superior surface of the weighted disk is aligned with the plane of the athlete's first 2 digits and no part of the athlete's hand is in contact with the weighted disk. The examiner releases the apparatus at predetermined, randomly assigned time intervals of between 2 and 5 seconds to prevent the athlete from anticipating the time of release. The athlete then catches the apparatus as quickly as possible after it begins to fall (Figure 1). In the event of an anticipatory grasp before the device is released, the examiner restarts the random-delay interval count before releasing the device. In the rare instances when an athlete drops the device, a “drop” trial is recorded and that trial is not used in the calculation of mean RTclin. All athletes were tested using the right hand with a protocol of 2 practice trials followed by 8 data-acquisition trials. The fall distance was measured from the superior surface of the weighted disk to the most superior aspect of the athlete's hand and was converted into a reaction time (in milliseconds) using the formula for a body falling under the influence of gravity (d = ½gt2), where d is distance, g is acceleration due to gravity, and t is time.
We calculated means and SDs for RTclin and RTcomp for each athlete. Mean RTclin was compared with mean RTcomp in all athletes using paired t tests. We chose to define RTclin and RTcomp variability for each participant as the SD of his individual clinical or computerized reaction time trials, respectively. We compared the variability in RTclin with the variability in RTcomp in all 94 athletes using paired t tests. Additional analysis was performed to compare mean RTclin with mean RTcomp and the variability in RTclin with the variability in RTcomp using paired t tests in 2 subgroups: athletes whose performance on the CogState Sport test battery passed the program's internal integrity checks and those whose performance did not pass. Mean RTclin and mean RTcomp were correlated using a Pearson correlation coefficient, r, in both subgroups.
Additional subgroup analyses were conducted using independent-samples t tests to compare both mean RTclin and mean RTcomp in those athletes who passed versus those who failed the CogState Sport program's integrity checks, those athletes with a self-reported history of concussion versus those without, and left-handed versus right-handed athletes. To identify a possible learning effect for RTclin and RTcomp, we performed linear regressions of RTclin and RTcomp by trial number and compared the mean slopes of the 94 least-squares regression lines for each measure with zero using 1-sample t tests. The null hypothesis of zero slope would indicate that mean RTclin and mean RTcomp did not change over trials and, therefore, would argue against a learning effect. Statistical analyses were conducted using either SPSS (version 14.0; SPSS Inc, Chicago, IL) or SAS (version 9.1; SAS Institute Inc, Cary, NC) for Windows.
A total of 94 male collegiate football players, with a mean age of 19.9 ± 1.5 years (range, 18 to 23 years), participated in this study. Of those, 68 passed the CogState Sport program's internal integrity check. In this subset of athletes, a correlation was seen between mean RTclin and mean RTcomp (r = 0.445, P < .001). This relationship was not found in the 26 athletes who did not pass the CogState Sport program's internal integrity check (r = −0.080, P = .698). In the overall sample of 94 athletes, mean RTclin was shorter than mean RTcomp (203 ± 20 milliseconds versus 268 ± 44 milliseconds, respectively; P < .001). Furthermore, RTclin had less variability, than RTcomp (26 ± 13 milliseconds versus 99 ± 58 milliseconds, respectively; P < .001). When mean RTclin and mean RTcomp were compared between athletes with and without valid CogState Sport sessions, group differences were found in mean RTcomp but not in mean RTclin (Table 1). Similarly, there was no difference in RTclin variability between athletes with and without valid CogState Sport test sessions, whereas variability in RTcomp was greater in athletes who did not pass the CogState Sport integrity checks (Table 2).
Of the 94 athletes studied, 37 reported a history of concussion. One athlete reported a history of 8 concussions, 2 reported 4 concussions, 2 reported 3 concussions, 9 reported 2 concussions, and 23 athletes reported 1 concussion. Of the 63 concussions reported, 23 involved loss of consciousness and 12 involved amnesia. No information was available regarding how much time had passed since these injuries; however, all athletes had fully recovered and had no concussion symptoms at the time of testing. When comparing the 37 athletes with a history of concussion to the 57 without such a history, no differences were detected in mean RTclin (201 ± 15 milliseconds versus 205 ± 23 milliseconds; P = .225) or mean RTcomp (269 ± 48 milliseconds versus 265 ± 41 milliseconds; P = .735). The 11 left-handed athletes had a shorter mean RTclin than the 83 right-handed athletes (188 ± 12 milliseconds versus 205 ± 20 milliseconds; P = .001), but no difference was detected in mean RTcomp between left-handed and right-handed athletes (265 ± 40 milliseconds versus 267 ± 44 milliseconds; P = .907).
With respect to a possible learning effect, the mean RTclin value by trial is illustrated in Figure 2. The mean slope of the least-squares linear regression lines for RTclin by trial number was −2.4 ± 5.4 milliseconds per trial. The mean slope of the least-squares linear regression lines for RTcomp was −0.92 ± 2.5 milliseconds per trial. Both slopes differed from zero (each P < .001), indicating that RTclin and RTcomp both changed across trials. Based on this regression analysis, the athletes involved in this study had an average improvement of 2.4 milliseconds per trial for RTclin and 0.92 milliseconds per trial for RTcomp.
The RTclin and RTcomp were positively correlated in those athletes who put forth a valid effort on their baseline CogState Sport test, as measured by passing the program's internal integrity check, but not in those athletes whose performance did not pass. Although this is not the only possible reason for the difference, failure to pass the CogState Sport integrity check may reflect poor effort or lack of motivation on the part of the athlete taking the test. We believe that this is the most likely explanation for integrity-check failure in our sample of athletes. Thus, RTclin appears to be a valid measure of reaction time in this population if RTcomp data that have passed the CogState Sport integrity check are the criterion standard for comparison. Similar results were obtained19,20 when RTclin was measured in a general population of healthy adults and compared with a different computerized test of reaction time that conformed to widely validated and accepted techniques for measuring reaction time and served as a criterion standard for comparison. The validity of RTclin is also supported by the finding that RTclin in left-handed athletes was faster than in their right-handed counterparts, a result consistent with previous reaction time research involving athletes.22 The presence of similar learning effects for RTclin and RTcomp further supports the validity of RTclin.
In this sample of 94 male football players, baseline performance on the clinical reaction time task appeared to be more consistent (ie, more uniform) than performance on the computerized reaction time task. The data support this finding in 2 ways. First, RTclin measurements showed less variability than RTcomp measurements, as determined by group SDs for the measurements. Second, RTcomp was increased in those athletes whose CogState Sport performance did not pass the program's internal integrity check compared with those who had valid CogState Sport test sessions. In contrast, athletes with and athletes without valid CogState Sport test performance demonstrated nearly identical RTclin measurements. One possible explanation for the more consistent performances on the clinical reaction time task is that this task is more intrinsically motivating than the computerized task and, therefore, athletes were more uniformly motivated during RTclin testing than during RTcomp testing. This possibility is consistent with the subjective impression of the study investigators, who noted that the athletes were clearly aware of the distance the clinical reaction time apparatus fell on each trial and became competitive with their previous performances during testing. These observations contrasted with those of the athletes' demeanors while taking the computerized test battery, in that they quietly performed the tasks without showing any outward signs of motivation to perform at their best. Moreover, in a nonathletic population (N = 31), RTclin was more motivating than a computerized measure of RT, although that computerized measure was not associated with CogState Sport.23
This clinical test of reaction time has potential clinical utility and may prove to be a valuable tool for athletic trainers and team physicians. It could be used in conjunction with a standard neurologic examination; standardized symptom checklists, such as the Standardized Assessment of Concussion24,25 or the Sport Concussion Assessment Tool26; or an assessment of postural stability, such as the Balance Error Scoring System,27,28 to improve the sports medicine practitioner's evaluation of athletes suspected of having sustained a concussion. The RTclin is not intended to serve as a replacement for more rigorous computerized neuropsychological test batteries for those athletes who have access to such evaluation techniques. However, when combined with sound clinical judgment, RTclin may be of value in monitoring recovery from a concussion in younger athletes who play for teams that do not have access to the more expensive computerized test batteries. Further study is warranted to investigate these possibilities.
This pilot study has several limitations. It was designed only to compare reaction times measured using the clinical reaction time apparatus with those measured via CogState Sport. Also, this study was not designed to assess the athletes' motivation. Further research should be conducted in athletic populations before firm conclusions are drawn regarding the role of motivation on baseline clinical reaction time assessment. In addition, we investigated only collegiate male football players. The study should be replicated in a more diverse group of athletes before generalizations to other athletic populations are made. Another limitation is that 2 of the 3 administrators of the clinical reaction time test were study authors; therefore, they were aware of the study hypothesis and were not blinded. It is unclear how this may have influenced the athletes' performance on such a rapid task, particularly given that the evaluators were not aware of the athletes' CogState Sport test results at the time of RTclin determination, but bias is possible. A further limitation is that all athletes were tested only once, including those athletes whose CogState Sport performance did not pass the program's internal integrity checks. Straume-Naesheim et al29 reported a slight learning effect for CogState Sport and advocated a double-baseline test protocol to eliminate this effect. Our data do, in fact, indicate a learning effect for the simple reaction time component of CogState Sport. Also, although our pilot reliability and validity RTclin testing did not identify a significant learning effect,19,20 a learning effect did appear to be present for RTclin. It would be valuable to determine the relationship between RTclin and RTcomp after testing both with a double-baseline protocol, including obtaining RTcomp values during subsequent valid CogState Sport test sessions for those athletes whose initial performance failed the integrity check process.
In summary, RTclin appears be a valid means of measuring baseline reaction time in a population of male collegiate football players and can be tested in an entire team as part of a routine preparticipation physical examination session. The athletes tested appeared to be intrinsically motivated by the clinical reaction time task, and this may have led to the observed improved consistency in RTclin measures as compared with RTcomp measures. The more consistent baseline values would be useful when making comparisons with postinjury values after a concussion. Further study is needed to determine whether RTclin is sensitive to the known effects of concussion on RT7–13 and whether this test method is feasible for concussed athletes on the sideline or in the training room immediately after an injury.
We thank Dr David Darby, chief medical officer for CogState Ltd, for the advice and assistance he provided relating to the CogState Sport program, and Steve Nordwall, MA, ATC, and his athletic training staff for their support in organizing and conducting this project. We also thank Dr Sri Krishna Chandran for his assistance with data imputation and organization. We acknowledge the support of our sponsors: The Foundation for Physical Medicine and Rehabilitation (Chicago, IL), which gave the 2007 New Investigator Award to J.T.E., and the University of Michigan Bone & Joint Injury Prevention & Rehabilitation Center (Ann Arbor, MI) for a pilot grant awarded to J.T.E. and J.K.R.