Influential Factors and Preliminary Reference Data for a Clinically Feasible, Functional Reaction Time Assessment: The Standardized Assessment of Reaction Time

We conducted a prospective, cross-sectional cohort study to assess preinjury factors and StART performance among healthy participants during preseason baseline screening. This study is reported in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology guidelines.²⁸ 

We enrolled a total of 89 (96.7%) individuals out of 92 who were enrolled in a larger study during the 2021–2022 academic year (Table 1). All participants were recruited at baseline testing from a single university's student-athlete body during fall 2021. Individuals were eligible if they completed the StART assessment and were undergoing baseline assessments as part of their college athletic preseason protocol. We excluded individuals who self-reported a history of learning disability, neurologic disorder, or psychiatric disorder, as these conditions and associated medications may affect reaction time. All participants provided written informed consent, and this protocol was approved by the Boston Children's Hospital Institutional Review Board.

Student-athletes completed health history forms and StART during the same day as part of their preseason baseline assessment. All baseline assessments were performed serially in designated areas, either individually or with 2 athletes at most in a testing area. The health history forms were given to participants to self-report demographic factors (eg, age, sex, race, and limb dominance), medical history (eg, concussion history, caffeine intake before testing, and hours of sleep the previous night), and sport history (eg, sport type, contact level, position, and years of participation). Primary sport was reported by participants and then categorized into sport contact level (contact, limited, or noncontact) based on an earlier study.²⁹  At least 1 member of the research team was always present to answer any questions about the form.

All StART trials were video and audio recorded (240 Hz with 720p resolution) on an iPad Pro (model A1876; Apple Inc) using the OnForm application³⁰  attached to a tripod 3.05 m away from participants to ensure that those of different heights could completely fit in the frame (Figure 1). The video frame rate was equivalent to or faster than that of criterion standard motion-capture cameras and allowed for high-fidelity movement detection.^13,31  A light-emitting diode penlight tip was placed in the camera frame and provided a time-synchronized visual stimulus for initiating reaction time (Figure 1A–C). Participants were instructed to “get set” for all StART trials, and then the penlight was illuminated randomly between 2 and 10 seconds after this preparatory cue (Figure 1D–F).

The StART consisted of 3 movements (standing, single-legged balance, and cutting) across 2 conditions (single and dual task), and 3 trials were completed per condition (18 trials total). The dual-task condition was serial subtraction by 6's and 7's (randomly selected) due to established cognitive loading and frequent use in dual-task literature.^13,25,31  Participants stood with their feet together and hands on hips as the starting position for standing trials; stood on their nondominant leg (limb they would not use to kick a ball) with hands on hips as they balanced for single-legged balance trials; and stood in a semisquatted, athletic stance with hands on hips for cutting trials.

For the standing and single-legged balance trials, participants moved their hands off their hips until their upper extremities were outstretched to a T position (ie, parallel to the ground) immediately after the penlight was activated. Cutting trials required them to rapidly accelerate from the starting position to the left- or right-side targets positioned 3.05 m away at 45° from the starting position by performing an athletic cutting motion (Figure 1J). All StART outcomes were calculated and scored as the time between light activation and movement. Specifically, the first frame of any upper extremity movement (eg, hands off hips, elbows bending, or fingers raising) was considered the reaction time for the standing and single-legged trials, and the first frame of any body movement (eg, foot pivoting, hands coming off hips, or torso or head deviating laterally) was considered the reaction time for the cutting trials. Reaction time was recorded using the OnForm time-between-events function to accurately convert frames between light and movement into seconds.

We conducted a sensitivity analysis of the pilot data to determine the minimal number of trials for measurement stability among StART outcomes, with all movement-condition combinations needing 2 or 3 trials each. The intra- and interrater reliability of each StART subtest was assessed in 8 healthy college students before and separate from the cohort described earlier by using intraclass correlation coefficients (ICCs).^32,33  The intrarater reliability across participants using ICC (2,k) models was as follow: single- (0.96) and dual- (0.99) task standing, single- (0.95) and dual- (0.99) task single-legged balance, and single- (0.99) and dual- (0.99) task cutting. The interrater reliability across subtests using ICC (2,k) models was as follows: single- (0.88) and dual- (0.98) task standing, single- (0.92) and dual- (0.88) task single-legged balance, and single- (0.95) and dual- (0.95) task cutting. Thus, acceptable reliability^32,33  was present among all StART outcomes and has since been expanded upon and independently reported in another study.³⁴ 

The entire StART assessment took approximately 5 minutes on average for participants to complete and approximately 10 minutes for the research team to analyze each person. The StART values were averaged to calculate 9 outcome scores: single- and dual-task standing, single-legged balance, and cutting reaction times (separately); single-task reaction time composite (all single-task trials); dual-task reaction time composite (all dual-task trials); and StART composite (all 18 trials).

To describe the cohort, we used descriptive statistics (frequencies and proportions, means and SDs) for health history characteristics. We computed a 3 (standing, single-legged balance, cutting)-by-2 (single and dual task) repeated-measures analysis of variance with post hoc t tests, deriving mean differences, 95% CIs, and Cohen d effect sizes to assess whether StART cognitive and movement differences were present. Effect sizes were interpreted as small (≤0.20), medium (0.21–0.79), or large (≥0.80).³⁵  Univariable general linear models were calculated to examine if preexisting factors (age [continuous], sex [dichotomous], hand and lower extremity dominance [dichotomous], concussion history [dichotomous], caffeine intake before testing [dichotomous]), hours of sleep the previous night (continuous), and sport contact level (3-level ordinal) were associated with StART. The StART preliminary reference data were binned and reported in the following percentile categories: <2 (very poor), 2–9 (poor), 10–24 (below average), 25–75 (broadly normal), 76–90 (above average), and >90 (superior).^36,37  All applicable general linear model assumptions were assessed with residual normality violations present among several StART outcomes, as is common for reaction-time measures.³⁸  Therefore, logarithmic transformations were performed on the affected StART outcomes, normality was successfully met before statistical analysis, and the outcomes were back transformed for reporting.^39,40  All statistical analyses were conducted using R (version 4.0.4; R Project for Statistical Programming), and the α level was set a priori at .05.

A total of 89 collegiate student-athletes completed the StART assessment (Table 1). A cognitive-by-movement task interaction (F_2,87 = 32.4, P < .001), main effect for cognitive task (F_2,87 = 800.4, P < .001), and main effect for movement task (F_2,87 = 27.3, P < .001) were observed for StART outcomes. Each cognitive-movement-condition was different from the others (P ≤ .03; d range = 0.44–4.87; Figure 2), except for single-task standing compared with single-task single-legged balance (mean difference = 2 milliseconds; 95% CI = −3, 8 milliseconds; P = .36; d = 0.19). The movement task effect indicated that the standing task produced a slower reaction time than single-legged balance (mean difference = 21 milliseconds; 95% CI = 15, 28 milliseconds; P < .001; d = 1.34), and the cutting task was slower than single-legged balance (mean difference = 28 milliseconds; 95% CI = 21, 36 milliseconds; P < .001; d = 1.49), but the standing task was not different from the cutting task (mean difference = −7 milliseconds; 95% CI = −16, 2 milliseconds; P = .14; d = 0.30). All single-task condition trials were faster than all dual-task condition trials (mean difference = 119 milliseconds; 95% CI = 109, 129 milliseconds; P < .001).

A summary of the univariable linear regressions between StART outcomes and preexisting factors is given in Table 2. Only 3 student-athletes were left-hand dominant and only 6 were left-leg dominant. Therefore, we did not analyze the association between limb dominance and StART performance. Only single-task (P = .02) and dual-task (P = .03) standing were associated with sex: female athletes had a slower reaction time by 15 milliseconds (95% CI = 2, 28 milliseconds; d = 0.51) and 28 milliseconds (95% CI = 2, 55 milliseconds; d = 0.47), respectively. No other StART outcome differences were observed for sex (P ≥ .056). Age was associated with the single-task cutting (P < .001) and single-task StART composite (P = .001). Every 1-year age increase was associated with a slower reaction time by 18 milliseconds (95% CI = 8, 27 milliseconds) for single-task cutting and 11 milliseconds (95% CI = 5, 17 milliseconds) for the StART single-task composite score. No other StART outcome differences were noted for age (P ≥ .057). Height, mass, concussion history, caffeine intake before testing, hours of sleep the previous night, and sport contact level were not associated with any StART outcomes (Table 2).

Preliminary reference value cutoffs for each StART outcome and the composite score are supplied in Table 3. The StART composite score was 287 ± 31 milliseconds, with a StART single-task composite of 227 ± 26 milliseconds and a StART dual-task composite of 347 ± 46 milliseconds. The movement subtests for the single-task and dual-task conditions were 219 ± 30 milliseconds and 366 ± 62 milliseconds for standing, 216 ± 28 milliseconds and 325 ± 51 milliseconds for single-legged balance, and 248 ± 40 milliseconds and 350 ± 58 milliseconds for cutting, respectively (Figure 2).

Our findings indicated that StART elicited a clear dual-task effect in almost every cognitive-motor combination. Sex and age were associated with StART performance. We present preliminary reference values (Table 3) that may be helpful when interpreting StART performance. However, these preliminary reference values are currently only intended for research and exploratory purposes until their validity can be confirmed and StART postconcussion deficits are understood.

From a theoretical viewpoint, each StART cognitive-motor combination would be expected to result in slightly altered reaction-time performance. Standing trials served as a relatively simple reaction-time assessment. Single-legged balance trials induced heightened sensorimotor coordination, as individuals with concussion have trouble integrating sensory information.^21,41,42  Cutting trials emulated sport-like functional movement that has been examined in laboratory settings by researchers using motion-capture equipment.^13,31  In our study, most StART components yielded unique reaction-time metrics, with moderate-to-large differences between tasks (Figure 2). Only the single-task standing and single-task single-legged balance comparisons were not different, and this result may indicate some StART redundancy. Note that we studied a sample of healthy collegiate athletes, and thus, results from the single-task standing and single-task single-legged balance assessments may differ after concussion due to acknowledged balance impairments acutely after injury.^43,44  Future investigations involving athletes experiencing concussions would help to better characterize the unique information offered by the StART subtests.

Reaction time can be affected by numerous intrinsic and extrinsic factors.^11,27  Our preliminary findings indicated that StART outcomes were overall resistant to many demographic factors and health history, except for age and sex. Age and sex are well-established factors associated with reaction times measured among collegiate athletes, with previous studies^11,27  showing that females and marginally older individuals typically displayed slower reaction times. Our findings partially align with this earlier work. We demonstrated that female athletes had slower single-task and dual-task reaction times than male athletes during the standing condition, although no other sex differences were evident. Older age was associated with a slower reaction time in the single-task cutting condition. Overall, the StART composite scores were minimally affected by age and sex (Table 2) and may have future postconcussion utility, but further exploration is warranted to understand if and which specific StART metrics elicit deficits after concussion. Future authors should evaluate a larger cohort to develop robust normative data and comprehensively understand the effects of age and sex on subtests. Based on our preliminary data, adjusting StART composite scores for age and sex effects may not be necessary.

The preliminary reference data we obtained provide valuable and efficient insights into estimated preinjury performance, especially when normative data are frequently used among most other concussion assessments.^6,10,37,45  Our relatively small cohort was enrolled across a single season and university, and therefore, providing robust normative data or exploring by age and sex was not possible. However, we supplied preliminary reference data to report initial estimates given that the StART composite outcomes were minimally affected by preexisting factors and approximated Gaussian distributions (Table 3). These values are intended for research and exploratory use only until validated, postconcussion impairments for this measure are determined, and we understand whether subtest or composite scores are the optimal metric for assessing deficits. Our results highlight the added cognitive processing time and added variance due to individual capabilities that occur when conducting reaction-time assessments under a dual-task paradigm. Researchers have observed similar increased reaction times and variances under dual-task conditions,^13,14  and thus, the signal-to-noise ratio may be important to consider.

Our study had several limitations. We examined a healthy collegiate student-athlete cohort from a single university across a variety of sports, so these findings may not be generalizable to other populations. The StART assessment may have practice or learning effects. Given that it was administered in a standardized order, practice effects might partially explain why single-legged balance displayed a faster reaction time than standing conditions. Lastly, numerous demographic factors, such as limb dominance and the specific sport played, were not assessed due to the limited sample size, and others, such as concussion history, ethnicity, and sport contact level, had small proportions or variances. Future investigators should examine StART among diverse cohorts across numerous settings to validate the accuracy of the estimates reported here.

We demonstrated that StART performance was unique across the cognitive-motor task combinations used and therefore may offer efficient insights into postconcussion examination. Age and sex were each associated with some StART outcomes. The StART combined composite was unaffected by age, sex, or any other demographic or health history factors we explored and, thus, may be a more robust dual-task, functional movement reaction-time measure. Yet before StART is implemented clinically, future research is necessary to determine if postconcussion deficits can be detected using StART or if StART provides additional diagnostic value beyond more commonly used measures.

The research reported in this work was supported by the National Institute of Neurological Disorders and Stroke of the National Institutes of Health Award No. R01NS100952, Dr Howell). Separate from this study, Dr Howell has received research support from the Eunice Kennedy Shriver National Institute of Child Health & Human Development (Award No. R03HD094560), the National Institute of Neurological Disorders and Stroke (Award Nos. R01NS100952, R03HD094560, and R43NS108823), MINDSOURCE Brain Injury Network, and the Tai Foundation. Dr Meehan receives royalties from ABC-Clio publishing for the sale of his book, Kids, Sports, and Concussion: A Guide for Coaches and Parents and Concussions; Springer International for the book Head and Neck Injuries in Young Athletes; and Wolters Kluwer for working as an author for UpToDate. His research is funded, in part, by philanthropic support from the National Hockey League Alumni Association through the Corey C. Griffin Pro-Am Tournament and a grant from the National Football League.