Detection of subtle changes in brain sensorimotor processes may enable clinicians to identify athletes who would derive the greatest benefit from interventions designed to reduce the risk for future injury and progressive neurologic or musculoskeletal dysfunction.
To develop a generalizable statistical model for identifying athletes who possess subtle alterations in sensorimotor processes that may be due to previous concussion.
Residential Olympic Training Center sports medicine clinic.
A primary cohort of 35 elite athletes and a secondary cohort of 40 elite athletes who performed identical tests the preceding year.
Two upper extremity tests of visual-motor reaction time and 2 tests of whole-body reactive agility were administered. The whole-body tests required lateral or diagonal responses to virtual-reality targets, which provided measures of reaction time, speed, acceleration, and deceleration.
Sport-related concussion history, which was reported by 54% (n = 19) of the athletes in the primary cohort and 45% (n = 18) of the athletes in the secondary cohort.
Univariable analyses identified 12 strong predictors of sport-related concussion history, which we combined to create a composite metric with maximum predictive value. Composite lateral asymmetry for whole-body reactive movements and persisting effects of previous musculoskeletal injury yielded a logistic regression model with exceptionally good discrimination (area under the curve = 0.845) and calibration (predicted-observed probabilities within 7 subgroups: r = 0.959, P = .001). Application of the derived model to compatible data acquired from another cohort of elite athletes demonstrated very good discrimination (area under the curve = 0.772) and calibration (within 8 subgroups: r = 0.849, P = .008).
Asymmetry in whole-body reactive movement capabilities may be a manifestation of a subtle abnormality in the functional connectivity of brain networks that might be relevant to previously reported associations between sport-related concussion history and musculoskeletal injury occurrence.
The strong association between whole-body reactive agility asymmetry and self-reported sport-related concussion history (SRC Hx) suggested that a subtle cognitive-motor impairment may persist long after complete resolution of overt signs and symptoms, thereby elevating the risk for future injury.
Persisting effects of previous musculoskeletal injury may explain whole-body reactive agility asymmetry in the absence of SRC Hx, or SRC Hx and previous musculoskeletal injury may have an interactive effect.
The model demonstrated very good calibration and strong discriminatory power for both the derivation and validation cohorts, which support its potential use for identifying individual athletes who are most likely to derive the greatest benefit from a targeted training intervention for improving whole-body reactive movement capabilities.
Emerging evidence1–4 strongly suggests that sport-related concussion (SRC) can have long-term adverse effects on neurocognitive function. Current clinical assessment methods do not appear to be sufficiently sensitive for detecting subtle changes in the functional connectivity of brain networks that have been documented using advanced neuroimaging and neurophysiological tests.5 A self-reported history of SRC (SRC Hx) has been associated with a fivefold relative risk for subsequent SRC,6 but the exact neurophysiological mechanisms have not been elucidated. An increasingly recognized consequence of SRC is a 2 to 3 times greater risk for musculoskeletal (MSK) injury,7 which is independent of MSK injury history (MSK Hx).8 Slowed information processing may represent an important long-term effect of SRC that interferes with the efficient performance of visually guided motor actions.8–12 Given that abnormalities have been found to persist beyond resolution of overt signs and symptoms postconcussion,13 the availability of a clinical test that provides evidence of impaired visual-motor performance capabilities could be valuable.
The term perception-action coupling refers to interdependencies between perceiving and acting within an environment that affords opportunities and imposes constraints.10 Neuromechanical responsiveness specifically describes the generation of forces to meet the demands of rapidly changing environmental challenges, which includes maintaining dynamic joint stability during exposure to unexpected external forces.14 Advanced neuroimaging methods have identified the temporoparietal cortex of the right hemisphere as a key area for interpreting visual inputs from both visual hemifields,15 as well as kinesthetic inputs from both extremities.16 Disruption of neural signaling from the right temporoparietal cortex to the prefrontal cortex could result in diminished, inaccurate, or absent responses to salient visual stimuli,17 thereby leading to suboptimal reactive capabilities.
Both upper extremity visual-motor reaction time (VMRT) and whole-body reactive agility (WBRA) test metrics have been shown to discriminate between elite athletes who self-reported versus those who denied SRC Hx.18 Wilkerson et al18 found that a 3-factor prediction model involving metrics relating to the peripheral-to-central VMRT ratio, left-to-right VMRT difference, and WBRA lateral-movement speed asymmetry provided 100% positive and 90% negative predictive values. Slow responsiveness to peripheral visual stimuli has been associated with disruption of white matter tracts in the corpus callosum, which may represent an indirect measure of the time required to transfer information between brain hemispheres.19 Right hemisphere specialization for processing visual-spatial information from both visual hemifields is responsible for asymmetric responsiveness to visual stimuli that normally favors the left visual hemifield.20 Disruption of attention network connectivity can produce neglect of stimuli in the left visual hemifield,21,22 which provides a plausible explanation for an observed reversal of responsiveness asymmetry that favored the right visual hemifield among athletes with SRC Hx.18
Researchers14,18,23 have demonstrated strong associations of dichotomized VMRT and WBRA metrics with both self-reported SRC Hx and subsequent MSK injury. Dichotomous categorization of continuous variables is a common predictive modeling procedure that provides an easily interpretable estimation of relative risk for a positive diagnostic or prognostic outcome.24 Model calibration is arguably the most important property of a predictive model and refers to estimation of an individual's absolute risk for a specified outcome.25 Unfortunately, determination of an optimal cut point for maximum discriminatory power typically has poor generalizability beyond the cohort used to derive the model,26 and calibration is rarely assessed and reported.24,25,27 Feature engineering describes a machine learning classification procedure that creates new predictive variables from an existing set to develop a model that is both simpler and more generalizable.28 In comparison with machine learning, regression modeling reflects human domain knowledge in model specification, and no evidence indicates that machine learning provides superior models.27 Therefore, the purpose of our study was to develop a simple, well-calibrated, and generalizable logistic regression model for clinical identification of elite athletes with neuromechanical performance deficiencies that could be due to persisting effects from a previous SRC.
A cohort of 35 healthy elite athletes who were temporarily residing at an Olympic training center volunteered to respond to survey questions and participate in tests of neuromechanical responsiveness (Table 1). All participants provided informed consent, and the Institutional Review Board of the University of Tennessee at Chattanooga approved the study.
The surveys used were the Sport Fitness Index,29 which contains questions about SRC Hx and MSK Hx, and the Depression Anxiety Stress Scale.30 The athletes performed 60-second VMRT tests during which they manually contacted randomly illuminated buttons on a height-adjustable board (model D2 System; Dynavision International, West Chester, OH) and WBRA tests requiring lateral or diagonal movements in response to the appearance of visual targets on a virtual-reality display (model TRAZER Sports Stimulator; Traq Global Ltd, Westlake, OH).
After a practice trial, each athlete performed 2 different versions of a 60-second VMRT dual-task test, both of which involved the Eriksen flanker test on a centrally located tachistoscope. One test required the athlete to orally identify the direction of the center arrow among 20 displays (>>>>>, <<<<<, >><>>, or <<><<), whereas another test used the center-arrow direction as a cue for the correct direction of 48 manual responses to pairs of illuminated buttons in corresponding locations on opposite sides of the board. For 1 WBRA test, athletes performed 20 lateral movements with 1.8 m of whole-body displacement to deactivate randomly displayed virtual-reality targets: 10 targets on the right and 10 targets on the left side of a 48- × 86-cm monitor. For another WBRA test, athletes performed 16 diagonal movements in combinations of right-left and forward-backward directions, which required 2.5 m of displacement to deactivate the targets. Time elapsed between target appearance and 0.2 m of body core displacement defined reaction time, which was averaged for the 10 trials in each direction. We also derived averaged values for speed, acceleration, and deceleration from the virtual-reality motion-analysis system. Asymmetry represented the absolute difference between performance values for opposite movement directions divided by the better of the 2 performance values (reaction time asymmetry, speed asymmetry, acceleration asymmetry, and deceleration asymmetry).
Statistical Analysis and Model Development
We quantified the association of each continuous variable with SRC Hx using receiver operating characteristic (ROC) analysis. Variables that demonstrated a clearly definable cut point on the ROC curve were converted into binary variables that were used to perform cross-tabulation analysis with calculation of the odds ratio (OR). Each binary predictor that demonstrated an OR ≥3 was entered as a continuous variable to assess its possible contribution to a multivariable logistic regression model. A backward-entry stepwise procedure was used to identify the simplest model with good discrimination and calibration. Entry of various combinations of the continuous variables continued until no improvement in model performance was apparent. Predicted probabilities assigned to the individual athletes were used to create 7 subgroups of equal size; each subgroup represented athletes with comparable predicted probability for SRC Hx. Predicted probabilities were plotted against the observed prevalence of SRC Hx within the subgroups to provide a visual representation of internal model calibration,24,31 and a bivariate correlation coefficient was calculated to quantify the relationship. To assess the generalizability of a derived model that included variables measured in an earlier study,20 the logistic regression intercept and β coefficients were used to calculate predicted probabilities for SRC Hx within a validation cohort of 40 elite athletes (Table 2). Model discrimination was considered very good if the predicted probability for SRC Hx yielded an area under the curve (AUC) >0.75.25 Assessment of external model calibration for the validation cohort of 40 athletes followed the same procedure used for internal model calibration, which involved the creation of 8 subgroups of equal size. All analyses were performed with SPSS (version 25; IBM Corp, Armonk, NY).
Sport-related concussion occurrence at 4.6 ± 5.3 years before testing (range = 3 months to 18 years) was reported by 54% (n = 19) of the athletes in the derivation cohort. The number of previous SRCs ranged from 1 to 3, with 32% (n = 6/19) reporting a single SRC and 68% (n = 13/19) reporting 2 or 3 SRCs. Concussion occurrence within the 12 months before the study was reported by 37% (n = 7/19). The results of the univariable analyses, with variables ordered by magnitude of AUC, are presented in Table 3. Asymmetry metrics for the WBRA tests represented 10 of the 12 strongest associations, with the strongest 2 being from the lateral test. Exploratory analyses identified a composite value derived from the average of reaction time asymmetry, speed asymmetry, acceleration asymmetry, and deceleration asymmetry for the WBRA lateral test as providing the best single-factor prediction model (AUC = 0.760, accuracy = 65.7%, Nagelkerke R2 = 0.310, Hosmer and Lemeshow goodness of fit P = .73). Using further exploratory analyses, we identified persisting effects of MSK Hx as an important covariate. We used a 6-level response option (persistent, frequent, occasional, infrequent, rare, or never) to the first question of the 10-item Sport Fitness Index to rate the extent to which moderate to severe joint or muscle injuries limited participation in sport-related activities over the past several years. Although the total score derived from the responses to all 10 items was a strong predictor of SRC Hx, the response to the first item improved logistic regression model performance to the greatest extent. The addition of MSK Hx as a covariate with composite lateral asymmetry provided a substantially improved model (AUC = 0.845, accuracy = 77.1%, Nagelkerke R2 = 0.485, Hosmer and Lemeshow goodness of fit P = .99).
Given that the WBRA lateral test data and Sport Fitness Index responses were available from a previous study,18 the 2-factor logistic regression model intercept (0.07) and β coefficients for composite lateral asymmetry (20.33) and MSK Hx (–0.98) were used to calculate predicted probabilities for 40 athletes who were not among those who composed the model derivation cohort. Sport-related concussion occurrence at 2.2 ± 2.4 years before testing (range, 1 month to 7 years) was reported by 45% (n = 18) of the athletes in the model validation cohort. The number of previous SRCs ranged from 1 to 8, with 39% (n = 7/18) reporting a single SRC and 61% (n = 11/18) reporting ≥2 SRCs. Concussion occurrence within the 12 months before the study was reported by 56% (n = 10/18).
Discrimination provided by the calculated probabilities was very good (AUC = 0.772, classification accuracy = 75.0%). Calibration plots for both the model derivation cohort (r = 0.959, P = .001; intercept = 0.001; β = .999) and the model validation cohort (r = 0.849, P = .008; intercept = 0.115; β = .881) are shown in Figure 1. A comparison of ROC curves derived from applying the same logistic regression model for calculating SRC Hx probability for individual athletes in both cohorts is provided in Figure 2. A comparison of prediction accuracy for composite lateral asymmetry cut points of ≥10% and ≥15%, without consideration of MSK Hx is presented in Table 4. To illustrate the practical implications of the study findings, ROC analysis was used to define optimal cut points for composite lateral asymmetry (≥18% versus <18%) and MSK Hx (adverse: persistent, frequent, or occasional response versus favorable: infrequent, rare, or never response) for the combined cohorts (n = 75). The prevalence of SRC Hx for combinations of the binary classifications of composite lateral asymmetry and MSK Hx is depicted in Figure 3.
The combination of WBRA asymmetry and self-reported persisting effects of previous MSK injury demonstrated a strong association with SRC Hx. Our findings are consistent with those reported by other investigators12,32 who used virtual reality and motion tracking to detect an effect of SRC on the whole-body movement capabilities of athletes. Slowing of neural processing speed is a well-documented, long-term effect of SRC, which almost certainly affects visually guided movements that require integration of cognitive and motor processes.11 Given that the volume and spatial distribution of white matter is not symmetric between brain hemispheres, diffuse axonal injury could logically be expected to slow neural processing to a greater extent in 1 hemisphere than the other. We identified the same ≥15-millisecond cut point for the dual-task VMRT right-left difference reported previously,18 but the strength of association did not meet the criterion of OR ≥ 3 used to identify the set of strongest predictors.
The combination of WBRA values for reaction time asymmetry, speed asymmetry, acceleration asymmetry, and deceleration asymmetry to create the continuous composite lateral asymmetry variable appears to have provided ≥10% and ≥15% binary classifications with good generalizability for different cohorts.26 Cut points differed substantially between the derivation and validation cohorts for separate analyses of the lateral WBRA metrics (reaction time asymmetry ≥30% versus ≥23%, speed asymmetry ≥10% versus ≥8%, acceleration asymmetry ≥12% versus ≥3%, deceleration asymmetry ≥14% versus ≥22%, respectively). We derived the composite lateral asymmetry cut point ≥18% from ROC analysis of continuous data for the combined cohorts to simplify test result interpretation. In contrast to the combination of WBRA metrics to create a composite variable, deconstruction of the Sport Fitness Index score to consider only its most discriminating item was used to identify an important secondary factor that substantially improved model discrimination and calibration.
Previous literature33,34 pertaining to functional asymmetry has been focused on different lower extremity performance capabilities represented as nondominant : dominant, injured : uninjured, or left : right ratios. Although bilateral differences in isolated measures of strength, power, postural balance, gait, jump landing, and hopping appear to be important injury risk factors, measures of dynamic whole-body activity reflect complex sensorimotor integration that controls interlimb coordination.35 The observed WBRA asymmetry does not simply represent a bilateral difference in extremity performance capabilities because both extremities contribute to the whole-body displacements in both lateral-movement directions. For example, deactivation of a right-side target typically involves a left extremity push-off, a right extremity landing, a left extremity landing with push-off, and a right extremity landing for reversal of motion back toward the center starting position.
Persisting effects of previous MSK injury, such as joint laxity, muscle weakness, loss of proprioceptive afference, and restricted range of motion, could certainly contribute to asymmetric whole-body movement capabilities. Therefore, the WBRA asymmetry we observed may have been due to persisting effects of MSK injury, persisting effects of SRC, or a combination of both factors. Among athletes who exhibited a composite lateral-movement asymmetry <18% and reported a favorable MSK Hx, the prevalence of SRC Hx was only 18.75% (n = 6/32). Among those who exhibited ≥18% asymmetry and reported a favorable MSK Hx, the prevalence of SRC Hx was 90.9% (n = 10/11). Among those who exhibited ≥18% asymmetry and reported an adverse MSK Hx, the prevalence of SRC Hx was 100% (n = 6/6). The cross-sectional design of this study did not allow for determination of the temporal order of previous SRC and MSK injury occurrences, but our results suggested that either type of injury could be responsible for asymmetric lateral-movement capabilities. Despite the possible confounding effect of MSK Hx, composite WBRA asymmetry for lateral movements appeared to have very good discriminatory value for identifying SRC Hx cases at asymmetry thresholds of 10% and 15% (Table 4).
The relatively small size of both the derivation and validation cohorts and an insufficient number of female athletes for conclusive determination of an effect of sex on the measures represent important limitations. Reliance on athlete self-report of SRC Hx can be viewed as a limitation, but the strength of the association might have been equal or greater if a definitive record of past injuries had been available. Another limitation is the lack of published data validating WBRA measures derived from the Kinect depth camera (Microsoft Corp, Redmond, WA), which is a component of the virtual-reality motion-tracking system. However, the Kinect depth camera provides 3-dimensional coordinate measurements within 3 m at a level of accuracy comparable with that of measurements derived from a multiple-camera system for locating reflective markers.36,37
Our laboratory recently assessed test-retest reliability for 3 WBRA test sessions separated by 24 hours.38 Eighteen college-aged participants completed a total of 40 movement patterns per test session (5 each for forward, backward, right, left, forward right, forward left, backward right, and backward left). Intraclass correlation coefficients were 0.536 for reaction time, 0.847 for speed, 0.919 for acceleration, and 0.948 for deceleration. The available evidence supported the validity and reliability of our WBRA measurements, but more research is needed to confirm that they are sufficiently precise for use as clinical indicators of impaired neural processing and elevated injury risk.
The long-term effect of SRC on visually guided motor actions may be an overlooked phenomenon that interferes with proper execution of complex movements.8–12 Given that SRC Hx appears to have a profound effect on the risk for subsequent SRC6 as well as the risk for MSK injury,7,8 assessment and training of neuromechanical responsiveness may prove to be an important advance in the prevention of sports injuries. Future research involving advanced neuroimaging and electrophysiological methods might identify neural correlates of whole-body performance asymmetries, which would provide a means to assess the potential for neuroplastic adaptations to specific training protocols.
Our finding of a strong association between WBRA asymmetry and self-reported SRC Hx suggested that subtle cognitive-motor impairment may persist long after complete resolution of overt signs and symptoms after SRC, which may elevate the risk for future injury. Persisting effects of previous MSK injury may explain WBRA asymmetry in the absence of SRC Hx, or SRC Hx and previous MSK injury may have an interactive effect. Our model demonstrated very good calibration and strong discriminatory power for both the derivation and validation cohorts, which support its potential use for identifying individual athletes who are most likely to derive the greatest benefit from a targeted training intervention for improving whole-body reactive movement capabilities.
We appreciate the valuable contributions made by Damon R. Sowers, MS, ATC, and Sydney L. Rubingh, MS, ATC, in acquisition of the data used to validate the logistic regression model.