Context:

Musculoskeletal injury-prediction methods vary and may have limitations that affect the accuracy of results and clinical meaningfulness.

Background:

Research examining injury risk factors is meaningful, but attempting to extrapolate injury risk from studies that do not prospectively assess injury occurrence may limit clinical applications. Injury incidence is a vital outcome measure, which allows for the appropriate interpretation of injury-prediction analyses; a lack of injury-incidence data may decrease the accuracy and increase the uncertainty of injury-risk estimates. Extrapolating results that predict an injury risk factor to predicting actual injuries may lead to inappropriate clinical decision-making models.

Conclusions:

Improved understanding of the limitations of injury-prediction methods, specifically those that do not prospectively assess injuries, will allow clinicians to better assess the clinical meaningfulness of the results.

As the emphasis on musculoskeletal injury prevention continues to grow,1,2  researchers35  have sought to identify or develop clinical tools that can be used to determine an individual's risk for injury during athletic participation. Injury-prediction and risk-factor–identification studies are performed using a variety of study designs and a multitude of statistics. Although some authors3,6  have identified statistical significance and clinical meaningfulness, it is important to consider methodologic design when determining the generalizability and clinical meaningfulness of the results. One methodologic flaw of particular concern occurs when investigators perform injury-prediction studies without prospectively tracking injuries and instead retrospectively identify group differences or predict the presence of previously established injury risk factors. Not only are researchers committing flaws, but the authors of systematic reviews of injury risk assessment studies often do not acknowledge them.7  Systematic reviews of injury risk-assessment studies are designed to evaluate findings in light of methodologic quality, yet the authors frequently fail to address methodologic flaws. Therefore, in this report, we will discuss flaws in injury-prediction studies and how those flaws may affect the clinical meaningfulness of the results.

Some investigators812  performed research to identify factors that predict injury without prospectively identifying injury incidence. Identifying group differences in retrospective case-control studies1012  is a current model that has been used to attempt to identify injury risk factors without prospectively identifying injury incidence. Injury-prediction studies using retrospective case-control study designs identify differences between injured and uninjured individuals after injury has occurred10 ; researchers then use the identified group differences to make inferences about injury risk. The often simple and relatively less demanding nature of these studies may be advantageous, but retrospective inferences about which characteristics increase injury risk are difficult to make and potentially inaccurate. In such designs, it is unclear whether group differences indicated an increased injury risk because the differences were not examined before the injuries occurred. It is possible that the differences were not present before the injuries and occurred as a response to the injuries.13  Despite these limitations, meaningful information can be gathered from these studies; the results can identify potential injury risk factors that warrant examination in a prospective manner.

An additional process used to identify potential injury risk factors is to ascertain factors that predict other, previously recognized risk factors for injury.8,9  Predicting injury risk factors effectively treats those risk factors as surrogates for injury, leading clinicians to infer that by predicting injury risk factors, they can also effectively predict injury.8  This process is concerning because it introduces additional error into the injury-risk–identification process that may reduce accuracy and precision and increase the uncertainty of an injury-risk estimate. Reduced precision and increased uncertainty of an estimate would be reflected in widened confidence intervals.14,15  Confidence intervals with coefficients of 95% indicate a range of values in which one could expect the true estimate to fall 95% of the time.14,15  Similarly, with measures of diagnostic ability, a 95% confidence interval associated with the sensitivity of a tool to identify individuals who will suffer injuries will be wider if that tool was developed to predict an injury risk factor instead of the injury itself.

To illustrate this point, consider the example in Figure 1 in which a clinical test (test X) was developed to predict individuals who are likely to have the injury risk factor of anterior-reach asymmetry greater than 4 cm on the Lower Quarter Y-Balance test. Because asymmetry greater than 4 cm is a risk factor for lower extremity injury,16  it may be tempting to claim that test X can also predict injury. This research model uses the Lower Quarter Y-Balance test as a surrogate for injury, with the claim that predicting performance on that test effectively predicts injury. This process is flawed, however, given the multiple levels of error introduced. Asymmetry greater than 4 cm on the Y-Balance test is not a perfect predictor of injury, in part because the test evaluates balance, which is only 1 aspect of multifactorial injury risk. Some individuals who are identified as being at increased risk for lower extremity injury will not be injured, and some who are identified as not having an increased risk will be injured. Also, error is associated with the ability of test X to identify Y-Balance test asymmetry, in part because multiple factors influence test asymmetry and test X does not assess all of those factors, resulting in some tested individuals being misidentified. Using a test (test X), which has some error, to identify individuals with a risk factor (Y-Balance performance) for injury, which also has error, results in compounded error associated with using test X to predict injury. This compounded error reduces test X's effectiveness and, in some cases, may make it an ineffective assessment of injury risk.

Figure 1. 

Diagram illustrating predictive relationships, and the error that occurs in the prediction process, among test X, Lower Quarter Y-Balance test asymmetry, and injury. A, Test X predicting injury directly. B, Y-Balance test asymmetry predicting injury directly. C, Test X predicting a risk factor for injury (Y-Balance test asymmetry). D, Test X indirectly predicting injury by predicting a risk factor for injury.

Figure 1. 

Diagram illustrating predictive relationships, and the error that occurs in the prediction process, among test X, Lower Quarter Y-Balance test asymmetry, and injury. A, Test X predicting injury directly. B, Y-Balance test asymmetry predicting injury directly. C, Test X predicting a risk factor for injury (Y-Balance test asymmetry). D, Test X indirectly predicting injury by predicting a risk factor for injury.

Close modal

An example of this concept in practice is found in a study8  that resulted in a clinic-based nomogram to identify female athletes at greater risk for anterior cruciate ligament (ACL) injury. Although the tool was created to identify individuals at greater risk for ACL injury, it was developed using a surrogate for injury: high knee-abduction moment (KAM). The nomogram predicted high KAM with 84% sensitivity and 67% specificity,8  but previous investigators17  had established that high KAM predicted ACL injury with 78% sensitivity and 73% specificity. Because the nomogram predicts high KAM with some error and high KAM predicts ACL injury with some error, the true extent to which the nomogram may actually predict ACL injury is unclear, and the nomogram is potentially less effective than high KAM itself. Authors18  who attempted to validate the nomogram using prospective injury data found it did not predict injury in the test population, demonstrating a possible lack of robustness in the tool. This failure to predict injury may be due to variations in study methods, but the difference between an injury risk factor and an actual injury cannot be ignored as a possible influence on predictive capability. Additionally, misunderstanding the implications of predicting injury risk factors may lead to inaccurate injury risk assessment and, subsequently, inappropriate clinical decision making.

An example of overextending results to infer implications about injury risk can also be found in a study19  that examined the effect of fatigue on reactive postural control. Researchers examined whether soccer-specific fatigue would impair dynamic balance in professional soccer players. An exercise protocol designed to replicate a soccer game was used to induce fatigue while participants, at various time points, completed balance tasks that required them to respond to perturbations.19  Exercise duration was significantly related to reaction time and center-of-gravity displacement, with slower reaction times and greater center-of-gravity displacement during the balance tasks conducted after longer exercise durations.19  The authors19  interpreted their findings to suggest that injury risk increased as exercise duration increased, specifically at the end of each half of a soccer game. Impaired reactive postural control was a surrogate for injury; because postural control decreased as exercise duration increased, exercising for longer durations was believed to place individuals at greater risk for injury (Figure 2). Although this may be true, the investigators did not examine injury incidence, and therefore, claims about injury risk should be tempered. However, these authors are not alone in making such claims, as researchers who do not measure injury potentially overextend their results by claiming direct injury implications.20,21 

Figure 2. 

Diagram illustrating relationships, and the error that occurs when establishing relationships, among exercise duration, reactive postural control, and injury. A, Exercise duration directly related to injury. B, Reactive postural control directly related to injury. C, Exercise duration directly related to reactive postural control. D, Exercise duration indirectly related to injury through a direct relationship with reactive postural control.

Figure 2. 

Diagram illustrating relationships, and the error that occurs when establishing relationships, among exercise duration, reactive postural control, and injury. A, Exercise duration directly related to injury. B, Reactive postural control directly related to injury. C, Exercise duration directly related to reactive postural control. D, Exercise duration indirectly related to injury through a direct relationship with reactive postural control.

Close modal

Despite the limitations of using predicting surrogates for injury, valuable information can still be gathered from studies that assess these surrogates. Finding a relationship between a test of interest and a surrogate for injury can provide the rationale for prospective studies that assess the ability of the test to itself predict injury. Yet until those prospective studies are performed, statements regarding the predictive capability of the test should be made with caution.

A variety of methodologic differences exist in the injury-prediction literature. Some authors retrospectively examined differences between injured and uninjured populations, whereas others predicted risk factors for injury rather than prospectively predicting injury occurrence. Although some benefits can be gained from these models, claims concerning injury-prediction capability should be tempered. When we consider risk factors for injury, our ability to predict actual injury is unknown and possibly less than it might otherwise be because of reduced accuracy and precision and increased uncertainty associated with compounded error. It is important for researchers, editors, reviewers, and clinicians to understand these concerns in order to ensure the correct interpretations of findings that may influence clinical practice.

1
Conley
KM,
Bolin
DJ,
Carek
PJ,
Konin
JG,
Neal
TL,
Violette
D.
National Athletic Trainers' Association position statement: preparticipation physical examinations and disqualifying conditions
.
J Athl Train
.
2014
;
49
(
1
):
102
120
.
2
ValovichMcLeod
TC,
Decoster
LC,
Loud
KJ,
et al.
National Athletic Trainers' Association position statement: prevention of pediatric overuse injuries
.
J Athl Train
.
2011
;
46
(
2
):
206
220
.
3
Gribble
PA,
Hertel
J,
Plisky
P.
Using the Star Excursion Balance Test to assess dynamic postural-control deficits and outcomes in lower extremity injury: a literature and systematic review
.
J Athl Train
.
2012
;
47
(
3
):
339
357
.
4
Lehr
ME,
Plisky
PJ,
Butler
RJ,
Fink
ML,
Kiesel
KB,
Underwood
FB.
Field-expedient screening and injury risk algorithm categories as predictors of noncontact lower extremity injury
.
Scand J Med Sci Sports
.
2013
;
23
(
4
):
E225
E232
.
5
Smith
CA,
Chimera
NJ,
Warren
M.
Association of Y balance test reach asymmetry and injury in Division I athletes
.
Med Sci Sports Exerc
.
2015
;
47
(
1
):
136
141
.
6
Wilkerson
GB,
Colston
MA.
A refined prediction model for core and lower extremity sprains and strains among collegiate football players
.
J Athl Train
.
2015
;
50
(
6
):
643
650
.
7
Aerts
I,
Cumps
E,
Verhagen
E,
Verschueren
J,
Meeusen
R.
A systematic review of different jump-landing variables in relation to injuries
.
J Sports Med Phys Fitness
.
2013
;
53
(
5
):
509
519
.
8
Myer
GD,
Ford
KR,
Khoury
J,
Succop
P,
Hewett
TE.
Development and validation of a clinic-based prediction tool to identify female athletes at high risk for anterior cruciate ligament injury
.
Am J Sports Med
.
2010
;
38
(
10
):
2025
2033
.
9
Myer
GD,
Ford
KR,
Foss
KD,
Rauh
MJ,
Paterno
MV,
Hewett
TE.
A predictive model to estimate knee-abduction moment: implications for development of a clinically applicable patellofemoral pain screening tool in female athletes
.
J Athl Train
.
2014
;
49
(
3
):
389
398
.
10
Jayanthi
NA,
LaBella
CR,
Fischer
D,
Pasulka
J,
Dugas
LR.
Sports-specialized intensive training and the risk of injury in young athletes: a clinical case-control study
.
Am J Sports Med
.
2015
;
43
(
4
):
794
801
.
11
Sturnick
DR,
Vacek
PM,
DeSarno
MJ,
et al.
Combined anatomic factors predicting risk of anterior cruciate ligament injury for males and females
.
Am J Sports Med
.
2015
;
43
(
4
):
839
847
.
12
Taunton
JE,
Ryan
MB,
Clement
DB,
McKenzie
DC,
Lloyd-Smith
DR,
Zumbo
BD.
A retrospective case-control analysis of 2002 running injuries
.
Br J Sports Med
.
2002
;
36
(
2
):
95
101
.
13
Mann
CJ.
Observational research methods. Research design II: cohort, cross sectional, and case-control studies
.
Emerg Med J
.
2003
;
20
(
1
):
54
60
.
14
Smith
CJ.
Interpreting confidence intervals
.
Phlebology
.
2012
;
27
(
3
):
141
142
.
15
Gardner
MJ,
Altman
DG.
Confidence intervals rather than P values: estimation rather than hypothesis testing
.
Br Med J (Clin Res Ed)
.
1986
;
292
(
6522
):
746
750
.
16
Plisky
PJ,
Rauh
MJ,
Kaminski
TW,
Underwood
FB.
Star Excursion Balance Test as a predictor of lower extremity injury in high school basketball players
.
J Orthop Sports Phys Ther
.
2006
;
36
(
12
):
911
919
.
17
Hewett
TE,
Myer
GD,
Ford
KR,
et al.
Biomechanical measures of neuromuscular control and valgus loading of the knee predict anterior cruciate ligament injury risk in female athletes: a prospective study
.
Am J Sports Med
.
2005
;
33
(
4
):
492
501
.
18
Goetschius
J,
Smith
HC,
Vacek
PM,
et al.
Application of a clinic-based algorithm as a tool to identify female athletes at risk for anterior cruciate ligament injury: a prospective cohort study with a nested, matched case-control analysis
.
Am J Sports Med
.
2012
;
40
(
9
):
1978
1984
.
19
Greig
M,
McNaughton
L.
Soccer-specific fatigue decreases reactive postural control with implications for ankle sprain injury
.
Res Sports Med
.
2014
;
22
(
4
):
368
379
.
20
Fagenbaum
R,
Darling
WG.
Jump landing strategies in male and female college athletes and the implications of such strategies for anterior cruciate ligament injury
.
Am J Sports Med
.
2003
;
31
(
2
):
233
240
.
21
Kristianslund
E,
Krosshaug
T.
Comparison of drop jumps and sport-specific sidestep cutting: implications for anterior cruciate ligament injury risk screening
.
Am J Sports Med
.
2013
;
41
(
3
):
684
688
.