Differences in various outcome measures have been identified between people who have sprained their ankles but have no residual symptoms (copers) and people with chronic ankle instability (CAI). However, the diagnostic utility of the reported outcome measures has rarely been determined. Identifying outcome measures capable of predicting who is less likely to develop CAI could improve rehabilitation protocols and increase the efficiency of these measures.
To determine the diagnostic utility and cutoff scores of perceptual, mechanical, and sensorimotor outcome measures between copers and people with CAI by using receiver operating characteristic curves.
Sports medicine research laboratory.
Twenty-four copers (12 men, 12 women; age = 20.8 ± 1.5 years, height = 173 ± 11 cm, mass = 78 ± 27 kg) and 24 people with CAI (12 men, 12 women; age = 21.7 ± 2.8 years, height = 175 ± 13 cm, mass = 71 ± 13 kg) participated.
Self-reported disability questionnaires, radiographic images, and a single-legged hop stabilization test.
Perceptual outcomes included scores on the Foot and Ankle Disability Index (FADI), FADI-Sport, and a self-report questionnaire of ankle function. Mechanically, talar position was quantified by measuring the distance from the anterior tibia to the anterior talus in the sagittal plane. Sensorimotor outcomes were the dynamic postural stability index and directional indices, which were calculated during a single-legged hop stabilization task.
Perceptual outcomes demonstrated diagnostic accuracy (range, 0.79–0.91), with 95% confidence intervals ranging from 0.65 to 1.00. Sensorimotor outcomes also were able to discriminate between copers and people with CAI but with less accuracy (range, 0.69–0.70), with 95% confidence intervals ranging from 0.37 to 0.86. The mechanical outcome demonstrated poor diagnostic accuracy (0.52).
The greatest diagnostic utility scores were achieved by the self-assessed disability questionnaires, which indicated that perceptual outcomes had the greatest ability to accurately predict people who became copers after their initial injuries. However, the diversity of outcome measures that discriminated between copers and people with CAI indicated that the causal mechanism of CAI is probably multifactorial.
Perception-based outcomes had the greatest ability to discriminate between copers and people with chronic ankle instability.
Sensorimotor-based outcomes discriminated between copers and people with chronic ankle instability but with less accuracy than perception-based outcomes.
Mechanical outcomes demonstrated a low level of diagnostic utility.
Lateral ankle sprains, which are common orthopaedic injuries,1 often result in chronic ankle instability (CAI), which is defined by many residual symptoms, including pain, episodes of giving way, recurrent injury, and decreased physical activity.2,3 Some people who have histories of lateral ankle sprain return to high-level activities (ie, jumping, pivoting) without recurrent injury or loss of function through an unidentified mechanism that allows them to function as if un-injured.4–8 These people are called copers. Understanding the differences between copers and people with CAI will help elucidate the underlying cause of CAI. To date, researchers have not identified differences in functional performance scores (eg, hop tests),4 the number of trials failed while completing a hop test protocol,4 ankle joint stiffness,8 fibula position (ie, positional fault),8 and several kinematic outcomes during various dynamic tasks between copers and people with CAI.5 However, differences in perceptual (eg, self-assessed questionnaires),4 mechanical (eg, ligament laxity),6 and sensorimotor (eg, balance and kinematic)5,7,8 outcomes have been identified between copers and people with CAI. Unfortunately, the diagnostic utility of the outcomes previously studied has rarely been determined.7
Quantifying the diagnostic utility and, most importantly, cutoff scores for outcome measures commonly used to study differences between copers and people with CAI is the first step in developing the framework for a classification scheme that clinicians could use after an initial lateral ankle sprain to help identify the people who are more and less likely to develop CAI and to lower the incidence of CAI4 through focused therapeutic intervention delivery for people more likely to develop CAI. A similar approach has been used to classify people with anterior cruciate ligament deficiency as copers or noncopers.9 Thus, determining the diagnostic utility and cutoff scores of outcome measures previously studied is feasible and important.
Traditional classification tools in medicine are typically binary (ie, healthy or diseased), but many outcome measures used in research and clinical settings produce a numeric value on a continuous scale (eg, range of motion, self-assessed disability, postural control). Thus, no particular value of sensitivity (true-positive rate) or specificity (true-negative rate) characterizes the overall accuracy of the test because the sensitivity and specificity of such an outcome measure change throughout the entire range of values.10 A receiver operating characteristic (ROC) curve is a statistical analysis technique that plots sensitivity (true-positive rate) against 1 – specificity (false-positive rate) to describe how sensitivity and specificity trade off over a measure's range of values. Furthermore, ROC curves can be used to determine appropriate cutoff score values for classification schemes used in future prospective investigations.10–13 Therefore, we designed our investigation to establish the framework for a classification scheme that clinicians could use to determine which people are more and less likely to develop CAI after an initial lateral ankle sprain (ie, to identify copers). Specifically, the purpose was to quantify the diagnostic utility and determine cutoff scores of outcome measures that have been shown to discriminate between copers and people with CAI.
This investigation was a secondary analysis of data previously collected as part of an investigation involving potential coping mechanisms after lateral ankle sprains. The data originally were collected to study group differences between un-injured controls, copers, and people with CAI, so most group means and standard deviations used in this investigation have been published.4,8,14
As previously described, data were collected using a single-blind case-control design.4,8,14 The primary author (E.A.W.) screened and grouped all participants and conducted data analysis.a Another author (K.E.N.) who was blinded to group membership collected and entered the data.a Twenty-four copers (12 men, 12 women; age = 20.8 ± 1.5 years, height = 173 ± 11 cm, mass = 78 ± 27 kg) and 24 people with CAI (12 men, 12 women; age = 21.7 ± 2.8 years, height = 175 ± 13 cm, mass = 71 ± 13 kg) volunteered. Participants were recruited from the student population at a large public university, and their ages ranged from 18 to 30 years.4,8,14 They participated in aerobic recreational activity at least 3 times each week for a total of 90 minutes and had a history of at least 1 unilateral lateral ankle sprain that necessitated immobilization or no weight bearing for at least 3 days.a Exclusion criteria were a history of previous ankle fractures, a history of head or acute lower extremity injury within the 3 months before the study, and participation in formal rehabilitation for the involved ankle.4,8,14,a People in the coper group had no pain, weakness, or instability in the involved ankle; had resumed all preinjury activities without limitation for at least 12 months before testing; and scored more than 22 of 48 points on the Ankle Joint Functional Assessment Tool (AJFAT).4,8,14,a People in the CAI group had at least 1 episode of giving way in the year before the study; had at least 1 recurrent sprain 3 to 6 months before the study; scored <22 on the AJFAT; perceived pain, instability, or weakness in the involved ankle and attributed the cause of those perceptions to their initial ankle injury; and had not returned to a preinjury level of activities.4,8,14,a Both a recent recurrent sprain and episodes of giving way were required to ensure that 2 of the most commonly reported symptoms of CAI were present in our selected sample. People in the CAI group had 5.1 ± 4.6 episodes of giving way in the year before the study and 1.3 ± 0.8 recurrent sprains 3 to 6 months before the study.4 A score of 22 on the AJFAT questionnaire, which other researchers have used,4,15 was chosen as the cutoff for the CAI group because Ross et al11 reported that this score successfully identified functional limitations in 100% of their participants with CAI.
All participants provided written informed consent, and the study was approved by the Institutional Review Board of the University of Florida.
Data collection took place in a single session. Radiographic images were taken at a university student health care center, whereas all other measures were collected in a biomechanics research laboratory. Testing sessions were counterbalanced using a Latin square design. Specific measurement tools and dependent variables that were not part of the inclusion criteria are described briefly here but were collected and reduced as previously described.4,8,14
Perceptual Outcome Measures
Self-reported disability was determined by instructing participants to complete the Foot and Ankle Disability Index (FADI), the FADI-Sport (FADI-S), and a self-report questionnaire of ankle function (SRQAF).4 The FADI, FADI-S, and SRQAF quantify the physical limitations and disability of the person completing the questionnaire. Although all 3 questionnaires pose similar questions, the number of items and weight of each item differ among questionnaires. For example, the FADI and FADI-S assign a maximum weight of 4 points to each question, which results in 104-point and 32-point scales, respectively. However, the 5 items on the SRQAF have different maximum weights (pain = 35, instability = 25, weight bearing = 20, swelling = 10, and gait pat- tern = 10). Thus, the magnitude of a person's CAI-related disability will vary among the questionnaires but will be greater on the SRQAF because of weighting and fewer included items. All 3 perceptual-based outcomes differed between established copers and people with CAI.4 Furthermore, between-limbs symmetry ratios for the FADI (r = 0.84, SEM = 1.0), FADI-S (r = 0.91, SEM = 1.1), and SRQAF (r = 0.85, SEM = 3.2) displayed good to excellent reliability and precision.
Mechanical Instability Outcome Measures
Wikstrom et al4 found that fibular position and ankle joint stiffness did not differ between copers and people with CAI. However, they did not calculate the sagittal-plane position of the talus. Therefore, structural alignment was determined using non–weight-bearing radiographic images that were collected with a Medio 30 CP-H (Phillips, Eindhoven, The Netherlands) with a 150-kV, microprocessor-controlled, high-frequency X-ray generator. Films were taken using a manual technique from 3.2 mAs at 64 kVp (average man) to 2.5 mAs at 62 kVp (average woman), with participants positioned on their sides and the hip, knee, and ankle placed in neutral positions.8,14,a Talar position was quantified by measuring the distance in millimeters between the most anterior margin of the inferior aspect of the tibia and the most anterior margin of the talar dome. Quantifying talar position in this manner has high intratester (intraclass correlation coefficient = 0.90) and intertester (intraclass correlation coefficient = 0.78) reliability.14
Sensorimotor Outcome Measures
We used ground reaction force data collected during a single-legged hop stabilization test to measure dynamic postural control as described in a previous investigation.8 Specifically, a triaxial force plate (model 4060; Bertec Corporation, Columbus, OH) sampling at a rate of 200 Hz captured the required ground reaction force data. To complete this test, participants started 70 cm from the center of the force plate and jumped off both legs to touch an overhead marker placed at 50% of each participant's maximum vertical leap before landing on the force plate with the test leg, stabilizing as quickly as possible, and maintaining this position for 3 seconds.a We defined the test leg as the involved limb. Force plate data were analyzed to calculate the dynamic postural stability index (DPSI) and directional indices as described in a previous study.8,a These indices assess the standard deviations of fluctuations around a zero point that then are divided by the number of data points in a trial so that higher scores indicate greater variability.a The medial-lateral stability index (MLSI), anterior-posterior stability index (APSI), and vertical stability index (VSI) correspond with the frontal (y), sagittal (x), and transverse (z) axes, respectively, of the force plate.a The DPSI is a composite of the ground reaction forces in all planes and thus is sensitive to force changes in each direction.a
The accuracy of the investigated outcome measures in identifying copers was calculated using the area under the curve (AUC) for ROC curves and the 95% confidence intervals (CIs) of the AUC. As indicated, an ROC curve illustrates the trade- off between sensitivity and specificity throughout a measure's entire range of values. Thus, an outcome measure with perfect accuracy would be depicted by an ROC running vertically from the origin (point 0,0) to 100% sensitivity (point 0,1) and then running horizontally to 100% specificity (point 1,1).10 A traditional academic point scale was used to classify the accuracy of the AUC and the 95% CIs of the AUC for discriminating between established copers and people with CAI, with 0.90 to 1.00 indicating excellent; 0.80 to 0.89, good; 0.70 to 0.79, acceptable; 0.60 to 0.69, poor; and 0.00 to 0.59, failure. The asymptotic α level was set a priori at .05.7,12 In addition, cutoff scores were quantified for all variables that were asymptotically different. Cutoff scores were determined by calculating the Youdin index (J) for each outcome value along the ROC curve, with the largest J value representing the cutoff score.16 The formula used to calculate J is given in Equation 1. To determine the clinical meaningfulness of the cutoff scores, likelihood ratios and their 95% CIs were calculated using the sensitivity and 1 – specificity data at the cutoff score point that the ROC curve analysis provided.17 Specifically, the ratio for a positive test result (LR+) is given in Equation 2, and the ratio for a negative test result (LR–) is given in Equation 3. For the purposes of this investigation, LR+ values that were more than 10 were interpreted as large and often conclusive; 5 to 10, moderate but usually important; 2 to 5, small and sometimes important; and 1 to 2, very small and usually unimportant.18,19 The LR– values that were less than 0.1 were interpreted as large and often conclusive; 0.1 to 0.2, moderate but usually important; 0.2 to 0.5, small and sometimes important; and 0.5 to 1, very small and usually unimportant.18,19
The means ± standard deviations, minimum and maximum values, and 95% CIs for the coper and CAI groups for each of the included outcome measures are provided in Table 1. Three figures display the calculated ROC curves. Perceptual outcomes are depicted in Figure 1 (self-assessed disability), the mechanical outcome (talar position) is shown in Figure 2, and sensorimotor outcomes (MLSI, APSI, VSI, DPSI) are illustrated in Figure 3. The resulting AUCs, 95% CIs, and asymptotic differences from the reference line for the included outcome measures are shown in Table 2. The quantified cutoff scores, the sensitivity and 1 – specificity values at the cutoff point, and the positive and negative likelihood ratios and their 95% CIs for the outcome measures that were asymptotically different from the reference line also are listed in Table 2. Six outcome measures (SRQAF, FADI, FADI-S, MLSI, APSI, DPSI) were asymptotically different from the reference line.
Our results indicated that perceptual (FADI, FADI-S, SRQAF) and some sensorimotor (MLSI, APSI, DPSI) outcomes can discriminate between copers and people with CAI. We hypothesize that these outcome measures represent part of the mechanism that allows copers to function as if uninjured and is absent in people who develop CAI. Furthermore, because various outcomes (perceptual and sensorimotor) successfully discriminated between groups, our findings support the theory that the causal mechanism of CAI is multifactorial. However, our most important finding was that the perceptual outcomes demonstrated the greatest ability to discriminate between copers and people with CAI, with accuracy point estimates ranging from 0.79 to 0.91 and 95% CIs ranging from 0.65 to 1.00. Indeed, the perceptual outcomes as a whole demonstrated higher AUC estimates, 95% CIs, and positive likelihood ratios and demonstrated lower negative likelihood ratios than the sensorimotor outcomes as a whole that were asymptotically different. These results indicated that perceptual outcomes had the greatest ability to accurately predict people who became copers after initial lateral ankle sprain.
Furthermore, the increased discriminatory accuracy of perception-based outcomes supports the anecdotal evidence that sensations of instability are the most commonly reported and perhaps the most serious symptom of CAI. Interestingly, the conscious perception of instability has not been discussed extensively in the literature despite the overwhelming prevalence of this symptom and the effect researchers and clinicians believe it can have on a patient's lifestyle. Buchanan et al20 illustrated the effect that the perception of instability can have on functional tasks. Specifically, people with CAI and healthy controls completed functional performance tasks and were asked whether their ankles felt unstable during the tasks. The initial results indicated no group differences in performance. However, in a secondary analysis comparing people who had CAI and ankles that felt unstable with people who had CAI and ankles that felt stable and with a healthy control group, they revealed that the participants who had CAI and ankles that felt unstable during the functional performance tests had performance deficits relative to both the healthy control group and participants who had CAI and ankles that felt stable. Functional performance differences between people with CAI and uninjured controls remain controversial in the literature because some investigators have not identified group differences,4,21,22 whereas others have found deficits23 or positive correlations between an ankle instability index and functional performance.24 However, only Buchanan et al20 conducted a secondary analysis to determine how the perception of instability affected the results. Using the perception of instability with traditional performance-based tests might increase the sensitivity of those tests. However, this is only speculative, and further research is needed to test this hypothesis.
Investigators using self-assessed disability questionnaires consistently have revealed disability in people with acute lateral ankle sprains relative to uninjured controls,25 people with CAI relative to copers,4,5 and people with CAI relative to un-injured controls.22,26–30 Whereas the means for the coper and CAI groups in our study were different from each other,4 the CAI group means of the FADI (95.2%) and FADI-S (92.9%) were higher than what typically is reported. For example, Hubbard et al29 reported CAI group means of 88.7% and 74.8% for the FADI and FADI-S, respectively. Similarly, Brown et al5 reported greater disability in people with mechanical instability (FADI = 89.1%, FADI-S = 76.6%) and functional instability (FADI = 94.2%, FADI-S = 81.5%) than we found in our investigation. Therefore, the people with CAI in our investigation demonstrated less self-assessed (ie, perceived) disability than those in the literature. The higher means we found might have altered the position of the cutoff scores and might have decreased the diagnostic utility of LR+ and LR– calculated from the cutoff scores. Indeed, the cutoff scores for the FADI (99.5%) and FADI-S (98.0%) indicated that if 1 survey item is scored less than the highest rating possible, the person should be considered to have CAI. Clearly, this recommendation is not reasonable, but the small to moderate LR+ and small LR– point estimates suggest that these cutoff scores are of some importance. The cutoff score for the SRQAF (93%) allows more than a single survey item to be scored less than the highest possible rating before a person should be considered to have CAI. Given the heavy weight assigned to each item, this cutoff score also appears to be unreasonable, but the calculated large LR+ and moderate LR– point estimates indicate that the SRQAF cutoff scores provide useful and important information. Overall, the point estimates of the perceptual outcomes indicated that using the cutoff scores probably will provide a moderate improvement in the probability of discriminating between copers and people with CAI well after the initial injury.18,19 However, the amount of improvement depends on the questionnaire used, and, based on our data, the SRQAF would be very useful, the FADI-S would be moderately useful, and the FADI would provide small and sometimes important information. Future investigations are needed to determine whether similar results would occur immediately after an acute lateral ankle sprain. If similar results are found, then requiring an athlete to complete these forms after injury could improve the health care provider's ability to predict whether the athlete is more or less likely to develop CAI.
Given the overall usefulness identified, additional research on a larger sample size with greater heterogeneity of self- assessed disability scores is needed to determine whether more reasonable cutoff scores (ie, lower percentages on the questionnaires) would provide equal or even greater information in discriminating between copers and people with CAI. This research also would help further establish how perception-based outcomes should fit within the framework of a classification scheme designed to determine which people are more and less likely to develop CAI after an initial lateral ankle sprain. Given the overall moderate LR+ and small to moderate LR– point estimates, we believe that perception-based outcomes, such as self-assessed disability questionnaires, should be used in future longitudinal research investigations designed to determine whether and when after injury each of the self-assessed disability questionnaires can discriminate successfully between people more and less likely to develop CAI. This recommendation is based on 3 points: (1) All 3 perceptual outcomes were asymptotically different in our investigation, (2) patient-oriented outcomes are inherently important to patients,27 and (3) Buchanan et al20 illustrated that the perception of instability can influence functional performance outcomes.
In addition to the perceptual outcomes, the MLSI, APSI, and DPSI were asymptotically different, albeit with lower accuracy point estimates (range, 0.69–0.70) and lower 95% CIs (range, 0.37–0.86) than the perception-based outcomes. However, the cutoff scores for the MLSI (0.04), APSI (0.13), and DPSI (0.30) are clinically reasonable because these scores fall well within the range of dynamic postural stability scores reported during a single-legged hop stabilization task and within the 95% CIs of the coper group means we reported. Copers have been shown to have higher MLSI scores than people with CAI,8 and higher DPSI and APSI scores have been reported consistently in people with CAI than uninjured controls.8,31 However, Wikstrom et al8 did not detect group differences in the APSI and DPSI between copers and people with CAI, making our results unexpected. Data exploration suggests that small group mean differences and high variability are probably the reason that APSI and DPSI group mean differences were not reported previously. However, ROC curves do not consider group mean differences or variability but are the end result of the data being rank ordered, which appears to better articulate the subtle differences between the groups in question.
In addition to lower AUC point estimates and 95% CIs, the included sensorimotor-based outcomes also generally had less meaningful LR+ and LR– point estimates than the perception-based outcomes. Indeed, the calculated LR+ and LR– 95% CIs indicated that using our cutoff scores for the MLSI, APSI, and DPSI will provide a very small to small increase in the probability of identifying copers.18,19 The number of degrees of freedom available to both copers and people with CAI might explain why sensorimotor-based outcomes had worse accuracy point estimates than perception-based outcomes had. Multiple degrees of freedom (eg, multiple joints within the lower extremity) enable people to produce various solutions to a particular task. With more degrees of freedom available, a person has a greater variety of solutions to accomplish a movement goal. Although more degrees of freedom offer flexibility, they are a major source of variability in movement patterns.32 Wikstrom et al4,8 suggested that copers have a greater number of movement solutions available and that this greater movement pattern flexibility has allowed copers to compensate (ie, prevent rein-jury) successfully for damage caused by the initial ankle sprain. However, more constraints (ie, fewer degrees of freedom) does not necessarily mean a decrease in performance because a person might have a highly efficient sensorimotor system within the decreased number of available degrees of freedom. Although we speculated that people with CAI still might be able to jump as far and as quickly as copers and uninjured controls because they can place greater emphasis on the remaining degrees of freedom available to them (eg, hip and knee joints), further research is needed to test this hypothesis.
If our hypothesis is true, then the people with CAI in our investigation might have used different movement patterns (ie, kinematics) to complete the single-legged hop stabilization task that resulted in similar outcomes (ie, DPSI scores). Thus, the distribution of group members within the rank order of MLSI, APSI, and DPSI scores might not be defined as clearly as within the rank order of the perception-based outcomes. Therefore, the accuracy estimates for the sensorimotor-based outcomes might have been affected negatively. As stated, the accuracy of the DPSI and other sensorimotor outcomes might be increased by combining them with questions of perceived instability, much like the method of Buchanan et al,20 but future research is needed to explore this possibility. Consistent with the perception-based outcomes, we recommend that researchers use the MLSI, APSI, and DPSI in future longitudinal investigations designed to determine whether and when after injury each of the included sensorimotor outcomes can discriminate successfully between people more and less likely to develop CAI despite the generally small LR+ and very small LR– point estimates. We make this recommendation because the MLSI, APSI, and DPSI were asymptotically different in our investigation despite a small sample size (24 per group).
Our investigation is a first step in a line of research designed to establish a framework for a classification scheme that clinicians could use to determine which people are more and less likely to develop CAI after an initial lateral ankle sprain. Our results have shown that perception-based outcomes represent the most distinguishing characteristic of CAI (ie, they have the greatest ability to discriminate between copers and people with CAI). More importantly, these findings make sense intuitively and confirm the anecdotal evidence that sensations of instability are not only a commonly reported symptom but an important symptom to consider when diagnosing and treating people with ankle conditions. However, only a small sample of potential variables were tested, and additional investigations are needed to determine which outcomes differ between copers and people with CAI and to determine the diagnostic utility of outcomes that identify group differences.
All 3 self-assessed disability questionnaires (perception-based outcomes) demonstrated a clear ability to discriminate between copers and people with CAI, with accuracy point estimates ranging from 0.79 to 0.91 and 95% CIs ranging from 0.65 to 1.00. Sensorimotor-based outcomes, such as the MLSI, APSI, and DPSI, also discriminated between copers and people with CAI but with less accuracy (range, 0.69–0.70), with 95% CIs ranging from 0.53 to 0.86. The mechanical outcome demonstrated a low level of diagnostic utility (0.52) and a 95% CI of 0.35 to 0.69, and it was not asymptotically different. Thus, perception-based outcomes demonstrated the greatest potential to be accurate predictors of the development of CAI after an initial lateral ankle sprain, but long-term prospective investigations are needed to confirm these findings.
aWikstrom EA, Tillman MD, Chmielewski TL, Cauraugh JH, Naugle KE, Borsa PA. Dynamic postural control but not mechanical stability differs among those with and without chronic ankle instability. Scand J Med Sci Sports. 2010;20(1):e137–e144. Used with permission from John Wiley and Sons.