Clinicians are urged to document patient-based outcomes during rehabilitation to measure health-related quality of life (HRQOL) from the patient's perspective. It is unclear how scores on patient-reported outcome instruments (PROs) vary over the course of an athletic season because of normal athletic participation.
Our primary purpose was to evaluate the effect of administration time point on HRQOL during an athletic season. Secondary purposes were to determine test-retest reliability and minimal detectable change scores of 3 PROs commonly used in clinical practice and if a relationship exists between generic and region-specific outcome instruments.
Twenty-three collegiate soccer athletes (11 men, 12 women).
At 5 time points over a spring season, we administered the Disablement in the Physically Active Scale (DPA), Foot and Ankle Ability Measure-Sport, and Knee Injury and Osteoarthritis Outcome Score (KOOS).
Time effects were observed for the DPA (P = .011) and KOOS Quality of Life subscale (P = .027). However, the differences between individual time points did not surpass the minimal detectable change for the DPA, and no post hoc analyses were significant for the KOOS-Quality of Life subscale. Test-retest reliability was moderate for the KOOS-Pain subscale (intraclass correlation coefficient = 0.71) and good for the remaining KOOS subscales, DPA, and Foot and Ankle Ability Measure-Sport (intraclass correlation coefficients > 0.79). The DPA and KOOS-Sport subscale demonstrated a significant moderate relationship (P = .018).
Athletic participation during a nontraditional, spring soccer season did not affect HRQOL. All 3 PROs were reliable and could be used clinically to monitor changes in health status throughout an athletic season. Our results demonstrate that significant deviations in scores were related to factors other than participation, such as injury. Finally, both generic and region-specific instruments should be used in clinical practice.
The Disablement in the Physically Active Scale, Foot and Ankle Ability Measure-Sport, and Knee Injury and Osteoarthritis Outcome Score scales are reliable instruments that have been used clinically to assess activity limitations and participation restrictions in collegiate athletes.
Significant changes in health-related quality of life are likely to be associated with an injury that restricts athletic participation or another factor rather than participation itself.
To capture all dimensions of health-related quality of life, clinicians should use both a generic and a region-specific instrument.
As evidence-based practice grows in the field of athletic training, clinicians are encouraged to document clinical outcomes to demonstrate the effectiveness of treatments or interventions to improve patient care.1 Patient-reported outcome instruments (PROs) are patient-centered outcomes used in clinical practice to capture the patient's perspective regarding physical impairment, functional limitations, and overall health-related quality of life (HRQOL).1 The HRQOL is a measure of a person's function in everyday life and an evaluation of his or her physical, psychological, and social aspects of health derived from personal beliefs, preferences, experiences, and expectations.2,3 Most often, HRQOL is measured using PROs, and numerous PROs have been created. These instruments are commonly classified into 3 categories: generic, region specific, and dimension specific.1 Generic PROs measure the patient's perception of his or her overall health and can capture a range of health-related problems.1 Examples of generic PROs are the Short-Form 124 and the Disablement in the Physically Active Scale (DPA).5,6 Region-specific PROs assess the patient's perception of function for a certain region of the body, such as the ankle or knee.1 Examples of region-specific PROs that are used in athletes are the Foot and Ankle Ability Measure-Sport (FAAM-S)7 and the Knee Injury and Osteoarthritis Outcome Score (KOOS).8
Health-related quality of life has not been studied thoroughly in athletes and is a topic of interest to athletic training researchers and clinicians.9–13 Throughout a traditional or nontraditional athletic season, athletes participate in different team activities that may contribute to an increase or decrease in HRQOL, regardless of the athlete's injury status. Given that athletes are involved in practices, games, agility training, conditioning, and weight training, we must consider the effect of continuous athletic participation on HRQOL. If participation in these activities can change HRQOL, use of these instruments after injury may be confounded by this phenomenon and influence how PRO scores are interpreted. Furthermore, when using PROs in clinical practice, it is important to know their test-retest reliability. The ability of an instrument to measure a change in health status depends on the instrument's ability to measure consistently over time. Assessing the test-retest reliability of these instruments will allow us to calculate the minimal detectable change (MDC). At this time, neither the test-retest reliability nor the MDC have been calculated for the DPA in healthy, physically active persons participating in intercollegiate athletics. In addition, multiple PROs can be used to assess different dimensions of HRQOL. Currently, we do not know if a relationship exists between generic and region-specific instruments. If a relationship does exist, athletic trainers may be able to use fewer PROs to assess HRQOL in their patients.
Our study had 3 purposes. The first purpose was to determine the test-retest reliability and calculate the MDC value of the DPA, FAAM-S, and KOOS subscales in collegiate soccer athletes with a history of lower extremity injury. We hypothesized that these instruments would demonstrate good test-retest reliability in this population. The second purpose was to determine the effect of administration time point on HRQOL scores in collegiate soccer players during an athletic season. We hypothesized that the administration time point would not affect HRQOL. Finally, we aimed to determine if a relationship exists between the DPA and the 2 region-specific instruments. We hypothesized that a strong positive relationship would exist between the generic and 2 region-specific instruments in an athletic population.
A prospective, repeated-measures design was used to determine the test-retest reliability and MDC of the DPA, FAAM-S, and KOOS subscales and to examine the effects of administration time point on HRQOL scores in collegiate soccer athletes over the duration of a spring soccer season. We recorded scores on the DPA, FAAM-S, and KOOS subscales at 5 time points: preseason I (T1), preseason II or baseline (T2), twice during the season (T3 and T4), and postseason (T5). Scores for T1 and T2 were collected 1 week apart and used to determine the test-retest reliability of each PRO. Scores for T2 through T5 were used to determine how administration time point affected PRO scores. Finally, scores from T1 were used to determine relationships between generic and region-specific instruments in this population. The elapsed time between measurements is shown in the Figure. The independent variable was time and the dependent variables were the scores on the PROs (DPA, FAAM-S, and KOOS subscales).
A total of 23 National Collegiate Athletic Association Division I collegiate soccer athletes (11 men, 12 women) participating in the 2013 spring season were initially included. Members of the team who were injured and were removed from all sport and physical activities at the beginning of the research study were excluded. Six participants reported that they were currently injured or had a documented injury during the spring season, and 1 person reported no history of injury on the demographic sheet. Therefore, of the 23 enrolled participants, 16 were available for analysis (Table 1). The most common injuries reported by the participants were ankle sprains (n = 16) and hamstrings/quadriceps strains (n = 5). Two participants reported a history of anterior cruciate ligament reconstruction, and 4 reported a history of concussion. This study was approved by the university's institutional review board. All participants signed an approved informed consent document before the study.
Participants reported for data collection at 5 time points (T1 through T5) over the duration of the spring soccer season. At the first data-collection session, the participants were asked to complete a demographic questionnaire. To assess general HRQOL and function related to the ankle and knee, participants completed 3 PROs: the DPA, FAAM-S, and KOOS subscales (1 for the right side and 1 for the left side) at each testing session. These PROs were completed using pen and paper and in a counterbalanced manner to reduce an order effect over time. All data-collection sessions were conducted on the same day of the week and at approximately the same time. In addition, throughout the spring season, the athletic trainers for each team documented injuries sustained and time lost using their electronic medical records system.
Disablement in the Physically Active Scale
The DPA is a 16-item generic, population-specific PRO that measures HRQOL across 4 domains: impairment, functional limitations, disability, and quality of life.5,6 The DPA is scored using a 5-point Likert scale, in which 1 represents no problem and 5 represents severe problem.5,6 Once each item and domain are scored, 16 points are subtracted from the final score. A higher score represents a higher level of disablement.6 The DPA scores range from 0 to 64. The DPA is a valid and reliable instrument (intraclass correlation coefficient [ICC] = 0.943) in patients with acute and persistent injuries.6
Foot and Ankle Ability Measure-Sport
The FAAM-S is an 8-item, region-specific PRO used to measure functional limitations and symptoms related to sport in patients with a history of foot and ankle injuries.7 The FAAM-S uses a 5-point Likert scale in which 0 represents unable to do and 4 represents no difficulty to rate the patient's ability to perform sport-related tasks, such as running, jumping, and landing.7 A higher score represents a higher level of foot and ankle function.14 The FAAM-S has demonstrated excellent test-retest reliability (ICC = 0.87) and internal consistency (α = .98).7
Knee Osteoarthritis Outcomes Score
The KOOS is a 42-item, region-specific PRO that evaluates a patient's functional status and HRQOL after knee injury in those at risk for developing posttraumatic osteoarthritis.8 The KOOS assesses 5 domains through different subscales: activities of daily living (KOOS-ADL), pain (KOOS-Pain), knee-related quality of life (KOOS-QOL), sport and recreation function (KOOS-Sport), and symptoms (KOOS-Symptoms).8 Each domain is measured on a 5-point Likert scale and each subscale is scored separately, with 100 being the highest score possible. Higher scores on the KOOS represent better function.8 The KOOS demonstrated high test-retest reliability in patients with a history of anterior cruciate ligament injury, meniscal injury, or posttraumatic osteoarthritis for all 5 subscales: KOOS-ADL (ICC = 0.75), KOOS-Pain (ICC = 0.85), KOOS-QOL (ICC = 0.86), KOOS-Sport (ICC = 0.81), and KOOS-Symptoms (ICC = 0.93).8
To examine the test-retest reliability and MDC, PRO scores from T1 and T2 were used because the participants were not active in formal spring soccer training during this week. To examine the effect of administration time point over the course of the season, we compared PRO scores across the T2 through T5 time points. Time point T2 occurred on the first day of the spring soccer season and was considered the baseline because it was the most recent time point before the spring soccer season. The PRO scores at T1 were used to examine the relationship between generic and region-specific instruments. Participants completed the FAAM-S and the KOOS for both the right and left ankle or knee, respectively. For all 3 statistical analyses, the side (right or left) tested on the FAAM-S and KOOS subscales was determined based on the side (right or left) with a history of injury as reported on the demographic questionnaire.
All statistical analyses were conducted using SPSS software (version 21; IBM Corporation, Armonk, NY). Because of the distribution of the data, nonparametric statistics were performed where applicable. Currently, no guidelines are available for dealing with missing items on the DPA. Therefore, missing items for the DPA were treated conservatively and replaced with the person's mean.15 Only 1 of 1280 items had to be completed using this method. Missing items for the FAAM-S and KOOS subscales were treated in accordance with the respective instrument's scoring instructions. Multiple imputation with 5 repetitions16 was used to replace missing time points (ie, missing total scores) within the data set for all PROs. Fewer than 11% (56/525) of the PRO scores in the data set were computed using multiple imputation. Missing data points or participants removed from analyses were attributed to reporting currently being injured or having no history of lower extremity injury at the T1 data-collection session, removal from the team, severe injury, injury that persisted through the season, or inability to attend the final data-collection session (Figure).
Descriptive statistics (mean ± SD) were calculated for each PRO for each testing session. Test-retest reliability was determined using ICCs (2,1). The ICCs were interpreted as weak (≤0.20), moderate (0.20–0.74), or good (≥0.75).17 The standard error of measurement (SEM) was calculated using the formula SEM = pooled SD * √1 – ICC. The MDC values were calculated using the SEM with the formula MDC = SEM * 1.96 * √2. We used separate Friedman tests to determine the effect of administration time point on each PRO (P ± .05). In the event of a significant time effect, post hoc comparisons were performed using Wilcoxon signed rank tests with a corrected P value (P < .0125) to compare differences from baseline (T2). Additionally, the MDC value calculated for each PRO was used to determine if the differences between time points exceeded the error associated with the measure. Finally, we calculated Spearman correlation the (r) and the coefficient of determination (r2) to identify the relationship between instruments (P < .05). Correlation coefficients were interpreted as weak (0.01 to 0.39), moderate (0.40 to 0.69), or strong (0.70 to 1.0).18
Test-Retest Reliability and MDC of the DPA, FAAM-S, and KOOS Subscales
A total of 16 participants (7 men, 9 women) were included in this analysis. Average scores for T1 and T2 for each PRO as well as ICCs, SEMs, and MDCs for each PRO can be found in Table 2. The reliability coefficient for the KOOS-Pain was interpreted as moderate and for the remaining scales was interpreted as strong.
Effect of Administration Time Point on PRO Scores
A total of 15 participants (6 men, 9 women) were included in this analysis. Descriptive statistics for each PRO (mean ± SD) can be found in Table 3. No main effects were identified for the FAAM-S (P = .672), KOOS-ADL (P = .070), KOOS-Pain (P = .284), KOOS-Sport (P = .972), or KOOS-Symptoms (P = .714). A time effect was noted for the DPA (P = .011): scores were lower at T5 than at T2 (P = .003). When examining the individual time points using the MDC, we found that the difference between T2 and T5 did not exceed the error associated with the measure. A time effect was detected for KOOS-QOL (P = .027), but no post hoc analyses were significant.
Injuries sustained during the season were recorded by the athletic trainers for each team using an electronic medical records system. Of the 16 participants, only 1 participant was removed because of a severe injury. A total of 6 participants had documented injuries that occurred during the spring season. We qualitatively compared the data for these participants with previous and subsequent data points around the time of injury and found no noticeable deviations in PRO scores. In addition, according to the activity logs, the participants did not miss any practice, agility, weight-lifting, conditioning, or PRO sessions. Therefore, these participants remained in the analysis, and we performed no statistical modifications on their data.
Relationships Between Generic and Region-Specific PRO Instruments
A total of 16 participants (7 men, 9 women) were included in this analysis (Figure). Mean ± SD scores for each instrument at T1 can be found in Table 2. One significant moderate relationship was noted between the DPA and the KOOS-Sport (r = −0.58, r2 = 33.6%, P = .02; Table 4). No other relationships were significant.
Test-Retest Reliability and MDC
We found moderate test-retest reliability for the KOOS-Pain and strong test-retest reliability for the DPA, FAAM-S, and other 4 KOOS subscales. Therefore, our hypothesis was confirmed: these instruments have good test-retest reliability and are suitable for clinical practice from this perspective. Participants completed the PROs during 2 testing sessions, 1 week apart, so that we could assess the test-retest reliability. We chose these time points, 1 week before the start of the season (T1) and the first day of the season (T2), based on the assumptions that the participants were not involved in any soccer-related activities and that their health status would not have changed during the 1-week period.
Establishing MDC values is important when using PROs in clinical practice. Understanding the variability associated with the measure allows clinicians to determine if a change in a patient's score was related to error or to a change in health status. The MDC is the smallest amount of change that needs to occur to demonstrate true change beyond the error associated with the measurement. The MDC calculated for the DPA in this study was ±12.48, indicating that an increase or decrease of 13 points on the DPA is needed to exceed the variability associated with the measure. Limited evidence is available regarding the reliability and MDC of the FAAM-S. Previously, an MDC of 15.5% was reported19 in a college-aged population with chronic ankle instability. This MDC is higher than our MDC of 7%; we believe the difference could be attributed to injury history because we do not know if any of our participants had a history of chronic ankle instability. Finally, our reliability coefficients for the KOOS subscales, aside from that for KOOS-Pain, are similar to those published previously.8 Some of our participants had a history of injury, which can predispose them to the development of posttraumatic osteoarthritis, similar to those participants in the Roos et al8 investigation.
Effect of Administration Time Point on PRO Scores
Our findings demonstrated a main effect of time for the DPA, with a difference between the T2 and T5 time points. However, based on the MDC, the difference between the time points did not exceed the error associated with the measure. Therefore, the differences were most likely due to error and not to true change. A main effect was present for the KOOS-QOL, but post hoc analyses revealed no differences between the time points. No other main effects for time were observed in the FAAM-S or remaining KOOS subscales. Hence, our hypothesis was confirmed: the administration time point did not affect generic or region-specific PROs, which assess physical impairments, activity limitations, and overall HRQOL. Participation in the spring athletic season alone did not influence general and specific health.
A recent study20 demonstrated similar findings in an identical population over the course of a spring season. Although these authors used different region-specific instruments (the Lysholm Scale and the International Knee Documentation Committee form), statistical differences indicated increased HRQOL as measured by the Lysholm Scale and International Knee Documentation Committee form at the midseason and postseason time points compared with the preseason time point.20 However, the differences did not exceed the variability associated with the MDC.20 Therefore, when combining our results with those established previously, the changes that exceeded the variability associated with the DPA, Lysholm Scale, and International Knee Documentation Committee form appear to be related to outside factors, such as sport injury, and not to continuous athletic participation by this population over a spring soccer season.
Relationship Between Generic and Region-Specific PROs
We demonstrated a moderate relationship between the DPA and the KOOS-Sport, yet only 33.6% of the variance was explained. We believe both instruments incorporate items that assess activity limitations and participation restrictions related to physical impairment for persons who function at a high activity level, such as collegiate soccer athletes. However, given the remaining insignificant relationships identified and the unexplained 67% of the variance, clinicians treating patients with lower extremity injuries should use both generic and region-specific instruments to assess HRQOL, determine the effectiveness of their treatments, and obtain the patient's perspective when developing treatment protocols. Numerous generic and region-specific PROs can be used in athletic training clinical practice. One barrier to implementing these instruments in clinical practice is determining how many and which instruments to use. For the purposes of this research study, we used 2 lower extremity region-specific PROs and 1 generic PRO, all of which are commonly used in physically active populations. Clinicians should continue to use both types of PROs when treating patients with ankle or knee conditions to accurately assess the effectiveness of treatments and the multiple dimensions of HRQOL in their patients.
This study was not without limitations. First, we acknowledge that the data were collected during a spring soccer season. Whether or not these same phenomena would be identified during a traditional season needs to be determined. In addition, participants were asked to recall their injury history, and therefore, some recall bias may be associated with their responses. Also, we only used lower extremity region-specific PROs, as most injuries in soccer affect the lower extremity.21,22 Future researchers could incorporate an upper extremity region-specific PRO to determine the effect of time on these instruments, particularly in overhead athletes.
The DPA, FAAM-S, and KOOS subscales are reliable instruments that can be used in clinical practice to assess activity limitations and participation restrictions in collegiate athletes. Additionally, participation in the nontraditional soccer season did not affect HRQOL scores when using the generic DPA and region-specific FAAM-S or KOOS. Ideally, PROs should be administered at the start of the season to provide a baseline assessment of HRQOL for all athletes and for use in injury-prevention or screening programs. However, our results demonstrate that clinicians can obtain baseline measures on these PROs for these uses during the athletic season. Additionally, changes in HRQOL are likely associated with other factors, such as an injury that restricts participation. Finally, clinicians should continue to use both a generic and region-specific instrument to capture all dimensions of HRQOL in their patients.
The investigators acknowledge and thank Cameron Powden, MSEd, ATC, and Kara Gorgos, MSEd, ATC, for their efforts in this investigation.