Generic patient-reported outcome (PRO) instruments are designed to capture health-related quality-of-life outcomes and to determine treatment effectiveness from the patient's perspective. Multiple generic PROs are used in clinical practice, and an investigation of the psychometric properties of these instruments in a high-functioning, physically active population is important for the future use of these instruments.
To determine the relationship among 3 generic PROs: the modified Disablement in the Physically Active (mDPA) Scale, the Patient-Reported Outcome Measurement Information System Physical Function (PROMIS-PF), and the Short Form 12 (SF-12) in physically active patients seeking treatment for a lower extremity health condition.
Cross-sectional study.
Athletic training clinical facility, physical therapy clinic.
One hundred patients seeking rehabilitation services for a lower extremity health condition.
All patients completed a demographic questionnaire and the 3 generic PROs at 1 time point during their rehabilitation: the mDPA-Total, mDPA-physical summary component (mDPA-PSC), mDPA-mental summary component (mDPA-MSC), the PROMIS-PF, and SF-12 mental component summary (SF-MCS) and physical component summary (SF-PCS). Separate Spearman rank (r) correlations were performed to assess the strength of the relationship among PRO instruments. The floor and ceiling effects were also examined.
A strong relationship was present between the SF-12 PCS and the mDPA-Total (r = −0.65), the mDPA-PSC (r = −0.64), and the PROMIS-PF (r = 0.65). Significant moderate relationships were identified between the mDPA-MSC and the SF-12 PCS (r = −0.43) and MCS (r = −0.53). Weak relationships were noted between the mDPA-Total and SF-12 MCS (r = −0.21) and the SF-12 MCS and mDPA-PSC (r = −0.10) and PROMIS-PF (r = 0.20).
The PROMIS-PF and mDPA had good convergent and divergent validity. Clinicians treating physically active patients should consider these instruments for use in clinical practice. Future researchers should examine additional psychometric properties of these instruments in physically active patients.
The modified Disablement in the Physically Active (mDPA) subscales had good convergent and divergent validity with the appropriate Short Form 12 subscales.
A ceiling effect was demonstrated for the mDPA-mental summary component scores.
Clinicians should consider the use of the Patient-Reported Outcome Measurement Information System Physical Function or the mDPA in clinical practice.
Patient-reported outcome (PRO) instruments are patient-based outcome measures that can be used in clinical practice to assess the effect of musculoskeletal injury on domains of health-related quality of life (HRQOL) and assist in decision making and goal setting during the provision of patient-centered care.1–4 Importantly, these outcomes allow for the incorporation of the patient's perspective when developing the treatment plan and in documenting treatment effectiveness.4,5 Despite the multiple instruments available and the known benefits of using PROs, 52% to 74% of the clinicians who commonly rehabilitate physically active patients do not incorporate these outcomes in patient care.6,7 The literature has suggested that athletic trainers (ATs)7 and physical therapists6 encounter barriers when using PRO instruments in practice. Athletic trainers who did not use PROs in practice reported barriers such as the time needed for clinicians to score and interpret them, the time it takes to complete the instruments in general, relevance to the patients they were treating, and lack of support.7 Similar results were also documented for physical therapists who did not use PROs in practice.6
Generic PROs are general in nature and are used in clinical practice to assess the effects of musculoskeletal injury on HRQOL across different patient populations and multiple conditions.5,8,9 The Short Form 36 (SF-36) and Short Form 12 (SF-12) are generic PROs commonly used in orthopaedic and sports medicine settings. The SF-12 measures general quality of life across 8 domains and is summarized in 2 subscales: the mental component summary (MCS) and physical component summary (PCS). However, these instruments require a user license, follow an intricate scoring rubric, and are based on population norms that may limit access, friendliness, and applicability in many clinical settings. Alternatively, the Disablement in the Physically Active (DPA) Scale10,11 and the Patient-Reported Outcome Measurement Information System Physical Function (PROMIS-PF)12 are 2 generic instruments that can be used to assess global detriments and may address several of the barriers associated with PRO implementation in athletic training clinical practice. The DPA was developed by ATs for physically active populations10,11 and has since been modified to include 2 component summary scores.11 The PROMIS-PF is a computerized adaptive test that uses a question bank to generate questions based on previous responses; thus, it can reduce the number of items certain patients have to answer when completing the forms.13 These 2 alternative forms may address some of the aforementioned barriers, while providing subjective information pertinent to patient-centered care in the athletic training facility.
Although many of the psychometric properties of the DPA and PROMIS-PF have been established, additional evidence regarding the convergent and divergent validity of these instruments in typically high-functioning, physically active patients seeking treatment for a musculoskeletal injury is warranted. Specifically, determining the relationship among these instruments and the established SF-12 would further support their validity for use in athletic training clinical practice. In addition, examining the floor and ceiling effects of these instruments in a physically active population seeking treatment for a musculoskeletal injury is important when considering adoption in practice. If patients continue to score at the floor or ceiling of the instrument, it may have limited ability to measure a change in practice and therefore may not be useful in the clinic. Thus, the purpose of our study was to determine the relationship among the SF-12 subscales and the modified DPA (mDPA) subscales and PROMIS-PF in physically active adults seeking treatment for 1 lower extremity condition. We hypothesized that the instruments measuring similar constructs would be strongly correlated; thus, clinicians could begin to use other generic quality-of-life instruments, such as the PROMIS-PF or mDPA, which are more applicable to the high-functioning, physically active population cared for in traditional athletic training clinical practice.
METHODS
Design
A cross-sectional design was used to examine the relationship among the SF-12 subscales, mDPA subscales, and PROMIS-PF in physically active adults who were seeking treatment for a lower extremity musculoskeletal condition. Specifically, these correlational analyses were performed to examine convergent validity (how well the 2 instruments measure the same construct) and divergent validity (how much the 2 instruments do not measure the same construct). A secondary analysis of floor and ceiling effects was also performed.
Population
Over 1 calendar year, participants were recruited through word of mouth and informational flyers at 3 athletic training clinics and 1 physical therapy clinic, all located in close geographic proximity. A total of 100 participants were included. Their demographic and health condition information can be found in Table 1. Participants were included if they identified themselves as intercollegiate or recreational athletes, were between the ages of 18 and 35, stated they were physically active (>90 minutes of physical activity per week) before their lower extremity health condition occurred, and were seeking health care services for the lower extremity musculoskeletal condition.
Procedures
After providing informed consent, the participants completed a demographic form and 3 PROs (mDPA, PROMIS-PF, SF-12) using a tablet (16 GB iPad; Apple Inc, Cupertino, CA). All research procedures were approved by 3 institutional review boards.
Instrumentation
Demographic Form
The demographic form collected information such as age, gender, and self-reported anthropomorphic measurements. Additional information included a general history of lower extremity injury, years of participation in the current mode of physical activity, and index injury information.
The Modified Disablement in the Physically Active Scale
The original DPA was designed by ATs for use among the physically active population. It measures function in 4 domains: impairment, functional limitations, disability, and quality of life.10,14 The original DPA is reliable in patients with acute and chronic injuries10 and in athletes who are participating in a nontraditional season, regardless of injury status.15 The mDPA consists of 2 summary components: the mental summary component (MSC) and the physical summary component (PSC).11 The mDPA-MSC (Cronbach α = 0.878) and the mDPA-PSC (Cronbach α = 0.941) demonstrated excellent internal consistency.11 The 16-item mDPA is scored on a 0 to 4 Likert scale, where 0 is no problem. The mDPA-Total score ranges from 0 to 64, the mDPA-MSC score ranges from 0 to 16, and the mDPA-PSC score ranges from 0 to 48; higher scores indicate increased disablement and decreased HRQOL.
The Patient-Reported Outcome Measurement Information System Physical Function, version 1.2
The PROMIS banks are a series of computerized adaptive tests (CATs) that are tailored to the patient's responses as he or she answers each question. Based on the patient's response to an individual question, an algorithm is used to select the next most appropriate question in the item bank.13 The use of a CAT decreases the burden on patients, as they are not required to answer all questions contained in a traditional PRO, thereby limiting the need to answer questions that may not pertain to his or her condition or provide relevant information.13,16 The PROMIS-PF was administered to all patients in this study. This CAT contains a 124-question item bank that assesses the patient's ability to accomplish various levels of physical activities.17 All PROMIS questions are scored on a 5-point Likert scale, and the final score is computed online through a norm-scoring method with a mean score of 50; a lower score represents decreased HRQOL relative to physical functioning.18 Given that this is a CAT, the scores were automatically calculated and provided in the Assessment Center (Northwestern University, Evanston, IL), thereby reducing the clinician burden to score the instrument. This bank has been used among patient populations with various musculoskeletal health conditions19 : upper extremity health conditions,17 foot and ankle conditions,20 and post-anterior cruciate ligament reconstruction.21
The Short Form 12
The SF-12, version 2 (4-week recall), is a licensed cost-for-use generic PRO adapted from the SF-36.22 The SF-12 comprises 12 items that assess physical and mental status across 8 health domains and is divided into the PCS and MCS subscales.22 The SF-12 is a reliable instrument among numerous adult populations and is scored using a norm-scoring method, with lower scores representing a decrease in HRQL.23 Scores were calculated using the SF scoring software (OptumInsight Life Sciences, Inc, Lincoln, RI). There is a paucity of literature on using the SF-12 in a physically active population with musculoskeletal injury.24
Statistical Analysis
Descriptive statistics, including the median and interquartile range, were calculated for all variables. Some patients elected not to complete all of the instruments. Therefore, the number of data points available for each instrument is provided in Table 2. Due to the non-normal distribution of the data, separate Spearman rank (r) correlations were performed to determine the relationship between all outcomes, and the coefficient of determination (r2) values were calculated to examine the shared variance between outcomes. Correlation coefficients were interpreted as strong (0.7–1.0), moderate (0.40–0.69), or weak (0.01–0.39).25 We performed these analyses to examine how well the 2 instruments measured the same construct (convergent validity) and how much the 2 instruments did not measure the same construct (divergent validity). Floor and ceiling effects were also assessed for the mDPA-Total, MSC, and PSC subscales, as these have not been previously studied in a patient population seeking treatment for a lower extremity musculoskeletal condition. A floor or ceiling effect was considered present if 15% or more of the patients scored the highest or lowest score possible, respectively.26,27 Finally, given the nature of the scoring for the SF-12 and PROMIS-PF, the number of patients who scored greater than the norm value of 50 was calculated. The α level was set a priori at P < .05. All analyses were performed using SPSS (version 22; IBM Corp, Armonk, NY).
RESULTS
The medians and interquartile ranges for all outcomes can be found in Table 2 and the Spearman rank correlation analyses in Table 3. The mDPA-PSC and PROMIS were moderately correlated with the SF-12 PCS, demonstrating good convergent validity (Table 3). The mDPA-PSC and PROMIS-PF were weakly correlated with the SF-12 MCS, reflecting good divergent validity. Finally, the SF-12 MCS and mDPA-MSC were moderately correlated, showing good convergent validity.
For the mDPA-Total, no floor or ceiling effect was evident, as none of the patients who completed this instrument had the highest possible score and only 1 patient (1%) had the lowest possible score. Similarly, the mDPA-PSC displayed no floor or ceiling effect, as none of the patients had the lowest possible score, and only 2 patients (2%) had the highest possible score. Finally, for the mDPA-MSC, we observed a ceiling effect, as 22 patients (22%) scored 0; however, no floor effect was present, as only 1 patient (1%) had the highest possible score. A total of 23 patients (25%) scored greater than 50 on the SF-12 PCS, while 77% scored greater than 50 on the SF-12 MCS. As for the PROMIS-PF, 11 patients (11%) scored greater than 50.
DISCUSSION
Our results indicated good convergent and divergent validity for the mDPA-Total, mDPA-PSC, PROMIS-PF, and SF-12 PCS among physically active patients seeking treatment for a lower extremity condition. Furthermore, our results indicated the mDPA-MSC had a ceiling effect, whereas the PCS did not have ceiling or floor effects. In addition, more than 50% of the patients scored greater than 50 on the SF-12 MCS, and only 11% scored greater than 50 on the PROMIS-PF. A post hoc power analysis indicated a sample size of 88 participants would have been sufficient to achieve a power of 90% with an α level of .05 and an r value of .30.
Our findings demonstrated the PROMIS-PF and mDPA subscales had acceptable convergent and divergent validity with the appropriate SF-12 subscales. Previous researchers28 identified a strong relationship between the PROMIS-PF and SF-36 PF (r = 0.91) in a large number of participants who were healthy or unhealthy and pursued various levels of physical activity. In addition, numerous authors21,29–32 have investigated the relationship between the PROMIS-PF and other PROs commonly used when treating patients with orthopaedic conditions. For example, strong to moderate correlations were present for the QuickDASH,17 International Knee Document Committee form,21 and Oswestry Disability Index.30 However, the physical activity levels of the patients included in these studies were often not provided or were unclear. In addition to demonstrating good convergent and divergent validity and finding that only 11% of patients exceeded a score of 50, other researchers20,21,29,30 determined the PROMIS-PF took less time to administer than the other instruments. Although we did not monitor the time to completion for the instruments used in this study, the average number of questions answered by the 100 participants was 4.36 (range = 4–12) on the PROMIS-PF, compared with the 16 items on the mDPA subscales and the 12 items on the SF-12 subscales. The smaller number of items on the PROMIS-PF instrument may have resulted in less time needed for completion, which is of great importance, as patient completion time and the time needed to score and analyze are frequently reported barriers to use of these instruments in practice.7 Given the strong correlation between the SF-12 subscales and the limited number of items on the PROMIS-PF, the latter can address the aforementioned barriers and provide beneficial information to clinicians.
The DPA was developed by ATs for use among physically active populations.10 We did not examine the convergent and divergent validity of the original instrument; however, the internal consistency, test-retest reliability, concurrent validity, and responsiveness were examined during the development of the instrument.10 This instrument has been used in studies of physically active participants with chronic ankle instability32,33 and those with a history of lower extremity injury.15 The mDPA was developed to identify subcomponents: the PSC and MSC.11 The mDPA has been used sparingly in the literature among physically active patients who were being treated for a foot and ankle condition.34 Our results indicate the mDPA had good convergent and divergent validity with the appropriate SF-12 subscales, demonstrating that the instrument was measuring similar constructs in both the physical and mental domains. The mDPA has 16 items, which is the largest number of the 3 instruments, yet the questions posed on the instrument are more directly related to high-functioning, physically active patients, as it was designed for these individuals.10 In addition, the instrument has 2 distinct subscales that provide clinicians a clear picture of both the physical and mental domains of health. Furthermore, the mDPA scoring was modified to make the instrument easier to score.11 These data in combination further support the use of this instrument in the intended population. However, we must note that the mDPA-MSC displayed a ceiling effect, which occurs when an instrument can no longer measure an increase in functional performance.35 The participants who completed the mDPA-MSC often scored the highest possible value, which means we would be unable to measure an increase in these self-reported areas over time. Therefore, clinicians should be cautious when providing this instrument to their patient populations and potentially use a secondary psychological or biopsychosocial outcome measure to accurately assess the patient's mental or psychological wellbeing as it relates to the injury.
Limitations
This study is not without limitations. First, we included patients with a variety of lower extremity conditions. However, we believe this contributes to the generalizability of our findings in relation to patients often treated in outpatient sports medicine physical therapy and athletic training clinics. Second, we surveyed patients at various times during their rehabilitation process and at various times during the treatment session. Given the nature of the study and our goal of determining the relationship among the instruments, we do not feel the lack of a standardized administration time influenced the results of the PROs. However, the time point when patients were seeking treatment could have affected the floor and ceiling measures. It is possible the patients participated during the end of their rehabilitation, when we would expect them to be near or at the floor or ceiling of the outcome measures. Furthermore, we did not classify injuries as acute or chronic or assess whether the patient had a previous history of the injury being treated. Thus, ceiling and floor effects should be viewed with caution, as a patient with a previous history and a patient with a chronic condition may have different ceiling or floor measures. In addition, we did not randomize the order of PRO completion; all patients completed the PROs in the exact same order. We do not believe this influenced our results or that we had an order effect, as we do not believe the questions on the instruments influenced the patient's responses to questions on subsequent instruments. Finally, we did not measure the time to completion for these instruments; therefore, we were unable to compare time to completion for each instrument.
CONCLUSIONS
Clinicians should use PROs to determine treatment effectiveness from the patient's perspective. We determined the mDPA and PROMIS-PF had good convergent and divergent validity with the SF-12. In addition, 11% of the patients scored higher than the average score of 50 for the PROMIS-PF, whereas the mDPA-MSC appeared to have a ceiling effect. These results suggest that the PROMIS-PF and mDPA should be considered for use as generic instruments in clinical practice. However, before 1 instrument can be selected and promoted for use across all providers and health conditions, future researchers should examine the test-retest reliability, internal consistency, and patient and clinician acceptability of both instruments.
ACKNOWLEDGMENTS
A portion of these results was presented at both the Eastern Athletic Trainers' Association Annual Meeting and the National Athletic Trainers' Association Annual Meeting. This study was funded by the Old Dominion University Research Foundation.