The increased emphasis on implementing evidence-based practice has reinforced the need to more accurately assess patient improvement. Psychometrically sound, patient-reported outcome measures are essential for evaluating patient care. A patient-reported outcome instrument that may be useful for clinicians is the Disablement in the Physically Active Scale (DPAS). Before adopting this scale, however, researchers must evaluate its psychometric properties, particularly across subpopulations.
To evaluate the psychometric properties of the DPAS in a large sample using confirmatory factor analysis procedures and assess structural invariance of the scale across sex, age, injury status, and athletic status groups.
Observational study.
Twenty-two clinical sites.
Of 1445 physically active individuals recruited from multiple athletic training clinical sites, data from 1276 were included in the analysis. Respondents were either healthy or experiencing an acute, subacute, or persistent musculoskeletal injury.
A confirmatory factor analysis was performed on the full sample, and multigroup invariance testing was conducted to assess differences across sex, age, injury status, and athletic status. Given the poor model fit, alternate model generation was used to identify a more parsimonious factor structure.
The DPAS did not meet contemporary fit index recommendations or the criteria to demonstrate structural invariance. We identified an 8-item model that met the model fit recommendations using alternate model generation.
The 16-item DPAS did not meet the model fit recommendations and may not be the most parsimonious or reliable measure for assessing disablement and quality of life. Use of the 16-item DPAS across subpopulations of interest is not recommended. More examination involving a true cross-validation sample should be completed on the 8-item DPAS before this scale is adopted in research and practice.
The 16-item Disablement in the Physically Active Scale (DPAS) did not meet the model fit recommendations and may not be the most parsimonious or reliable measure for assessing disablement and quality of life.
Given the model fit concerns and measurement noninvariance, clinicians should use the DPAS with caution.
Although the DPAS Short Form-8, a more parsimonious model, was also identified in this study, more research involving a true cross-validation sample should be completed before this scale is fully adopted in research and practice.
The health care professions have experienced an increased emphasis on implementing evidence-based practice, which has reinforced the need to more accurately assess patient improvement. Practitioners often rely on clinician-generated evidence, such as strength measures, because these criteria provide objective data thought to measure changes in health status.1 However, a critique of this process is that these measures do not consider the patient experience and may not indicate meaningful improvement from a patient's perspective,2 making psychometrically sound, patient-reported outcome (PRO) measures essential for evaluating the quality of patient care and clinical practice.3,4
The need to assess improvement from the patient's perspective has led to the development of PRO instruments for assessing specific regions of the body, different conditions, or other domain-specific measures (eg, functional limitations, quality of life). Researchers have attempted to balance sensitivity with applicability,5 with the primary objective of accurately measuring change relevant to patients given their clinical circumstances and goals.4 The data produced by this process should inform patients, clinicians, and third-party entities about the quality of care provided and whether patients perceive a return to their expected levels of health.2,4 Effective use of PROs may lead to improved decision making in clinical practice and improved communication among all stakeholders.6 Furthermore, the efficient use of PROs may increase understanding of a patient's clinical circumstances and goals and guide the development of a treatment plan, thereby improving the efficiency of health care provided while allowing for greater implementation of patient-centered care.2,6–11
A PRO instrument designed specifically for physically active populations is the Disablement in the Physically Active Scale (DPAS).12,13 The DPAS is modeled after the Short Form-36 (SF-36), which is often considered a criterion-standard generic PRO instrument.12,13 The need for the DPAS was partially driven by data indicating that the SF-36 may not be an appropriate instrument for athletes or the physically active population. When comparing SF-36 scores based on physical activity level (eg, athlete versus nonathlete), investigators14,15 have reported that age-matched control populations produced different responses on the scale. Variance in responses among these populations is a common problem with PRO instruments because athletes often have lower levels of baseline disability than nonathletic patients.16,17 As a result of this difference, athletic patients frequently have lower ceilings by which to demonstrate improvement with repeated administration of PRO instruments.16 Therefore, even commonly used, valid instruments might need to be reassessed before administration in these populations if the validation process did not include high-level physical functioning patients, such as athletes.17
The DPAS, in contrast, may be a viable alternative because it was designed for and tested in athletes, with a framework for assessing the disablement process12 while using items that measure components (eg, physical, mental) similar to those found in the SF-36.18 The potential concern with using the DPAS is that few studies have been conducted on the psychometric properties of the scale, and in most of these, the authors have sampled what is considered a small, homogeneous athletic population12,18 based on contemporary recommendations for scale development.19,20 Furthermore, psychometric assessment guidelines for instrument refinement and validation have indicated that invariance testing is an important component for determining instrument validity.19,20 When the scale is used to assess patient improvement or conduct research, factorial stability must exist across different populations. An instrument intended to be administered in a heterogeneous population must have established measurement properties that are equivalent in various subgroup populations (eg, sex, age, activity level, injury classification).20 Without establishing the invariance of an instrument, one cannot assume that the items measure the underlying constructs comparably across groups, as has been found with the SF-36.16,17 Using an instrument that meets the measurement recommendations for invariance testing is important for clinicians or researchers who want to track individual changes over time, compare differences across groups (eg, active versus physically active, pediatric versus geriatric) at a single time or with repeated measures, or assess treatment effectiveness based on patient outcomes measured using the instrument.19–21
Currently, this level of psychometric analysis has not been conducted on the DPAS. The measurement properties of the scale have never been evaluated to determine how sex, age, activity level, or stage of musculoskeletal injury (ie, acute, subacute, or chronic) of respondents completing the scale affect model fit. In addition, further psychometric assessment of overall model fit is needed because of the limited study22 to date of the scale involving a large, heterogeneous sample. Therefore, the purposes of our study were to evaluate the psychometric properties of the DPAS in a large sample using confirmatory factor analysis (CFA) procedures and assess invariance (ie, equal factor variances, equal factor covariance, and equal means) of the scale across sex, age, injury status (injured or healthy), and athletic status (competitive, recreational, or occupational athlete).
METHODS
Participants
A convenience sample of athletic trainers recruited participants from 20 athletic training clinics and 2 outpatient rehabilitation clinics in the United States for 1 year. The athletic trainers recruited competitive, recreational, and occupational athletes who were injured, as well as those who were uninjured, to volunteer for the study. Volunteers who were physically active were included in the study; those with chronic pain were excluded (Table 1).12,22,23 Individuals who were not in the specified age group (age range = 14–40 years) or were not in one of the specified athletic status classifications (competitive, recreational, or occupational) were also excluded from data collection. Participants were grouped by predefined physical activity categories (competitive athlete [eg, National Collegiate Athletic Association student-athlete], recreational athlete [eg, an individual who is active on a university-sponsored recreation team], occupational athlete, [eg, an individual who engages in weight lifting]) that established the athletic status classification (Table 2).
All participants provided written informed consent, and the study was approved by our institutional review board. When applicable, minors provided assent, and their legal guardians provided consent before data collection.
Instrumentation
The survey packet consisted of the DPAS and a demographic questionnaire that was completed at the initial visit with an athletic trainer. The DPAS is a PRO scale thought to measure 4 factors using 16 items. The first 3 constructs—impairments, functional limitations, and disability—are first-order factors of the higher-order construct, disablement. Items 1 through 4 are designed to tap into the impairment construct, items 5 through 9 into the functional limitations construct, and items 10 through 12 into the disability construct. Items 13 through 16 are designed to address the quality-of-life construct, which is hypothesized to covary with the higher-order disablement construct (Figure 1). Respondents rate each item on a Likert scale that ranges from 1 (no problem) to 5 (severely affected). The scores for each item are summed, and 16 points are subtracted from the summed total to produce a final score. A person's total score ranges from 0 (ie, floor) to 64 points.12
Hypothesized model of the Disablement in the Physically Active Scale (DPAS). Abbreviations: d, disturbance variable; v, unique variance.
Hypothesized model of the Disablement in the Physically Active Scale (DPAS). Abbreviations: d, disturbance variable; v, unique variance.
The athletic trainer working with the participant collected deidentified participant characteristics that consisted of injury category (ie, healthy, acute, subacute, or persistent), athletic status (eg, competitive, recreational, or occupational athlete), age, sex, sport, general injury location (ie, lower extremity, spine, upper extremity), specific injury location (eg, head or neck, shoulder or arm, ankle or foot), and type of injury (eg, arthritis, neuroma, strain, sprain, postsurgery). The athletic trainer was permitted to explain terms or phrases used on the DPAS (eg, cardiovascular endurance) or characteristic items and to answer questions if confusion arose while participants completed the paperwork; this type of clinician-patient communication is expected when providing effective patient-centered care. The collected DPAS data and characteristic data were entered into Qualtrics software (Qualtrics, LLC, Provo, UT) by the collecting athletic trainer.
Data Analysis
Data Cleaning
Data were downloaded from Qualtrics for analysis using SPSS (version 25.0; IBM Corp, Armonk, NY) and Analysis of Moment Structure (AMOS; version 25.0; IBM Corp). Missing data were treated conservatively, and data from respondents who did not answer at least 90% (15/16) of the DPAS items were removed from the dataset. Any remaining missing data were replaced with the rounded mean score of the respective item for analysis.19 However, missing demographic data were left as missing values. Data cleaning involved assessment of the univariate distributions of all variables to verify normal distribution with low levels of skewness and kurtosis. Multivariate outliers were identified using the Mahalanobis distance at P < .001.19,24,25
Confirmatory Factor Analysis
A CFA on the second-order DPAS was conducted in AMOS software. In the second-order model, the 3 latent variables—impairments, functional limitations, and disability—created a second-order latent variable, disablement, which then was covaried with the first-order variable, quality of life. An additional CFA was conducted on the first-order model that removed the second-order latent variable, disablement, and instead, covaried all 4 first-order latent variables (ie, impairments, functional limitations, disability, and quality of life). Conducting a first-order CFA model allowed assessment of correlations among the latent constructs of the DPAS, which was not possible in a second-order model. For both CFA models, goodness-of-fit indices were evaluated using contemporary a priori values. The more contemporary model fit indices evaluated were the Comparative Fit Index (CFI; ≥0.95), Tucker-Lewis Index (TLI; ≥0.95), root mean square error of approximation (RMSEA; ≤0.06), and Bollen Incremental Fit Index (IFI; ≥0.95).19,26,27 Additionally, modification indices were examined to identify local fit concerns (eg, cross-loadings) and potential model misspecification. The likelihood ratio statistic (χ2 statistic) was also calculated, but because it is heavily influenced by sample size, it was not used as the primary assessment of model fit.19,20
Invariance Testing
The multigroup invariance analysis was conducted using AMOS. Confirmatory factor analysis invariance testing (ie, configural, metric, and scalar) was applied to assess model fit across groups.19,20 Model fit was compared using the CFI difference test (CFIDIFF), with a cutoff of 0.01, and the χ2 difference test (χ2DIFF), with a cutoff of P = .01.19,28 Given the sensitivity of the χ2DIFF to sample size,28 the CFIDIFF held greater weight in our decisions regarding model fit. If a model exceeded the χ2DIFF but met the CFIDIFF, invariance testing continued. Structural invariance for the DPAS was assessed across groups by sex, age according to development stages (adolescents, aged 14–18 years; emerging adulthood, aged 19–25 years; and early adulthood, aged 25–40 years),29 injury status (injured or healthy), and athletic status (ie, competitive athlete and combined recreational or occupational athletes).
Alternate Model Generation
Because the model fit for the 16-item DPA scale did not meet contemporary recommendations,19,26,27 we performed alternate model generation using AMOS to identify a more parsimonious factor structure. Modification indices and factor loadings, along with assessment of the items and theoretical fit, were used to guide item removal.19,20,30 We also used bivariate correlations between items and Cronbach α to guide decisions about item removal within subdimensions.19,31,32 One modification was made at a time, and global and local fit were assessed after item removal. The final model fit was assessed using the recommendations19,26,27 described in the CFA procedures. A bivariate correlation analysis was conducted on the composite scores of the DPAS and the new model to determine if the proposed model explained an acceptable amount of the variance in responses on the DPAS; an acceptable percentage of the variance explained was set at r ≥ 0.90 (R2 = 0.81).33
RESULTS
Preliminary Analysis
A total of 1445 individuals took the survey. Thirty-six individuals were missing responses to >10% of the DPAS items and their data were, therefore, removed from the dataset. A total of 133 individuals reported scores that indicated either univariate (z scores ≥ 3.4) or multivariate (Mahalanobis distance ≥ 32.0) outliers. Thus, data from 1276 participants (667 [52.3%] males, 600 [47.0%] females; age = 20.8 ± 4.4 years) remained for analysis. Participants were grouped by sex, age, injury classification, and athletic status (Table 3).
Confirmatory Factor Analysis
The CFA of the 16-item DPAS indicated marginal but not preferred model fit to the sample data. The goodness-of-fit indices approached but did not meet recommended values (CFI = 0.936, TLI = 0.923, RMSEA = 0.081, IFI = 0.936; Figure 2). Factor loadings from individual items to their respective first-order latent variables ranged from 0.60 to 0.89. Path coefficients between the second-order latent variable, disablement, and the first-order latent variables were very high, ranging from 0.94 to 1.02 (Figure 2). Correlations among the first-order latent variables—impairment, functional limitations, and disability—were very high, ranging from 0.89 to 0.97 (Figure 3). Additionally, modification indices indicated that several items had meaningful cross-loadings and suggested alternative specifications that could maximize fit and parsimony.
Confirmatory factor analysis of the 16-item Disablement in the Physically Active (DPA) scale. Abbreviations: d, disturbance variable; v, unique variance. Adapted with permission.12,22 χ2 = 927.755; degrees of freedom = 100; χ2 statistic/degrees of freedom ratio = 9.278; Comparative Fit Index = 0.936; Tucker-Lewis Index = 0.923; root mean square error of approximation = 0.081; Bollen Incremental Fit Index = 0.936; P < .001.
Confirmatory factor analysis of the 16-item Disablement in the Physically Active (DPA) scale. Abbreviations: d, disturbance variable; v, unique variance. Adapted with permission.12,22 χ2 = 927.755; degrees of freedom = 100; χ2 statistic/degrees of freedom ratio = 9.278; Comparative Fit Index = 0.936; Tucker-Lewis Index = 0.923; root mean square error of approximation = 0.081; Bollen Incremental Fit Index = 0.936; P < .001.
First-order confirmatory factor analysis of the 16-item Disablement in the Physically Active Scale (DPAS). Abbreviations: d, disturbance variable; v, unique variance. Adapted with permission.12,22 χ2 = 904.827; degrees of freedom = 98; χ2 statistic/degrees of freedom ratio = 9.233; Comparative Fit Index = 0.937; Tucker-Lewis Index = 0.923; root mean square error of approximation = 0.080; Bollen Incremental Fit Index = 0.937; P < .001.
First-order confirmatory factor analysis of the 16-item Disablement in the Physically Active Scale (DPAS). Abbreviations: d, disturbance variable; v, unique variance. Adapted with permission.12,22 χ2 = 904.827; degrees of freedom = 98; χ2 statistic/degrees of freedom ratio = 9.233; Comparative Fit Index = 0.937; Tucker-Lewis Index = 0.923; root mean square error of approximation = 0.080; Bollen Incremental Fit Index = 0.937; P < .001.
Invariance Analysis
Sex Subgroups
In the sample, 1267 participants reported their sex (male = 667, female = 600), and these data were included in the analysis. Baseline models for males and females indicated acceptable but not ideal model fit (CFI = 0.931 and 0.943, respectively), with both groups exceeding the RMSEA cutoff of ≤0.06 (Table 4). The initial model (equal form) demonstrated acceptable but not ideal fit (CFI = 0.936; χ2 = 1001.70; RMSEA = 0.057; Table 4). The metric model (ie, equal loadings) met both the CFIDIFF and the χ2DIFF (CFI = 0.936; χ2 = 1019.32). Satisfactory metric invariance criteria warranted examination of the equal first-order latent variables. Both the CFIDIFF and χ2DIFF criteria were met (Table 4); therefore, males and females exhibited similar variability on the first-order DPAS latent variables. The scalar model (ie, equal indicator intercepts) exceeded the χ2DIFF but did meet the criteria for the CFIDIFF, indicating that it was invariant and allowed for assessment of means between groups. The equal latent means slightly exceeded the χ2DIFF but did meet the CFIDIFF, indicating no apparent differences between levels of disablement and quality of life between males and females.
Age Subgroups
In the sample, 1151 individuals reported their ages, and these data were included in the analysis. Individuals were categorized into 1 of 3 age groups: ages 14 to 18 years (n = 332), ages 19 to 24 years (n = 665), or ages 25 to 40 years (n = 154). Baseline models for age groups indicated acceptable but not ideal model fit (CFI range = 0.924–0.940); all groups exceeded the RMSEA cutoff of ≤0.06 (Table 5). The initial equal form model again demonstrated acceptable but not ideal fit (CFI = 0.937; χ2 = 1036.04; RMSEA = 0.047; Table 5). The metric model (ie, equal loadings) met both the CFIDIFF and the χ2DIFF. Satisfactory metric invariance criteria warranted examination of the equal latent variances. Both the CFIDIFF and χ2DIFF criteria were met (Table 5); thus, all age groups exhibited similar variability on the DPAS first-order latent variables. The scalar model (ie, equal indicator intercepts) exceeded the χ2DIFF but did meet the CFIDIFF. The invariant scalar model warranted assessment of the means between groups. The equal latent means slightly exceeded the χ2DIFF but did meet the CFIDIFF, indicating similar levels of disablement and quality of life among age groups.
Injury Classification Subgroups
In the sample, 1271 reported their injury classification, and these data were included in the analysis. Individuals were categorized into 2 groups: injured (n = 1035) or healthy (n = 236). Baseline models for injury status indicated less than ideal fit: neither group met the RMSEA cutoff of ≤0.06 or the CFI cutoff of ≥0.95, with model fit slightly exceeding a CFI of >0.90 (Table 6). The initial equal form model demonstrated less than ideal fit (CFI = 0.905; χ2 = 1002.19; RMSEA = 0.057; Table 6). The metric model (ie, equal loadings) slightly exceeded the χ2DIFF but did meet the CFIDIFF.
Satisfactory metric invariance criteria warranted examination of the equal latent variances of the DPAS. When testing equal factor variances, both the CFIDIFF and χ2DIFF noninvariant criteria were exceeded (Table 6). When variances were not constrained to be equal, the injured subsample exhibited substantially more variability of the first-order latent variables, impairment (healthy variance = 0.18, injured variance = 0.23), functional limitations (healthy variance = 0.20, injured variance = 0.58), disability (healthy variance = 0.12, injured = 0.49), and quality of life (healthy variance = 0.31, injured = 0.61). The scalar model (ie, equal indicator intercepts) exceeded the CFIDIFF and the χ2DIFF, suggesting item-level bias among injury classification subgroups (Table 6).
Athletic Status Classification Subgroups
In the sample, 1271 reported their athletic status classification. and these data were included in the analysis. Individuals were categorized into 2 groups: competitive athlete (n = 677) or combined recreational or occupational athlete (n = 594). Baseline models for athletic status classification indicated acceptable but not ideal fit, with both groups exceeding the RMSEA cutoff of ≤0.06 (Table 7). The initial equal form model demonstrated acceptable but not ideal fit (CFI = 0.937; χ2 = 1004.69; RMSEA = 0.057; Table 7). The metric model (ie, equal loadings) met both the CFIDIFF and χ2DIFF.
Satisfactory metric invariance criteria warranted examination of the equal latent variances. The χ2DIFF was exceeded; however, the CFIDIFF noninvariant criteria were met (Table 7), indicating that both athletic classification groups exhibited similar variability on the first-order latent variables of the DPAS. The scalar model (ie, equal indicator intercepts) exceeded the χ2DIFF but did meet the CFIDIFF criteria. The invariant scalar model warranted assessment of the means between groups. The equal latent means slightly exceeded the χ2DIFF but did meet the CFIDIFF test, indicating similar levels of disablement and quality of life among athletic classification subgroups.
Alternate Model Generation
The CFA of the 16-item DPAS indicated marginal fit to the data (CFI = 0.936, TLI = 0.923, RMSEA = 0.081, IFI = 0.936; Figure 2); therefore, we used alternate model generation to create a more parsimonious scale. Assessment of modification indices suggested several meaningful cross-loadings and alternative specifications that could maximize fit and parsimony. Given the high correlation values (>0.95), the impairments, functional limitations, and disability factor items were combined into 1 factor, renamed physical. The final solution resulted from the removal of 8 items, consisted of 2 factors (ie, physical and quality of life) that contained 4 items each, and met recommended goodness-of-fit indices (CFI = 0.993, TLI = 0.989, RMSEA = 0.036, IFI = 0.993; Figure 4). All factor loadings were different, ranging from 0.69 to 0.86, and the correlation between physical and quality of life was 0.40 (P < .001). The Cronbach α for both factors met recommended values (physical: α = .84; quality of life: α = .83). Cumulative scores for the 8-item DPAS were highly correlated (r = 0.98, P < .001, R2 = 0.96) with the cumulative scores for the 16-item DPAS.
First-order 8-item alternative Disablement in the Physically Active Scale (DPAS) model. Abbreviations: d, disturbance variable; v, unique variance. Adapted with permission.12,22 χ2 = 49.718; degrees of freedom = 19; χ2 statistic/degrees of freedom ratio = 2.617; Comparative Fit Index = 0.993; Tucker-Lewis Index = 0.989; root mean square error of approximation = 0.036; Bollen Incremental Fit Index = 0.993; P < .001.
First-order 8-item alternative Disablement in the Physically Active Scale (DPAS) model. Abbreviations: d, disturbance variable; v, unique variance. Adapted with permission.12,22 χ2 = 49.718; degrees of freedom = 19; χ2 statistic/degrees of freedom ratio = 2.617; Comparative Fit Index = 0.993; Tucker-Lewis Index = 0.989; root mean square error of approximation = 0.036; Bollen Incremental Fit Index = 0.993; P < .001.
DISCUSSION
The purpose of our study was to assess the psychometric properties of the 16-item DPAS, a frequently used clinical assessment tool. The psychometric properties were evaluated in a large, heterogeneous sample across sex, age, injury classification, and athletic status groups. We used contemporary CFA and structural equation modeling procedures to more rigorously examine the DPAS for model fit and multigroup invariance.19,28 Our results suggested that the 16-item DPAS does not meet contemporary fit index recommendations, does not meet the criteria to demonstrate invariance, and should be modified to produce a more parsimonious and psychometrically sound instrument that will improve instrument precision and reduce the item-response burden for clinicians and patients.
Confirmatory Factor Analysis
The overall findings of our CFA are consistent with previous results,22 indicating that the 16-item scale does not meet model fit recommendations, and the solution presents with model misspecification and constructs that have multicollinearity bordering on singularity.19,20,24 Therefore, item removal is necessary to effectively measure the unique subconstructs (ie, impairments, functional limitations, and disability) of disablement8,12,34 that were originally intended in the creation of the DPAS.12,35 Without item modification (eg, item removal, rewriting items to improve clarity, reducing overlapping examples for patients),36 items from different subconstructs will continue to measure the same phenomenon, or participants will continue to struggle to interpret the intended differences among the items.
Invariance Testing
Invariance testing may be conducted for a number of reasons, such as to ensure an instrument's items are being interpreted similarly across groups (eg, injured and healthy individuals), underlying constructs (eg, functional limitation, disablement) are being measured similarly across groups, or measurement properties are being maintained across repeated measures.19,28 To our knowledge, we are the first to assess any of the invariance procedures involving the DPAS. The initial (eg, CFA) results indicated that the model did not meet more stringent goodness-of-fit standards that would typically prohibit invariance testing under the more contemporary guidelines.19,28 Given the common use of the DPAS in clinical practice, conducting invariance testing using less rigorous criteria was deemed relevant. The DPAS met the less rigorous invariance criteria (ie, the underlying constructs were being measured and interpreted similarly) across age, sex, and athletic status groups but did not meet the criteria for injury status. The DPAS was not invariant in this analysis, suggesting that the disablement and quality-of-life constructs and items did not have the same meaning across group membership (ie, healthy and injured). The differences may result from the items or examples included with the items having different meanings to people, depending on their current health status. Therefore, differences in scores (ie, healthy people had lower levels of disablement than injured people) based on this group membership cannot be solely attributed to injury status. The results indicated that the differences were being confounded by measurement artifact (eg, item interpretation, error), and drawing inferences based on group membership differences (ie, healthy versus injured) is not supported.19
Additionally, although testing between groups appeared to pass the less stringent criteria, several concerns with the DPAS were raised during the process. Acceptable but not preferred model fit indices were observed for all groups (ie, no models met the more stringent global fit guidelines: CFI ≥ 0.95, TLI ≥ 0.95, RMSEA ≤ 0.06, IFI ≥ 0.95),19,27 and a number of concerns were present (eg, high cross-loadings, high correlations among first-order latent variables) within the models. Although we did not modify the 16-item DPAS models (eg, releasing a constraint) during our invariance testing procedures, researchers could assess individual items and constructs to identify the most problematic items, remove problematic items, and attempt to improve model fit through the invariance testing process.19,20
Alternate Model Generation
Using alternate model generation, we identified a more parsimonious model that met the more stringent model fit standards.19,27 The 8-item DPAS exceeded the model fit recommendations19,20 and matched the proposed DPAS Short Form-8 that has been previously published.22 The final model contained 2 factors, physical and quality of life. The physical factor combined items originally in the impairment and functional limitations factors. All items from the disability factor were eliminated, whereas all 4 items in the quality-of-life factor were retained. The problems noted in the 16-item CFA and multigroup invariance (eg, high cross-loadings, high correlations among first-order latent variables) tests were not present in the 8-item version. Furthermore, despite the number of items being reduced by 50%, the 8-item version accounted for an acceptable amount of the variance in participants' responses on the DPAS. The high correlation (r = 0.98, P < .001, R2 = 0.96) between the 2 versions of the scale indicated substantial overlap between them and provided evidence that the short-form version captures the same information as the complete scale. The correlation value in our study exceeded the previously published correlation value (r = 0.94, P ≤ .001, R2 = 0.88) between the DPAS and the DPAS Short Form-8.22
Implications for Clinical Practice and Research
The effective use of PROs in clinical practice requires instruments that measure multiple constructs relevant to patient care (eg, health status, functional limitations)7,12,34 ; however, the constructs should be unique, and the scale should produce precise measurement of those constructs.19,20 Clinically, the identification of an 8-item instrument was relevant because the solution resolved the model fit concerns found in the original DPAS, improved the measurement precision for assessing the proposed constructs, had substantial overlap with the original scale, and reduced barriers (eg, contained fewer items) for clinicians and patients.4,22,37 Our invariance results also provided insight regarding differences between certain groups (ie, healthy versus injured) that are clinically relevant for using the DPAS to track health improvements. The findings supplied preliminary evidence that differences between certain groups may be related to the instrument (eg, measurement error) and not the respondents' stages of health (ie, health improvements over time). Further research (eg, longitudinal invariance testing) is needed to assess group differences using repeated measures and ensure that the changes experienced result from treatment outcomes indicating improved health and not measurement error.19,20 In addition, our results support previous findings18,22 that showed the distinctness of the quality-of-life and disablement constructs, which indicates clinicians should examine construct scores individually and not as a summative measure of disablement when the quality-of-life and disablement constructs are unique dimensions within the scale.
Limitations and Future Research
Although we tested a fairly diverse population, the participants were recruited from a relatively small group of clinics from across the United States. Further, our convenience sample primarily consisted of young, physically active participants; therefore, we do not know if the DPAS is invariant in other (eg, geriatric) populations who were not tested. Additionally, fairly conservative standards were used to assess model fit19,28 ; however, none of the models would have met the stricter recommendation guidelines. Whereas our invariance testing results identified potential concerns with the DPAS, further invariance testing (eg, longitudinal testing) is needed to fully elucidate the potential group differences and scale precision for measuring health status improvement over time. Yet future attention should likely be focused on the modified DPAS (ie, DPAS Short Form-8) because it does not have the same concerns as identified in the original DPAS by previous authors22 or us. The DPAS Short Form-8 model22 was also identified using alternate model generation in our sample. Whereas the 8-item version is promising, a true cross-validation analysis with invariance testing is still needed before adoption. Therefore, the DPAS Short Form-8 findings from our study should be interpreted with caution until more psychometric analyses (eg, invariance testing, longitudinal testing) are conducted in a cross-validation sample.30 Future researchers should identify whether the model meets the strict criteria when participants respond to only the 8 items and if the new model meets the recommended guidelines for invariance testing.
CONCLUSIONS
The 16-item DPAS did not meet model fit recommendations and may not be the most parsimonious or reliable measure for assessing disablement and quality of life. Given the model fit concerns and measurement noninvariance, clinicians should use the DPAS with caution. Whereas we also identified the DPAS Short Form-8, a more parsimonious model, additional research in a true cross-validation sample should be completed before this scale is fully adopted in research and clinical practice.