Context

The increased emphasis on implementing evidence-based practice has reinforced the need to more accurately assess patient improvement. Psychometrically sound, patient-reported outcome measures are essential for evaluating patient care. A patient-reported outcome instrument that may be useful for clinicians is the Disablement in the Physically Active Scale (DPAS). Before adopting this scale, however, researchers must evaluate its psychometric properties, particularly across subpopulations.

Objective

To evaluate the psychometric properties of the DPAS in a large sample using confirmatory factor analysis procedures and assess structural invariance of the scale across sex, age, injury status, and athletic status groups.

Design

Observational study.

Setting

Twenty-two clinical sites.

Patients or Other Participants

Of 1445 physically active individuals recruited from multiple athletic training clinical sites, data from 1276 were included in the analysis. Respondents were either healthy or experiencing an acute, subacute, or persistent musculoskeletal injury.

Main Outcome Measure(s)

A confirmatory factor analysis was performed on the full sample, and multigroup invariance testing was conducted to assess differences across sex, age, injury status, and athletic status. Given the poor model fit, alternate model generation was used to identify a more parsimonious factor structure.

Results

The DPAS did not meet contemporary fit index recommendations or the criteria to demonstrate structural invariance. We identified an 8-item model that met the model fit recommendations using alternate model generation.

Conclusions

The 16-item DPAS did not meet the model fit recommendations and may not be the most parsimonious or reliable measure for assessing disablement and quality of life. Use of the 16-item DPAS across subpopulations of interest is not recommended. More examination involving a true cross-validation sample should be completed on the 8-item DPAS before this scale is adopted in research and practice.

Key Points
  • The 16-item Disablement in the Physically Active Scale (DPAS) did not meet the model fit recommendations and may not be the most parsimonious or reliable measure for assessing disablement and quality of life.

  • Given the model fit concerns and measurement noninvariance, clinicians should use the DPAS with caution.

  • Although the DPAS Short Form-8, a more parsimonious model, was also identified in this study, more research involving a true cross-validation sample should be completed before this scale is fully adopted in research and practice.

The health care professions have experienced an increased emphasis on implementing evidence-based practice, which has reinforced the need to more accurately assess patient improvement. Practitioners often rely on clinician-generated evidence, such as strength measures, because these criteria provide objective data thought to measure changes in health status.1  However, a critique of this process is that these measures do not consider the patient experience and may not indicate meaningful improvement from a patient's perspective,2  making psychometrically sound, patient-reported outcome (PRO) measures essential for evaluating the quality of patient care and clinical practice.3,4 

The need to assess improvement from the patient's perspective has led to the development of PRO instruments for assessing specific regions of the body, different conditions, or other domain-specific measures (eg, functional limitations, quality of life). Researchers have attempted to balance sensitivity with applicability,5  with the primary objective of accurately measuring change relevant to patients given their clinical circumstances and goals.4  The data produced by this process should inform patients, clinicians, and third-party entities about the quality of care provided and whether patients perceive a return to their expected levels of health.2,4  Effective use of PROs may lead to improved decision making in clinical practice and improved communication among all stakeholders.6  Furthermore, the efficient use of PROs may increase understanding of a patient's clinical circumstances and goals and guide the development of a treatment plan, thereby improving the efficiency of health care provided while allowing for greater implementation of patient-centered care.2,611 

A PRO instrument designed specifically for physically active populations is the Disablement in the Physically Active Scale (DPAS).12,13  The DPAS is modeled after the Short Form-36 (SF-36), which is often considered a criterion-standard generic PRO instrument.12,13  The need for the DPAS was partially driven by data indicating that the SF-36 may not be an appropriate instrument for athletes or the physically active population. When comparing SF-36 scores based on physical activity level (eg, athlete versus nonathlete), investigators14,15  have reported that age-matched control populations produced different responses on the scale. Variance in responses among these populations is a common problem with PRO instruments because athletes often have lower levels of baseline disability than nonathletic patients.16,17  As a result of this difference, athletic patients frequently have lower ceilings by which to demonstrate improvement with repeated administration of PRO instruments.16  Therefore, even commonly used, valid instruments might need to be reassessed before administration in these populations if the validation process did not include high-level physical functioning patients, such as athletes.17 

The DPAS, in contrast, may be a viable alternative because it was designed for and tested in athletes, with a framework for assessing the disablement process12  while using items that measure components (eg, physical, mental) similar to those found in the SF-36.18  The potential concern with using the DPAS is that few studies have been conducted on the psychometric properties of the scale, and in most of these, the authors have sampled what is considered a small, homogeneous athletic population12,18  based on contemporary recommendations for scale development.19,20  Furthermore, psychometric assessment guidelines for instrument refinement and validation have indicated that invariance testing is an important component for determining instrument validity.19,20  When the scale is used to assess patient improvement or conduct research, factorial stability must exist across different populations. An instrument intended to be administered in a heterogeneous population must have established measurement properties that are equivalent in various subgroup populations (eg, sex, age, activity level, injury classification).20  Without establishing the invariance of an instrument, one cannot assume that the items measure the underlying constructs comparably across groups, as has been found with the SF-36.16,17  Using an instrument that meets the measurement recommendations for invariance testing is important for clinicians or researchers who want to track individual changes over time, compare differences across groups (eg, active versus physically active, pediatric versus geriatric) at a single time or with repeated measures, or assess treatment effectiveness based on patient outcomes measured using the instrument.1921 

Currently, this level of psychometric analysis has not been conducted on the DPAS. The measurement properties of the scale have never been evaluated to determine how sex, age, activity level, or stage of musculoskeletal injury (ie, acute, subacute, or chronic) of respondents completing the scale affect model fit. In addition, further psychometric assessment of overall model fit is needed because of the limited study22  to date of the scale involving a large, heterogeneous sample. Therefore, the purposes of our study were to evaluate the psychometric properties of the DPAS in a large sample using confirmatory factor analysis (CFA) procedures and assess invariance (ie, equal factor variances, equal factor covariance, and equal means) of the scale across sex, age, injury status (injured or healthy), and athletic status (competitive, recreational, or occupational athlete).

Participants

A convenience sample of athletic trainers recruited participants from 20 athletic training clinics and 2 outpatient rehabilitation clinics in the United States for 1 year. The athletic trainers recruited competitive, recreational, and occupational athletes who were injured, as well as those who were uninjured, to volunteer for the study. Volunteers who were physically active were included in the study; those with chronic pain were excluded (Table 1).12,22,23  Individuals who were not in the specified age group (age range = 14–40 years) or were not in one of the specified athletic status classifications (competitive, recreational, or occupational) were also excluded from data collection. Participants were grouped by predefined physical activity categories (competitive athlete [eg, National Collegiate Athletic Association student-athlete], recreational athlete [eg, an individual who is active on a university-sponsored recreation team], occupational athlete, [eg, an individual who engages in weight lifting]) that established the athletic status classification (Table 2).

Table 1

Inclusion and Exclusion Criteria for Participant Activity Level, Injury, and Pain Type

Inclusion and Exclusion Criteria for Participant Activity Level, Injury, and Pain Type
Inclusion and Exclusion Criteria for Participant Activity Level, Injury, and Pain Type
Table 2

Definitions for Participant Athletic Status Stratificationa

Definitions for Participant Athletic Status Stratificationa
Definitions for Participant Athletic Status Stratificationa

All participants provided written informed consent, and the study was approved by our institutional review board. When applicable, minors provided assent, and their legal guardians provided consent before data collection.

Instrumentation

The survey packet consisted of the DPAS and a demographic questionnaire that was completed at the initial visit with an athletic trainer. The DPAS is a PRO scale thought to measure 4 factors using 16 items. The first 3 constructs—impairments, functional limitations, and disability—are first-order factors of the higher-order construct, disablement. Items 1 through 4 are designed to tap into the impairment construct, items 5 through 9 into the functional limitations construct, and items 10 through 12 into the disability construct. Items 13 through 16 are designed to address the quality-of-life construct, which is hypothesized to covary with the higher-order disablement construct (Figure 1). Respondents rate each item on a Likert scale that ranges from 1 (no problem) to 5 (severely affected). The scores for each item are summed, and 16 points are subtracted from the summed total to produce a final score. A person's total score ranges from 0 (ie, floor) to 64 points.12 

Figure 1

Hypothesized model of the Disablement in the Physically Active Scale (DPAS). Abbreviations: d, disturbance variable; v, unique variance.

Figure 1

Hypothesized model of the Disablement in the Physically Active Scale (DPAS). Abbreviations: d, disturbance variable; v, unique variance.

Close modal

The athletic trainer working with the participant collected deidentified participant characteristics that consisted of injury category (ie, healthy, acute, subacute, or persistent), athletic status (eg, competitive, recreational, or occupational athlete), age, sex, sport, general injury location (ie, lower extremity, spine, upper extremity), specific injury location (eg, head or neck, shoulder or arm, ankle or foot), and type of injury (eg, arthritis, neuroma, strain, sprain, postsurgery). The athletic trainer was permitted to explain terms or phrases used on the DPAS (eg, cardiovascular endurance) or characteristic items and to answer questions if confusion arose while participants completed the paperwork; this type of clinician-patient communication is expected when providing effective patient-centered care. The collected DPAS data and characteristic data were entered into Qualtrics software (Qualtrics, LLC, Provo, UT) by the collecting athletic trainer.

Data Analysis

Data Cleaning

Data were downloaded from Qualtrics for analysis using SPSS (version 25.0; IBM Corp, Armonk, NY) and Analysis of Moment Structure (AMOS; version 25.0; IBM Corp). Missing data were treated conservatively, and data from respondents who did not answer at least 90% (15/16) of the DPAS items were removed from the dataset. Any remaining missing data were replaced with the rounded mean score of the respective item for analysis.19  However, missing demographic data were left as missing values. Data cleaning involved assessment of the univariate distributions of all variables to verify normal distribution with low levels of skewness and kurtosis. Multivariate outliers were identified using the Mahalanobis distance at P < .001.19,24,25 

Confirmatory Factor Analysis

A CFA on the second-order DPAS was conducted in AMOS software. In the second-order model, the 3 latent variables—impairments, functional limitations, and disability—created a second-order latent variable, disablement, which then was covaried with the first-order variable, quality of life. An additional CFA was conducted on the first-order model that removed the second-order latent variable, disablement, and instead, covaried all 4 first-order latent variables (ie, impairments, functional limitations, disability, and quality of life). Conducting a first-order CFA model allowed assessment of correlations among the latent constructs of the DPAS, which was not possible in a second-order model. For both CFA models, goodness-of-fit indices were evaluated using contemporary a priori values. The more contemporary model fit indices evaluated were the Comparative Fit Index (CFI; ≥0.95), Tucker-Lewis Index (TLI; ≥0.95), root mean square error of approximation (RMSEA; ≤0.06), and Bollen Incremental Fit Index (IFI; ≥0.95).19,26,27  Additionally, modification indices were examined to identify local fit concerns (eg, cross-loadings) and potential model misspecification. The likelihood ratio statistic (χ2 statistic) was also calculated, but because it is heavily influenced by sample size, it was not used as the primary assessment of model fit.19,20 

Invariance Testing

The multigroup invariance analysis was conducted using AMOS. Confirmatory factor analysis invariance testing (ie, configural, metric, and scalar) was applied to assess model fit across groups.19,20  Model fit was compared using the CFI difference test (CFIDIFF), with a cutoff of 0.01, and the χ2 difference test (χ2DIFF), with a cutoff of P = .01.19,28  Given the sensitivity of the χ2DIFF to sample size,28  the CFIDIFF held greater weight in our decisions regarding model fit. If a model exceeded the χ2DIFF but met the CFIDIFF, invariance testing continued. Structural invariance for the DPAS was assessed across groups by sex, age according to development stages (adolescents, aged 14–18 years; emerging adulthood, aged 19–25 years; and early adulthood, aged 25–40 years),29  injury status (injured or healthy), and athletic status (ie, competitive athlete and combined recreational or occupational athletes).

Alternate Model Generation

Because the model fit for the 16-item DPA scale did not meet contemporary recommendations,19,26,27  we performed alternate model generation using AMOS to identify a more parsimonious factor structure. Modification indices and factor loadings, along with assessment of the items and theoretical fit, were used to guide item removal.19,20,30  We also used bivariate correlations between items and Cronbach α to guide decisions about item removal within subdimensions.19,31,32  One modification was made at a time, and global and local fit were assessed after item removal. The final model fit was assessed using the recommendations19,26,27  described in the CFA procedures. A bivariate correlation analysis was conducted on the composite scores of the DPAS and the new model to determine if the proposed model explained an acceptable amount of the variance in responses on the DPAS; an acceptable percentage of the variance explained was set at r ≥ 0.90 (R2 = 0.81).33 

Preliminary Analysis

A total of 1445 individuals took the survey. Thirty-six individuals were missing responses to >10% of the DPAS items and their data were, therefore, removed from the dataset. A total of 133 individuals reported scores that indicated either univariate (z scores ≥ 3.4) or multivariate (Mahalanobis distance ≥ 32.0) outliers. Thus, data from 1276 participants (667 [52.3%] males, 600 [47.0%] females; age = 20.8 ± 4.4 years) remained for analysis. Participants were grouped by sex, age, injury classification, and athletic status (Table 3).

Table 3

Participant Characteristics (N = 1276)

Participant Characteristics (N = 1276)
Participant Characteristics (N = 1276)

Confirmatory Factor Analysis

The CFA of the 16-item DPAS indicated marginal but not preferred model fit to the sample data. The goodness-of-fit indices approached but did not meet recommended values (CFI = 0.936, TLI = 0.923, RMSEA = 0.081, IFI = 0.936; Figure 2). Factor loadings from individual items to their respective first-order latent variables ranged from 0.60 to 0.89. Path coefficients between the second-order latent variable, disablement, and the first-order latent variables were very high, ranging from 0.94 to 1.02 (Figure 2). Correlations among the first-order latent variables—impairment, functional limitations, and disability—were very high, ranging from 0.89 to 0.97 (Figure 3). Additionally, modification indices indicated that several items had meaningful cross-loadings and suggested alternative specifications that could maximize fit and parsimony.

Figure 2

Confirmatory factor analysis of the 16-item Disablement in the Physically Active (DPA) scale. Abbreviations: d, disturbance variable; v, unique variance. Adapted with permission.12,22  χ2 = 927.755; degrees of freedom = 100; χ2 statistic/degrees of freedom ratio = 9.278; Comparative Fit Index = 0.936; Tucker-Lewis Index = 0.923; root mean square error of approximation = 0.081; Bollen Incremental Fit Index = 0.936; P < .001.

Figure 2

Confirmatory factor analysis of the 16-item Disablement in the Physically Active (DPA) scale. Abbreviations: d, disturbance variable; v, unique variance. Adapted with permission.12,22  χ2 = 927.755; degrees of freedom = 100; χ2 statistic/degrees of freedom ratio = 9.278; Comparative Fit Index = 0.936; Tucker-Lewis Index = 0.923; root mean square error of approximation = 0.081; Bollen Incremental Fit Index = 0.936; P < .001.

Close modal
Figure 3

First-order confirmatory factor analysis of the 16-item Disablement in the Physically Active Scale (DPAS). Abbreviations: d, disturbance variable; v, unique variance. Adapted with permission.12,22  χ2 = 904.827; degrees of freedom = 98; χ2 statistic/degrees of freedom ratio = 9.233; Comparative Fit Index = 0.937; Tucker-Lewis Index = 0.923; root mean square error of approximation = 0.080; Bollen Incremental Fit Index = 0.937; P < .001.

Figure 3

First-order confirmatory factor analysis of the 16-item Disablement in the Physically Active Scale (DPAS). Abbreviations: d, disturbance variable; v, unique variance. Adapted with permission.12,22  χ2 = 904.827; degrees of freedom = 98; χ2 statistic/degrees of freedom ratio = 9.233; Comparative Fit Index = 0.937; Tucker-Lewis Index = 0.923; root mean square error of approximation = 0.080; Bollen Incremental Fit Index = 0.937; P < .001.

Close modal

Invariance Analysis

Sex Subgroups

In the sample, 1267 participants reported their sex (male = 667, female = 600), and these data were included in the analysis. Baseline models for males and females indicated acceptable but not ideal model fit (CFI = 0.931 and 0.943, respectively), with both groups exceeding the RMSEA cutoff of ≤0.06 (Table 4). The initial model (equal form) demonstrated acceptable but not ideal fit (CFI = 0.936; χ2 = 1001.70; RMSEA = 0.057; Table 4). The metric model (ie, equal loadings) met both the CFIDIFF and the χ2DIFF (CFI = 0.936; χ2 = 1019.32). Satisfactory metric invariance criteria warranted examination of the equal first-order latent variables. Both the CFIDIFF and χ2DIFF criteria were met (Table 4); therefore, males and females exhibited similar variability on the first-order DPAS latent variables. The scalar model (ie, equal indicator intercepts) exceeded the χ2DIFF but did meet the criteria for the CFIDIFF, indicating that it was invariant and allowed for assessment of means between groups. The equal latent means slightly exceeded the χ2DIFF but did meet the CFIDIFF, indicating no apparent differences between levels of disablement and quality of life between males and females.

Table 4

Goodness-of-Fit Indices for the Measurement Invariance Analyses Across Sex

Goodness-of-Fit Indices for the Measurement Invariance Analyses Across Sex
Goodness-of-Fit Indices for the Measurement Invariance Analyses Across Sex

Age Subgroups

In the sample, 1151 individuals reported their ages, and these data were included in the analysis. Individuals were categorized into 1 of 3 age groups: ages 14 to 18 years (n = 332), ages 19 to 24 years (n = 665), or ages 25 to 40 years (n = 154). Baseline models for age groups indicated acceptable but not ideal model fit (CFI range = 0.924–0.940); all groups exceeded the RMSEA cutoff of ≤0.06 (Table 5). The initial equal form model again demonstrated acceptable but not ideal fit (CFI = 0.937; χ2 = 1036.04; RMSEA = 0.047; Table 5). The metric model (ie, equal loadings) met both the CFIDIFF and the χ2DIFF. Satisfactory metric invariance criteria warranted examination of the equal latent variances. Both the CFIDIFF and χ2DIFF criteria were met (Table 5); thus, all age groups exhibited similar variability on the DPAS first-order latent variables. The scalar model (ie, equal indicator intercepts) exceeded the χ2DIFF but did meet the CFIDIFF. The invariant scalar model warranted assessment of the means between groups. The equal latent means slightly exceeded the χ2DIFF but did meet the CFIDIFF, indicating similar levels of disablement and quality of life among age groups.

Table 5

Goodness-of-Fit Indices for the Measurement Invariance Analyses Across Age Groups

Goodness-of-Fit Indices for the Measurement Invariance Analyses Across Age Groups
Goodness-of-Fit Indices for the Measurement Invariance Analyses Across Age Groups

Injury Classification Subgroups

In the sample, 1271 reported their injury classification, and these data were included in the analysis. Individuals were categorized into 2 groups: injured (n = 1035) or healthy (n = 236). Baseline models for injury status indicated less than ideal fit: neither group met the RMSEA cutoff of ≤0.06 or the CFI cutoff of ≥0.95, with model fit slightly exceeding a CFI of >0.90 (Table 6). The initial equal form model demonstrated less than ideal fit (CFI = 0.905; χ2 = 1002.19; RMSEA = 0.057; Table 6). The metric model (ie, equal loadings) slightly exceeded the χ2DIFF but did meet the CFIDIFF.

Table 6

Goodness-of-Fit Indices for the Measurement Invariance Analyses Across Injury Classification

Goodness-of-Fit Indices for the Measurement Invariance Analyses Across Injury Classification
Goodness-of-Fit Indices for the Measurement Invariance Analyses Across Injury Classification

Satisfactory metric invariance criteria warranted examination of the equal latent variances of the DPAS. When testing equal factor variances, both the CFIDIFF and χ2DIFF noninvariant criteria were exceeded (Table 6). When variances were not constrained to be equal, the injured subsample exhibited substantially more variability of the first-order latent variables, impairment (healthy variance = 0.18, injured variance = 0.23), functional limitations (healthy variance = 0.20, injured variance = 0.58), disability (healthy variance = 0.12, injured = 0.49), and quality of life (healthy variance = 0.31, injured = 0.61). The scalar model (ie, equal indicator intercepts) exceeded the CFIDIFF and the χ2DIFF, suggesting item-level bias among injury classification subgroups (Table 6).

Athletic Status Classification Subgroups

In the sample, 1271 reported their athletic status classification. and these data were included in the analysis. Individuals were categorized into 2 groups: competitive athlete (n = 677) or combined recreational or occupational athlete (n = 594). Baseline models for athletic status classification indicated acceptable but not ideal fit, with both groups exceeding the RMSEA cutoff of ≤0.06 (Table 7). The initial equal form model demonstrated acceptable but not ideal fit (CFI = 0.937; χ2 = 1004.69; RMSEA = 0.057; Table 7). The metric model (ie, equal loadings) met both the CFIDIFF and χ2DIFF.

Table 7

Goodness-of-Fit Indices for the Measurement Invariance Analyses Across Athletic Status

Goodness-of-Fit Indices for the Measurement Invariance Analyses Across Athletic Status
Goodness-of-Fit Indices for the Measurement Invariance Analyses Across Athletic Status

Satisfactory metric invariance criteria warranted examination of the equal latent variances. The χ2DIFF was exceeded; however, the CFIDIFF noninvariant criteria were met (Table 7), indicating that both athletic classification groups exhibited similar variability on the first-order latent variables of the DPAS. The scalar model (ie, equal indicator intercepts) exceeded the χ2DIFF but did meet the CFIDIFF criteria. The invariant scalar model warranted assessment of the means between groups. The equal latent means slightly exceeded the χ2DIFF but did meet the CFIDIFF test, indicating similar levels of disablement and quality of life among athletic classification subgroups.

Alternate Model Generation

The CFA of the 16-item DPAS indicated marginal fit to the data (CFI = 0.936, TLI = 0.923, RMSEA = 0.081, IFI = 0.936; Figure 2); therefore, we used alternate model generation to create a more parsimonious scale. Assessment of modification indices suggested several meaningful cross-loadings and alternative specifications that could maximize fit and parsimony. Given the high correlation values (>0.95), the impairments, functional limitations, and disability factor items were combined into 1 factor, renamed physical. The final solution resulted from the removal of 8 items, consisted of 2 factors (ie, physical and quality of life) that contained 4 items each, and met recommended goodness-of-fit indices (CFI = 0.993, TLI = 0.989, RMSEA = 0.036, IFI = 0.993; Figure 4). All factor loadings were different, ranging from 0.69 to 0.86, and the correlation between physical and quality of life was 0.40 (P < .001). The Cronbach α for both factors met recommended values (physical: α = .84; quality of life: α = .83). Cumulative scores for the 8-item DPAS were highly correlated (r = 0.98, P < .001, R2 = 0.96) with the cumulative scores for the 16-item DPAS.

Figure 4

First-order 8-item alternative Disablement in the Physically Active Scale (DPAS) model. Abbreviations: d, disturbance variable; v, unique variance. Adapted with permission.12,22  χ2 = 49.718; degrees of freedom = 19; χ2 statistic/degrees of freedom ratio = 2.617; Comparative Fit Index = 0.993; Tucker-Lewis Index = 0.989; root mean square error of approximation = 0.036; Bollen Incremental Fit Index = 0.993; P < .001.

Figure 4

First-order 8-item alternative Disablement in the Physically Active Scale (DPAS) model. Abbreviations: d, disturbance variable; v, unique variance. Adapted with permission.12,22  χ2 = 49.718; degrees of freedom = 19; χ2 statistic/degrees of freedom ratio = 2.617; Comparative Fit Index = 0.993; Tucker-Lewis Index = 0.989; root mean square error of approximation = 0.036; Bollen Incremental Fit Index = 0.993; P < .001.

Close modal

The purpose of our study was to assess the psychometric properties of the 16-item DPAS, a frequently used clinical assessment tool. The psychometric properties were evaluated in a large, heterogeneous sample across sex, age, injury classification, and athletic status groups. We used contemporary CFA and structural equation modeling procedures to more rigorously examine the DPAS for model fit and multigroup invariance.19,28  Our results suggested that the 16-item DPAS does not meet contemporary fit index recommendations, does not meet the criteria to demonstrate invariance, and should be modified to produce a more parsimonious and psychometrically sound instrument that will improve instrument precision and reduce the item-response burden for clinicians and patients.

Confirmatory Factor Analysis

The overall findings of our CFA are consistent with previous results,22  indicating that the 16-item scale does not meet model fit recommendations, and the solution presents with model misspecification and constructs that have multicollinearity bordering on singularity.19,20,24  Therefore, item removal is necessary to effectively measure the unique subconstructs (ie, impairments, functional limitations, and disability) of disablement8,12,34  that were originally intended in the creation of the DPAS.12,35  Without item modification (eg, item removal, rewriting items to improve clarity, reducing overlapping examples for patients),36  items from different subconstructs will continue to measure the same phenomenon, or participants will continue to struggle to interpret the intended differences among the items.

Invariance Testing

Invariance testing may be conducted for a number of reasons, such as to ensure an instrument's items are being interpreted similarly across groups (eg, injured and healthy individuals), underlying constructs (eg, functional limitation, disablement) are being measured similarly across groups, or measurement properties are being maintained across repeated measures.19,28  To our knowledge, we are the first to assess any of the invariance procedures involving the DPAS. The initial (eg, CFA) results indicated that the model did not meet more stringent goodness-of-fit standards that would typically prohibit invariance testing under the more contemporary guidelines.19,28  Given the common use of the DPAS in clinical practice, conducting invariance testing using less rigorous criteria was deemed relevant. The DPAS met the less rigorous invariance criteria (ie, the underlying constructs were being measured and interpreted similarly) across age, sex, and athletic status groups but did not meet the criteria for injury status. The DPAS was not invariant in this analysis, suggesting that the disablement and quality-of-life constructs and items did not have the same meaning across group membership (ie, healthy and injured). The differences may result from the items or examples included with the items having different meanings to people, depending on their current health status. Therefore, differences in scores (ie, healthy people had lower levels of disablement than injured people) based on this group membership cannot be solely attributed to injury status. The results indicated that the differences were being confounded by measurement artifact (eg, item interpretation, error), and drawing inferences based on group membership differences (ie, healthy versus injured) is not supported.19 

Additionally, although testing between groups appeared to pass the less stringent criteria, several concerns with the DPAS were raised during the process. Acceptable but not preferred model fit indices were observed for all groups (ie, no models met the more stringent global fit guidelines: CFI ≥ 0.95, TLI ≥ 0.95, RMSEA ≤ 0.06, IFI ≥ 0.95),19,27  and a number of concerns were present (eg, high cross-loadings, high correlations among first-order latent variables) within the models. Although we did not modify the 16-item DPAS models (eg, releasing a constraint) during our invariance testing procedures, researchers could assess individual items and constructs to identify the most problematic items, remove problematic items, and attempt to improve model fit through the invariance testing process.19,20 

Alternate Model Generation

Using alternate model generation, we identified a more parsimonious model that met the more stringent model fit standards.19,27  The 8-item DPAS exceeded the model fit recommendations19,20  and matched the proposed DPAS Short Form-8 that has been previously published.22  The final model contained 2 factors, physical and quality of life. The physical factor combined items originally in the impairment and functional limitations factors. All items from the disability factor were eliminated, whereas all 4 items in the quality-of-life factor were retained. The problems noted in the 16-item CFA and multigroup invariance (eg, high cross-loadings, high correlations among first-order latent variables) tests were not present in the 8-item version. Furthermore, despite the number of items being reduced by 50%, the 8-item version accounted for an acceptable amount of the variance in participants' responses on the DPAS. The high correlation (r = 0.98, P < .001, R2 = 0.96) between the 2 versions of the scale indicated substantial overlap between them and provided evidence that the short-form version captures the same information as the complete scale. The correlation value in our study exceeded the previously published correlation value (r = 0.94, P ≤ .001, R2 = 0.88) between the DPAS and the DPAS Short Form-8.22 

Implications for Clinical Practice and Research

The effective use of PROs in clinical practice requires instruments that measure multiple constructs relevant to patient care (eg, health status, functional limitations)7,12,34 ; however, the constructs should be unique, and the scale should produce precise measurement of those constructs.19,20  Clinically, the identification of an 8-item instrument was relevant because the solution resolved the model fit concerns found in the original DPAS, improved the measurement precision for assessing the proposed constructs, had substantial overlap with the original scale, and reduced barriers (eg, contained fewer items) for clinicians and patients.4,22,37  Our invariance results also provided insight regarding differences between certain groups (ie, healthy versus injured) that are clinically relevant for using the DPAS to track health improvements. The findings supplied preliminary evidence that differences between certain groups may be related to the instrument (eg, measurement error) and not the respondents' stages of health (ie, health improvements over time). Further research (eg, longitudinal invariance testing) is needed to assess group differences using repeated measures and ensure that the changes experienced result from treatment outcomes indicating improved health and not measurement error.19,20  In addition, our results support previous findings18,22  that showed the distinctness of the quality-of-life and disablement constructs, which indicates clinicians should examine construct scores individually and not as a summative measure of disablement when the quality-of-life and disablement constructs are unique dimensions within the scale.

Limitations and Future Research

Although we tested a fairly diverse population, the participants were recruited from a relatively small group of clinics from across the United States. Further, our convenience sample primarily consisted of young, physically active participants; therefore, we do not know if the DPAS is invariant in other (eg, geriatric) populations who were not tested. Additionally, fairly conservative standards were used to assess model fit19,28 ; however, none of the models would have met the stricter recommendation guidelines. Whereas our invariance testing results identified potential concerns with the DPAS, further invariance testing (eg, longitudinal testing) is needed to fully elucidate the potential group differences and scale precision for measuring health status improvement over time. Yet future attention should likely be focused on the modified DPAS (ie, DPAS Short Form-8) because it does not have the same concerns as identified in the original DPAS by previous authors22  or us. The DPAS Short Form-8 model22  was also identified using alternate model generation in our sample. Whereas the 8-item version is promising, a true cross-validation analysis with invariance testing is still needed before adoption. Therefore, the DPAS Short Form-8 findings from our study should be interpreted with caution until more psychometric analyses (eg, invariance testing, longitudinal testing) are conducted in a cross-validation sample.30  Future researchers should identify whether the model meets the strict criteria when participants respond to only the 8 items and if the new model meets the recommended guidelines for invariance testing.

The 16-item DPAS did not meet model fit recommendations and may not be the most parsimonious or reliable measure for assessing disablement and quality of life. Given the model fit concerns and measurement noninvariance, clinicians should use the DPAS with caution. Whereas we also identified the DPAS Short Form-8, a more parsimonious model, additional research in a true cross-validation sample should be completed before this scale is fully adopted in research and clinical practice.

1. 
Abrams
D,
Davidson
M,
Harrick
J,
Harcourt
P,
Zylinski
M,
Clancy
J.
Monitoring the change: current trends in outcome measure usage in physiotherapy
.
Man Ther
.
2006
;
11
(1)
:
46
53
.
2. 
Evans
TA,
Lam
KC.
Clinical outcomes assessment in sport rehabilitation
.
J Sport Rehabil
.
2011
;
20
(1)
:
8
16
.
3. 
Hankemeier
DA,
Popp
JK,
Walker
SE.
Familiarity with and use of clinical prediction rules and patient-rated outcome measures
.
Athl Train Sports Health Care
.
2017
;
9
(3)
:
108
123
.
4. 
Nicholas
P,
Hefford
C,
Tumilty
S.
The use of the Patient-Specific Functional Scale to measure rehabilitative progress in a physiotherapy setting
.
J Man Manip Ther
.
2012
;
20
(3)
:
147
152
.
5. 
Stucki
G,
Liang
MH,
Fossel
AH,
Katz
JN.
Relative responsiveness of condition-specific and generic health status measures in degenerative lumbar spinal stenosis
.
J Clin Epidemiol
.
1995
;
48
(11)
:
1369
1378
.
6. 
Valier
ARS,
Jennings
AL,
Parsons
JT,
Vela
LI.
Benefits of and barriers to using patient-rated outcome measures in athletic training
.
J Athl Train
.
2014
;
49
(5)
:
674
683
.
7. 
Jette
A.
Outcomes research: shifting the dominant research paradigm in physical therapy
.
Phys Ther
.
1995
;
75
(11)
:
965
970
.
8. 
Jette
DU,
Halbert
J,
Iverson
C,
Miceli
E,
Shah
P.
Use of standardized outcome measures in physical therapist practice: perceptions and applications
.
Phys Ther
.
2009
;
89
(2)
:
125
135
.
9. 
Childs
JD,
Cleland
JA.
Development and application of clinical prediction rules to improve decision making in physical therapist practice
.
Phys Ther
.
2006
;
86
(1)
:
122
131
.
10. 
Michener
LA.
Patient- and clinician-rated outcome measures for clinical decision making in rehabilitation
.
J Sport Rehabil
.
2011
;
20
(1)
:
37
45
.
11. 
Valovich McLeod
TC,
Snyder
AR,
Parsons
JT,
Bay
RC,
Michener
LA,
Sauers
EL.
Using disablement models and clinical outcomes assessment to enable evidence-based athletic training practice, part II: clinical outcomes assessment
.
J Athl Train
.
2008
;
43
(4)
:
437
445
.
12. 
Vela
LI,
Denegar
CR.
The Disablement in the Physically Active Scale, part II: the psychometric properties of an outcomes scale for musculoskeletal injuries
.
J Athl Train
.
2010
;
45
(6)
:
630
641
.
13. 
Hoch
JM,
Druvenga
B,
Ferguson
BA,
Houston
MN,
Hoch
MC.
Patient-reported outcomes in male and female collegiate soccer players during an athletic season
.
J Athl Train
.
2015
;
50
(9)
:
930
936
.
14. 
McAllister
DR,
Motamedi
AR,
Hame
SL,
Shapiro
MS,
Dorey
FJ.
Quality of life assessment in elite collegiate athletes
.
Am J Sports Med
.
2001
;
29
(6)
:
806
810
.
15. 
Huffman
GR,
Park
J,
Roser-Jones
C,
Sennett
BJ,
Yagnik
G,
Webner
D.
Normative SF-36 values in competing NCAA intercollegiate athletes differ from values in the general population
.
J Bone Joint Surg Am
.
2008
;
90
(3)
:
471
476
.
16. 
Fritz
JM,
Clifford
SN.
Low back pain in adolescents: a comparison of clinical outcomes in sports participants and nonparticipants
.
J Athl Train
.
2010
;
45
(1)
:
61
66
.
17. 
Hsu
JE,
Nacke
E,
Park
MJ,
Sennett
BJ,
Huffman
GR.
The Disabilities of the Arm, Shoulder, and Hand questionnaire in intercollegiate athletes: validity limited by ceiling effect
.
J Shoulder Elbow Surg
.
2010
;
19
(3)
:
349
354
.
18. 
Houston
MN,
Hoch
JM,
Van Lunen
BL,
Hoch
MC.
The development of summary components for the Disablement in the Physically Active scale in collegiate athletes
.
Qual Life Res
.
2015
;
24
(11)
:
2657
2662
.
19. 
Kline
RB.
Principles and Practice of Structural Equation Modeling. 4th ed
.
New York, NY
:
Guilford Publications;
2015
.
20. 
Brown
TA.
Confirmatory Factor Analysis for Applied Research. 2nd ed
.
New York, NY
:
Guilford Publications;
2014
.
21. 
Chen
FF.
Sensitivity of goodness of fit indexes to lack of measurement invariance
.
Struct Equ Modeling
.
2007
;
14
(3)
:
464
504
.
22. 
Baker
RT,
Burton
D,
Pickering
MA,
Start
A.
Confirmatory factor analysis of the Disablement in the Physically Active Scale and preliminary testing of short-form versions: a calibration and validation study
.
J Athl Train
.
2019
;
54
(3)
:
302
318
.
23. 
Strong
J,
Unruh
AM,
Wright
A,
Baxter
GD.
Pain: A Textbook for Therapists
.
Edinburgh, United Kingdom
:
Churchill Livingstone;
2002
.
24. 
Jöreskog
K,
Sörbom
D.
LISREL 8: Structural Equation Modeling with the SIMPLIS Command Language
.
Chicago, IL
:
Scientific Software International;
1993
.
25. 
Tabachnick
BG,
Fidell
LS.
Using Multivariate Statistics. 4th ed
.
Needham Heights, MA
:
Allyn & Bacon;
2001
.
26. 
Bryant
FB,
Yarnold
PR.
Principal-components analysis and exploratory and confirmatory factor analysis
.
In:
Grimm
LG,
Yarnold
PR,
eds.
Reading and Understanding Multivariate Statistics
.
Washington, DC
:
American Psychological Association;
1995
:
99
136
.
27. 
Hu
L,
Bentler
PM.
Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives
.
Struct Equ Modeling
.
1999
;
6
(1)
:
1
55
.
28. 
Byrne
B.
Structural Equation Modeling with AMOS: Basic Concepts, Applications, and Programming. 3rd ed
.
New York, NY
:
Routledge;
2016
.
29. 
Sigelman
CK,
Rider
EA.
Life-Span Human Development. 9th ed
.
Boston, MA
:
Cengage Learning;
2017
.
30. 
Schermelleh-Engel
K,
Moosbrugger
H,
Müller
H.
Evaluating the fit of structural equation models: tests of significance and descriptive goodness-of-fit measures
.
Methods Psychol Res
.
2003
;
8
(2)
:
23
74
.
31. 
Cohen
L,
Manion
L,
Morrison
K.
Research Methods in Education. 6th ed
.
New York, NY
:
Routledge;
2007
.
32. 
Pesudovs
K,
Burr
JM,
Harley
C,
Elliott
DB.
The development, assessment, and selection of questionnaires
.
Optom Vis Sci
.
2007
;
84
(8)
:
663
674
.
33. 
Ware
J
Kosinski
M,
Keller
SD. A
12-item short-form health survey: construction of scales and preliminary tests of reliability and validity
.
Med Care
.
1996
;
34
(3)
:
220
233
.
34. 
Verbrugge
LM,
Jette
AM.
The disablement process
.
Soc Sci Med
.
1994
;
38
(1)
:
1
14
.
35. 
Vela
LI,
Denegar
C.
Transient disablement in the physically active with musculoskeletal injuries, part I: a descriptive model
.
J Athl Train
.
2010
;
45
(6)
:
615
629
.
36. 
Dillman
DA,
Smyth
JD,
Melani Christian L. Internet, Phone, Mail and Mixed-Mode Surveys: The Tailored Design Method. 4th ed
.
Hoboken, NJ
:
John Wiley & Sons;
2014
.
37. 
Westaway
MD,
Stratford
PW,
Binkley
JM.
The Patient-Specific Functional Scale: validation of its use in persons with neck dysfunction
.
J Orthop Sports Phys Ther
.
1998
;
27
(5)
:
331
338
.