Abstract

This article presents the results of a pilot study to evaluate the psychometric properties of the Escala Pessoal de Resultados (EPR), the Portuguese version of the Personal Outcomes Scale (POS), to assess quality of life (QOL) in people with intellectual disability (ID) in Portugal. We present the adaptation and validation processes of the EPR. Data were collected from 1,264 adults with intermittent and limited levels of support needs and respective proxies. Results of the research are discussed in requisites of the validity and reliability. The different coefficients illustrate that this scale may be a useful addition to the QOL assessment in Portuguese people with ID.

In the last decade, considerable attention has been centered upon the quality of life (QOL) construct of the Personal Outcomes Scale (POS). There is a growing interest in the QOL's measurement for populations with intellectual disability (ID). People with ID often experience problems concerning their participation in society and are in danger of being excluded from opportunities that typically are available to others. Therefore, adults with ID need to have greater control in their lives and should have the opportunity to voice their choices, perspectives, and concerns (Bonham et al., 2004; Schalock, Gardner, & Bradley, 2007; Shogren & Broussard, 2011; Turnbull, Turnbull, Wehmeyer, & Park, 2003). The QOL assessment is increasingly being used as a personal outcome measure, and is focused on the collection of information from the person in question about his or her own life (Bonham et al., 2004; Brown, Schalock, & Brown, 2009; Cummins, 2005; Phillips, 2006; Schalock, 2004; Schalock et al., 2007; Schalock & Verdugo, 2002; Verdugo, Schalock, Keith, & Stancliffe, 2005; Wang, Schalock, Verdugo, & Jenaro, 2010).

The measurement of QOL allows for the weighting of domains and indicators according to the significance or the value assigned by each person (Schalock & Verdugo, 2002). It should also reflect each person's individual and cultural life experiences (Claes, van Hove, van Loon, Vandevelde, & Schalock, 2010; Schalock, 2004; Schalock et al., 2007; Schalock & Verdugo, 2002; van Loon, Van Hove, Schalock, & Claes, 2009). Varying numbers of QOL domains have been reported in literature; however, the number of domains is less important than the acknowledgment that each structure must include a multielement framework and that all domains must represent the QOL's construct (Cummins, 2005; Schalock, 2004; Schalock & Verdugo, 2002; Verdugo et al., 2005).

The QOL is a multidimensional concept that includes subjective and objective measures (Bonham et al., 2004; Brown et al., 2009; Claes et al., 2010; Cummins, 2005; Phillips, 2006; Schalock, 2004; Schalock et al., 2007; Schalock & Verdugo, 2002; van Loon et al., 2009; Verdugo et al., 2005; Wang et al., 2010), and is a hierarchical construct (Wang et al., 2010). Nevertheless, according to the World Health Organization (WHO), the QOL is based on subjective measures and can be understood as the satisfaction, the perception, and the interpretation by each person of the various facets of his or her own life (Skevington, Lotfy, & O'Connell, 2004; WHOQOL Group, 1998a).

In the last 2 decades, many instruments have been developed for QOL assessment. This measurement is essential for research and intervention (Camfield & Skevington, 2008; Claes et al., 2010; Finlay & Lyons, 2001; Hartley & MacLean, 2006; Kayes & McPherson, 2010; Kimberlin & Winterstein, 2008; Li, Tsoi, Zhang, Chen, & Wang, 2013; McGillivray, Lau, Cummins, & Davey, 2009; Schalock, 2004; Schwartz & Rabinovitz, 2003; van Loon et al., 2009). However, it is not clear which scale is suitable for evaluating QOL in different situations. Subjective measures (self-report) only may not be enough to assess QOL and the evaluation of individual programs (Hensel, Rose, Kroese, & Banks-Smith, 2002; Schalock, 2004). As a consequence, this study investigates the validity and reliability of the Escala Pessoal de Resultados (EPR; the Portuguese version of the POS), which involves subjective and objective QOL indicators, predicting that the scale will be an appropriate measure for Portuguese people with ID. The instrument is based on eight core domains set out to assess QOL in people with ID, namely: personal development, self-determination, interpersonal relations, social inclusion, rights, emotional well-being, physical well-being, and material well-being (Schalock & Verdugo, 2002).

The original scale of the EPR is the POS, which advocates international guidelines to assess QOL in people with ID (Claes et al., 2010; van Loon, van Hove, Schalock, & Claes, 2008, 2009). The POS was developed for the purpose of assessing specific indicators associated with each of the eight QOL domains that have been validated in a series of cross-cultural studies (Bonham et al., 2004; Claes et al., 2010; Jenaro et al., 2005; Schalock & Verdugo, 2002; van Loon et al., 2009; Wang et al., 2010). The POS scale (in the form of the EPR) was selected because of its validity, reliability, and frequent use in studies about ID. However, to date, it has not been validated in Portuguese adults with ID.

Current research emphasizes the validity and the reliability of the psychometric properties of the EPR. We hypothesized that: (1) the scale should demonstrate the construct validity through the intercorrelations between domain scores and total scores and the eight-factor model should be supported by the exploratory factor analysis (EFA); (2) the scale should demonstrate evidence of concurrent validity with a related measure of similar construct; (3) the scale should demonstrate a good test-retest reliability; (4) alpha's coefficients and split-half should demonstrate evidence of internal consistency; and (5) inter-respondent reliability coefficients should be acceptable.

Method

Sample

The sample comprised of 1,264 adults with intermittent (n  =  769; 60.84%) and limited levels of support needs (n  =  495; 39.16%). The age of the participants ranged from 18 to 66 years (M  =  31.36). Regarding gender, 600 participants were female (47.47%) and 664 were male (52.53%). Living arrangements included homeownership (n  =  93; 7.36%), family home (n  =  924; 73.10%), and institutions (n  =  247; 19.54%). Eighty-seven adults with ID (6.88%) did not have an occupation (were only with the family at home), while the others worked a paid job (n  =  138; 10.92%) or were involved in professional training (n  =  481; 38.05%) or occupational activities centers (n  =  558; 44.15%).

The support staff that participated for the report of others portion of the EPR (n  =  224) was connected to 45 Portuguese institutions. There was a mix of psychologists, social workers, special education teachers, and psychomotor/occupational therapists or monitors. Age ranged from 22 to 64 (M  =  41.46), and 179 were female (79.91%), while 45 were male (20.09%). The second part of the EPR (the report of others portion) also involved family members (n  =  109). Age of included family members ranged from 21 to 89 years (M  =  57.37); 74 were female (67.90%) and 35 were male (32.10%). These participants were mothers (n  =  56; 51.38%), fathers (n  =  24; 22.02%), siblings (n  =  15; 13.76%), and others members of family (n  =  14; 12.84%).

Measures

The Portuguese EPR scale is divided into two parts (a self-report and a report of others). The self-report part has a set of questions to be answered directly by people with ID. The report of others part has the same indicators, but should be completed by a proxy—a family member or a caregiver who knows the person for at least 2 years.

A total of 48 questions are presented in each part of the EPR, with six questions per domain. Answers on the scale are reported on a 3-point Likert type scale (i.e., frequent, sometimes, and never). Two new items about importance and satisfaction were added to EPR in the self-report scale, which differs from the original POS. The question about importance is hierarchical (i.e., 1 corresponding to the most important domain and 8 to the lesser important). Satisfaction on each domain is rated through a 3-point Likert scale (i.e., very satisfied, not too satisfied, and unsatisfied).

Procedures

Ethical approval for the study was provided by the ethic committee of Centro Hospitalar de São João. The study was carried out in compliance with the Helsinki Declaration (World Medical Association, 2008). Participants signed an informed consent prior to the entry of the study and were free to leave at any time. The selection of participants was based on the following criteria: formal diagnosis of intermittent or limited level of support needs of ID, and 18 years or more on age.

All EPR items were administered to each participant in accordance with the guidelines of the original scale (van Loon et al., 2008). The interviewers of the self-report and report of others measures were always the same. They were qualified professionals who had been previously trained to administer the EPR. In the self-report portion, the EPR was always administered through an interview. Before the participants answered the questions, the interviewers read standard instructions and completed sociodemographic information. Statistical Package for Social Sciences (SPSS), version 21.0, was used for data analysis.

Results

Validity

Content validity

The cross-cultural adaptation and the content validity research were developed through five stages: (1) translation, (2) back translation, (3) comparison between the versions in Portuguese and in English, (4) content validity by group of experts, and (5) focus group (Verdugo et al., 2005). A content specialist reviewed the initial translation in the target language and made necessary corrections. Products were evaluated by 10 experts who agreed, after doubts and language clarification, on a pre-final version of the scale with items based on Portuguese lifestyle and culture. The experts considered all of the EPR's questions relevant (n  =  98), but 14 items were identified as needing revision in regard to level of clarity, simplicity, and ambiguity (Simões & Santos, 2014). The scores of scale content validity index-universal agreement, scale content validity index-average, and Cohen's Kappa showed adequate evidence of the EPR's content validity (Simões & Santos, 2014).

Construct validity

To evaluate the extent to which the domains of the EPR actually assess the QOL construct, Pearson's correlations were calculated between the eight domains of the EPR and the total score and domains. The scores of Pearson's coefficients are considered moderate (.40 to .69), high (.70 to .89), and very high (above .90; Pestana & Gageiro, 2005). As can be noted in Table 1, domain-to-domain correlations in the self-report portion showed at least one moderate correlation. The higher score was observed between the interpersonal relations and social inclusion domains (r  =  .62). The correlations between the total self-report score and the domain scores were moderate to high. Higher Pearson's coefficients among total QOL score and domain scores were observed in self-determination (r  =  .72), interpersonal relations (r  =  .72), social inclusion (r  =  .75), and material well-being (r  =  .71). In the same way, significant domain-to-domain coefficients were confirmed in the report of others portion (see Table 2). Correlations among the domain scores and the total QOL scores were high on the same four domains of the self-report measure, and ranged from .44 (physical well-being) to .78 (social inclusion).

Table 1

Intercorrelations of Domains (Self-Report)

Intercorrelations of Domains (Self-Report)
Intercorrelations of Domains (Self-Report)
Table 2

Intercorrelations of Domains (Report of Others)

Intercorrelations of Domains (Report of Others)
Intercorrelations of Domains (Report of Others)

The structure of the EPR was analyzed with an EFA to find out if a potential framework of the POS may emerge from the sample data on Portuguese population. We employed the maximum likelihood (ML) estimation method, with a Promax rotation. To evaluate the sampling adequacy, we used the Kaiser-Meyer-Olkin (KMO) index and the Bartlett's test of sphericity (Pestana & Gageiro, 2005). The initial factor pattern/structure matrix, including all 48 items, showed 12 factors in the self-report portion and 11 in the report of others part. Nineteen items were removed from the self-report measure and 14 from the report of others measure. The results of the EFA showed that these unretained items had salient loadings on two or more factors (Brown, 2006), with values ≥ .32 on at least two factors (Costello & Osborne, 2005; Worthington & Whittaker, 2006), or failed to exceed the cut-off point of .40 (DeVon et al., 2007; Henson & Roberts, 2006; Matsunaga, 2010). The factor pattern/structure matrix is presented in Table 3 and Table 4. In the self-report portion, results of the EFA supported eight factors, accounting for 51.33% of the variance (see Table 3), and the eigenvalue of the ninth unretained factor was 1.01. The report of others portion (see Table 4) also revealed eight factors accounting for 56.62% of the overall variance. Criteria of factors retained included multiple strategies, namely: eigenvalues greater than 1, scree plot, and components loadings above .40. We removed items with low communalities and rejected a factor with fewer than three items (Brown, 2006; Costello & Osborne, 2005; Henson & Roberts, 2006; Pestana & Gageiro, 2005; Worthington & Whittaker, 2006). Although the results seem to support the eight-factor structure proposed by the POS (van Loon et al., 2008, 2009), the EFA did not confirm all items in the Portuguese sample.

Table 3

EPR's Factor Pattern/Structure Matrix (Self-Report)

EPR's Factor Pattern/Structure Matrix (Self-Report)
EPR's Factor Pattern/Structure Matrix (Self-Report)
Table 4

EPR's Factor Pattern/Structure Matrix (Report of Others)

EPR's Factor Pattern/Structure Matrix (Report of Others)
EPR's Factor Pattern/Structure Matrix (Report of Others)

Concurrent validity

To determine the concurrent validity, two scales were administered to the same person, specifically: the EPR (self-report measure only) and the World Health Quality of Life-BREF (WHOQOL-BREF; Canavarro et al., 2007; Skevington et al., 2004; WHOQOL Group, 1998b). Concurrent validity evaluates the relationship between an instrument and previous standard scores obtained by another test (Claes et al., 2010; DeVon et al., 2007; Kimberlin & Winterstein, 2008; Pasquali, 2007; Schalock et al., 2007; van Loon et al., 2009). Consequently, the correlation between different dimensions of two scales shows how the same construct is measured in the same subjects (Cicchetti, 1994; DeVon et al., 2007; Kimberlin & Winterstein, 2008; Pasquali, 2007). Both scales were assessed to 216 adults with ID (M age  =  31.38; 128 with intermittent and 88 with limited levels of support needs). The WHOQOL-BREF was chosen because it was the only QOL instrument standardized to the Portuguese population (Canavarro et al., 2007), and has been used in several studies. The WHOQOL-BREF was developed as a short version of the WHOQOL-100 (also validated in Portugal by Canavarro et al., 2009) to be used in situations where respondent burden must be minimized (Canavarro et al., 2007; Skevington et al., 2004; WHOQOL Group, 1998b). This scale has a different QOL framework than the EPR because it consists of four domains, namely: the physical, the psychological, the social, and the environmental. While the EPR includes both subjective and objective measures (Claes et al., 2010; van Loon et al., 2008, 2009), the WHOQOL-BREF is based only on subjective indicators related to personal expectations and subjective experiences (Canavarro et al., 2007; Skevington et al., 2004; WHOQOL Group, 1998b). Table 5 summarizes the Pearson's correlation coefficients between the EPR (self-report measure) and the WHOQOL-BREF. Both scales had moderate correlations between: emotional well-being in EPR and the physical (r  =  .41; p < .001) and psychological (r  =  .43; p < .001) domains in the WHOQOL-BREF; physical well-being in EPR and physical (r  =  .49; p < .001) and psychological (r  =  .48; p < .001) domains in WHOQOL-BREF.

Table 5

Concurrent Validity Coefficients

Concurrent Validity Coefficients
Concurrent Validity Coefficients

Reliability

Test-retest reliability

In order to determine the test-retest reliability, the EPR was re-administered, by the same interviewer to the same person and proxy, 2 to 3 weeks following its initial administration. A total of 160 people with ID (ages ranging from 18 to 64 years; M  =  32.23; 77 with intermittent and 83 with limited levels of support needs) and respective support staff (ages ranging from 31 to 56 years; M  =  38.17) were assessed. The period of 2 to 3 weeks between administrations was considered optimal for minimizing the likelihood that raters could remember previous responses or that the person's behavior could change (DeVon et al., 2007; Kimberlin & Winterstein, 2008). Pearson correlations were calculated to quantify the test-retest reliability (Andresen, 2000; Cicchetti, 1994; DeVon et al., 2007; Li et al., 2013; McGillivray et al., 2009; Noonan, Miller, & Noreau, 2009). We assessed the repeatability and the stability scores of the EPR's QOL index and domains, which were all statistically significant. Table 6 illustrates that the test-retest evaluation showed high correlations in the self-report portion for all domains, except for the domain emotional well-being (r  =  .67). Correlations for the report of others measure were higher in all domains, when compared with the self-report measure. They ranged from high (r  =  .82 in interpersonal relations) to very high (r  =  .93 in rights). Test-retest reliability was also analyzed for the additional questions on importance and satisfaction. Results can be seen in Table 7. On the question of importance, Pearson correlations were weak in the emotional well-being domain (r  =  .32) to moderate in the other seven domains. On the question of satisfaction, Pearson correlations ranged from weak (r  =  .35, personal development) to high (r  =  .73, interpersonal relations). Items with lower test-retest coefficients were rewritten using simple, clear, brief, and everyday language that could be understood by adults with ID. The way in which a question is phrased is of crucial importance (Andresen, 2000; Bonham et al., 2004; Finlay & Lyons, 2001; Hartley & MacLean, 2006; Li et al., 2013; McGillivray et al., 2009). Thus, some insight of acquiescence was also analyzed in the pretest from 160 adults with ID and respective proxies. According to the results, no participant responded to all the items with a favorable answer. In the self-report portion, people with ID indicated the favorable response to 50.3%, neutral to 32.9%, and negative to 16.8% of the questions.

Table 6

Test-Retest Reliability

Test-Retest Reliability
Test-Retest Reliability
Table 7

Test-Retest Reliability of the Importance and Satisfaction Questions

Test-Retest Reliability of the Importance and Satisfaction Questions
Test-Retest Reliability of the Importance and Satisfaction Questions

Internal consistency reliability

To evaluate the internal consistency, Cronbach's alpha coefficients and split-half were calculated (Pestana & Gageiro, 2005). According to several authors, Cronbach's alpha above .80 may be considered excellent (Andresen, 2000; Hartley & MacLean, 2006; Li et al., 2013; Noonan et al., 2009; Robinson, Shaver, & Wrightsman, 1991), between .70 and .80 is adequate, and below .70 is inadequate (Andresen, 2000; Li et al., 2013; Noonan et al., 2009). Following the Cronbach's alpha guidelines, the overall instrument demonstrated good indexes of reliability, and the QOL index ranged from .87 (self-report measure) to .90 (report of others measure). The report of others portion showed higher values of internal consistency at the domain level, compared to the self-report part (see Table 8). In both parts of the EPR, the physical well-being domain has the lowest score (α  =  .46 on self-report; α  =  .55 on report of others). The highest coefficient of the self-report scale was observed in rights (α  =  .77), and social inclusion had the highest score in the report of others scale (α  =  .80).

Table 8

Cronbach Alpha's Coefficients

Cronbach Alpha's Coefficients
Cronbach Alpha's Coefficients

According to Pestana and Gageiro (2005), after randomly dividing all items into split-halves (split-half reliability), different indexes were estimated: the Cronbach's alpha for each part, the correlation between forms, and the global reliability coefficients (Spearman-Brown and Guttman). As shown in Table 9, the first and the second half ranged from .74 (self-report measure) to .89 (all scale). The correlations between each half were high. Spearman-Brown and Guttman coefficients were equal or above .90.

Table 9

Split-Half Coefficients

Split-Half Coefficients
Split-Half Coefficients

Inter-rater reliability

In order to evaluate the inter-respondent reliability, a report of others measure was conducted by interviewing one institutional support worker (n  =  14; M age  =  37.28) and one family member (n  =  63; M age  =  52.75) for each person that completed the self-report measure (n  =  63; M age  =  32.23). The advantage of the inter-respondent reliability is that it provides the ranking scores by independent evaluators and gives values of interexaminer agreement (Andresen, 2000; Cicchetti, 1994). As shown in Table 10, inter-respondent values ranged from moderate (r  =  .40) to excellent (r  =  .88), and the degree of scores differs between the three groups. Reliability normative standards were used to evaluate these coefficients, namely: poor < .40, moderate to good ≥ .40 < .75, and excellent ≥ .75 (Andresen, 2000; Li et al., 2013; Noonan et al., 2009). Cicchetti (1994) reported similar scores (i.e., poor < .40, fair ≥ .40 < .60, good ≥ .60 < .75, and excellent ≥ .75). Higher correlations, in general, were observed between support staff versus adults with ID or family members.

Table 10

Inter-Respondent Reliability Coefficients

Inter-Respondent Reliability Coefficients
Inter-Respondent Reliability Coefficients

Discussion

An ideal scale to assess QOL “should be one which is valid, reliable, with low administer burden, and cultural-specific to the target population involved” (Li et al., 2013, p. 266). The present research is a pilot study to determine the EPR as a valid and a reliable tool to evaluate the QOL in Portuguese people with ID.

Concerning the construct validity, the domain-to-domain scores of the EPR of both parts had shown at least one moderate correlation. These results were similar to those found in the original study (van Loon et al., 2009). We reiterate that the “domains should be intercorrelated, but not highly intercorrelated” (van Loon et al., 2009, p. 30).The correlations domain-total scale ranged from moderate to high and were statistically significant (p < .001). In fact, the physical well-being domain had lower correlations (r  =  .35) in previous research (Claes et al., 2010; van Loon et al., 2009). According to Hartley and MacLean (2006) and Robinson et al. (1991), with r ≥ .60 in at least two related measures or r ≥ .70 in one related measure, the construct validity is strong, and is moderate when r  =  .50−.59 in at least two measures and r  =  .69−.60 in one related measure. The expected scores showed empirical construct validity, since correlations between domain-total scale were higher than .70 in four domains for both EPR portions. Consequently, there are different indicators for the QOL domains, but eight subscales that relate to each other and assess the same construct.

The EFA seems to identify the factors that have substantive relevance and theoretical/empirical meaning (Brown, 2006; Matsunaga, 2010). Simultaneously, it is an essential step of validity and we emphasize the importance of this information, as there are no data available on the original POS scale (Claes et al., 2010; van Loon et al., 2009). Despite the fact that we identified eight factors (Schalock & Verdugo, 2002) in a Portuguese sample that supported the original scale, the final factor pattern/structure matrix presented shortcomings. For example, we retained two factors in the self-report scale and one factor in the report of others scale with only two items. However, “it is possible to retain a factor with only two items if the items are highly correlated (i.e., r > .70) and relatively uncorrelated with other variables” (Worthington & Whittaker, 2006, p. 821), and the three factors retained showed this principle. Additionally, some retained items presented low communality coefficients, but they were important for the factor pattern/structure matrix. We highlight that “if an item has a communality of less than .40, it may either a) not be related to the other items, or b) suggest an additional factor that should be explored” (Costello & Osborne, 2005, p. 4). Finally, although the scale presented the same indicators on each domain, for both parts, we observed differences in the factor pattern/structure matrix between the self-report and the report of others measures. The results reflected different perceptions between people with ID and proxies, and it seems that the indicators of each domain do not necessarily have to be the same for both parts. For those reasons, it is necessary for a confirmatory factorial analysis to contribute more evidence of the Portuguese QOL's construct in the field of ID.

Regarding concurrent validity, the domains scores of the EPR were weakly (r  =  .14) to moderately (r  =  .49) correlated with the domains scores of the WHOQOL-BREF. It is difficult to stipulate scores for the concurrent validity purposes, as it depends on the characteristics of the tests used, yet the scores should not be equal to 1 (identical construct) or 0 (different construct; Cicchetti, 1994). According to DeVon et al. (2007), the concurrent validity is substantial and high when coefficients are equal or superior to .45. The concurrent validity of the POS was performed with the GENCAT-Scale (Verdugo, Arias, Gómez, & Schalock, 2008). Although these two scales were based on the same eight domains defined by Schalock and Verdugo (2002), the correlations ranged between material well-being (r  =  .23) to self-determination (r  =  .79; Claes et al., 2010; van Loon et al., 2009).

The concurrent validity between the EPR and the WHOQOL-BREF (Canavarro et al., 2007; Skevington et al., 2004; WHOQOL Group, 1998a) was a challenge, as the scales have different domains and indicators (subjective/objective versus subjective). In the present research, the relationship between the objective/subjective measures (EPR) and the personal sense of well-being (WHOQOL-BREF) was modest. It is interesting to emphasize that the subjective indicators are related to well-being and its importance, while the other indicators advocate circumstances and experiences of objective life, as well as the traditional social goals (Brown et al., 2009; Schalock, 2004; Verdugo et al., 2005). On the one hand, the satisfaction is not consistent with the objective scores (Gardner & Carran, 2005). On the other hand, the objective indicators do not capture the variability and contextual information (Kayes & McPherson, 2010). The literature also highlights the difficulty of comparing subjective and objective measures (Brown et al., 2009; Camfield & Skevington, 2008; Fakhoury & Priebe, 2002; Hensel et al., 2002; Kusel et al., 2007; Phillips, 2006). This fact is consistent with previous studies that have compared the WHOQOL-BREF with other QOL scales, in which weak to moderate correlations were observed (Fakhoury & Priebe, 2002; Kusel et al., 2007).

In addition, the instruments created for a certain population can be inappropriate for another (Finlay & Lyons, 2001; Kayes & McPherson, 2010; McGillivray et al., 2009). The WHOQOL-BREF is a generic tool for assessing QOL (Skevington et al., 2004). In this way, the EPR is an essential scale to measure subjective and objective QOL indicators in Portuguese adults with ID. In the field of ID, only the subjective measures may not be suitable to assess QOL and practitioners' interventions (Hensel et al., 2002). Schalock (2004) considers that the subjective indicators are important to determine the satisfaction of people with ID, but the objective measures should also be used for evaluating environmental design and service programs.

The scores presented good test-retest reliability and results of the two administrations were correlated. All of Pearson's correlations were high to very high (Pestana & Gageiro, 2005), except for the emotional well-being domain in the self-report portion. According to Hartley and MacLean (2006), the test-retest reliability, with 1 to 3 weeks between the applications, are considered moderate (r  =  .80 and ≤ .89) and strong (r ≥ .90). Based on those guidelines, the EPR can be evaluated as a valid and generic QOL scale from this psychometric point of view. The QOL index correlation was moderate (r  =  .89) in the self-report scale and strong (r  =  .92) in the report of others scale. This information is important, as there are no test-retest reliability data available on the original POS (Claes et al., 2010; van Loon et al., 2009).

The two questions that were added to the EPR's self-report portion, regarding importance and satisfaction about the domains, were analyzed separately. The test-retest coefficients showed a slight evidence of repeatability or stability. On the one hand, the questions seem to be based on feelings, opinions, and subjective interpretations (Gardner & Carran, 2005; Kayes & McPherson, 2010) that may have affected the test-retest scores. We emphasize that “in QOL theory, emotions—and related satisfaction judgments—are seen as adaptive in that they provide continuous feedback on progress toward personal goals” (Frisch, 2006, p. 18). However, it seems that stability is lower because the importance and the satisfaction of the eight QOL domains are related to personal circumstances, experiences, and goals. On the other hand, the hierarchical question requires abstract thinking and complex concepts (Finlay & Lyons, 2001; Hartley & MacLean, 2006; Li et al., 2013). For most people with ID, it was difficult to prioritize the domains and differentiate the eight points of response. As a result, these items will not be considered for the EPR's QOL index.

Concerning internal consistency, the lowest Cronbach's alpha scores were observed in the physical well-being and interpersonal relations domains. Alpha coefficients in most domains were acceptable and the scale seems to be reliable in the field of ID (Andresen, 2000; Hartley & MacLean, 2006; Li et al., 2013; Noonan et al., 2009; Robinson et al., 1991; Schalock et al., 2007). Data indicated that the reliability is slightly similar to the original version. Cronbach's alpha on self-report scale was .89 in 79 adults with ID (Claes et al., 2010) and .88 with 399 participants (van Loon et al., 2009). Internal consistency of the report of others scale was respectively .86 (Claes et al., 2010) and .85 (van Loon et al., 2009). The domains ranged from rights (n  =  79; α  =  .50) to social inclusion (n  =  79; α  =  .81), and from material well-being (n  =  399; α  =  .25) to self-determination (n  =  399; α  =  .75; Claes et al., 2010; van Loon et al., 2009). Bonham et al. (2004) also reported lower internal consistency in the report of others measure compared to the self-report measure. Scores of our study contrast with these results, as the report of others portion has a higher internal consistency.

According to the guidelines of Andresen (2000), Cicchetti (1994), Li et al. (2013), and Noonan et al. (2009), inter-respondent coefficients of the domains ranged from moderate to excellent. The results were consistent with the POS, which ranged from .29 to .79 (Claes et al., 2010; van Loon et al., 2009). Our study reflected higher scores between staff members and people with ID in the personal development, self-determination, social inclusion, and rights domains. Simultaneously, we noted a better agreement between key workers and family members in the interpersonal relations and well-being domains (emotional, physical, and material). Schwartz and Rabinovitz (2003) pointed out that staff tended to overestimate life satisfaction by desire for positive outcomes of their work, and so concluded that parents seem to be better judges. However, further investigations should be performed to understand if these results can be related to the expectations of Portuguese families with this population.

The EPR is important and useful because it comprises all QOL principles of conceptualization, assessment, and intervention in the field of ID. Policy and practice have implications in the QOL of adults with ID, and should encourage individual experiences in all contexts of life. This research introduces the EPR as a tool with psychometric properties for assessing QOL in people with ID, from their own perspectives. However, self-report ratings must be complemented by objective measures (report of others). The EPR data is essential to ensure that the needs, decisions, and choices of people with ID are included in Portuguese programs and delivery of services. Adults with ID should be considered within QOL measurement and intervention plans. The QOL assessment should also be used together with other measures, such as the Adaptive Behavior Scale (Nihira, Leland, & Lambert, 1993) and the Supports Intensity Scale (Thompson et al., 2004) to increase planning designs.

Limitations

Our study illustrates the challenge of assessing QOL in people with ID, with both self-report and report of others ratings. Nevertheless, there were limitations and the findings should be cautiously analyzed. As a convenience sample was used, further research is needed to confirm the results in other levels of ID. Unfortunately our research did not provide data on alternative and parallel forms of the EPR, to avoid the burden of participants. Although the selection of the indicators was based from previous literature (Schalock & Verdugo, 2002) and from previous scale (Claes et al., 2010; van Loon et al., 2008, 2009), future research should include additional items in the domains retained by the EFA, such as rights (self-report and report of others measures), and emotional well-being (self-report measure). Future confirmatory factorial analysis research should be performed to prove the EPR structure, and help to understand the inclusion (or not) of the items with low communality coefficients. The psychometric properties of the domains such as interpersonal relations and physical well-being are also in need of improvement in future research. Nevertheless, this article represents an initial research to provide the QOL's construct from theory to practice in Portuguese adults with ID.

Acknowledgments

The authors would like to thank Robert Schalock for reviewing the article, spending considerable time and effort, and providing valuable comments.

References

References
Andresen
,
E.
(
2000
).
Criteria for assessing the tools of disability outcomes research
.
Archives of Physical Medicine and Rehabilitation
,
81
,
S15
S20
. .
Bonham
,
G.
,
Basehart
,
S.
,
Schalock
,
R.
,
Marchand
,
C.
,
Kirchner
,
N.
,
&
Rumenap
,
J.
(
2004
).
Consumer-based quality of life assessment: The Maryland Ask Me! Project
.
Mental Retardation
,
42
(
5
),
338
355
. .
Brown
,
T.
(
2006
).
Confirmatory factor analysis for applied research
.
New York, NY
:
The Guilford Press
.
Brown
,
I.
,
Schalock
,
R.
,
&
Brown
,
R.
(
2009
).
Quality of life: Its application to persons with intellectual disabilities and their families-introduction and overview
.
Journal of Policy and Practice in Intellectual Disabilities
,
6
(
1
),
2
6
. .
Camfield
,
L.
,
&
Skevington
,
S.
(
2008
).
On subjective well-being and quality of life
.
Journal of Health Psychology
,
13
(
6
),
764
775
. .
Canavarro
,
M.
,
Serra
,
A.
,
Simões
,
M.
,
Rijo
,
D.
,
Pereira
,
M.
,
&
Gameiro
,
S.
Carona
,
C.
(
2009
).
Development and psychometric properties of the World Health Organization Quality of Life Assessment Instrument (WHOQOL-100) in Portugal
.
International Journal Behavioral Medicine
,
16
,
116
124
. .
Canavarro
,
M.
,
Simões
,
M.
,
Serra
,
A.
,
Pereira
,
M.
,
Rijo
,
D.
,
&
Quartilho
,
M.
Carona
,
C.
(
2007
).
Instrumento de avaliação da qualidade de vida da Organização Mundial de Saúde: WHOQOL-BREF
.
In
Simões
,
M.
,
Machado
,
C.
,
Gonçalves
,
M.
,
editors.
Almeida
,
L.
,
editor
.
Avaliação psicológica: Instrumentos validados para a população portuguesa
.
77
100
Coimbra, Portugal
:
Quarteto Editora
.
Cicchetti
,
D.
(
1994
).
Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology
.
Psychological Assessment
,
6
(
4
),
284
290
. .
Claes
,
C.
,
van Hove
,
G.
,
van Loon
,
J.
,
Vandevelde
,
S.
,
&
Schalock
,
R.
(
2010
).
Quality of life measurement in the field of intellectual disabilities: Eight principles for assessing quality of life-related personal outcomes
.
Social Indicators Research
,
98
,
61
72
.
http://dx.doi.org/ 10.1007 /s11205-009-9517-7
.
Costello
,
A.
,
&
Osborne
,
J.
(
2005
).
Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis
.
Practical Assessment, Research and Evaluation
,
10
(
7
),
1
9
.
Cummins
,
R.
(
2005
).
Moving from the quality of life concept to a theory
.
Journal of Intellectual Disability Research
,
49
(
10
),
699
706
. .
DeVon
,
H.
,
Block
,
M.
,
Moyle-Wright
,
P.
,
Ernst
,
D.
,
Hayden
,
S.
,
&
Lazzara
,
D.
Kostas-Polston
,
E.
(
2007
).
A psychometric toolbox for testing validity and reliability
.
Journal of Nursing Scholarship
,
39
(
2
),
155
164
. .
Fakhoury
,
W.
,
&
Priebe
,
S.
(
2002
).
Subjective quality of life: Its association with other constructs
.
International Review of Psychiatry
,
14
,
219
224
.
Finlay
,
W.
,
&
Lyons
,
E.
(
2001
).
Methodological issues in interviewing and using self-report questionnaires with people with mental retardation
.
Psychological Assessment
,
13
(
3
),
319
335
. .
Frisch
,
M.
(
2006
).
Quality of life therapy: Applying a life satisfaction approach to positive psychology and cognitive therapy
.
Hoboken, NJ
:
John Wiley & Sons
.
Gardner
,
J.
,
&
Carran
,
D.
(
2005
).
Attainment of personal outcomes by people with developmental disabilities
.
Mental Retardation
,
43
(
3
),
157
174
. .
Hartley
,
S.
,
&
MacLean
,
W.
(
2006
).
A review of the reliability and validity of Likert-type scales for people with intellectual disability
.
Journal of Intellectual Disability Research
,
50
,
813
827
. .
Hensel
,
E.
,
Rose
,
J.
,
Kroese
,
B.
,
&
Banks-Smith
.
J.
(
2002
).
Subjective judgements of quality of life: A comparison study between people with intellectual disability and those without disability
.
Journal of Intellectual Disability Research
,
46
(
2
),
95
107
. .
Henson
,
R.
,
&
Roberts
,
J.
(
2006
).
Use of exploratory factor analysis in published research: Common errors and some comment on improved practice
.
Educational and Psychological Measurement
,
66
(
3
),
393
416
. .
IBM SPSS Statistics for Windows (Version 21.0) [Computer software]
.
Armonk, NY
:
IBM Corp
.
Jenaro
,
C.
,
Verdugo
,
M.
,
Caballo
,
C.
,
Balboni
,
G.
,
Lachapelle
,
Y.
,
&
Otrebski
,
W.
(
2005
).
Cross-cultural study of person-centered quality of life domains and indicators: A replication
.
Journal of Intellectual Disability Research
,
49
(
10
),
734
739
.
Kayes
,
N.
,
&
McPherson
,
K.
(
2010
).
Measuring what matters: Does ‘objectivity’ mean good science
?
Disability and Rehabilitation
,
32
(
12
),
1011
1019
. .
Kimberlin
,
C.
,
&
Winterstein
,
A.
(
2008
).
Validity and reliability of measurement instruments used in research
.
American Journal of Health-System Pharmacy
,
65
(
1
),
2276
2284
. .
Kusel
,
Y.
,
Laugharne
,
R.
,
Perrington
,
S.
,
McKendrick
,
J.
,
Stephenson
,
D.
,
Stockton-Henderson
,
J.
,
&
Burns
,
T.
(
2007
).
Measurement of quality of life in schizophrenia: A comparison of two scales
.
Social Psychiatry and Psychiatric Epidemioly
,
42
,
819
823
. .
Li
,
C.
,
Tsoi
,
E.
,
Zhang
,
A.
,
Chen
,
S.
,
&
Wang
,
C.
(
2013
).
Psychometric properties of self-reported quality of life measures for people with intellectual disabilities: A systematic review
.
Journal of Developmental and Physical Disabilities
,
25
,
253
270
. .
Matsunaga
,
M.
(
2010
).
How to factor-analyze your data right: Do's, don'ts, and how-to's
.
International Journal of Psychological Research
,
3
(
1
),
97
110
.
McGillivray
,
J.
,
Lau
,
A.
,
Cummins
,
R.
,
&
Davey
,
G.
(
2009
).
The utility of the Personal Wellbeing Index Intellectual Disability Scale in an Australian sample
.
Journal of Applied Research in Intellectual Disabilities
,
22
,
276
286
. .
Nihira
,
K.
,
Leland
,
H.
,
&
Lambert
,
N.
(
1993
).
Adaptive Behavior Scale-Residential and Community edition (ABS-RC: 2)
.
Austin, TX
:
Pro-Ed Publishing
.
Noonan
,
V.
,
Miller
,
W.
,
&
Noreau
,
L.
(
2009
).
A review of instruments assessing participation in persons with spinal cord injury
.
Spinal Cord
,
47
,
435
446
. .
Pasquali
,
L.
(
2007
).
Validade dos testes psicológicos: Será possível reencontrar o caminho
?
Psicologia: Teoria e Pesquisa
,
23
,
99
107
.
Pestana
,
M.
,
&
Gageiro
,
J.
(
2005
).
Análise de dados para ciências sociais: A complementaridade do SPSS
.
Lisboa, Portugal
:
Edições Sílabo
.
Phillips
,
D.
(
2006
).
Quality of life: Concept, policy and practice
.
London, England
:
Routledge
.
Robinson
,
J.
,
Shaver
,
P.
,
&
Wrightsman
,
L.
(
1991
).
Criteria for scale selection and evaluation
.
In
Robinson
,
J.
,
Shaver
,
P.
,
editors
.
Wrightsman
,
L.
,
editor.
Measures of personality social psychological attitudes
.
1
15
San Diego, CA
:
Academic Press, Inc
.
Schalock
,
R.
(
2004
).
The concept of quality of life: What we know and do not know
.
Journal of Intellectual Disability Research
,
48
(
3
),
203
216
. .
Schalock
,
R.
,
Gardner
,
J.
,
&
Bradley
,
V.
(
2007
).
Quality of life for people with intellectual and other developmental disabilities: Applications across individuals, organizations, communities, and systems
.
Washington, DC
:
American Association on Intellectual Disability
.
Schalock
,
R.
,
&
Verdugo
,
M.
(
2002
).
Handbook on quality of life for human service practitioners
.
Washington, DC
:
American Association on Mental Retardation
.
Schwartz
,
C.
,
&
Rabinovitz
,
S.
(
2003
).
Life satisfaction of people with intellectual disability living in community residences: Perceptions of the residents, their parents and staff members
.
Journal of Intellectual Disability Research
,
47
(
2
),
75
84
. .
Shogren
,
K.
,
&
Broussard
,
R.
(
2011
).
Exploring the perceptions of self-determination of individuals with intellectual disability
.
Intellectual and Developmental Disabilities
,
49
(
2
),
86
102
. .
Simões
,
C.
,
&
Santos
,
S.
(
2014
).
Cross cultural adaptation, validity and reliability of the Escala Pessoal de Resultados
.
Social Indicators Research
,
119
(
2
),
1065
1077
. .
Skevington
,
S.
,
Lotfy
,
M.
,
&
O'Connell
,
K.
(
2004
).
The World Health Organization's WHOQL-BREF quality of life assessment: Psychometric properties and results of the international field trial
.
Quality of Life Research
,
13
,
299
310
.
Thompson
,
J.
,
Bryant
,
B.
,
Campbell
,
E.
,
Craig
,
E.
,
Hughes
,
C.
,
Rotholz
,
D.
,
&
Wehmeyer
,
M.
(
2004
).
Supports Intensity Scale: Users manual
.
Washington, DC
:
American Association on Mental Retardation
.
Turnbull
,
H.
,
Turnbull
,
A.
,
Wehmeyer
,
M.
,
&
Park
,
J.
(
2003
).
A quality of life framework for special education outcomes
.
Remedial and Special Education
,
24
(
2
),
67
74
. .
van Loon
,
J.
,
van Hove
,
G.
,
Schalock
,
R.
,
&
Claes
,
C.
(
2008
).
Personal Outcomes Scale: A scale to assess an individual's quality of life
.
Ghent, Netherlands
:
Stichting Arduin
.
van Loon
,
J.
,
van Hove
,
G.
,
Schalock
,
R.
,
&
Claes
,
C.
(
2009
).
Personal Outcomes Scale: Administration and standardization manual
.
Ghent, Netherlands
:
Stichting Arduin
.
Verdugo
,
M.
,
Arias
,
B.
,
Gómez
,
L.
,
&
Schalock
,
R.
(
2008
).
Formulari de l'escala GENCAT de qualitat de vida. Manual d'aplicació de l'escala GENCAT de qualitat de vida
.
Barcelona, Spain
:
Departament d'Acció Social i Ciutadania, Generalitat de Cataluña
.
Verdugo
,
M.
,
Schalock
,
R.
,
Keith
,
K.
,
&
Stancliffe
,
R.
(
2005
).
Quality of life and its measurement: Important principles and guidelines
.
Journal of Intellectual Disability Research
,
49
(
10
),
707
717
. .
Wang
,
M.
,
Schalock
,
R.
,
Verdugo
,
M.
,
&
Jenaro
,
C.
(
2010
).
Examining the factor structure and hierarchical nature of the quality of life construct
.
American Journal on Intellectual and Developmental Disabilities
,
115
(
3
),
218
233
. .
WHOQOL Group
. (
1998a
).
The World Health Organization Quality of Life Assessment (WHOQOL): Development and general psychometric properties
.
Social Science & Medicine
,
46
,
1569
1585
.
WHOQOL Group
. (
1998b
).
Development of the World Health Organization WHOQOL-BREF quality of life assessment
.
Psychological Medicine
,
28
,
551
558
.
World Medical Association
. (
2008
).
Declaration of Helsinki: Ethical principles for medical research involving human subjects
. .
Worthington
,
R.
,
&
Whittaker
,
T.
(
2006
).
Scale development research: A content analysis and recommendations for best practices
.
The Counseling Psychologist
,
34
(
6
),
806
838
. .

Author notes

Cristina Simões and Sofia Santos, University of Lisbon, Portugal; and Claudia Claes, University College Ghent.