Abstract

Low language ability and response bias are frequently cited as impediments to valid responding to items on interview schedules. Structured interviews with a random sample of 154 adults with mental retardation showed that around two thirds of respondents were either unable to respond or exhibited response bias. There was a significant difference in scores on the Adaptive Behavior Scale between those who exhibited response bias and those who did not. In cases of nonresponse or response bias, the substitution of the respondent with a proxy respondent has been proposed as an alternative method of gathering subjective data. In this study concordance between individuals' responses and those of proxies was low on a subjective measure and high on an objective scale.

Editor in charge: Steven J. Taylor

The need to evaluate major service reform has meant that interest in quality of life as it relates to people with mental retardation has increased since the early 1970s, and especially since the late 1980s (Hughes, Hwang, Kim, Eisenman, & Killian, 1995). There has been a progression in the quality of life literature from an initial lack of unanimity about how the broader concept of quality of life should be defined to an emerging consensus (Hatton, 1998). In the last few years, techniques for assessing the satisfaction of people with mental retardation with various aspects of their lives have gained increasing prominence. However, there are a number of potentially problematic issues regarding eliciting subjective responses, which are especially pertinent in the case of people with mental retardation.

The term quality of life is generally considered to encompass the breadth of life experience. Most commentators agree that quality of life should reflect various important domains of life (e.g., Campbell, Converse, & Rodgers, 1976; Cummins, 1997; Felce, 1997; Parmenter, 1988; Schalock, 1996). In their review of 87 empirical studies on quality of life published between 1970 and 1993, Hughes et al. (1995) identified 14 frequently cited quality of life domains: social relationships and interaction; psychological well-being and personal satisfaction; employment; self-determination, autonomy, and personal choice; recreation and leisure; personal competence, community adjustment and independent living skills; residential environment; community integration; support services received; individual and social demographic indicators; personal development and fulfillment; social acceptance, social status, and ecological fit; physical and material well-being; and civic responsibility. The majority of these quality of life domain areas have been addressed in many comprehensive evaluations of services for people with mental retardation (Brown, Bayer, & Brown, 1992; Burchard, Hasazi, Gordon, & Yoe, 1991; Conroy & Bradley, 1985; Felce, 1989; Lowe & de Paiva, 1991).

Although there appears to be some consensus about the domains that are relevant, there has been considerable debate about how quality of life assessment should be undertaken. For example, Hughes et al. (1995) found that 1,243 measures had been used to assess quality of life. The principal point of contention has been about whether it is possible to measure quality of life objectively, or whether, by definition, quality of life is synonymous with subjective appraisal. Borthwick-Duffy (1992) presented three perspectives on quality of life defined in terms of (a) life conditions, (b) satisfaction with life conditions, and (c) a combination of life conditions and satisfaction. A fourth perspective (d), favored by many recent authors, takes a multidimensional view of quality of life as a combination of life conditions and satisfaction weighted by personal value (e.g., Campbell et al. 1976; Cummins, 1997; Felce, 1997; Parmenter, 1988; Schalock, 1996).

Subjective appraisal (i.e., rating satisfaction) has been a cornerstone of research on the quality of life of the general population (e.g., Andrews, 1974; Campbell et al., 1976). However, objective assessment has dominated service evaluation in the mental retardation field (e.g., Emerson, 1985; Emerson & Hatton, 1996). There are a number of reasons for this lack of research on the satisfaction of people with mental retardation. By virtue of the cognitive and language deficits of individuals with mental retardation (e.g., comprehension, memory, and expression), obtaining their views, particularly about complex abstract issues, is obviously difficult. Sigelman, Budd, Winer, Schoenrock and Martin (1982) identified four standards that could be applied in judging the validity of information gained directly from people with mental retardation.

The first standard is responsiveness (i.e., the proportion of interviewees who could answer a question, regardless of the truth or inaccuracies of answers). Sigelman, Schoenrock, Spanhel, Hromas, Winer, Budd, and Martin (1980) found that responsiveness varied as a function of the level of respondent IQ, and Sigelman, Winer and Schoenrock (1982) found that responsiveness varied as a function of the way questions were framed. For example, yes–no questions about activities yielded higher responses than did yes–no questions about subjective phenomena, such as happiness. The latter type of question was answered more frequently than verbal multiple-choice questions or open-ended questions, both of which are cognitively more demanding. Our first aim in the current study was to investigate responsiveness, in particular, how responsiveness related to language ability and to levels of adaptive behavior.

The second standard was test–retest reliability (i.e., correspondence between answers to the same questions on two occasions). Test–retest reliability figures reported by Sigelman et al. (1983) were extremely variable. On average, a sample of institutionalized children gave consistent responses to yes–no questions about activities asked on two occasions 87% of the time. However, the consistency of answers to multiple-choice questions presenting four levels of happy and sad faces and enquiring about satisfaction with living circumstances was only 46%. Heal and Sigelman (1996) commented that it is difficult to isolate levels of test–retest reliability from levels of validity. For example, the higher levels of test–retest reliability found in the first sample may be evidence of a tendency to answer yes–no questions affirmatively rather than evidence of good test–retest reliability.

Sigelman, Budd et al.'s (1982) third standard was consistency–correspondence between responses to questions whose wording or format differed but whose meaning remained the same. The most common form of inconsistency among people with mental retardation is the tendency to acquiesce, that is, to say yes to questions regardless of their content. Naysaying, the tendency to say no to all questions is also a problem, although naysaying is less common. Heal and Sigelman (1996) reported a number of studies in which researchers investigated acquiescence. For example, Sigelman and her colleagues (1981b) found that levels of acquiescence for various items about choice ranged from 44% to 71%, whereas naysaying occurred up to 12% of the time. High levels of acquiescence among people with mental retardation have been reported frequently in other studies (e.g., Harner, 1991; Heal & Rubin, 1993; Heal & Chadsey-Rusch, 1985; Novak, Heal, Pilewski, & Laidlaw, 1989).

Another form of systematic response bias is recency bias, whereby the respondent has a tendency to choose the last option offered in response to an either–or question or multiple-choice question. It can be detected by seeing whether the respondent chooses the second option on both of two either–or questions that have the same content but differ with respect to the order of options. The opposite, primacy, may also occur. Sigelman, Budd, Spanhel, and Schoenrock (1981a) reported that recency and primacy biases characterized an average of 28% of responses to either/or questions in three samples. Heal and Rubin (1993) reported recency bias at 10% in their study of 91 adults with mental retardation.

One strategy to reduce response bias has been to precede self-report scales with questions designed to screen responses for bias (Kishi, Teelucksingh, Zollers, Park-Lee, & Meyer, 1988). Stancliffe (1995b) found that only 2.1% of respondents who passed the lie/acquiescence item on Kishi et al.'s (1988) decision-making scale subsequently displayed acquiescence.

The possibility of biased responding exists whenever anyone undertakes a self-report questionnaire or interview. The likelihood of bias occurring is greater among people with mental retardation (Heal & Sigelman, 1990), although it is by no means inevitable (Cattermole, Jahoda, & Markova, 1988; Voelker, Shore, Brown-More, Hill, & Perry, 1990). Given the range of adaptive functioning among people with mental retardation, particularly with respect to language, information on the relationship between response bias and ability would be valuable to those wishing to assess life satisfaction using a self-report methodology. The current study aimed to investigate levels of acquiescence and recency bias and to demonstrate how these forms of response bias related to respondent ability.

The final standard applied by Sigelman, Budd et al. (1982) in judging the quality of information received from interviewees with mental retardation was its agreement with information received from other sources. The use of proxy respondents, such as staff or parents, is one possible solution when the type of problems mentioned above result in nonresponse or responses that are of dubious validity. The opinions and judgments of significant others regarding quality of life issues are often used as the basis of service delivery (McVilly, Burton-Smith, & Davidson, 2000). Several researchers have attempted to evaluate consumer–proxy agreement by comparing proxy responses about people who can respond for themselves with self-reports from these same people. Research on this issue has left uncertainty as to whether proxy responses can substitute for self-reporting. For example, in their analysis of agreement between the responses of staff and those whom they supported, Budd, Sigelman, and Sigelman (1981) reported agreement at only 52% for a yes–no checklist and 60% for an open-ended question about participation in various sports. Jenkinson, Copeland, Drivas, Scoon, and Yap (1992a) found that staff reported significantly less involvement of people with mental retardation in decision-making than these people reported themselves. Other researchers have cast doubt on the accuracy of proxy responses, particularly in the case of questions about satisfaction with various aspects of their lives (e.g., Berkson & Romer, 1980; Burnett, 1989; Rapley, Ridgeway, & Beyer, 1998; Stancliffe, 1995b; Voelker et al., 1990). However, there is also evidence of consumer:proxy agreement (e.g., McVilly et al., 2000; Schalock & Keith, 1993; Stancliffe, 1999; Voelker et al., 1990). Furthermore, as Stancliffe (2000) has pointed out, “It remains an open question as to whether findings of agreement between proxies and self-reports from verbal individuals can be generalized to non-verbal people with more profound mental retardation who cannot respond for themselves” (p. 90).

In addition to investigation of levels of responsiveness and response bias and their relation to adaptive behavior, in the current study we examined the degree of correlation between the responses of people with mental retardation and those of staff responding on their behalf.

Method

Sampling

The sample was drawn from the southern half of Wales and a territory of equivalent population size in southwest England. Eleven of the 22 Welsh local authorities (statutory bodies responsible for commissioning services) and eight English local authorities were selected on the basis that they were within reasonable travelling distance from the research base.

The contracting officer in each authority was asked for information about all staffed houses serving six or fewer adults with mental retardation. This yielded a total of 585 settings. The decision to focus on small scale staffed housing was taken because these are a significant form of residential service provision in Britain for adults with mental retardation. The mean occupancy of staffed residential homes in Wales is about 3 people (Perry, Beyer, Felce, & Todd, 1998). In England it is 6.3 people, with 45% of homes catering for three or fewer people (Department of Health, 2000). There is no evidence to suggest, or reason to believe, that the population served in small scale settings differs significantly from the general population of people with mental retardation. Indeed, the characteristics of individuals living in small-scale settings are likely to be representative of the general population of people with mental retardation.

Having numbered each house, we used a personal computer to select 88 settings randomly (44 in England and 44 in Wales) in proportion to the pattern of statutory, voluntary, and private provision in the total population of services. A random stratified sample of 80 settings was achieved. This represented 14% of all the settings in the selected areas. Data relating to setting process and structure were collected in all 80 settings. The results of analyses of these data were reported by Perry, Lowe, Felce, and Jones (2000). Because of the impracticality of collecting outcome data in all 80 settings, a subsample of 47 settings with 154 people being supported was selected on the basis that the range of ability levels among those supported and representation of provider agencies remained consistent with the initial sample. In the current paper, the results relate to the final sample.

Interviews were undertaken in interviewees' homes by the first author or a trained data collector. It was not possible to examine interrater reliability on the subjective measures because of the potentially confounding effect of poor test–retest reliability. However, there was evidence of reliable coding of subjective data, derived from the second data collector coding tape-recorded interviews blind. The time taken to administer the scales varied considerably between interviewees. For those people who proceeded beyond the pretesting phase of the scales (see below), it took about 20 minutes on average to administer each of the British Picture Vocabulary Scale and Choice Questionnaires, and 30 minutes to administer the Comprehensive Quality of Life Scale.

Participants and Setting Descriptors

Of the 47 settings, 25 were provided by the voluntary sector and 16 were privately operated. In the United Kingdom, these sectors are sometimes referred to as the independent sector. Four were run by local authorities and 2 were health authority settings (statutory government bodies). Setting size averaged 3.3 people with a range from 1 to 5. Fifty eight of the 154 people interviewed were women and 96 were men. The average age was 46 years. The average total raw score on the Adaptive Behavior Scale (see below) across all 154 people being supported was 185 out of a possible 307. Individual Adaptive Behavior Scale raw scores ranged from 51 to 290.

Measurement

The Adaptive Behavior Scale (Part One) (ABS) (Nihira, Leland, & Lambert, 1993)

The ABS consists of 73 items spanning 10 domains of adaptive behaviour. A total ABS raw score was calculated by combining all domain scores, with the exception of Vocational Activity. This was omitted because the items relate to work outside the residential setting. Therefore, the maximum possible raw score was 307. An overall percentile rank score was then calculated. Interrespondent reliability was checked by obtaining independent assessments from two staff members for a sample of 29 people being supported (12% of the initial sample). It was calculated in three separate ways in an attempt to take into account the fact that for some of its items a checklist format is used, whereas for others, there is a scale format. First, interrespondent reliability was calculated by dividing the number of times staff members agreed exactly on an item by the total number of items and multiplying by 100. Second, lower item scores of the respective pairs were divided by the higher, converted to percentages and averaged. Third, the difference in item scores were expressed as proportions of the possible range, converted to percentages and deducted from 100%. Regardless of how it was calculated, interrespondent reliability was similar (range = 75% to 81%).

The British Picture Vocabulary Scale (second edition) (Dunn, Dunn, Whetton, & Burley, 1997)

This instrument was designed primarily to measure receptive vocabulary. Pointing to evidence that vocabulary subtests have proved to be among the most important contributors to comprehensive tests of intelligence (Elliot, 1983, 1990), the authors note that the British Picture Vocabulary Scale may also be used as a measure of cognitive ability. It is also appropriate for use with people with severe speech or language difficulties because responses can be gestural.

Normative scores for the current sample were standardized against the oldest reference group listed in the manual, which was comprised of adolescents between 15.5 and 15.67 years of age. The manual also provides tables for the derivation of age equivalent scores, indicating the age at which a particular raw score is the average.

Dunn et al. (1997) reported a median alpha coefficient of .86 for raw scores across all age groups. Evidence of content validity, construct validity, and concurrent validity is also provided by the authors.

The Choice Questionnaire (Stancliffe, 1995a)

This questionnaire was developed to assess the degree of choice exercised by people with mental retardation. It contains 26 items that relate to six domains: Domestic Activities, Staff and the Other People You Live With, Money and Spending, Health, Social Activities, Community Access and Personal Relationships, Work/Day Activities, and Overall Choice. It can be completed by a care provider, or the individual with mental retardation. Whether the respondent was the care provider or the person being supported, the Choice Questionnaire was administered as a structured interview. For all items the respondent is requested to indicate which of three possible levels of choice is available. In essence, the three levels range from no choice to complete choice, with some choice constituting the middle level. For example, the first item is, “Who decides what time you go to bed?” The three response options are: “I decide for myself,” “I usually decide with help. Sometimes others tell me,” or “I have a set bedtime” or “Others mostly tell me when to go to bed.” Items concern factual information about choice as opposed to appraisals of satisfaction or dissatisfaction with choice. In this sense, although responses are supplied by the person with mental retardation, it should be classified as an objective scale.

When people with mental retardation are interviewed, 4 additional items precede the 26 items of the main questionnaire (see below). Two of these were designed to test for evidence of the recency effect in the responses of those supported, the other two were designed to assess acquiescent responding. The recency items were modeled on similar questions used by Sigelman and her colleagues (1983). The main items of the questionnaire are not administered if the pretest items show evidence of either type of response bias.

Recency items:

(a) Do you live by yourself or with other people?

______ By myself  ______ With others

(b) Do you live with other people or by yourself?

______ By myself  ______ With others

Actual situation: lives alone/with others

Acquiescence items:

Most mornings do you pick what clothes to wear?

______ Yes  ______ Sometimes  ______ No

Most mornings does someone else tell you what clothes to wear?

______ No  ______ Sometimes  ______ Yes

In the current study, these preliminary questions were put to everyone who was responsive to a very simple question about themselves.

Stancliffe and Parmenter (1999) reported that Cronbach's alpha was .81 for self reports and .90 for staff responses. Test–retest correlations for total scores were reported as .95 and .89 for self-report and staff report, respectively. Interrater reliability for staff report was .86. These results are based on the self-reports of 30 individuals (15 males, 15 females) with mental retardation. Their mean age was 35.6 years, and, according to results on the Peabody Picture Vocabulary Test–Revised, 18 had severe mental retardation, 4 had moderate disability, and 8 had mild. In the current study, exact item-by-item interrespondent agreement for the scale administered to carer givers was 68%.

Comprehensive Quality of Life Scale–Intellectual/Cognitive Disability (5th ed.)—(ComQol–ID) (Cummins, 1997)

The version of the ComQol used in the current study, the ComQol-ID was designed specifically for use with people with mental retardation. It measures seven domains objectively and subjectively: Material Wellbeing, Health, Productivity, Intimacy, Safety, Community, and Emotional Wellbeing. Three items related to each of the seven objective domains. The objective scale was administered to staff in the current study and is not relevant to the issues of self-reporting or proxy responding; therefore, the administration of the objective scale is not described here and scores on it are not reported. A comprehensive pretesting procedure is undertaken prior to administration of the subjective scale in order to ensure that respondents are not prone to acquiescence, that they understand the concepts of satisfaction and importance, and that the response choices they are eventually given are appropriate to their level of ability.

An acquiescent responding scale is administered first in which the following four questions are asked:

1. (After pointing to the person's watch or some item of clothing) Does that (watch) belong to you?

2. Do you make all your own clothes and shoes?

3. Have you seen the people who live next door?

4. Did you choose who lives next door?

Questions 2 and 4 require negative responses to demonstrate that acquiescence is absent. If positive responses are provided to these items, no further testing takes place.

The respondent is then tested for discriminative competence. There are three phases of testing that are designed to determine whether the person can (a) identify items in order of magnitude; (b) use a scale by matching to a concrete reference; and (c) use a scale with an abstract reference. Within each phase, the tasks progress in complexity from binary choice to a choice involving five elements. For example, the binary choice for the first phase was between a picture of a large cube (or block) and a small one. The most complex task of the second phase was to match five blocks of varying sizes to five steps of varying sizes. The tasks of the third (abstract reference) phase used two, three, and five steps, respectively, each accompanied by a pictorial representation of the concept of importance. The respondent is required to demonstrate that he or she can associate these with varying levels of importance.

At each phase, respondents have to respond correctly, at least using the binary choice, in order to proceed to the next phase. Thus, if they do not respond correctly to all three phases, they do not complete the subjective scale. For those who pass the pretest, the maximum level of discriminative competence displayed during the third phase determines the level of choice provided in the importance subscale. This comprised one item on each of the seven quality of life domains. For example, “How important to you is how healthy you are?” The respondents then point to the point on the scale that they feels best reflects their opinion. Continuing with the example of health, a person who had only managed to discriminate between a binary choice at the third phase of pretesting would point to one of two icons to indicate how important they felt their health was, whereas someone who had been able to discriminate at five levels would point to one of the five anchors of a five-point Likert scale.

There was only one phase of pretesting for the satisfaction domain. The first two phases were omitted on the grounds that a respondent reaching this stage of pretesting already would have demonstrated the ability to use a scale with an abstract reference. For the satisfaction domain pretest, respondents had to discriminate between pictures of faces with expressions ranging from happy to sad. As with pretesting for the importance domain, respondents had to make a binary choice initially and, if successful, they progressed to a choice between three, and then five faces. The satisfaction scale was comprised of one item for each of the domains. Each was prefaced with the phrase: “How happy or sad do you feel about ____________.”

The final section of the ComQol allows a care provider to respond to the importance and satisfaction subscales on behalf of the person with learning disabilities. The care provider uses the 5-point importance and satisfaction scales, respectively, to respond to the questions. Care providers were asked to respond from the point of view of the person being supported. If they felt unable to do this for particular items, those items were coded as missing.

The ComQol manual contains a scale conversion table so that respondent ratings of importance or satisfaction can be represented on a scale of one to five, irrespective of whether two, three, or five-point Likert scales were used. Scale conversion is not based on standardized scores.

Reliability of the ComQol

Cummins (1997) reported Cronbach's alpha coefficients for the seven domains of the subjective scale as .48 for importance and .65 for satisfaction. These values were derived from Cummins (1992), in which the third edition of the scale was used with 59 individuals who had mental retardation (Slosson mean age equivalent 5.4 ± 1.68) living in small group homes, 61% of whom were male.

Results

Levels of Response Bias When Using Subjective Quality of Life Measures

Rank ABS scores and nonparametric statistical procedures were used in the analysis because not all variables were normally distributed nor was variance homogeneous for all variables. Figure 1 shows the percentage of respondents who were unable to respond to the pretest items of the Choice Questionnaire and ComQol. A nonresponse was considered to occur if the respondent said nothing throughout the presentation of pretest items or if none of their responses was related to the questions asked. Figure 1 also shows the percentage of respondents who exhibited response bias (see ComQol and Choice Questionnaire descriptions above), classified by ABS level. The horizontal axis represents people grouped by rank ABS score tenths. All 15 people in the bottom two tenths (rank ABS scores up to 20%) did not respond to the questions on either the Choice Questionnaire or the ComQol. Eighty five percent of people with rank scores between 21 and 50 also did not respond, and all of the remainder showed acquiescence bias and most showed recency bias. In the five upper tenths (rank ABS scores between 51 and 100), nonresponse rates declined progressively from 38% through 30%, 25%, 14% to 0%. Acquiescence bias affected between a half and two thirds of the remaining respondents, with rank ABS scores in the two tenths between 51 and 70. Above this level, response biases were shown by only a minority. Respondents were, on the whole, slightly less vulnerable to the recency effect than to acquiescence.

Figure 1

Percentage of people who exhibited response bias, during pretesting on the Choice Questionnaire (CQ) and the ComQol (Comprehensive Quality of Life Scale)

Figure 1

Percentage of people who exhibited response bias, during pretesting on the Choice Questionnaire (CQ) and the ComQol (Comprehensive Quality of Life Scale)

There was a significant difference between those who acquiesced on the ComQol and those who did not in terms of total rank ABS scores, scores on the language development domain of the ABS, and British Picture Vocabulary Scale scores. The same was true for those who acquiesced on the Choice Questionnaire and those who did not, and those who exhibited response bias on the Choice Questionnaire and those who did not. Test statistics are shown in Table 1.

Table 1

Mann-Whitney Test Results

Mann-Whitney Test Results
Mann-Whitney Test Results

In general, then, only people whose ABS scores were in the top three tenths were likely to be able to complete these scales without acquiescing. The average nonresponse rate for this group was 14% and average “pass” rates for the remaining respondents were 85% and 81% on the Choice Questionnaire and ComQol, respectively. However, it can be seen from Table 2 that there was a wide range in the three ability measures among people who did not show response bias. Among this group scores on the ABS language development domain tended to be high, with raw scores averaging 37 out of a possible 43. However, there were examples of people with scores as low as 26/43 who did not exhibit response bias. Similarly, total ABS raw scores averaged 244 out of a possible 307, although the bottom of the range was 171. The range of British Picture Vocabulary Sale raw scores among respondents who did not show response bias was similarly large (33 to 142). The mean score among those who passed the pretests (89) converts to an equivalent language age of 9.33 years,

Table 2

Characteristics of People (N = 44) Who Did Not Show Response Bias on the CQ or the ComQol

Characteristics of People (N = 44) Who Did Not Show Response Bias on the CQ or the ComQol
Characteristics of People (N = 44) Who Did Not Show Response Bias on the CQ or the ComQol

Comparison Between Staff Proxy Responses and Those of the People Being Supported

The scores of the people being supported were not significantly correlated with scores based on staff proxy responses on either the importance or satisfaction dimensions of the ComQol. In view of the overall degree of disparity between these two sources, the relationship between the responses of people with mental retardation and those of proxies on each of the 16 ComQol items was investigated. There was only one significant correlation of the 16, and this is consistent with what might be expected by chance given a 5% critical region. In contrast, the correlation between the scores of those being supported and staff total scores on the Choice Questionnaire was highly significant, n = 56, rho =.74, p <.01.

Discussion

Evaluation of residential services has taken a broadening view over time of the quality of life of people with mental retardation. Measures have tended to proliferate without a move towards consensus as to what can and should be measured. Notwithstanding the evident difficulties that many people with mental retardation have with understanding and using language, there is increasing advocacy for obtaining the views of people with mental retardation about their lives, particularly if quality of life is seen as a primarily subjective construct. There is also a debate about the role of objective measurement among quality of life researchers.

The self-report scales could not be used properly by two thirds of the sample, which was randomly selected and, therefore, likely to be representative of the population of people with mental retardation in staffed housing services in the United Kingdom. Because the sample spanned the tenths of the ABS rank scores, which were normed on a large United States sample, inferences cannot be made beyond the United Kingdom. It does not seem to be possible to use the self-report measures with anyone with average rank ABS scores below 50%. No one with average rank ABS scores in the first quartile (up to 25%) responded to the questions in the Choice Questionnaire or ComQoL. Nonresponse was also high among people with average rank ABS scores in the second quartile (26% to 50%), and all those who did respond showed response bias. We note that the sample may not be representative of the population of people with mental retardation in other countries.

In general, the measures could only be used with individuals who had scores in the top three tenths. The difference in ABS scores found between those who acquiesced and those who did not is consistent with previous studies of response bias both among people with mental retardation (e.g., Sigelman, Budd, Spanhel, & Schoenrock, 1981b) and in the general population (Gudjonsson, 1990). The average British Picture Vocabulary Scale scores for people able to use the scales translated to an age equivalent of over 9 years, which gives some indication of the high receptive language and cognitive abilities required, a level that precludes use by people with severe and profound mental retardation.

In view of the high level of response bias reported, one might question the validity of screening procedures in which four or fewer items are used. The accuracy with which these pretests distinguish between those who are likely to be free of response bias and those who are not is unknown. It may be the case that they were too stringent or too lenient. There is certainly scope for further research to compare brief tests such as those used in the current study with those designed to examine response bias in more depth. However, in the interests of user-friendliness, the length and complexity of pretests should be considered alongside their validity, especially when they are to be used as a precursor to fairly lengthy self-report scales. In a similar vein, although it does not detract from our purpose in this study, which was to investigate responsiveness and response bias, one might question the utility of an acquiescent scale for a measure such as the ComQol, which relies on multiple-choice Likert scales.

Further modification of response format may have the potential to increase the proportion of people with mental retardation who can respond without bias, although it should be remembered that one of the measures already employed iconic formats. It may be possible to teach people with mental retardation how certain icons relate to concepts. To a certain extent this is done in the ComQol insofar as the respondent learns to point to icons as a means of expression. However, caution is required if investigators are to avoid running the risk of denying the impact of intellectual disability on language comprehension and use. It would be a mistake to attend solely to ease of nonverbal expression, as in pointing to one of a range of iconic representations, without giving equal consideration to how to communicate the nature of the question and whether it is or could be understood. Receptive and expressive language skills may differ in an individual, in some cases considerably so. However, in general, they are reasonably similar. Many subjective quality of life questions appeal to abstract constructs, such as satisfaction, happiness, or contentment, and address constructs such as choice, friendship, and health. Understanding of constructs is a relatively sophisticated language stage, and it is this which might best be viewed as setting a limiting criterion to the people from whom subjective opinions can be elicited. Although it is clearly possible to employ nonverbal means of expression, it is much more difficult to pose questions in a way that is similarly independent of spoken language. It is clearly illogical to recognize absence of verbal ability in the expressive domain while continuing to ask relatively demanding questions verbally. Rigorous standards would suggest that investigators substantiate that the questions posed are understood by respondents in the same way as the questioner understands them. Confirmation of verbal understanding is not provided by pointing to an iconic representation. It is to some extent provided by the response bias screening question incorporated into the two scales investigated.

The need for independent corroboration that questions have been understood is illustrated by an example from the current study. When one man, who was reasonably articulate and who had passed all the ComQol pretests, was asked how happy or sad he felt about the things he owned (material wellbeing), he began to list his possessions. His response was uninterpretable as a comment on whether he was or was not satisfied. In this case, his verbal response to the interviewer demonstrated that he was not answering the question as posed. If the man had responded by pointing to one of the iconic representations of happiness, his lack of understanding would not have been apparent.

At a certain level of severity of disability, adoption of alternative methods of data collection appears to be necessary in order to assess the quality of life of people with mental retardation. One such method is the use of staff or carer proxy responses. However, proxy responses on the ComQol were not significantly correlated with self-reported views among people who passed the screening tests. This is consistent with much of the literature on third-party responding to measures of satisfaction (e.g., Jenkinson, Copeland, Drivas, Scoon, & Yap, 1992b; Stancliffe, 1995b; Wehmeyer & Metzler, 1995). These findings, therefore, provide no rationale for the use of staff as proxy respondents in relation to subjective issues. Further research is required to investigate whether consumer:proxy concordance varies as a function of the nature of the relationship between consumers and proxies. For example, it may be that more accurate proxy responses would be made by people such as family members or citizen advocates who have had long-term relationships with those on behalf of whom they are responding.

Despite the lack of correlation found between proxy responses and those of people with mental retardation on the ComQol, there was a high degree of association between the scores obtained on the Choice Questionnaire when administered to either staff or the people being supported. This is consistent with Stancliffe and Parmenter's (1999) finding of a lack of significant consumer–proxy differences using the Choice Questionnaire and Stancliffe's (1999) report of “substantial” positive correlation between consumers and staff on the Empowerment factor of the Quality of Life Questionnaire (Schalock & Keith, 1993). One could argue that the Quality of Life Questionnaire and the Choice Questionnaire are really objective measures because although individuals with mental retardation are the respondents, on the whole, respondents are asked about objective facts rather than about levels of satisfaction. Thus staff may be adequate proxy respondents in relation to interviews that solicit objective facts but not when appraisal of the satisfaction of people being supported is concerned. If this is the case, it may be prudent for quality of life assessment and research to be confined to objective methods for those individuals who are unable to voice their own opinions. It remains an open question as to whether proxy responses that might not reflect consumers' views accurately, are preferable to a total absence of responses.

In this study we have examined only two of the several barriers to the effective use of self-report measures with people who have mental retardation. Only brief reference has been made to other potential barriers, such as the impact of interviewee experience, social desirability, adaptation and personal temperament. The importance of facilitating the direct involvement of as many people with mental retardation as possible in the quality assurance process is beyond doubt. However, it is equally important to exercise caution when interpreting or acting upon self-reported data, particularly that which is subjective, because of the distinct possibility of invalid results. In addition, the exclusive use of self-report and subjective methodologies will inevitably exclude people with severe mental retardation. Thus, there is a compelling argument for the continued use of objective measures in the quality assurance process.

Note: This research was conducted under a grant from the Department of Health, Research and Development Division. We are grateful to all those who participated in the study.

References

References
Andrews
,
F.
1974
.
Social indicators of perceived quality of life.
Social Indicators Research
1
:
279
299
.
Berkson
,
G.
and
D.
Romer
.
1980
.
Social ecology of supervised communal facilities for mentally disabled adults: 1. Introduction.
American Journal on Mental Deficiency
85
:
219
228
.
Borthwick-Duffy
,
S.
1992
.
Quality of life and quality of care in mental retardation.
In L. Rowitz (Ed.), Mental retardation in the year 2000. New York: Springer-Verlag
.
Brown
,
R.
,
M.
Bayer
, and
P.
Brown
.
1992
.
Empowerment and developmental handicaps: Choices and quality of life.
Toronto: Captus
.
Budd
,
E. C.
,
C. K.
Sigelman
, and
L.
Sigelman
.
1981
.
Exploring the outer limits of response bias.
Sociological Focus
14
/
4
:
297
307
.
Burchard
,
S.
,
J.
Hasazi
,
L.
Gordon
, and
J.
Yoe
.
1991
.
An examination of lifestyles and adjustment in three community residential alternatives.
Research in Developmental Disabilities
12
:
127
142
.
Burnett
,
P.
1989
.
Assessing satisfaction in people with an intellectual disability living in community based residential settings.
Australian Disability Review
1
:
14
19
.
Campbell
,
A.
,
P. E.
Converse
, and
W. L.
Rodgers
.
1976
.
The quality of American life.
New York: Russell Sage Foundation
.
Cattermole
,
M.
,
A.
Jahoda
, and
I.
Markova
.
1988
.
Leaving home: The experience of people with mental handicap.
Journal of Mental Deficiency Research
32
:
47
57
.
Conroy
,
J.
and
V.
Bradley
.
1985
.
The Pennhurst longitudinal study: A report of five years of research and analysis.
Philadelphia: Temple University, Developmental Disabilities Center
.
Cummins
,
R.
1992
.
Comprehensive Quality of Life Scale: Intellectual disability.
Melbourne, Australia: Deakin University
.
Cummins
,
R.
1997
.
The Comprehensive Quality of Life Scale: Intellectual disability (5th ed).
Melbourne, Australia: Deakin University
.
Department of Health.
2000
.
Community care statistics 1999: Residential personal social services for adults, England.
London: Author
.
Dunn
,
L.
,
L.
Dunn
,
C.
Whetton
, and
J.
Burley
.
1997
.
The British Picture Vocabulary Scale (2nd ed).
Windsor: NFER-NELSON
.
Elliot
,
C.
1983
.
The British Ability Scales—Technical handbook (manual 2).
Windsor: NFER-NELSON
.
Elliot
,
C.
1990
.
Differential Ability Scales: Introductory and technical information.
San Antonio, TX: Psychological Corp
.
Emerson
,
E.
1985
.
Evaluating the impact of deinstitutionalization on the lives of mentally retarded people.
American Journal of Mental Deficiency
90
:
277
288
.
Emerson
,
E.
and
C.
Hatton
.
1996
.
Deinstitutionalization in the UK and Ireland: Outcomes for Service Users.
Journal of Intellectual and Developmental Disability
21
:
17
37
.
Felce
,
D.
1989
.
Staffed housing for adults with severe and profound mental handicaps: The Andover Project.
Kidderminster, UK: BIMH
.
Gudjonsson
,
G.
1990
.
The relationship of intellectual skills to suggestibility, compliance and acquiescence.
Personality and Individual Differences
5
:
303
314
.
Harner
,
C.
1991
.
Assessing the satisfaction of adults with mental retardation living in the community.
Unpublished doctoral dissertation, University of Illinois, Urbana
.
Hatton
,
C.
1998
.
Whose quality of life is it anyway? Some problems with the emerging quality of life consensus.
Mental Retardation
36
:
104
115
.
Heal
,
L. W.
and
J.
Chadsey-Rusch
.
1985
.
The Lifestyle Satisfaction Scale (LSS)—Assessing individuals' satisfaction with residence, community setting, and associated services.
Applied Research in Mental Retardation
6
:
475
490
.
Heal
,
L.
and
S.
Rubin
.
1993
.
Biases in responses during the interviews of individuals with mental retardation.
Paper presented at the annual meeting of the American Association on Mental Retardation, Washington, DC
.
Heal
,
L.
and
C.
Sigelman
.
1990
.
Methodological issues in measuring the quality of life of individuals with mental retardation.
In R. Schalock (Ed.), Quality of life: Perspectives and issues (pp. 161–176). Washington, DC: American Association on Mental Retardation
.
Heal
,
L.
and
C.
Sigelman
.
1996
.
Methodological issues in quality of life measurement.
In R. Schalock (Ed.), Quality of life—Volume 1: Conceptualization and measurement (pp. 91–104). Washington DC: American Association on Mental Retardation
.
Hughes
,
C.
,
B.
Hwang
,
J.
Kim
,
L.
Eisenman
, and
D.
Killian
.
1995
.
Quality of life in applied research: A review and analysis of empirical measures.
American Journal on Mental Retardation
99
:
623
641
.
Jenkinson
,
J.
,
C.
Copeland
,
C.
Drivas
,
H.
Scoon
, and
M.
Yap
.
1992a
.
Decision making by community residents with an intellectual disability.
Australia and New Zealand Journal of Developmental Disabilities
18
:
1
8
.
Jenkinson
,
J.
,
C.
Copeland
,
V.
Drivas
,
H.
Scoon
, and
M.
Yap
.
1992b
.
Decision making by community residents with an intellectual disability.
Australia and New Zealand Journal of Developmental Disabilities
18
:
1
8
.
Kishi
,
G.
,
B.
Teelucksingh
,
N.
Zollers
,
S.
Park-Lee
, and
L.
Meyer
.
1988
.
Daily decision-making in community residences: A social comparison of adults with and without mental retardation.
American Journal on Mental Retardation
92
:
430
435
.
Lowe
,
K.
and
S.
de Paiva
.
1991
.
NIMROD: An overview.
London: Her Majesty's Stationary Office
.
McVilly
,
K.
,
R.
Burton-Smith
, and
J.
Davidson
.
2000
.
Concurrence between subject and proxy ratings of quality of life for people with and without intellectual disabilities.
Journal of Intellectual Disability Research, 44, 19–39
.
Nihira
,
K.
,
H.
Leland
, and
N.
Lambert
.
1993
.
Adaptive Behavior Scale (2nd ed).
Washington, DC: American Association on Mental Deficiency
.
Novak
,
A.
,
L.
Heal
,
M.
Pilewski
, and
T.
Laidlaw
.
1989
.
Apartment placement from a community ICFMR.
Paper presented at the annual meeting of the American Association on Mental Retardation, San Francisco
.
Perry
,
J.
,
S.
Beyer
,
D.
Felce
, and
S.
Todd
.
1998
.
Strategic service change: Development of core services in Wales, 1983–1995.
Journal of Applied Research in Intellectual Disabilities
11
/
1
:
15
33
.
Perry
,
J.
,
K.
Lowe
,
D.
Felce
, and
S.
Jones
.
2000
.
Characteristics of community housing services for people with learning disabilities: A stratified random sample of statutory, voluntary, and private agency provision.
Health and Social Care in the Community
8
/
5
:
307
315
.
Rapley
,
M.
,
J.
Ridgeway
, and
S.
Beyer
.
1998
.
Staff:staff and staff:client reliability of the Schalock and Keith (1993) Quality of Life Questionnaire.
Journal of Intellectual Disability Research
42
:
37
42
.
Schalock
,
R.
(
Ed.
).
1996
.
Quality of life. Volume 2: Application to persons with disabilities.
Washington, DC: American Association on Mental Retardation
.
Schalock
,
R.
,
G.
Bonham
, and
C.
Marchand
.
2000
.
Consumer based quality of life assessment: A path model of perceived satisfaction.
Evaluation and Program Planning
23
/
1
:
75
85
.
Schalock
,
R.
and
K.
Keith
.
1993
.
Quality of Life Questionnaire.
Worthington, OH: IDS
.
Sigelman
,
C. K.
,
E. C.
Budd
,
C. L.
Spanhel
, and
C. J.
Schoenrock
.
1981a
.
Asking questions of retarded persons: A comparison of yes no and either or formats.
Applied Research in Mental Retardation
2
/
4
:
347
357
.
Sigelman
,
C. K.
,
E. C.
Budd
,
C. L.
Spanhel
, and
C. J.
Schoenrock
.
1981b
.
When in doubt, say yes: Acquiescence in interviews with mentally-retarded persons.
Mental Retardation
19
/
2
:
53
58
.
Sigelman
,
C. K.
,
E. C.
Budd
,
J. W.
Winer
,
C. J.
Schoenrock
, and
P. W.
Martin
.
1982
.
Evaluating alternative techniques of questioning mentally retarded persons.
American Journal of Mental Deficiency
86
:
511
518
.
Sigelman
,
C.
,
C.
Schoenrock
,
E.
Budd
,
J.
Winer
,
C.
Spanhel
,
P.
Martin
,
S.
Hromas
, and
G.
Bensberg
.
1983
.
Communicating with mentally retarded persons: Asking questions and getting answers.
Lubbock: Texas Tech University, Research and Training Center in Mental Retardation
.
Sigelman
,
C.
,
C.
Schoenrock
,
C.
Spanhel
,
S.
Hromas
,
J.
Winer
,
E.
Budd
, and
P.
Martin
.
1980
.
Surveying mentally retarded persons: Responsiveness and response validity in three samples.
American Journal of Mental Deficiency
84
:
479
486
.
Sigelman
,
C.
,
J.
Winer
, and
C.
Schoenrock
.
1982
.
The responsiveness of mentally retarded persons to questions.
Education and Training of the Mentally Retarded
17
:
120
124
.
Stancliffe
,
R.
1995a
.
Choice and decision making and adults with intellectual disability.
Macquarie University, Sydney, Australia
.
Stancliffe
,
R. J.
1995b
.
Assessing opportunities for choice-making: A comparison of self-reports and staff reports.
American Journal on Mental Retardation
99
:
418
429
.
Stancliffe
,
R. J.
1999
.
Proxy respondents and the reliability of the Quality of Life Questionnaire Empowerment factor.
Journal of Intellectual Disability Research
43
:
185
193
.
Stancliffe
,
R. J.
2000
.
Proxy respondents and quality of life.
Evaluation and Program Planning
23
/
1
:
89
93
.
Stancliffe
,
R.
and
T.
Parmenter
.
1999
.
The Choice Questionnaire: A scale to assess choices exercised by adults with intellectual disability.
Journal of Intellectual and Developmental Disability
24
/
2
:
107
132
.
Voelker
,
S.
,
D.
Shore
,
C.
Brown-More
,
L.
Hill
, and
J.
Perry
.
1990
.
Validity of self-report of adaptive behavior skills by adults with mental retardation.
Mental Retardation
28
:
305
309
.
Wehmeyer
,
M.
and
C.
Metzler
.
1995
.
How self-determined are people with mental retardation? The national consumer survey.
Mental Retardation
33
:
111
119
.

Author notes

Authors:Mr. Jonathan Perry, MSc ( perry@cf.ac.uk) and David Felce, PhD ( felce@cf.ac.uk), Professor, Welsh Centre for Learning Disabilities Applied Research Unit, Meridian Court, North Road, Cardiff, CF14 3BG, Wales, United Kingdom