Self-reported physical activity (PA) behavior with assistance from a secondary source has previously been used with adults with an intellectual disability (ID). Limited evidence of reliability and validity have been provided for this approach. This study examined evidence of convergent (CV) and discriminant (DV) validity for self-report with assistance from a secondary source as a measure of PA in adults with ID. PA of 37 participants with ID were assessed using (a) self-report, (b) accelerometers, and (c) pedometers. The multitrait-multimethod (MTMM) analysis was used to evaluate validity. MTMM analysis revealed high reliability among variables, low to strong CV, and moderate DV. The study outcomes provide initial convergent and discriminant validity evidence for this measure of PA in adults with ID.
Physical activity (PA) is positively associated with health benefits and decreased risk in mortality and morbidity associated with many chronic diseases and conditions (U.S. Department of Health and Human Services [USDHHS], 2008). Physical activity is a complex behavior that can be examined from multiple dimensions, which include physical fitness, energy expenditure, and sedentary behavior. Because of the complexity in PA behavior, a variety of measurement tools have been used to examine this behavior. Currently, self-report measures of physical activity remain the most practical and economical method to assess PA in populations both with and without disabilities (Masse & de Niet, 2012; Temple, Frey, & Stanish, 2006; Warms, 2006).
The need for surveillance and measurement of physical activity behavior in adults with intellectual disability (ID) living in the community has become an important health priority (Fujiura & RRTC Expert Panel on Health Measurement, 2012; U.S. Department of Health and Human Services, 2002; U.S. Department of Health and Human Services, 2010). The use of self-report to capture the physical activity of adults with ID has been used in previous research (Beange, McElduff, & Baker, 1995; Draheim, Williams, & McCubbin, 2002b; Harada & Siperstein, 2009; Matthews et al., 2011; Messent, Cooke, & Long, 1998; Robertson et al., 2000). Likely because of issues related to level of cognitive function in this population, researchers have relied on the use of proxy or secondary sources to assist people with ID in their responses to self-report PA questionnaires. Proxy reporting of PA behaviors of adults with ID involves care providers answering questions on behalf of the person. Self-reporting of PA with assistance from a secondary source involves individuals with ID being asked questions directly and answering for themselves. Secondary sources are categorized as either a family member, care provider, or group home staff that assist people with ID (if needed or requested) with questions pertaining to quantitative judgments and time questions (Finlay & Lyons, 2001; Fujiura & RRTC Expert Panel on Health Measurement, 2012). This approach aligns more with empowerment initiatives for people with ID and aids in some of the methodological issues with self-report.
Studies that have used self-report with assistance from secondary sources to measure PA in people with ID have either not examined or provided validity evidence for using this approach (Cervantes & Porretta, 2010; Stanish, Temple, & Frey, 2006). Cervantes and Porretta's (2010) recent review of the literature examining PA measurement among people with disabilities found that few studies reported validity evidence. It is questionable if previous research using self-report with assistance has captured the accurate PA behavior of adults with ID without adequate support of validity evidence (Finlay & Lyons, 2001; Fujiura & RRTC Expert Panel on Health Measurement, 2012; Temple et al, 2006). Accurate assessment of PA of people with ID is important for inclusion into larger scale epidemiological, surveillance, and intervention studies (Fujiura & RRTC Expert Panel on Health Measurement, 2012; Masse & de Niet, 2012; Matthews et al., 2011; U.S. Department of Health and Human Services, 2010).
This study examined the validity evidence for self-report with assistance from a secondary source to measure PA in adults with ID living in the community using the multitrait-multimethod (MTMM) analysis (Campbell & Fiske, 1959; Rowe & Mahar, 2006). The MTMM analysis examines the relationship between convergent validity (evidence that measures of the same construct correlate highly) and discriminant validity (evidence that measures of different constructs do not correlate as highly as measures of the same construct). We hypothesized that self-report with assistance from a secondary source to measure PA would correlate with objective measures of PA (accelerometers and pedometers), providing evidence of convergent validity (CV). We also hypothesized that the two objective measures of PA would correlate highly providing further evidence of CV. Finally, we hypothesized that a self-report with assistance measure of fat intake in adults with ID would not correlate as much with a self-report with assistance measure of PA and objective measures of PA, providing evidence of discriminant validity (DV).
Thirty-seven participants with ID age 19–74 years participated in the study. Eleven of the participants had Down syndrome. Participants were recruited through county offices of Developmental Disabilities Services, area Arc offices in a northwestern state, and private and state-operated assisted living programs. Because of confidentiality policies of these service agencies, researchers were not allowed to recruit participants directly until the service agencies performed an initial screening of interested participants. Service agencies were asked to identify eligible participants based on the following criteria: (a) having an intellectual disability as defined by the American Association on Intellectual and Developmental Disabilities (Luckasson et al., 2002); (b) residence in a community setting; and (c) independent ambulation. Interested participants were identified and their contact information was forwarded to the first author. The living arrangements of the participants were distributed accordingly: Six (16%) participants lived with family; 21 (57%) lived in group homes; and 10 (27%) lived semi-independently. Table 1 provides descriptive information on the participants. Before data collection, informed consent from all participants was obtained, and all procedures were approved by the investigator's Institutional Review Board for the Protection of Human Subjects.
Thirty-two secondary sources assisted the participants with ID during the interview-administered survey questions. These secondary sources were instructed to assist the participants with responses only when participants requested or when participants responded with “I don't know.” Secondary sources assisted the adults with ID in recalling duration of physical activities and frequency of food intake. Twenty-four (75%) of the secondary sources spent ≥ 20+ hr and ≥ 3 days a week with the participants they assisted. Twenty-five (78%) of the secondary sources did not live with the participant. Twenty-three (72%) of the secondary sources were either care providers or group home staff. Twenty-eight (88%) of the secondary sources that assisted with self-report measures assisted for both administration of the measures. Previous studies (Draheim et al., 2002b; Draheim, Williams, & McCubbin, 2003; Stanish & Draheim, 2005) have used self-report with assistance from a secondary source as a measure of PA through the Third National Health and Nutrition Examination Survey (NHANES) III (National Center for Health Statistics, 1994) in adults with ID based on two assertions. Adults with ID residing in the community routinely participate in certain physical activity on daily and weekly basis. Secondary sources that assist with self-report measures are more likely to assist with the daily activity schedules of the adults with ID and are therefore knowledgeable of their habitual PA behaviors.
Weight (kilograms) and height (centimeters) were measured with participants dressed in lightweight clothing. Body mass index (BMI) was calculated according to the formula: body mass (kg) divided by height squared (m2). Age, gender, intellectual disability condition (Down syndrome and non-Down syndrome), and weekly working hours were collected on all participants.
The PA questionnaire section of the NHANES III (National Center for Health Statistics, 1994) was used to assess regular PA habits. The PA survey was administered by the first author through an interview with the participant and the secondary source. The original mode of administration of the NHANES III PA survey is interviewer-administered, and this has been the most used form of self-report for adults with ID given the methodological issues mentioned previously (Finlay & Lyons, 2001). Contents of the interviewer-administered survey on PA determined if participants had walked (jogged or run) 1 mile or more at a time without stopping; had ridden a bicycle; had swum; had participated in aerobics or aerobic dance, other dancing, calisthenics, or floor exercise; had done gardening or yard work; or had lifted weights during the past week. For each positive response, participants were asked to indicate duration of activities, such as how many times and how long they performed the activity. The NHANES III PA survey has been used in previous research to measure PA behavior in adults with ID (Draheim et al., 2002b; Draheim et al., 2003; Stanish & Draheim, 2005).
Accelerometers and pedometers were used as objective measures of PA for the participants. The Actiwatch accelerometer (Mini Mitter, Bend, OR) is a small, lightweight, limb or waist worn, activity monitoring device. The Actiwatch has been used in previous research to measure PA of people with developmental disabilities (Kim & Yun, 2009). The Omron HJ-112 pedometers (Omron Healthcare, Vernon Hills, IL) were used to measure the number of steps that each participant accumulated. The Omron HJ-112 has been used in previous research to measure walking activity of people with ID, including Down syndrome (Kim & Yun, 2009; Pitchford & Yun, 2010).
Dietary Fat Intake
To complete evaluation of the MTMM analysis, discriminant validity evidence was needed; and the results from the Block Fat Screener (NurtitionQuest, Berkeley, CA) were used for evaluating evidence of DV. The screener was used to examine dietary fat intake. This brief screening tool includes 17 questions, and it is designed to rank people from low to high with regard to their usual fat intake during the week or month. The screener includes the top sources of fat as determined by national surveys and recent research (Block, Gillespie, Rosenbaum, & Jenson, 2000). For the purposes of analyses, a corresponding number from 1 to 3 was assigned to the low to high rankings, respectively. Fat intake served as the second construct that was used to examine the strength of validity evidence for self-reported PA with assistance from a secondary source in adults with ID. The Block Fat Screener has been used previously with adults with ID (Draheim, Williams, & McCubbin, 2002a).
Participants wore an elastic belt consisting of an accelerometer and pedometer located on their right waist or hip during waking hours, except for when performing water activities, for two 7-day periods. Pedometers were sealed to prevent tampering. Accelerometers were exchanged every 2 to 3 days, depending on location of participant from the researcher, and pedometers were unsealed during the exchange time to record previous step counts and check for potential tampering. Following the first 7 days of wearing the accelerometer and pedometer, the NHANES III PA survey and Block Fat Screener were administered to the participants with ID and secondary source. These procedures were repeated following the second 7 consecutive days of wearing the monitors. The time between first and second administration of the two self-report surveys was an average of 7 days with a range between 2 and 14 days.
Descriptive statistics were used to summarize all demographic information, and evidence of validity was evaluated using the MTMM analysis. The MTMM analysis is a systematic procedure for evaluating the evidence of validity of a measurement instrument by examining the pattern of multiple validity and reliability coefficients (Campbell & Fiske, 1959; Rowe & Mahar, 2006). It examined the strength of the convergent evidence, discriminant evidence, and reliability evidence.
In MTMM, the reliability coefficients should be greater than the CV and DV coefficients. The CV coefficients should be greater than DV coefficients. The relationship between self-reported PA and objective measures of PA (CV 1) coefficients represent the agreement between two attempts to measure the same construct through different methods and should be higher than the DV coefficients. Because two objective measures of PA were used in this study, a relationship between pedometer and accelerometer measures of PA (CV 2) coefficient was also determined and expected to be higher than CV 1 and the DV coefficients. The DV coefficients represent the relationship between different constructs using similar and dissimilar methods of measurement. Among DV coefficients, coefficients from the relationship between self-reported PA and fat intake (DV 1) should be higher than coefficients from the relationship between self-reported fat intake and objective measures of PA (DV 2) (Campbell & Fiske, 1959; Yun & Ulrich, 2002). When the expected pattern of reliability and validity coefficients is apparent, the validity evidence becomes stronger. Because the researchers were only interested in establishing evidence of validity for the PA measure, a CV coefficient for fat intake was not included.
Reliability coefficients were calculated from a two-way random analysis of variance (ANOVA). Intraclass correlation (ICC) was used to examine interview-1 to interview-2 reliability in self-reported PA bouts and fat intake, total number of activity counts per 7-day observation, and total number of steps taken per 7-day observation. The usual desired value for a multiple day intraclass correlation is about 0.8 (Baranowski & de Moor, 2000). One participant's Week 1 data were collected over 8 consecutive days because of a nonwear day occurrence; and two participants' Week 2 data were collected over 8 consecutive days because of a nonwear day occurrence. In both occurrences, the nonwear day was removed and the additional day was added to complete the analysis. Pearson-Product Moment correlation coefficients were calculated to examine the association between self-reports with assistance variables (reported PA bouts per interview and reported fat intake per interview), accelerometers, and pedometers. The correlation coefficients interpretive magnitudes were considered low (0.3), moderate (0.5), and strong (0.7). Analyses were performed using SPSS 15.0 (Statistical Package for the Social Sciences, Inc, Chicago, IL).
The reliability coefficients, CV coefficients, and DV coefficients are provided in Table 2. The reliability coefficient for self-report with assistance as a measure of PA was ICC (2, 2) = 0.80. The ICC (2, 2) means that a two-way random ANOVA model was used with an average of two measures. The CV 1 coefficients for self-reported PA with accelerometers and pedometers were r = 0.34 (p < 0.05) and r = 0.52 (p < 0.01), respectively. The results indicate a low and moderate positive relationship as expected. The CV 2 coefficient between the objective measures of PA was r = 0.85 (p < 0.01) indicating a strong positive relationship as expected. The DV 1 coefficient between self-reported fat intake and PA was r = −0.37 (p < 0.05) indicating a low negative relationship that was not expected. The DV 2 coefficients between self-reported fat intake and the objective measures of PA (accelerometers and pedometers) were r = 0.00 and r = −0.23, respectively. These results indicate a minimum relationship, as expected.
The primary goal of this study was to examine the convergent and discriminant evidence of validity for self-report with assistance from a secondary source as a measure of PA in adults with ID. Examination of the pattern of reliability and validity coefficients of the MTMM analysis reveals strong reliability evidence, low to strong convergent evidence, and moderate discriminant evidence for PA assessment using self-report with assistance in adults with ID. The reliability coefficients were all within the desirable range with the objective measures of PA in adults with ID demonstrating the highest values. Previous researchers have reported stronger evidence of reliability for PA from objective measurement tools in populations without disability (Bassett, Cureton, & Ainsworth, 2000; Macfarlane, Lee, Ho, Chan, & Chan, 2006; Matthews, Moore, George, Sampson, & Bowles, 2012; Sallis & Saelens, 2000; Tudor-Locke, Ainsworth, Thompson, & Matthews, 2002; Tudor-Locke, Williams, Reis, & Pluto, 2002; Tudor-Locke, Williams, Reis, & Pluto, 2004). The results from this study support the argument that objective measurements have better psychometric properties than subjective measurement tools. The CV coefficients were greater than all of the DV coefficients, except for the DV 1 coefficient. This exception has been explained as a prevalent threat to the MTMM analysis and results from common method variance. Common method variance is defined as the overlap in variance between two variables ascribed to the type of measurement instrument used rather than due to a relationship between the underlying constructs (Drost, 2011).
The low to moderate positive relationships of CV 1 coefficients were supported from outcomes previously reported by Draheim et al. (2003), who demonstrated a similar association between self-reported PA with assistance through the NHANES III survey and certain cardiovascular disease risk factors (e.g., systolic and diastolic blood pressure) in adults with ID. These outcomes were also similar to previous studies that have indicated similar coefficient values between self-report measures of PA, accelerometers, and pedometers in populations without disability (Bassett et al., 2000; Macfarlane et al., 2006; Sallis & Saelens, 2000; Tudor-Locke, Ainsworth, et al., 2002; Tudor-Locke, Williams, et al., 2002; Tudor-Locke, Williams, et al., 2004). One main explanation for the observed CV 1 relationships exhibited in studies with populations with ID and without disability is that self-report measures of PA are often employed in applications for which their use was not supported by validity evidence (Masse & de Niet, 2012; Matthews et al., 2012; Rimmer, 2006). This is surely the case for populations with ID because of the dearth of contextually specific self-report measures of PA (Fujiura & RRTC Expert Panel on Health Measurement, 2012). The study demonstrated a moderate positive relationship among self-report with assistance from a secondary source through the NHANES III PA survey and pedometer outcomes. A previous study by Stanish and Draheim (2005) that assessed walking activity using a pedometer and the NHANES III PA survey in adults with ID found no significant correlations between pedometer step counts and any of the NHANES III PA survey variables (including total number of PA bouts per week reported). Reasons for the different outcomes from the two studies could include the duration of observation (7 days vs. 14 days); definition of PA bouts; and type of pedometer used. The moderate evidence of self-report fat intake with accelerometer and pedometer DV is supported from outcomes previously reported that indicated that PA was not significantly associated with BMI (Fujiura, Fitzsimons, Marks, & Chicoine, 1997) or abdominal obesity (Draheim et al., 2002a) in adults with ID.
This study provides further evidence that self-report with assistance from a secondary source as a measure of PA through the NHANES III survey is a reliable procedure to use with adults with ID. Our self-report with assistance measure of PA reliability coefficient was similar to the value previously reported by Stanish and Draheim (2005). This study provides further evidence that pedometer outcomes are a reliable measure of walking activity in people with ID. Our pedometer reliability coefficient value was similar to values previously reported by Stanish (2004) and Pitchford and Yun (2010), who examined the accuracy of pedometers and walking activity in adults with ID (including Down syndrome). The study also provides evidence that accelerometer outcomes are a reliable measure of PA in adults with ID. Our accelerometer reliability value was higher than the value (ICC = 0.87) previously reported by Frey (2004) who compared PA levels between adults with and without ID.
The MTMM approach provides evidence to validate the adequacy and appropriateness of decisions made from measurement outcomes. This approach afforded the researchers an opportunity to utilize multiple measures (self-report, accelerometers, and pedometers) to examine the PA behaviors of adults with ID. The use of multiple measures to examine PA has been a point of emphasis in terms of improving the quality of PA research with populations with (including ID) and without disability, especially for self-reports used in large epidemiologic studies (Cervantes & Porretta, 2010; Matthew et al., 2012; Temple et al., 2006; Warms, 2006; Yun & Ulrich, 2002). This is the first study to our knowledge to have a collection period a minimum of 14 days for accelerometer and pedometer data. The extended period of time for collection coupled with the reliability and validity coefficients provides evidence that more longitudinal studies of PA behavior in adults with ID can be undertaken.
Limitations of this study include the following. The participants were not randomly selected, but volunteered. The sample of the present study included a higher proportion of adults with Down syndrome than what is generally observed in the U.S. population of adults with ID. No statistically significant differences were found, however, when researchers examined group differences on all variables of interest (self-report variables, activity counts, and step counts). These data were collected during the summer months, when PA habits likely differ from those occurring during colder and inclement seasons. Seasonal variation in participation may influence the estimates of prevalence of PA via questionnaires (Draheim et al., 2002b; Draheim et al., 2003; Temple et al., 2006).
Reliability and validity issues of the responses of people with ID to self-report measures of PA—whether completed independently, with assistance from a secondary source, or through proxy–have been identified as a pressing research need in the literature (Draheim et al., 2002b; Draheim et al., 2003; Fujiura & RRTC Expert Panel on Health Measurement, 2012; Matthews et al., 2011; Temple et al., 2006). Self-report with assistance from a secondary source as a measure of PA in adults with ID was highly reliable. The results indicated low to moderate CV 1 evidence and strong CV 2 evidence. The results indicated moderate DV 1 and DV 2 evidence. Results also indicated that accelerometer and pedometer outcomes are reliable and that significant relationships exist among the outcomes as measures of PA in adults with ID. Further research is needed on the development of contextually specific self-report measures of PA for adults with ID, which could strengthen validity evidence. Through estimating measurement validity in PA research, we may be able to better quantify progress toward meeting national physical activity objectives and to better evaluate our efforts toward reducing PA disparities between those with and without ID (Matthews et al., 2011; Rimmer, 2006, U.S. Department of Health and Human Services, 2010).
This research was supported by the Arc of Washington Trust Fund.
Marquell Johnson, University of Wisconsin-Eau Claire; Joonkoo Yun, Oregon State University, Corvallis; and Jeffrey A. McCubbin, Colorado State University, Fort Collins.
Address correspondence concerning this article to Marquell Johnson, Department of Kinesiology, McPhee Physical Education Center 227, University of Wisconsin-Eau Claire, 105 Garfield Avenue, P.O. Box 4004, Eau Claire, WI 54702-4004 USA (email: firstname.lastname@example.org).