Reviews of research on deinstitutionalization show that investigators have focused primarily on adaptive behavior changes of “movers,” while paying minimal attention to “stayers.” Analysis of their research also revealed some methodological problems. We assessed 150 movers and 150 stayers in 1994, before deinstitutionalization began in 1997. We matched samples on seven distinctive variables, which were again assessed at similar intervals of 3, 15, and 27 months after movement. Movers made significant gains in independence and household skills, while stayers incurred losses in social skills and cognitive competencies. Teaching domestic skills and high autonomy were the best predictors of final self-care and multicognitive competencies, after controlling for initial baseline scores.
The number of persons living in American state-operated institutions has declined from 193,000 in 1967 (the peak year) to about 43,000 in 2002 (Coucouvanis, Polister, Prouty, & Lakin, 2003; Lerman, 1982). An additional 31,000 individuals lived in institutions that were privately operated—an increase from the 26,000 residents in private institutions in 1970 (Coucouvanis et al., 2003; Lerman, 1982). The initial momentum for deinstitutionalization was spurred by the documented deleterious effects of institutions on the lives of residents. Summarizing the literature between the 1960s and the 1980s, Larson and Lakin (1989) cited “evidence of high mortality rates, decreased measured intelligence and loss of adaptive abilities over time, and more frequent disordered personalities among institution populations” (p. 325). One response to these negative findings was an effort to reform institutions with the assistance of federal Medicaid funds. A second response was the movement of persons into “normalized, least restrictive environments” outside institutions (Lerman, 1982, p. 159).
As the movement out of institutions accelerated, many empirical studies were launched in order to document the impact on persons who left the institutions. In the past several years, there have been a number of reviews of research studies on the impacts of deinstitutionalization in America (Kim, Larson, & Lakin, 2001; Larson & Lakin, 1989; Lynch, Kellow, & Wilson, 1997), the United Kingdom (Emerson & Hatton, 1996), and Australia (Young, Sigafoos, Suttie, Ashman, & Grevell, 1998). The reviewers of these studies have primarily focused on the impacts on those who left institutions (“movers”) while paying much less attention to those persons who remained in large facilities (“stayers”). These reviewers have agreed that there are likely to be significant improvements in adaptive behaviors of people who leave institutions and move into smaller community residences. However, the reviewers of the research studies reported a limited understanding of the diverse impact patterns that might occur for stayers, when compared to movers.
In a search of the literature, we located only one recent study in which the researchers examined the longitudinal effects on adaptive behaviors of persons remaining in a state-operated institution in Minnesota in the 1990s (Stancliffe & Hayden, 1998). Using an aggregate measure of adaptive behaviors, Stancliffe and Hayden found that there were no significant changes over a period of 4 years, even for those persons who moved into smaller living units within the institution. The absence of any negative impact may be due to the fact that upgrading of living units, staffing, and programming had occurred within state institutions since the infusion of federal Medicaid funds in the late 1960s. However, it is also possible that the aggregate or global measure of adaptive behaviors used was not sufficiently sensitive to identify changes in specific types of adaptive competencies.
The problem of measurement sensitivity also arises when analyzing the reviews of research studies of movers. An analysis of the reviews also reveals that other methodological issues, such as rigorous comparative designs and precision matching, may be lacking in studies in which researchers assessed changes in adaptive behaviors of movers compared to stayers.
Kim et al. (2001) conducted the largest systematic review of research about deinstitutionalization (N = 250 studies). They found that only 38 of these studies (15%) met minimal standards of research adequacy. Selection criteria included (a) a minimum of 5 participants, (b) basic demographic information; (c) adult subjects only, (d) inclusion of baseline data while participants were residing in institutions, (e) posttest periods of at least 6 months, and (f) use of the same adaptive measures at baseline and posttest(s). Because Kim et al. were interested in assessing only studies conducted in the United States, the final number of reviewed studies was reduced to 33. They found that only 12 of the 33 studies included a contrast group of movers and a matched group of institutional stayers. Out of these 12 more rigorous studies, 8 contained reports of statistically significant improvement compared to the control group for a measure of overall adaptive behavior or self-care skills. In the remaining 4 studies, investigators reported positive gains for community residents, but these gains were not significant at the .05 level. Using the best American studies as a benchmark, Kim et al. found that improved mover outcomes were reported in about two thirds of the studies. In their review, however, they did not offer any assessment of what happened to the comparison groups of stayers; no within-group analyses of stayers were presented.
Besides the 12 contrast-group studies, Kim et al. (2001) also identified 21 other American research evaluations that were longitudinal but lacked an adequate institutional comparison group of stayers. Analyses of the longitudinal outcomes for overall adaptive behaviors revealed that 14 of the 21 (or two thirds) were statistically significant in a positive direction and one was significant in a negative direction.
In addition to the variability in methodological rigor, assessments of outcomes may also differ in the measurement of adaptive behaviors. Kim et al. (2001) found that in the 33 American studies, investigators used nine adaptive behavior instruments. In a meta-analysis that included 7 of the studies that Kim et al. believed to be more rigorous, plus 4 others, Lynch et al. (1997) found that the type of instruments used had a substantial impact on the types of statistical effects found in the studies. Recent researchers have illustrated how a global index of adaptive behaviors can refer to a combined score on four distinct behavioral domains: (a) motor skills, (b) social and communication skills, (c) personal living skills, and (d) community living skills (Heller, Miller, & Hsieh, 2002; Stancliffe, Hayden, Larson, & Lakin, 2002). The 18 to 21 items employed for each domain and overall scoring are unique to the broad independence aggregate meaning of the Inventory for Client and Agency Planning (ICAP), developed by Bruininks, Hill, Weatherman, and Woodcock (1986).
In contrast to the ICAP measures of adaptive behaviors, other researchers have relied on a modified form of the American Association on Mental Retardation (AAMR) Adaptive Behavior Scale— ABS (Conroy & Elks, 1999; Conroy & Feinstein, 1993; Conroy & Seiders, 1994). The 32 items used by Conroy and his colleagues to assess overall adaptive behaviors, when analyzed, can refer to the domains of Cognition, Self-Care, Communication, Social Skills, and Mobility. Except for the exclusion of Cognition, there appears to be some degree of overlap between the ICAP and the modified ABS domains; however, there are large differences in the number of items used for each domain and in the total number of items (32 vs. 72 to 84, respectively).
The diverse approaches to assessing adaptive behaviors, however, appear to share one common assumption, namely, that it is useful to combine the scores on distinctive domains as an additive measure that can produce an overall mean score of adaptive behaviors. Although this type of global approach may have its adherents, it is incapable of detecting different changes that might occur within one or more of the domains. Not all of the contrast studies reviewed by Kim et al. (2001) use an aggregated measure of adaptive behaviors, but it is noteworthy that in 9 of the 12 contrast studies, the researchers relied on this approach.
Instead of assuming that the use of an aggregate measure of adaptive behaviors is warranted, it appears reasonable to test the proposition that the domains of Motor Skills, Self-Care Skills, and other domains “go together” as a unitary measure. Factor analyses of scores on all the domains could be used to test whether all of the domains loaded on a single dimension or whether a domain, such as Motor Skills, was an independent factor. When this approach was used in New Jersey with a random sample of 1,190 persons living in institutions in 1994, Jagannathan, Gaboda, and Murray (1995) found strong empirical evidence that Motor Skills was, indeed, a distinctive domain that did not load with other adaptive behaviors. Recently, Conroy, Spreat, Yuskauskas, and Elks (2003) reanalyzed 31 adaptive behaviors used to assess people with disabilities in Oklahoma and found that these behaviors referred to three discrete factors.
In addition to testing whether an aggregated measurement of adaptive behaviors is warranted, it is also important to improve the matching criteria used in any evaluation of movers and stayers. The best studies in which researchers use contrast—or comparison—groups are prospective, and they use distinct baseline measures that are under their control. One measure used in most studies relies on matching movers and stayers on levels of mental retardation, which are reported by institutional personnel and often assessed at varying time periods. As Felce and Emerson (2001) noted:
Matching by level of mental retardation, which appears to have been the basis of several studies, rather than by scores on a more finely grained assessment scale, may be considered inadequate given the evidence of association between adaptive behavior and outcomes. (p. 80)
Instead of assessing persons into gross categories of retardation, such as borderline, severe, or profound, it is much more useful for researchers to measure specific cognitive abilities related to space, time, color, numbers, words, and communication skills before persons move out of institutions, and then use these scores as a “fine-grained” matching criterion. Measures of cognitive abilities or competencies can then be repeatedly assessed over time as a distinctive domain that may or may not change for movers and matched stayers.
We based the present study on a prospective research design in which we matched 150 movers and stayers on age and gender and relied on the use of distinctive assessments of abilities in cognition, communication, social–emotional functioning, self-care, and mobility as well as challenging behaviors toward others and self. All matched persons in the study were assessed in 1994 before there was a decision to close an institution in the state of New Jersey. After movers left the institution in 1997, we assessed them and their matches at similar repeated intervals over a 27-month period in order to identify any differences in one or more types of adaptive behaviors. Our major purpose in this study was to first identify the outcomes that can occur when there is an opportunity to implement a controlled longitudinal research design of movers and stayers with more precise measures of adaptive behaviors and improved matching criteria. After assessing patterns of changes within and between samples, we systematically searched for variables that best predict (or explain) the adaptive behavior scores using hierarchical regression analyses. The results are presented, therefore, in separate descriptive and explanatory sections.
The primary data source for constructing matched groups of movers and stayers was the New Jersey Client Assessment Form–hereafter called the Assessment Form developed by staff of the Developmental Disabilities (DD) Planning Institute in 1994 after reviewing existing forms used by the states of California, Connecticut, and New York (Lerman et al., 1995). The items constructed for the Assessment Form were initially conceptualized as measuring six specific domains or areas of functioning, by relying on the responses of knowledgeable caregivers for a random sample of 1,190 persons residing in eight New Jersey Developmental Disabilities (DD) Centers. The forms were self-administered and completed in 20 to 30 minutes under the supervision of DD Planning Institute staff members. All data analyses were conducted using SAS software Version 6.12. A series of factor analyses (employing orthogonal and oblique rotations via Varimax and Promax procedures, respectively, and loadings of .50 or greater) revealed that each of the distinct domains could be empirically measured.
The six domains and their characteristics are as follows:
Cognitive Abilities: 16 items referring to whether the consumer was able to associate time with events and actions, demonstrate spatial/perceptual abilities, and display number awareness as well as writing and reading skills in the past 4 weeks. Responses were coded as yes (1) or no (0).
Communication: 11 items referring to clarity of speech as well as other expressive and receptive communication skills in the past 4 weeks. Responses were coded as yes (1) or no (0).
Self-Care and Independent Living Skills: 21 items referring to degrees of independence in performing basic skills of eating, drinking, dressing, and toileting as well as community and household skills in the past 4 weeks. Responses were coded as ability to start and finish without prompts or help (3), requires mainly verbal prompts (2), requires lots of hands-on help (1), and not able or had no opportunity (0).
Social–Emotional Functioning: 13 items referring to demonstrations of sociability, friendliness, and caring for others in the past 4 weeks. Responses were coded as often (2), occasionally (1), and never (0).
Mobility/Motor Control: 8 items referring to ability to demonstrate physical capabilities in the past 4 weeks. Responses were coded as independently (2), needed help (1), and not able (0) as well as 2 items assessing the ability to walk independently (scored 0 to 4), and wheelchair independence (scored 0 to 4).
Special Behaviors: 24 items referring to disruptive and aggressive behaviors toward self and others during the past year. Responses were coded as yes (1) or no (0).
After we statistically analyzed the initial domains, we conducted additional second-order factor analyses in order to examine any possible overlap between factors associated with each domain of functioning (using Varimax and Promax rotation procedures and loading criteria at 0.50 or greater). Second-order factor analyses of the items that constituted the initial domains of Cognition, Communication, and Social–Emotional Functioning revealed that there was indeed overlap among these three areas. All of the 40 items actually referred to a single broader domain of Multicognitive Functioning (see Factor 1 in Table B that follows Appendix A).
The 10 items referring to ambulation, motor control, and physical skills continued to function as a distinctive single domain as a second factor. Mobility skills did not load above .50, with the multicognitive items in the second-order factor analyses. In contrast, the 21 self-care items loaded at a high level on both the Multicognitive and Mobility domains. The Self-Care domain was, therefore, conceived as an overlapping domain, dependent on both multicognitive and mobility competencies. In a test–retest study conducted over 15 to 18 months using the same staff caregivers, we confirmed the strong stability of each of the domains (see Table B following Appendix A for more detail).
Separate second-order factor analyses of the special behavior items revealed that two specific domains were assessed: (a) 13 items referring to aggressive and inappropriate behaviors toward others, such as like hitting and threatening others; and (b) 11 items referring to harmful and inappropriate behaviors toward self, such as head-banging and excessive scratching.
This study of adaptive behavior competencies was part of a larger evaluation research design of the closure of a New Jersey DD Center, the North Princeton Developmental Center–hereafter called North Princeton. We designed this evaluation to assess program similarities and differences as well as the quality of life and well-being of movers compared to stayers. As noted earlier, a 1994 sample of 1,190 persons from each of the eight New Jersey DD Centers, including North Princeton, were assessed using the New Jersey Client Assessment Form. A representative sample of North Princeton persons constituted the mover sample and the random sample of those remaining in the other seven DD Centers constituted the pool of persons who could be matched as the stayer sample (see Appendix B for more detail).
The major characteristics of the final mover sample, as well as their individual stayer matches, are presented in Table 1. Using the characteristics of the final mover sample (n = 150) as the basis for matching, the stayer sample was constructed by finding persons in the other DD Centers who had been assessed in 1994 using the following characteristics in sequential order:
Gender: An exact gender match was found for each mover, resulting in an equal proportion of males and females (59% and 41%, respectively) in the mover and stayer samples.
Age: For the final matching for age, we used three stratification groups—young (18 to 34 years), middle (35 to 49 years), and mature (50 years and over). As Table 1 reveals, the mean age of the movers and stayers was 52.4 and 52.2, respectively, a nonsignificant difference.
Competency measures: After obtaining comparable age and gender groupings, we sequentially matched each group by multicognitive, self-care, and mobility scores. The matching was done by comparing the raw scores or rank orders for the competency measures based on the distribution of 1994 scores. If a match on the exact raw scores could not be achieved, then movers and stayers were matched by their decile ranks on each specific competency score. Table 2 displays how comparable the movers and stayers were, based on the final average raw scores for each group. It is evident that the average scores for mover and stayer samples were quite close. Any observed differences were not statistically significant.
Behavior scores. After matching on the previous five characteristics, we then matched participants using the raw scores for behaviors toward others and self scores. Again, decile rankings were used when needed to find stayers that were comparable to the final mover sample members. The raw scores of movers and stayers depicted in Table 2 are quite comparable, and differences were not statistically significant.
Interview Procedures and Analytical Design
In order to carefully monitor changes in movers, we conducted assessments at the following intervals after they left North Princeton (beginning in January 1997): 3, 9, 15, and 27 months. By assessing individuals at 6-month intervals for the first 15 months after placement, we were able to determine whether there were initial short-term changes in their competencies. Because there was prior evidence in the test–retest study of reliability that persons living in the institutions were unlikely to significantly change their competencies in 15 to 18 months, we assessed the stayers at 3, 15, and 27 months, which was on or about the same dates as their matched movers were assessed.
All data for movers and stayers postclosure were collected by interviewers trained and under the supervision of the DD Planning Institute staff. Data about competencies and other measures were obtained in face-to-face interviews with staff caregivers who knew the movers and stayers well. Caregivers in the community knew the movers for 3 months at Time 3, 15 months at Time 5, and 27 months at Time 6. The caregivers of stayers knew them for much longer periods of time. This interview procedure was chosen because we could not be certain ahead of time about the literacy levels of our informants and whether records would be uniformly available in the community. We wanted to maintain similar conditions for all interviews, and we chose the first data-collection time of 3 months (Time 3) in order to determine whether the measures would maintain their reliability in an interview rather than a self-administered format. Therefore, the competency measures were again submitted to a series of confirmatory factor analyses in which we used only 3-month data for both movers and stayers. A factor analysis of each competency measure for the combined samples of movers and stayers confirmed that all 40 multicognitive items, all 21 self-care items, and all 10 mobility/motor skills items were statistically still an integral part of each factor. In contrast, the special behavior items linked to each factor were not confirmed using the last 3 months as a time referent (instead of the past year) and, therefore, were not be used in any of the remaining analyses.
At the first 3-month assessment, we found that 40 of the movers who left North Princeton went to other institutions, such as other DD Centers, nursing homes, or private institutions. In order to properly compare the postinstitutional competencies of movers who actually moved into community residences with matched stayers who remained in institutions, we had to exclude both the 40 movers who went into institutions and their matches. The remaining 110 community movers could then be compared to their matched stayers in order to assess any changes in competencies from the first baseline measurements in 1994 (Time 1) to subsequent assessments in 1997–1998 (Time 3), in 1998–1999 (Time 5), and in 2000–2001 (Time 6).
The analyses that were conducted are presented in four stages. The first analyses was focused on identifying the pattern of change, if any, for each competency within each residential setting type. For analyses of change within residential setting type, we relied on repeated measures MANOVA tests that included the baseline (Time 1) and data at 3 months (Time 3), 15 months (Time 5), and 27 months (Time 6). These analyses tested whether the means differed significantly over time within each sample (see Table 2).
In the second set of analyses, we combined both samples for each competency and tested for differences of time only, residence only, and interaction of time and residence. Based on these repeated tests, differences between community and institution scores were examined for each distinct time period (see Tables 3).
In the third set of analyses, we focused solely on identifying changes within domains for two of the competencies (Multicognition and Self-Care) that displayed significant differences between residential setting types. We used these analyses to examine more closely which specific behaviors within a domain changed and when the changes occurred.
The primary statistical procedure employed in these three analyses was the repeated Multiple Analysis of Variance test—MANOVA (Walker, 2002). The MANOVAs were conducted after testing to determine whether the repeated measures had equal correlations among the pairs of repeated measures. For virtually all of the comparisons, the test for unequal correlations, the Mauchly sphericity test, was statistically significant. Therefore, all F values reported that involved repeated-measure main effects or interactions were on multivariate tests. Based on these overall tests, we conducted more detailed differences within domains and between time periods for specific sample types, using the baseline measure of 1994 (Time 1) as the referent point for all comparisons.
The fourth set of analyses we used were hierarchical multivariate regression procedures to locate the environmental and program variables that best predicted the scores at Times 3, 5, and 6.
Changes Over Time Within Residential Type
Table 2 provides the mean scores for matched pairs of community movers and institutional stayers at four time periods for the three major competency measures: (a) multicognition, (b) self-care, and (c) mobility. At the 1994 baseline year (Time 1), there were no significant differences between the matched samples for any competency measure. Even though the number of matched pairs had been reduced from 150 to 110, at this time period, there were also no significant differences between community and institution pairs on the other matching variables of age, behaviors toward others, and behaviors toward self. Assessment of the mean competency scores over time for each residential setting type, however, reveals that diverse patterns of change occurred. In reviewing these findings, we note that the number of matched pairs for each time period progressively decreased due to deaths and missing data (for more details on deaths, see Lerman, Apgar, & Jordan, 2003). Table 2 depicts the mean scores of the maximum number of matched pairs for each time period for each competency.
In Table 2, examination of the institutional multicognition scores of stayers reveals a continued reduction in mean scores from 22.8 (Time 1) to 19.7 (Time 3) to 19.2 (Time 5) and then down to 18.3 (Time 6). The community movers did not exhibit this type of pattern. There was a fluctuation upward at Time 3, but a slight move downward at Time 5 (24.1). The final score at Time 6 was slightly lower than at Time 1 (23.2). A statistical analysis of these overall changes in the multicognition scores across time, within each residential type, controlling for comparable persons across time, indicates that the decrease in scores for stayers was statistically significant, p < .0001. In contrast, the fluctuation of the community movers since Time 1 was not statistically significant.
Table 2 reveals a different pattern for the changes in self-care. The institutional stayers appear to have decreased scores over time. However, repeated analyses show that when the same persons are compared over time, any fluctuation of the mean self-care scores of institutional stayers is not statistically significant. The community movers, in contrast, exhibited a shift upward in their scores over time. Statistical analysis confirmed that this change within the community sample was significant, p < .0001. Therefore, the pattern of self-care change appears significantly different for movers and stayers. Table 2 further reveals that both residential types had a decrease in mobility since 1994, p < .0001.
Assessment of Changes Between Residential Settings: Multicognition
Table 2 provides evidence that multicognitive scores became lower over time for consumers living in institutions but remained fairly comparable over time for those in community residences. In order to assess whether these trends resulted in significant differences between the groups, it is necessary to simultaneously control for changes over all time periods since Time 1 while assessing changes between residential setting types. Table 3 shows the results of these analyses using repeated MANOVAs to assess any differences.
Table 3 depicts changes during three time periods and was focused on determining whether the patterns are consistent at (a) Times 1 and 3 only, (b) Times 1, 3, and 5 only, and (c) Times 1, 3, 5, and 6 simultaneously. For each time comparison, the analysis determined whether there were significant differences if only time changes were considered, if only residential differences were considered, and if the combination of time and residential changes were considered. The combination of time and residential type is the crucial test of whether there was a significant difference in the scores of those in community versus institutional settings over time. For each time period analysis, we employed the maximum number of matched pairs.
Significant change in time only occurred in the comparisons that included Times 5 and 6, but not when Times 1 and 3 were analyzed alone. A similar pattern occurred for residence type only. However, the interaction of time and residence type was significant for all multicognitive time comparisons. This finding indicates that significant differences in multicognitive scores emerged over time between the residential types.
These analyses provide convincing evidence that the significant lowering of multicognitive scores of consumers remaining in institutions ultimately made them significantly different in this area from those who moved into community residences. Remaining institutionalized from 1994 to 2000– 2001 had negative consequences. Community movers did not make any significant gains or losses in multicognition; rather, they were likely to maintain their 1994 levels of multicognition over time.
Assessment of Changes Between Residential Settings: Self-Care
Table 2 shows that consumer self-care scores increased over time for community movers but did not change for their institutional matches. In order to determine whether changes within residential setting types yielded significant differences between consumers, we conducted repeated MANOVAs. The results are depicted in Table 3.
Analyses are repeated for three time comparisons in Table 3 in order to confirm the consistency of the findings. Differences for time only occurred for two of the three time comparisons. There were no distinct residence-type only differences for any time comparisons. However, there were strong interactions of time and residence type for all of the time comparisons.
These analyses provide convincing evidence that the significant increase of self-care scores of consumers residing in community residences make them markedly different from those who remained in DD Centers. Remaining in institutions from 1994 to 2000–2001 had no significant impact on self-care scores of stayers. Community movers, however, increased their competencies over time.
Assessment of Changes Between Residential Settings: Mobility
Table 3 shows the changes in mobility between residential settings over time. Analysis of time only differences in the three time comparisons reveals significant differences in all comparisons. However, assessments of residence type only and interaction of time and residence type reveal no significant difference for any time comparison.
These analyses provide convincing evidence that the significant decrease in mobility within both residential setting types over time yields time-only differences. There were no significant differences between consumers in community and institutional settings regarding the degree of loss. Individuals in both residential setting types experienced comparable mobility loss between 1994 and 2000–2001.
Identifying Types of Self-Care Changes
In a meta-analysis of research studies on deinstitutionalization, Lynch et al. (1997) found that the strongest improvements in adaptive behaviors occurred with instruments that included domestic self-care activities. This general finding about self-care has been confirmed in the present study, but the evidence does not actually specify which types of activities changed and whether some of the gains might have occurred within the institution before placement in the community actually occurred. Over 3 years elapsed between the 1994 baseline (Time 1) measures and the first postplacement assessment at Time 3 (in 1997–1998). The public announcement that North Princeton was scheduled for closure and residents were to be moved took place in April 1995, about 6 months after completing Time 1 measures. It is conceivable that more competent residents may have realized that they had something to look forward to, and dedicated staff may have begun to talk about and perhaps prepare them for noninstitutional living.
In order to assess the specific types of self-care activities that changed and when changes actually occurred, we used all of the available surveys conducted by the DD Planning Institute for the entire North Princeton population. Surveys were conducted at North Princeton in 1996, with comparable competency instruments, in order to provide information to service providers about persons who might be candidates for receiving services in the community. Focusing solely on those former North Princeton residents in our sample who were destined to become movers in community residences, was possible to conduct a comparison between 1994 (Time 1) and 1996 (Time 2)—prior to moving into the community. Unfortunately, comparable information for the matched institutional stayers was not available. However, it is not necessary because we were interested in determining when the community movers gained self-care skills and whether all types of self-care changed to an equal degree.
In addition to the Time 2 data, additional data were also available on community movers 9 months after leaving North Princeton. These measurements constitute Time 4 assessments. Using all of the available data for community movers, we found that a more complete time series could be constructed from 1994 to 1996 (Time 1–Time 2), 1996 to 1997– 1998 (Time 2–Time 3), 1997 to 1997–1998 (Time 3–Time 4), and 1998–1999 to 2000–2001 (Time 5– Time 6). Table 4 provides the information for assessing differences within the community movers over these time periods. In addition to the data on total self-care scores, tests of differences are also provided on three subindexes (basic self-care, independence, and household skills) contained within the overall index of self-care. According to the meta-analysis of Lynch and her colleagues, it is the independent and household subindexes that should exhibit the most changes over time.
The three self-care subindexes that are used in Table 4 contain the following items:
Basic subindex (7 items): feed self, drink from cup, toileting/bladder, toileting/bowel, dress self, move in familiar setting, chewing/swallowing food
Independence subindex (9 items): clean room, do laundry, care for own clothes, use money/not count change, use money/count change, identify items to buy, order food in public, choose/buy own items
Household subindex: (5 items): shop for simple needs, prepare food, use stove/microwave, wash dishes, use public transportation
Table 4 shows that the results of testing total self-care scores for all time comparisons, including the time period within the institution (Time 1 vs. Time 2); all scores were statistically significant. However, none of the time comparisons for the basic subindex are statistically significant, including Time 1 versus Time 2. In contrast, all of the time comparisons, with the exception of Time 1 versus Time 2, were statistically significant for the subindexes of independence and household skills. The findings indicate that change occurred within the community for the self-care skills of independence and household skills but not for basic self-care skills. These gains in independence and household skills are the primary sources of the gains in the overall scores depicted in earlier analyses.
The findings in Table 4 also reveal that the Time 1 versus Time 2 comparison did not yield a significant difference for any of the three subindexes. The only change that may have occurred in the institution (between Times 1 and 2) is for the total score, but not when all of the self-care scores were disaggregated.
The findings regarding self-care appear to support and expand the findings of Lynch and her colleagues (1997). Even though we did not precisely match movers and stayers on their subindex scores, we conducted a reanalysis of Time 1 scores in order to validate using the 1994 subindex scores as a baseline measure. This analysis revealed that there were no significant differences between movers and stayers in 1994 for basic self-care, independence, or household skills (table not shown).
Identifying Types of Multicognitive Changes
A few of the studies of deinstitutionalization have also assessed changes in academic, communication, and social skills. In their review, Kim et al. (2001) found that in 4 of the 12 contrast studies, the investigators assessed changes in the two types of cognitive skills, and in 7 of the contrast studies, the researchers assessed social skills. They reported significant improvements by movers in 2 of the 4 academic skill comparisons, 1 of the 4 communication skill comparisons, and 6 of the 7 social skill comparisons. In a meta-analysis of 11 studies, Lynch et al. (1997) reported “modest effects” for both types of cognitive skills and social skills compared to the larger effects found for self-care skills. The authors in both reviews did not report that repeated measures were employed; rather, studies were based on single assessments at two time periods.
In contrast, the present study contained four time comparisons for all assessments of movers and stayers, and we found that there were no overall changes in multicognitive competencies for the mover sample. There was, however, a quite significant loss in multicognition between 1994 and 2000–2001 for the institutional stayer group. Assessments of the types of cognitive competencies that changed for stayers and determining when these changes occurred are the focus of the analysis that follows.
In constructing the multicognitive index of 40 items, we relied on the high correlations between abstract cognitive skills (e.g., knowledge of time, color, and numbers) and the skills of communication and social–emotional understanding. The specific items used in the construction of each subindex, each of which can be used to more precisely locate which skills may have changed over time, were as follows:
Cognitive (16 items): associate event with hour; tell time to 5 minutes; associate events with past, present, and future; remember events a month ago; sort by color; sort by size; sort by shape; find way around building; say numbers by rote; do simple addition and subtraction; print/write single words; print/write sentences; spell name correctly; read simple words; read simple sentences
Communication (11 items): use few words, links to objects; use many words, links to objects; ask simple questions; tell stories, jokes; answer yes/no; speech understood by strangers; speech understood by known person; understand one-step direction; understand two-step direction; understand a joke/story; understand yes/no
Social–Emotional (13 items): interact with staff, interact with peers, respond appropriately, interact in activities, interact in imitative manner, offer help to others, show consideration for others' feelings, manage affairs when needed, able to take turns, show caution with strangers, recognize own limitations, talk about future plans, take care of others' belongings
Table 5 presents the findings or the subindexes of Multicognition as well as the overall measure for Time 1 versus Times 3, 5, and 6, respectively, for the institutional sample. The overall finding for the total multicognitive measure repeats the some of the results from Table 2. It is evident that each time comparison yields a strong significant finding that the overall skills have become lower. The strongest F value for the Time 1 versus Time 6 comparison indicates the continued lowering of skills over time.
The abstract cognitive items display a quite different pattern. There was an overall time period difference, p < .02, but this was primarily due to the Time 1 versus Time 6 comparison. There were no discernible differences occurring at the 3 months and 15 months follow-up time periods.
The communication items did not reveal any significant difference in the overall time periods or in any of the other specific time comparisons. The last column in Table 5, referring to social–emotional skills, exhibits the most significant findings. Not only was the F value quite high in the overall time period, but each time comparison also yielded a very high F value. The lowering of social skills appears to have plateaued by 15 months because there was very little difference between the F values at Time 1 versus Time 5 and Time 1 versus Time 6.
In summary, it appears that the strongest loss in multicognitive skills for all time periods was in the area of social–emotional skills. The much more modest change in cognitive skills occurred solely in the Time 1 versus Time 6 comparisons, and there were no significant changes in communication skills. Even though matching was not done on any of these subindexes, a re-examination of the matched movers and stayers at Time 1 reveals that there were no significant differences between the samples in 1994 for the subindexes of cognitive, communication, or social skills (table not shown).
Identifying the Variables Influencing Self-Care Scores
In the meta-analysis study conducted by Lynch and her colleagues (1997), the authors offered the following hypothetical scenario for the “likely explanation” for the stronger statistical effect for gains in domestic self-care scores by movers: “The intimate nature of small communities settings, as opposed to congregate care facilities, makes it more likely that deficits in these areas will be noticed and remediated” (p. 260).
According to this line of reasoning, small size is important because it facilitates more intimate and informal contacts and relationships that can be used by staff members to notice and remediate deficits in self-care competencies. Felce and Emerson (2001), in a more recent analytical review of a variety of studies conducted in the United Kingdom, America, and Australia suggested the following types of influences that were worthy of attention in explaining gains in adaptive behaviors: (a) greater exposure of consumers to intensive support, (b) stronger staff orientation towards providing consumer choice and community participation, (c) greater emphasis on staff procedures and performance that are focused and organized, (d) small residence size as an indirect effect of facilitating other influences, and (e) provision of increased opportunity to learn and exhibit skills. These reviewers appear to place a greater stress on remediation that is “focused and organized” than Lynch et al. (1997) suggested, but it is unlikely that Lynch and her colleagues would disagree with the other potential influences.
By considering these views, we were able to review the interviews conducted at 3, 15, and 27 months and identify specific variables that could serve as indicators of potential program and environmental differences between community and institutional residences. These variables can serve as candidates for finding the best predictors of self-care scores. The following variables were found to be significantly associated with types of residential placements (at a bivariate level): (a) Day programs as an indicator of a focused and organized program was much more likely to be associated with movers, p < .002. (b) Lower utilization of social control as an indicator of staff procedures was much more likely to be associated with movers, p < .001. This measure was constructed out of 4 items referring to use of manual restraints, loss of privileges, use of point systems with demerits, and use of room restrictions or time outs (άs = .68, .63, and .71 at 3, 15, and 27 months, respectively). (c) Staff orientations that stress a stronger belief in noninstitutional sites as a promoter of choice, community participation, independence, and 7 other quality of life goals were more likely to be associated with staff working with movers, p < .0001. (A total of 10 items constituted a single factor and had an alpha above .90 for all time periods for all mover and stayer staff members.) (d) Teaching of basic skills in an informal or formal manner, as an indicator of remediation performance, was more likely to be associated with movers, p < .02. Three items referring to eating, dressing, and hygiene were employed, ά = .56. (e) Teaching of domestic skills in an informal or formal manner, as an indicator of remediation performance, was more likely to be associated with movers, p < .0001. Two items (cooking and shopping in the past 3 months) were used, ά = .63. (f) Speech therapy, as an indicator of remediation performance, was more likely to be associated with stayers, p < .005. (g) Changes in staff members working with consumers between 3 and 15 and 15 and 27 months, as an indicator of a decrease in consistent staff support, were more likely to be associated with movers, p < .0001. (h) Residency size of 5 or fewer persons was more likely to be associated with movers, p < .0001. (i) Staff eating with residents, as an indicator of intimacy and intensive support, was more likely to be associated with movers, p < .0001. (j) More staff on duty during dinner per resident, as an indicator of providing more support, was more likely to be associated with movers, p < .0001. (j) Permitting consumers to have more autonomy by making decisions about their lives on a daily basis (e.g., waking up or going to bed, having visitors, choosing clothes, and spending their own money) was more likely to be associated with movers, p < .0001. (All 10 items used in this index constituted a single factor and had alphas of .87 to .89 for 3 time periods). (k) Permitting consumers to experience greater community participation (e.g., going to stores, restaurants, movies, or parks) was more likely to be associated with movers, p < .0001. (All 7 items used in this index constituted a single factor and had an alpha of .84 to .87 for all time periods).
These 12 environmental and program variables were examined separately in order to find the best independent influences that could account for a significant portion of variability in self-care scores for the combined sample. After controlling for all Time 1 matching variables and any other Time 1 personal characteristics that might also be associated with self-care scores at 3 months, we entered these variables into a hierarchical regression analysis. In order to identify the best model for consistency, we also conducted hierarchical regression analyses at 15 and 27 months. In summary, the distinctive regression analyses were conducted as follows: (a) All Time 1 matching variables were entered in a separate analysis with 2 Time 1 personal variables that were found to be significantly associated with movers—having a psychiatric diagnosis and/or epilepsy or a seizure disorder. The best variables associated with Time 3 self-care scores were then chosen as the Block 1 control variables for the analyses at 3, 15, and 27 months, using a significance level of .05 as the decision criterion. (b) A separate regression analysis was then conducted to identify the best of the 12 environmental/program variables at 3 months, p < .05, without using any control variables. (c) Using the best Time 1 control variables as Block 1 and the best program variables at Time 3 as Block 2, we conducted a combined hierarchical regression analysis to identify the best model for predicting self-care scores at Time 3. (d) The best Block 1 and Block 2 variables at Time 3 were then used to determine whether the influences were similar at Times 5 and 6, using the appropriate time measures for each time period.
The best Block 1, or control, variables for predicting Time 3 self-care scores proved to be the initial baseline self-care scores at Time 1 as well as the Time 1 scores for mobility and age. Out of the 12 environmental/program variables assessed as the best ones to be used as Block 2 in a combined hierarchical regression analysis, only 2 proved to be independent predictors of self-care at Time 3: informal or formal teaching of domestic skills and high autonomy scores. All of the remaining 10 candidates proved to be unable to meet the .05 criterion when entered into separate regression analyses with these best Block 2 predictors. This outcome does not necessarily mean that size of residence, eating with consumers, or community participation are not important but, rather, that their influences in this study may be indirect in facilitating the likelihood that informal or formal teaching and the granting of higher autonomy will have a direct, and potentially significant, impact on the prediction of self-care scores. As an example, in a distinct regression analysis of informal or formal teaching, we found that eating with consumers was the best predictor of remediation via teaching (table not shown).
Table 6 provides a summary of the significant statistical findings where the best Block 1 and Block 2 variables are confirmed as well as where Block 1 is entered alone. At 3 months, Block 1 variables accounted for 68.6% of the variability in self-care scores. Block 2 variables added 10.8% when accounting for the Time 3 scores. To eliminate multicollinearity as a problem, we made sure that in all analyses the variance influence factor was within acceptable bounds for this and other time-period analyses.
At 15 months (Time 5) and 27 months (Time 6), the additional contributions of teaching and autonomy were higher than at 3 months—15.2% and 13.9%, respectively. When all of the models were adjusted for the number of variables employed in the model, the adjusted R2 was between 78.9% and 79.8% for all of the models. These findings indicate that the models as a whole, with all 5 variables, were strong predictors of self-care scores at all time periods for the entire sample employed in this study.
Although the initial Block 1 scores are clearly the best predictors, it is important to note that the relative importance of Time 1 self-care scores as an independent influence, as indicated by the standardized beta values, was appreciably reduced when the Block 2 variables were introduced at all time periods (from .74 to .52 at Time 3, .72 to .46 at Time 5, and .71 to .47 at Time 6). In addition, although persons with higher mobility and younger ages at Time 1 were also independent predictors at each time period, it is also clear that the Block 2 beta values were of greater importance at all time periods. Although the informal or formal teaching of domestic skills and the support of increased autonomy contributed modestly to the explanatory power of the total model (via R2 contribution), their relative importance as independent predictors of self-care scores was particularly strong at Times 5 and Time 6 because their beta values were over one half the size of the Time 1 beta values of the initial self-care scores (i.e., .46 to .26 and .26 and .47 to .24 and .25, respectively).
Identifying the Variables Influencing Multicognition
In our search for the best model for predicting multicognition scores for the three time periods, we employed the same variables and logic as were used in the self-care analyses. The best Block 1 control variable was, as expected, multicognition scores at Time 1. No other matching or individual variable reached a level of statistical significance of .05, after excluding self-care at Time 1 as exhibiting high multicollinearity with multicognition at Time 1. After examining the separate regression analyses of the 12 environmental/program variables that might be associated with Time 3 multicognition scores, we determined that informal or formal teaching of domestic skills and high autonomy scores were the only significant predictor candidates for the Block 2 variables.
Table 7 shows that multicognition at Time 1 was a very strong predictor as the sole Block 1 variable at all time periods, with R2 contributions ranging from 64.5% to 70.8%. There was some diminution in the standardized beta values when the Block 2 variables were entered at each time period, but the independent contribution of the Time 1 scores remained quite strong at each time period (.63 to .64). The contribution of the Block 2 variables to the total model was statistically significant, but less than 10%, ranging from 6.7% to 8.2%. However, teaching domestic skills and higher autonomy scores, variables associated with increased self-care scores, appear to have a significant independent effect on multicognitive competencies at all time periods.
The majority of earlier researchers have relied on an aggregate measure of adaptive behaviors and have assisted the field in advancing our understanding of the outcomes of deinstitutionalization. However, our findings here provide strong empirical evidence that disaggregating the multiple domains of adaptive behaviors can be quite useful in understanding the changes in competencies over time of persons with developmental disabilities. Using three broad competency domains instead of only a single aggregate, we clearly found that a study in which a carefully matched sample is employed can reveal three distinctive patterns of changes over time within and between samples of movers and stayers. Examining distinctive patterns of change over time of stayers has been virtually absent in the recent literature, except for the study by Stancliffe and Hayden (1998) of adaptive behaviors. Although studies of movers have certainly been more plentiful, these investigators have tended to focus primarily on whether movers have gained more than stayers, with only a minimal interest in contrasting patterns of change for specific types of competencies.
The present study provides empirical evidence that the matched movers did indeed significantly improve in their self-care skills, specifically in the areas of independent and household competencies, but not in basic skills. The evidence for movers regarding the adaptive behavior domains of Mobility and Multicognition revealed declines in the former and no changes in the latter. Using an aggregated approach to measurement, we would not have been able to identify these three differential impacts over time.
If we had focused solely on movers, however, we would have missed one of the most salient findings of this evaluation, namely, the significant loss by stayers in their multicognitive competencies, particularly in the areas of social skills and, to a lesser extent, the subdomain of cognitive competencies. The stayers experienced these losses while not changing their basic self-care and their limited independent and household skills. Like the movers, they also experienced a significant loss in mobility skills as they got older between 1994 and 2000– 2001.
The loss in mobility by both groups over a span of 6 to 7 years is not surprising, given that their average age in 1994 was 52 years. The significant gain in self-care scores by movers is also not surprising because reviewers of research studies in America and the United Kingdom have concluded that this outcome is likely to occur in about two thirds of the studies. The fact that the gains by the community movers tended to level off about 12 to 15 months after leaving the institution is also congruent with other well-designed studies. In the United Kingdom, this leveling effect of adaptive behavior gains is termed a plateau effect (Emerson & Hatton, 1996). The contrast–longitudinal studies reviewed by Kim et al. (2001) were not coded for repeated measures and, therefore, offer no evidence about plateau effects.
The two important descriptive findings in the present study, namely, the loss in multicognitive competencies by institutional stayers over a 6- to 7-year period and the significant gains in self-care competencies by community movers, have important implications for public policy as well as deinstitutionalization research. An evaluation study of a shorter duration might not be long enough to carefully document the long-term consequences of remaining in institutions. Remaining in institutions for 6 to 7 years by persons who were similar to community movers appears to be quite costly in human terms. With this study we have provided the state of New Jersey with empirical evidence that failure to accelerate the rate of deinstitutionalization may have a negative impact on those remaining in DD Centers. In addition, remaining institutionalized also denies these individuals an opportunity to improve their self-care skills.
Although one could argue that the North Princeton community movers and their institutional matches do not represent the state's institutional population, our findings offer the closest approximation of what has occurred, and is likely to continue, over time. This approximate outcome is likely because the institutional matches were selected from all of the remaining 7 DD Centers in the state. It is also possible that similar outcomes have occurred, and will continue to occur, in other states. Similar longitudinal studies with prospective designs are needed using repeated measures of independent domains of adaptive behaviors in order to test this hypothesis.
It is important to note that carrying out this type of design also incurs methodological threats to the validity of the findings, particularly in the attrition of the initial sample through deaths, movement into noncommunity facilities, and incomplete responses for some items within each domain. Although a study of mortality revealed that there were no more deaths among movers when high-risk variables were controlled, attrition due to death reduced the number of matched pairs (Lerman et al., 2003). Attrition resulted in the number of matched pairs being reduced substantially from 110 matched movers and stayers in 1994 to 72 to 80 matched pairs for postclosure analyses. It is possible that the results would have been different if the number of pairs had not decreased. Unfortunately, this type of sample attrition is a risk associated with a prospective repeated research design over a 6- to 7-year period.
The search for identifying the best predictors for self-care and multicognition scores highlighted the importance of two types of environmental and program variables: the informal or formal teaching of domestic skills and autonomy in daily activities. The acquisition of increased competencies by consumers can occur in a less formal and regulated environment and be facilitated by smaller living units, where staff shop, cook, and eat with residents, and have an orientation that promotes autonomous behaviors. It is these nonprofessional activities that may be of greater importance than formal Individual Habilitation Plan protocols and procedures employed in American institutions.
Theoretically, the teaching of domestic skills and the encouragement of greater autonomy could occur in an institutional setting as well as in a small community residence. As Stancliffe and Hayden (1998) noted, attempts to create smaller living units within institutions have occurred. However, the larger and more formal and regulated settings appear significantly less able to provide the opportunities where staff and consumers jointly engage in the everyday activities of shopping, cooking, and eating together on a consistent basis. Therefore, we were not surprised that Stancliffe and Hayden reported that no significant changes in adaptive behaviors occurred over time in these settings.
Aside from the evidence that institutions are significantly less likely to be associated with the conditions that facilitate improved self-care competencies and the maintenance of multicognitive skills, this study has documented that social–emotional capabilities, the ability to relate to others, decreased significantly over time when a person was institutionalized. The empirical evidence of this and other studies, together with the legal support for deinstitutionalization promulgated by the Olmstead decision (1999), presents a challenge to all states that continue to maintain congregate residences for persons with developmental disabilities.
Note:This research was funded by the State of New Jersey's Division of Developmental Disabilities. However, the interpretation of the findings is based solely on the views of the authors and does not necessarily represent those of the sponsoring organization or any of its staff.
In order to test the reliability of these findings over time, we conducted a special test–retest study on a random sample of 350 of the original 1,190 consumers. The same staff caregivers were asked to complete the assessments 15 to 18 months later. Table B provides evidence that the distinct factor structures remained quite stable over time. In addition, the test–retest correlation coefficients between Time 1 and Time 2 were .94 for multicognition, .98 for self-care and .94 for mobility–motor skills. The special behavior domains had lower levels of stability because the Time 1 versus Time 2 correlation coefficients were .78 for behaviors toward others and .73 for behaviors towards self (Jagannathan, Cammasso, Lerman, & Cook, 1997). These results indicate that the behavioral measures were only moderately reliable compared to the strong results for the competency measures (i.e., cognitive abilities, communication, and self-care skills, social emotional functioning, and mobility/motor control capabilities).
In addition to retesting the factor structures obtained with the new 3-month data, we compared the correlations of the combined samples in 1994 (Time 1) and 1997–1998 (Time 3) to assess the likelihood that the measures were strongly correlated using a new collection method and new raters after a period of 3 to 4 years (depending on when movement out of North Princeton occurred). The correlation for each competency measure for the combined samples between Times 1 and 3 were .83 for multicognition, .85 for self-care, and .88 for mobility/motor skills. Given their high intercorrelations over time, it is readily apparent that the measures maintained a strong degree of stability—despite the changes in the modality of obtaining information from caregivers and the lapse of time.
The initial mover sample assessed in 1994 consisted of 171 persons, including an oversampling of residents of medical cottages. By Spring of 1996, however, only 150 North Princeton persons remained in the institution; a total of 21 persons had been relocated prior to the initiation of closure placements scheduled for January 1997 or had died. Therefore, the proposed mover sample consisted of 150 persons who were residents of North Princeton in Spring 1996.
By January of 1997, further attrition of the proposed mover sample had occurred, so that 140 of the 150 persons eligible for placements remained. Therefore, an additional 10 persons were randomly chosen from the pool of remaining eligible movers to comprise the final mover sample. These 10 persons were assessed with the Assessment Form instrument used in a 1996 study of the North Princeton population, and these assessments were used for finding matching stayers. The remaining 140 movers had 1994 and 1996 assessments, but for matching purposes, only the 1994 assessment data were used to find their stayer matches. This was deemed necessary because we only had 1994 data for the pool of stayers who had not been included in the 1996 test–retest study of instrument reliability. Of the 150 person final mover sample, there were 14 persons who resided in a medical cottage prior to January 1997.
Authors: Paul Lerman, DSW, Co-Principal Investigator, Dawn Hall Apgar, PhD, Director and Co-Principal Investigator (Hall@ADM.NJIT.EDU) and Tameeka Jordan, MA, Data Management Supervisor, Developmental Disabilities Planning Institute, New Jersey Institute of Technology, Center for Architecture and Building Science Research, 323 Martin Luther King Blvd., Newark, NJ 07102