Evidence-based practice is an emerging standard for therapeutic interventions and services in the fields of psychology (American Psychological Association, 2002, 2005), medicine (Institute of Medicine, 2001), and education (American Educational Research Association, 2006; National Research Council, 2002). Recent calls for renewed emphasis on treatments based on empirical evidence of effectiveness has resulted in the articulation of clearer standards for research designs and methods for the synthesis of research findings to determine effectiveness of treatments (Odom et al., 2005; Slavin, 2008). The field of intellectual disabilities has a strong tradition of research designed to inform practice and promoting evidence-based treatments as the standard of care for persons with intellectual and/or developmental disabilities (Jacobson & Mulick, 1996), and recent calls for use of evidence-based treatments has been welcomed by the field.
The articulation of the evidence base for interventions and services for persons with intellectual and developmental disabilities is particularly timely given recent advances in treatments. For example, after a series randomized clinical trials, McCracken et al. (2002) found that risperidone is an effective treatment of behavior and core symptoms associated with autism spectrum disorders. Additional multisite trials have been initiated to evaluate promising treatment protocols that combine behavioral and pharmacological treatments (McDougle et al., 2005). Established treatments, such as parent training for behavior management, have been extended to children with developmental disabilities in studies designed to demonstrate their effectiveness in treating behaviors typical of this population and extending the demonstrated efficacy of parent-training interventions for disorders other than children with autism spectrum disorders (e.g., McIntyre, 2008; Plant & Sanders, 2007).
Individuals with intellectual and developmental disabilities have diverse needs and may require a wide range of supports across their life span. As a result, the focus, contexts, and integration of treatments for this population may be especially challenging. For example, the integration of functional analyses of behavior with communication interventions has shown consistently positive outcomes for individuals with significant disabilities across a range of ages and populations as well as settings such as schools, homes, and work (Hagopian, Fisher, Sullivan, Acquisto, & LeBlanc, 1998; Halle, Ostrosky, & Hemmeter, 2006; Mancil, 2006; Mirenda, 1997; O'Reilly et al., 2010). The varied contexts of adult living have been an area of increasing interest in the field, resulting in a growing number of evidence-based treatments that address the issues faced by adults with intellectual disabilities and their caregivers. In addition to the functional communication training studies, there have been promising treatments on functional analyses to address significant behavior directed toward self and others (Iwata, Dorsey, Slifer, Bauman, & Richman, 1982; Mace, 1994; O'Neill, Horner, Albin, Storey, & Sprague, 1990). Studies on examination of strategies to promote self-determination (Davis & Wehmeyer, 1991; Wehmeyer & Schwartz, 1997) and person-centered planning for the benefit of individuals with intellectual disabilities and their families (Chambers & Childre, 2005; Heller, Miller, Hsieh, & Sterns, 2000; O'Brien, 2002) have produced promising indicators that not only treatments addressing specific behaviors but also interventions to improve the quality of life for individuals can be evaluated to determine guidelines for evidence-based practices. The examination of individual variations in response to treatment indicates that treatment engagement, as well as the specific features of treatments, affects outcomes, suggesting the need for methods to model the impact of treatment engagement on outcomes.
Concomitant with the increased emphasis on evidence-based treatments, there has been increasing attention to the quality of the design and methods of individual studies contributing to the evidence bases. Guidelines for assessing the internal validity, external validity, and outcomes of experimental studies have been developed for both single subject and group design methods. Two relatively early and important sets of standards for evaluating empirical research were developed in school psychology (Kratochwill & Stoiber, 2002, for American Psychological Association Division 16) and special education (Odom et al., 2005, for the Division for Research, Council for Exceptional Children). The standards promulgated by the What Works Clearinghouse (U.S. Department of Education, 2002) for evaluating educational interventions have also played a role in shaping the standards for publication and evaluation of group experimental design studies in related areas.
With emerging standards for design and outcomes of individual studies has come a concurrent emphasis on methods for aggregating the findings of individual studies in order to determine the magnitude of change that might be expected from a given treatment as well as the quality of evidence contributed. In group design, methods for aggregation of findings from studies have focused on determining the extent to which studies meet the established standards for internal and external validity, relying heavily on the use of randomized control treatment designs as the gold standard. Outcomes have largely been characterized by comparing the effects of the treatment and control groups and calculating an effect size to indicate the relative effectiveness of the treatment. Because effect sizes are a type of standardized measure, they may be averaged and weighted for sample size to yield a single estimate of the effects of a treatment across studies, following the general methods first outlined by Hedges (1981) and frequently applied in meta-analyses (Lipsey & Wilson, 2001).
Although standards for evaluating the quality of individual single subject design studies have been laid out, guidelines for the number of studies needed in order to judge a practice as evidence based (Horner et al., 2005) and finding a common metric for characterizing the aggregated effects of single subject design studies (Wolery, Busick, Reichow, & Barton, 2010) have proven more difficult. Both the time series nature of data and the standards for demonstrating adequate design and effects within the single subject method (i.e., stable baselines, followed by a shift in level and/or trend associated with the introduction of the treatment, with an adequate number of demonstrations across persons, settings, or behaviors) make calculating the magnitude of behavior change within and across baselines conceptually and mathematically difficult. Although a number of methods for aggregating the effects of single subject design studies have been offered in recent years (Ma, 2006; Parker & Hagan-Burke, 2007; Parker, Hagan-Burke, & Vannest, 2007; Scruggs, Mastropieri, & Casto, 1987), none of these has been widely accepted as adequately representing the standards of single subject research design methods (Gast, 2010). This is an important issue related to research with persons who have intellectual disabilities because our field has many areas where single subject research designs are the predominate method for investigating the effectiveness of treatments (e.g., function-based assessments of behaviors, social skills training, video modeling, friendship promotion, self-determination and choice, treatment of self-injurious behavior).
In the context of emergent standards for individual studies and the aggregation of findings from studies to assess the extent of evidence in support of specific treatments, there are three particular issues that face researchers in the field of intellectual disabilities. First, the majority of research involving individuals in this population does not use experimental methods. How and whether it is possible to use descriptive studies to inform evidence-based practice is an important issue for consideration. Although standards for evaluating individual experimental studies using groups or single subject design methods have been offered, and there are some methods for aggregating data from studies using these methods, there has been relatively little written about standards for correlational designs (see, however, Thompson, Diamond, McWilliam, Snyder, & Snyder, 2005). In a field where the majority of published studies are descriptive in nature, almost nothing has been offered about the how and when such studies should be considered as contributing to the evidence base for effective practice. For example, should well-designed studies in which researchers describe access to services, satisfaction with a service, or report systematic needs assessments be considered in developing recommendations for evidence-based practice (e.g., Bailey, Hebbler, Olmsted, Raspa, & Bruder, 2008, from the Early Childhood Evaluation Outcomes Center)? Should only evidence of effectiveness be considered in the development of policy and practices in intellectual disabilities? If such systematic information is to be included, then the field must take steps to develop guidelines for the evaluation of these types of nonexperimental studies and suggest principles for the use of such data in supporting recommendations for practice.
Second, there are many areas of treatment and practice for persons with intellectual disabilities where there is simply not sufficient high quality experimental evidence from which specific recommendations for practice can be made. For example, Browder, Wakeman, Spooner, Ahlgrim-Delzell, and Algozzine (2006) found no published studies of reading instruction for persons with intellectual disabilities who have limited spoken language. Although more recent work (Browder, Ahlgrim-Delzell, Courtade, Gibbs, & Flowers, 2008) has expanded the available evidence, there are not sufficient studies to meet the standards for recommending an evidence-based practice (Gersten et al., 2005). Many important areas for treatment in intellectual disabilities have an empirical basis that includes a relatively small number of single subject studies with few or no systematic replication of a specific treatment strategy and few or no group experimental studies comparing treatment strategies. Outside of the studies involving interventions for individuals with autism spectrum disorders, the replicated evidence for treatments using single subject or group design methods is sparse. Although there are important exceptions (e.g., differential reinforcement procedures in the treatment of feeding problems, function-based treatments to reduce severe behavior problems), there are many areas where primary demonstrations of effectiveness, replications, and groups comparison studies are needed (e.g., early interventions for children with multiple disabilities, instructional strategies for teaching generative communication using alternative and augmented modes, instructional strategies for teaching mathematic skills, effective strategies for family support, school-wide positive behavior support).
Third, given the nature of available evidence, it may be important for our field to determine whether we agree with and endorse the guidelines for quality and quantity of studies required to recommend a practice as evidence based. With the limited evidence available, it may be necessary to adopt a tiered system for characterizing the existing evidence (e.g., no evidence, limited but promising evidence, adequate evidence; see Odom et al., 2005). Alternatively, it may be important that we are relatively conservative in recommending standards for evidence than others have been. For example, both the Division 16 guidelines and the CEC Division for Research recommendations consider generalization and maintenance of the effect of an intervention or treatment, but studies may be judged to be high quality and practices recommended as evidence based without multiple strong demonstrations of generalization and maintenance. Given the importance of robust, durable, and meaningful treatment outcomes for persons with intellectual disabilities, assessment of generalization, maintenance, as well as the social validity and acceptability of treatments are central to the evaluation of the usefulness of a treatment.
In sum, we are at an important point in the development of evidence-based standards for interventions with and treatments for persons with intellectual disabilities. On the one hand, evidence- based treatments are a foremost concern for our field and the presentation of guidelines for evaluating the quality and outcome of individual studies and the aggregated outcomes of sets of studies on a specific treatment offers an important advance for our field. On the other hand, we have a relatively modest portfolio of experimental studies in which researchers specifically addressed important treatments of persons with intellectual disabilities; much of the existing experimental evidence is based on single subject designs, and there are challenges in aggregating and synthesizing these studies. Therefore, the emerging standards for evidence-based treatments may not sufficiently address important issues such as generalization, maintenance, and social validity that must be considered in recommending effective treatment.
In this context, we proposed to the AJIDD editor and to the executive council of Division 33 of the American Psychological Association (Intellectual and Developmental Disabilities) that a special section on evidence-based practices in intellectual and developmental disabilities should be undertaken. In the Call for Papers, the AJIDD editor invited four general types of studies appropriate to the current dialogue on evidence-based practices in psychology and related areas: (a) original reviews of the literature, with an emphasis on quantitative syntheses; (b) meta-analyses of interventions; (c) discussions of standards for research designs, methods, and evidence synthesis related to the evaluation of evidence-based practices and discussion; and (d) discussions of standards for practice directly related to research evidence. Of eight papers submitted, two were accepted for publication. The papers ranged widely in topic, method, and style, but all authors reviewed or synthesized literature on a treatment or practice for persons with intellectual disabilities. The wide-ranging topics and methods, and the varied responses to the methods of evaluation by the AJIDD reviewers, provided an interesting and provocative picture of where we are as a field, consistent with the discussion of emergent standards for individual studies, the relatively small number of high quality studies on important treatments, the lack of agreement about standards for aggregating the outcomes of single subject research, and difficulty in evaluating all types of literature reviews and syntheses.
The papers published in this special section are useful examples of the type of review and syntheses that can contribute to establishing the evidence base for treatments for individuals with intellectual disabilities. The first paper, “Twenty Years of Communication Intervention Research With Individuals Who Have Severe Intellectual and Developmental Disabilities,” is the product of a systematic review of research on communication interventions for persons with severe disabilities by members of the National Joint Committee for the Communicative Needs of Persons With Severe Disabilities (NJC). In this review Snell et al. (2010) apply the standards set for the National Research Council (2002) report on the standards for educational research to more than 10 studies of communication interventions that included persons with significant intellectual disabilities. The authors of this ambitious paper conclude that there is sufficient evidence to support communication intervention for persons with severe intellectual disabilities based on reported outcomes of the interventions and sufficient studies with adequate research designs. Using a broad-based analysis of the empirical literature and a guide for policy and practice for this population, the researchers clearly fulfill their purposes. They also point out some important limitations of the research as reported (missing or limited information about population characteristics, intervention procedures, and outcomes that could not be reliably coded) and again point to the lack of randomized clinical trials, particularly trials comparing theoretically or conceptually diverse interventions for particular populations (e.g., comparisons of oral and sign-based primary communication interventions for older adults with Down syndrome). Both the content of the paper and the authors' report of what is absent in the literature and what could not be addressed in their review will be important to the next generation of research and practice in communication disorders.
The second paper by Eldevik and colleagues (2010) is a more narrowly focused review of early intensive behavioral interventions for children with autism. Unlike other reviews of autism intervention (e.g., meta-analysis of intensive behavioral intervention by Reichow & Wolery, 2009; narrative review of comprehensive approaches by Rogers & Vismara, 2008), Eldevik et al. used individual participant data from 16 group design studies of behavioral intervention for children with autism. Reliable change indices, absolute risk reduction, and the number needed to treat were the quantitative metrics for judging intervention change in IQ and adaptive behavior for children in three groups: behavioral intervention, control group, and a comparison group. Although not without methodological limitations, this novel approach to aggregating and analyzing individual participant data in group design studies makes important contributions to the evidence base in early intensive behavioral interventions for children with autism. Furthermore, in using the reliable change index and number needed to treat, the authors provide important information that may be useful for public health and policy decisions.
Together, these papers represent two important points on the continuum of research reviews and syntheses that are required in the field of intellectual disabilities. There is certainly a need for a range of papers that would complete this continuum, as outlined in the original Call for Papers. Our concern is that few such papers have been written to date, perhaps because the task is daunting or the research literature too limited or varied in method or focus to yield systematic reviews that meet contemporary standards. It is our hope that this will be the first of several special sections on evidence-based practices, and we invite inquiries and submissions for future sections.
