In his commentary “against therapy,” Sturmey (2005) concluded that “given the hundreds of studies documenting the effectiveness and efficiency of behavioral interventions, behavioral approaches must remain the preferred treatment option for people with mental retardation” (p. 56). The evidence on which he makes this conclusion are the nine controlled studies in Prout and Nowak-Drabik's (2003) meta-analysis, Carr et al.'s (1999) narrative review, two meta-analytic reviews of single-case experimental design studies (Didden, Duker, & Korzilius, 1997; Scotti, Evans, Meyer, & Walker, 1991), clinical practice guidelines (New York Department of Health, 1999a, 1999b, 1999c), and a meta-analysis of choice-making interventions (Shoegren, Faggalla-Luby, Bae, & Wehmeyer, 2004). Sturmey pointed out that these authors documented a very large number of empirical studies of behavior interventions, including those for mental health problems, and an absence of well-conducted empirical studies for nonbehavioral interventions. However, Sturmey did not provide any critical analysis or commentary on the evidence base for behavioral interventions. In this commentary I draw attention to a number of significant aspects of this evidence base that practitioners need to be aware of when formulating behavioral interventions.
Sturmey (2005) is right in his assertion that there is a lack of evidence for nonbehavioral interventions. This was the conclusion of two recent reviewers of cognitive–behavioral and psychodynamic interventions (Beail, 2003; Willner, 2005). However, the research reviewed in Beail (2003) and Willner (2005) reflected a process of development from descriptive case reports to empirically evaluated case series and, more recently, attempts at controlled designs, a progression known as the hour-glass model (Roth & Fonagy, 1996). This differs markedly to the evidence base for behavioral interventions, which largely consists of hundreds of reports of single-case experimental designs in which the participants act as their own control. Carr, Innis, Blakeley-Smith, and Vasdev (2004) described the ethical, practical, and conceptual reasons why group designs are so uncommon. However, because participants in behavioral intervention research are “not randomly assigned to treatments, the validity of our conclusions may be jeopardized” (Didden et al., 1997, p. 397).
Didden et al.'s (1997) meta-analysis is an extension of Scotti et al.'s (1991). Thus, they cover the same and additional studies. Didden et al. expanded the number of journals included and extended the time period. This present commentary is focused on these analyses due to their extensive coverage and the existence of an audit trail provided by the detailed tables in their reports.
For Whom Are Behavioral Interventions Effective?
Examinations of characteristics of the participants in the studies reviewed suggest some significant limits on the generalizability and transferability of the evidence base.
The majority of the participants in the hundreds of studies were children. Scotti et al. (1991) reported that 76% of the participants were children. Didden et al. (1997) did not report age ranges but did note that the average age of participants was 16 years. Further, the New York Department of Health (1999a, 1999b, 1999c) publications, quoted by Sturmey (2005), only cover children up to 3 years of age. Thus, adults with mental retardation are underrepresented.
Level of functioning
Didden et al. (1997) found 34 presenting problems, but over half of the participants presented with self-injurious behavior or stereotypy. When grouped for the meta-analysis with other internally maladaptive behaviors, the percentage of such behaviors rose to 69.6. This is perhaps not surprising because the majority of the participants also had severe or profound levels of mental retardation; for this population internally maladaptive behaviors are a major reason for clinical intervention. Also, in most cases the behavior being measured occurred at a high frequency.
Of Didden et al.'s (1997) 34 topographies, only 3 could be classified as a mental health as opposed to a behavior problem. Taken together, these 3 topographies accounted for 1.25% of the data in the analysis. Neither Scotti et al. (1991) nor Didden et al. included mental health presentations as a variable in their analyses.
Didden et al. (1997) and Scotti et al. (1991) did include externally destructive behaviors as a category in their analyses. In the psychotherapy literature reviewed by Beail (2003), anger featured as a major outcome variable. In the cognitive–behavioral approach, anger is treated as a mediating variable for aggression. Thus, the goal of therapy is a reduction in self-reported feelings of anger. Anger was not one of Didden et al.'s (1997) topographies, but aggression was represented (9.3%). However, Scotti et al. and Didden et al. concluded that such behaviors tend to be less successfully treated than are internally maladaptive behaviors.
Range of Interventions
In their more extensive review, Didden et al. (1997) found 64 primary treatment procedures. They grouped these into four categories, one of which was pharmacology. However, this was the smallest category, accounting for only 6.78% of the interventions. Thus, the results of the meta-analysis are based on three broad groupings of behavioral interventions. More detailed analyses of treatment procedures were not possible due to missing data for many cells. Another feature of the data on which the meta-analysis was conducted was the number of aversive interventions. Scotti et al. (1991) noted an increase use in these methods over time, but Didden et al. did not comment on this. Aversive techniques accounted for 25% of the behavioral interventions. In addition, further interventions were listed that are based on physical restraint or punishment. Such interventions have been subjected to increasing criticism and have become associated with inhumane treatment and as being out of step with current service values and philosophies. A meta-analysis carried out today would most likely show a significant decline in published reports of aversive- and punishment-based interventions.
Intervention Location and Standards of Practice
Scotti et al. (1991) reported that 95% of the interventions were implemented and evaluated in nonintegrated settings. Similarly, Carr et al. (1999) and Didden et al. (1997) noted the lack of community-based interventions. Scotti et al. concluded that “Overall, the standards of practice revealed by the published literature can only be described as highly disillusioning” (p. 254). Didden et al. confirmed Scotti and colleagues' finding of a positive relation between functional analysis and treatment outcome. However, Carr et al., Didden et al., and Scotti et al. all identified a lack of functional analyses being carried out (almost 80%). Scotti et al. found all levels of treatment intrusiveness being applied to all levels of behavior severity. Only 22% of the researchers showed any evidence of trying to employ a hierarchy of least intrusive alternative treatments. Also, they noted that women were more likely to be subjected to aversive interventions. They were also concerned about the lack of procedural documentation, especially where intrusive interventions were concerned.
Didden et al. (1997) and Scotti et al. (1991) also commented on methodological weaknesses in the studies they reviewed. Scotti et al. noted that only 30% of the investigators reported any attempt at generalization, and Carr et al. commented that reports concerning generalization were largely anecdotal. Scotti et al. also criticized the researchers for their lack of follow-up data. Over half did not report any, and of those that who did, it was for 6 months or less. Didden et al. critiqued the meta-analytic measure, percentage of nonoverlapping data, that were used. They pointed out that by employing this measure, investigators failed to consider magnitude of change. Thus, similar percentages of nonoverlapping data scores represent different magnitudes of change. Also, because authors of many single-case reports did not include phase mean and standard deviations or raw data, the issue of magnitude of change could not be included in their analysis.
In this commentary I have tried to draw attention to some of the limitations of the evidence base for behavioral interventions. The evidence base has a number of features that clinicians need to consider when formulating interventions and for future research. The majority of the participants were children with severe and profound levels of mental retardation presenting with high frequency internally maladaptive behaviors. Thus, the evidence base for this client group is good. However, such claims have to be tempered by the fact that over 25% of the behavioral interventions were aversive- or punishment-based. Further, aversive interventions were not more effective, so there is an evidence base as well as an ethical and humanistic base for not using them.
The evidence base has limited applicability to treatments for adults with mild mental retardation and those who present with externally destructive behaviors. This client group seems to be receiving greater attention in the cognitive–behavioral and psychodynamic literature (Beail, 2003; Willner, 2005). Further, contrary to Sturmey's (2005) claim, there is very little evidence for the effectiveness of behavioral treatments for mental health problems.
The majority of the interventions in the evidence base were implemented in segregated settings, with very few community-based interventions. The evidence base may not generalize or transfer to community-based settings. Whittaker (1993) has argued that we may need to develop new approaches regarding aggression, but this would seem to be the case for the treatment of many other challenging behaviors in community settings. Whitaker also drew attention to the fact that behaviors treated in the extant behavioral literature tended to be high frequency, whereas in the community they tended to be low frequency but high impact. Also, Carr et al. (2004) noted that as we enter a new era emphasizing community-based approaches for dealing with challenging behavior, much flexibility and innovation will be required for both research design and measurement strategies.
In this commentary, I have suggested that there is no room for complacency with regard to the evidence base for behavioral interventions. The evidence base has a number of positive aspects but also limitations regarding transferability and generalizability. These limitations make it difficult to accept Sturmey's (2005) generalization that behavioral approaches remain the preferred treatment option for all people with mental retardation. Clinicians should use the evidence base more critically and with greater specificity when formulating interventions. However, we also need to continue to be creative and explore the use of interventions that have been researched as effective with the general population. The lack of evidence does not mean that nonbehavioral interventions are ineffective. Further, people with mental retardation have a right to a similar range of therapies that are used in the general population. We also need to adapt and develop behavioral interventions that have potential application in community-based settings and continue to research our endeavours to develop the evidence base.
Author: Nigel Beail, PhD, Professor, Barnsley Learning Disability Service, The Keresforth Centre, Keresforth Close, Barnsley, S70 6RS, United Kingdom. Nigel.firstname.lastname@example.org