In its historic 2002 ruling (Atkins v. Virginia), the U.S. Supreme Court established that persons who are intellectually disabled cannot receive the death penalty because their compromised functioning precluded them from acting with adequate culpability. The Supreme Court articulated a definition of intellectual disability that is being used in some form in all states: significantly subaverage general intelligence that concurs with adaptive behavior deficits and was manifested in the developmental period of life (Ellis et al., 2004).
Adaptive behavior has come to be interpreted in various ways in the legal arena. In many states (e.g., CA, DE, ID, IL, MS, NC, OH, OK), adaptive behavior is considered in terms of the 10 skill areas cited in the American Association on Intellectual and Developmental Disabilities' (AAIDD; then the American Association on Mental Retardation [AAMR]) manual, Mental Retardation: Definition, Classification and Systems of Support (Luckasson et al., 1992): communication, community use, functional academics, home living, health and safety, leisure, self-care, self-direction, social, and work. In other jurisdictions, adaptive behavior is conceptualized as the three “dimensions” introduced in the revised AAIDD manual (Luckasson et al., 2002): conceptual skills, social skills, and practical skills (e.g., VA, PA). However, in numerous states, neither the legislature nor the judiciary has decided which aspects of adaptive behavior should be measured (e.g., AZ, CO, LA, NV, TX). In these locales, examiners are free to use any instrument they choose for evaluating the adaptive behavior of Atkins v. Virginia (2002) claimants. One of the choices has been the Street Survival Skills Questionnaire (SSSQ; Linkenhoker & McCarron, 1993). In this article, we clarify why the SSSQ should not be used to establish the adult adaptive behavior of criminal offenders who have filed a claim of intellectual disability.
Purpose of the SSSQ
The SSSQ (Linkenhoker & McCarron, 1993) was not designed for determining whether a person's adaptive behavior is adequate or deficient for the purpose of diagnosing intellectual disability. As explained by its developers, “The Street Survival Skills Questionnaire (SSSQ) was specifically constructed for a developmentally disabled population” (Linkenhoker & McCarron, 1993, p. 4).
The SSSQ was created for use as part of McCarron and Dial's (1986) approach for evaluating the functional strengths and weaknesses of “the mentally disabled” (McCarron & Dial, 1986). It was formulated to assess nine “components,” or skill areas: basic concepts, functional signs, tools, domestics, health and safety, public services, time, monetary, and measurements. It is clear from these descriptors that this instrument gauges a narrow slice of overall adaptive behavior because it does not address areas such as self-care, self-direction, use of leisure time, or social skills. Instead, because a critical purpose of McCarron and Dial's evaluation is discerning readiness of persons with intellectual disabilities for community-based employment, the SSSQ concentrates on measuring skills that constitute the practical dimension of the 2002 manual's (Luckasson et al., 2002) conception of adaptive behavior. On the basis of their SSSQ scores, clients are classified in terms of functional levels to permit more informed decisions about living and habilitation requirements and their readiness for the normalizing experience of community employment. As explained by Linkenhoker and McCarron (1993), the SSSQ was intended for four applications: “(1) classification with respect to mental retardation [level], (2) vocational program placement, (3) residential program placement, and (4) training strategies” (p. 35).
The 2002 test manual (Luckasson et al., 2002) reports that SSSQ (Linkenhoker & McCarron, 1993) scores correlated substantially (r = .68) with clients' work proficiency 1 year later (measured with the San Francisco Vocational Competency Scale; Levine & Elzey, 1968), which supports its use for vocational planning with persons who are intellectually disabled.
Norming of the SSSQ
Because the SSSQ (Linkenhoker & McCarron, 1993) was intended for persons who are developmentally disabled, its difficulty level was set to accommodate these individuals, and it was normed on a seemingly representative group of 400 persons with severe to mild mental disabilities, whose Peabody Picture Vocabulary Test (Dunn, 1965) IQ averaged 58. Prior to its 1983 publication, the SSSQ was also administered to three groups of adolescents who were not developmentally disabled, in efforts to expand its applicability to the assessment of “normal adolescents and adults who are characterized by lower intellectual functioning, psychiatric disorders, social disadvantagement or a history of prolonged institutional living” (Linkenhoker & McCarron, 1993, p. 9). One of these was “a normal adolescent group” of 200 individuals from “two secondary school pre-vocational programs located in Indiana and Texas” (p. 12), whose ages ranged between 14 and 18 years. For unknown reasons, the SSSQ developers referred to this group as “normal adults” and “average adults.”
On the basis of those youths' scores, two tables were constructed: One table converted raw scores for the nine SSSQ (Linkenhoker & McCarron, 1993) areas into scaled scores, with a mean of 10 and a standard deviation of 3, and the other converted total SSSQ scores into standard scores, with a mean of 100 and a standard deviation of 15 (Linkenhoker & McCarron, 1993, Tables 21 and 22, pp. 33–34). These tables have fostered a belief that the SSSQ was normed on adults who were not intellectually disabled and that the SSSQ is therefore a reliable and valid instrument for establishing the adult adaptive behavior of Atkins v. Virginia (2002) claimants (e.g., Fisher & Samson, 2003; Reynolds et al., 2003; Stevens & Price, 2006). However, as evident from study of the SSSQ manual, this test was not normed on adults, and the cited reliability and validity data were derived from the scores of the standardization group of persons with developmental disabilities.
Adequacy of the SSSQ Norms for Diagnostic Purposes
It is well accepted in the psychological assessment community that a test's standardization sample must be large enough to provide reliable data and that it must be representative of the population to which it will be applied (e.g., Anastasi & Urbina, 1997). Thus, for example, the self-report version of the Adaptive Behavior Assessment System-II (ABAS-II; Oakland & Thomas, 2003) was normed on 990 adults in seven age groups that were stratified by geographic location, sex, race/ethnicity, and educational level in accord with the 1999 U.S. Census. Such a large sample provides reasonable assurance that the resultant norms adequately portray the adaptive behavior of the average person at various ages. However, the SSSQ's (Linkenhoker & McCarron, 1993) “normal/average adults” were 200 7th- to 12th-grade students from two U.S. schools, which cannot be considered to represent the adult U.S. population. In that regard, the current AAIDD manual (Luckasson et al., 2002) advises the following: “The validity of scales normed primarily on individuals in segregated school, work, or living arrangements may be limited to context that are useful for programming but are not acceptable for diagnostic purposes” (p. 83).
Because the SSSQ's (Linkenhoker & McCarron, 1993) “normal/average adults” norms were not derived from either normal or average adults, much less on a sufficient and representative number of them, its use to establish any aspect of the adaptive behavior of adults who have not already been diagnosed as being developmentally disabled is contraindicated.
Reliability and Validity of the SSSQ for Diagnostic Purposes
The SSSQ (Linkenhoker & McCarron, 1993) manual presents neither reliability coefficients nor measurement error for the “normal/average adults” group, and there appear to exist no published data that indicate that SSSQ scores are valid indexes of general adaptive behavior. Table 2 of the SSSQ manual (Linkenhoker & McCarron, 1993, p. 12) shows that the reliability coefficients and measurement error it presents were computed on a group of scores for which the total features a 30-point standard deviation. Because the standard deviation of the “normal/average adults” data is reported to be 22, the reliability indexes in that table could not have been derived from their test scores. The SSSQ reliability data must therefore have been calculated from the scores of those persons with mental disabilities on whom the instrument was standardized. The reliability of scores produced by “normal/average adults” is therefore unknown, which, among other things, precludes calculation of the measurement error inherent in them. It is therefore impossible to ascribe a confidence level to SSSQ standard scores that are derived from the “normal/average adults” raw-score conversion tables. In practical terms, this means that the SSSQ scores produced by persons who have not been diagnosed as being developmentally disabled, such as Atkins v. Virginia (2002) claimants, are unreliable.
The validity studies cited in the SSSQ (Linkenhoker & McCarron, 1993) manual were also not conducted with the “normal/average adult” group's data. The manual shows only that the SSSQ scores produced by persons who are developmentally disabled were useful for various habilitation purposes. For example, the correlation between such individuals' SSSQ total raw score and a measure of vocational performance designed for persons with developmental disabilities—the San Francisco Vocational Competency Scale (Levine & Elzey, 1968)—was moderate, which implies acceptable concurrent validity. However, the SSSQ subtest scores of those persons did not correlate significantly with their scores on the Self-Help, Socialization, or even Occupation Scales of a broader adaptive behavior measure, Gunsburg's (1963) Progress Assessment Chart (as cited in Linkenhoker & McCarron, 1993, Table 19). In view of that finding, Haring (1992) expressed concerns about the SSSQ's validity as a measure of overall adaptive behavior and advised that it be used only in conjunction with more comprehensive instruments. Subsequently, Janniro et al. (1994) found no significant (r = .12) relationship between the SSSQ and Vineland Adaptive Behavior Scales' (Sparrow, Balla, & Cicchetti, 1984) total scores in a group of youth with developmental disabilities. Since that time, it has come to be generally accepted that the SSSQ is not a measure of overall adaptive behavior and that it is best suited for its designed use, predicting “entry or retention of competitive, gainful employment among people with mental retardation” (Reschly et al., 2002, p. 173).
Expressed Rationale for Using the SSSQ in
Atkins v. Virginia Contexts
Validity studies that contraindicate the SSSQ's (Linkenhoker & McCarron, 1993) diagnostic utility and lack of reliability data for adults who are not disabled would seem sufficient to establish that this test is unsuited for assessing the adaptive behavior of adults. Nonetheless, some still consider the SSSQ to be a legitimate diagnostic tool in Atkins v. Virginia (2002) contexts (Stevens & Price 2006), and research has been cited in efforts to portray this instrument as being an appropriate diagnostic tool. Giller et al.'s study (1986) found the SSSQ to be comparable with the AAIDD's Adaptive Behavior Scales (ABS; Lambert et al., 1974), in terms of capacity to discriminate adequate from deficient adaptive behavior (Reynolds et al., 2003). However, that study merely showed that the SSSQ classified persons who are mentally disabled as well as the ABS in terms of adaptive behavior quality. That study did not address whether the SSSQ discerned adaptive behavior deficits as well as the ABS or whether the SSSQ can assess accurately the adaptive behavior of adults who are not disabled. Moreover, its authors concluded that the SSSQ provides a “measure of specific adaptive behavior” (p. 67), not a comprehensive assessment.
Some have claimed that Janniro et al.'s study (1994), cited above, which was also conducted with persons who are developmentally disabled, found the SSSQ (Linkenhoker & McCarron, 1993) to be comparable with the Vineland Adaptive Behavior Scales (Reynolds et al., 2003; Sparrow et al., 1984). Yet, Janniro et al. determined the opposite: “The two tests are not measuring the same general construct” (p. 194). In addition, Janniro et al. pointed out that “none of the questionnaire content covers social skills, moral understanding, or interpersonal relationships” (p. 194), and likewise advised against using the SSSQ as the sole assessment instrument, even with individuals who are intellectually disabled.
Thus, there do not appear to be any published data that establish that the SSSQ (Linkenhoker & McCarron, 1993) validly measures any aspect of adaptive behavior of adults who are not intellectually disabled. Even with persons who are intellectually disabled, those who have researched this test, including its developers, have found that it assesses a narrow group of behaviors of a select segment of the population, so that the test's total score is not a valid index of anyone's overall adaptive behavior.
Consequences of Using the SSSQ in
Atkins v. Virginia Contexts
Because the SSSQ (Linkenhoker & McCarron, 1993) was designed for persons who are severely to mildly mentally disabled, it is an “easy” test that “has little or nothing to do with adapting to real-world environments or tasks” (Greenspan, 2006, p. 214). Our review of the total raw scores obtained by the three groups of “normal/average” adolescents confirms that expectation. The highest possible total raw score is 216, and the manual reports that this group (i.e., 14 to 18 year olds) obtained a mean score of 200, with a standard deviation of 22 (Linkenhoker & McCarron, 1993, p. 32). This finding portrays a ceiling effect, which occurs because many youth obtained perfect or near-perfect scores. Those who earned the highest possible scores—who “hit” the ceiling—could have scored higher if the test had allowed it. However, because the ceiling prevented them from doing so, they were grouped together with less capable youth. As a result, those who actually had much less ability than the most able did not produce scores that were significantly lower (i.e., statistically different). For example, children aged 9 through 12 averaged a total score of 190 (Linkenhoker & McCarron, 1993, Table A). These data indicate that 9 year olds performed as well on the SSSQ as “normal/average adults.” Relatedly, the authors found that “persons with severe neurological disabilities” (Linkenhoker & McCarron, 1993, p. 37) had to obtain SSSQ raw scores of “179+” before they were likely succeed in very basic community employment. With a standard deviation of 22, a score of 179+ is not significantly different from 190 or even 200. Thus, the data reported by Linkenhoker and McCarron indicate that persons who function like typical defendants with mild mental retardation will produce total SSSQ scores that are not meaningfully lower than those of “normal/average adults.” As a result, the SSSQ is unlikely to discern significant deficits in adaptive behavior among criminal defendants who are mentally retarded and will misclassify many or most as possessing adequate adaptive behavior.
With respect to how a test can be applied, the American Psychological Association (APA) guidelines for “competent and responsible” test use caution that
(N)ormative data or decision making rules may not be accurate when (a) important characteristics of the examinee are not represented in the norm group, (b) administration or scoring procedures do not follow those used in standardizing the test, (c) characteristics of the test may affect its utility for the situation (e.g., ceiling or floor effects), (d) the test contains tasks that are not culturally relevant to the test taker, or (e) the validity evidence does not support decisions made on the basis of the test scores. (Turner et al., 2001, p. 1101)
Use of the SSSQ (Linkenhoker & McCarron, 1993) to establish the adult adaptive behavior of Atkins v. Virginia (2002) claimants thus disregards psychometric standards that the psychological testing community considers to be critical. The SSSQ was designed to assess prevocational skills of persons who are developmentally disabled, not the adaptive behavior of those being evaluated for the possibility of developmental disabilities. Norming with a representative adults who are not disabled was never conducted. Reliability of the SSSQ scores of persons who are not mentally disabled is unknown. A pronounced ceiling effect makes it virtually impossible to discriminate persons who function at the higher levels of developmental disability from those who are not disabled. Moreover, this instrument has not been shown to be a valid measure of overall adaptive behavior, even for persons who are developmentally disabled.
The SSSQ's (Linkenhoker & McCarron, 1993) shortcomings illustrate why Speart (1999) warned that no instrument can adequately serve all measurement purposes. The SSSQ was designed for persons with “severe neurological disabilities.” To be applicable to that population, its difficulty had to be adjusted to an ability level that accommodates those with IQs in the 20s and 30s (Linkenhoker & McCarron, 1993, Table 1). As a result, the SSSQ is an easy test for anyone capable of the most basic kinds of work in the community, those with mild mental retardation. Its use in Atkins v. Virginia (2002) contexts thus penalizes criminal defendants with mental retardation for the adaptive behavior they have been able to acquire. It is very important, especially now that psychometrically sound instruments like the ABAS-II (Burns, 2005) provide broad measurement coverage, that the SSSQ not be used to diagnose developmental disabilities. Representations of SSSQ data as indexes of a criminal defendant's adaptive behavior must therefore be rebutted vigorously as a misuse of the instrument.
Authors: George C. Denkowski, PhD (firstname.lastname@example.org), Clinical Psychologist, and Kathryn M. Denkowski, EdD, Psychologist, Private Practice, 1701 River Run, Suite 1105, Fort Worth, TX 76107