Abstract

The definition of intellectual disability, according to the American Association on Intellectual and Developmental Disabilities, includes the assumption that adequate supports should improve a person's functioning. Consequently, support needs have to be assessed to plan services for persons with intellectual disability. The Supports Intensity Scale (SIS; J. R. Thompson et al., 2004) is a standardized instrument for assessing support needs and their intensity. This study was designed to estimate the interrespondent, interinterviewer coefficients of the French version of the SIS. Approximately 40 persons with intellectual disabilities from Quebec, a Canadian province, participated in this study. For each participant, 2 respondents and 2 interviewers were identified and 3 French SIS questionnaires were filled out. Results are presented and discussed compared with those obtained with the original, English-based SIS.

Intellectual disability may be defined as the product of interactions between a person's skills and the demands of his or her environment (Luckasson et al., 2002). Support is a key element in understanding intellectual disability because providing adequate and sustained supports enhances a person's functioning and reduces the gap between the person's skills and the environmental expectations from him or her. Support is a multidimensional concept. It could be defined as “resources and strategies that aim to promote the development, education, interests, and personal well-being of a person and that enhance individual functioning” (Luckasson et al., 2002, p. 151). Supports may be provided by multiple sources, including the natural environment and professional services. Planning supports for a person with an intellectual disability requires a systematic and reliable analysis of his or her needs and should be person centered (Thompson et al., 2002). To conduct accurate analysis, instruments are needed to determine support intensity needs (MacMillan, Gresham, & Siperstein, 1993). However, most of the available instruments assess support needs based on adaptive and/or challenging behaviors exhibited by the person but do not assess multidimensional support needs (Lamoureux, 2006).

The Supports Intensity Scale (SIS; Thompson et al., 2004) is the only instrument that assesses support needs across most life activity areas, which are as follows: home living, community living, lifelong learning, employment, health and safety, social, and protection and advocacy activities. For Sections 1 (life activity area) and 2 (protection and advocacy area), support needs are assessed on three different 4-point Likert scales that evaluate the frequency and duration of daily support as well as the type of support requested. For the third section (exceptional medical and behavioral support needs), the SIS also permits assessment of support needs based on four medical areas (respiratory care, feeding assistance, skin care, and other exceptional medical needs) and four behavioral areas (externally directed destructiveness, self-directed destructiveness, sexual problem behaviors, and other challenging behaviors). Exceptional medical or behavioral support needs are assessed on a 3-point scale from 1 (none) to 3 (extensive). The SIS provides scores for each domain, as well as an index score—the Support Needs Index (SNI)—which indicates the overall support needs based on results for the first section. Four levels of intensity may be considered in support planning (Luckasson et al., 2002; Luckasson et al., 1992). Supports could be (a) intermittent: punctual; (b) limited: of a determined length; (c) extensive: frequent but of a limited length and in several environments; or (d) pervasive: constant and in several environments.

The SIS is designed to assess support intensity needs for persons with intellectual disability aged 16 years and older. Administration of the SIS requires 30 to 45 min. It should be completed through a semistructured interview with a person who has known the assessed person for at least 3 months and has had the opportunity to observe him or her in different settings. The SIS User's Manual (Thompson et al., 2004) indicates that interviews should be conducted by a professional who has completed at least a 4-year degree program and has several years of direct work experience with persons with intellectual disability. Respondents could be the person him- or herself or someone living or working with the person whose support needs are being assessed, such as a parent, a direct-care staff member, a case manager, or a teacher.

The SIS areas were developed through a literature review and a Q-sort method process. Four field tests were also conducted. The SIS appears to be a reliable instrument because of its good to excellent psychometric properties (Thompson et al., 2004). Good reliability properties are necessary to determine if observed individual differences are indeed due to individual characteristics rather than bias in data collection. Cicchetti and Sparrow (1981) provided guidelines in interpretation of reliability coefficients. It is assumed that a correlation coefficient lower than .40 is poor; one between .40 and .59 is fair; .60 to .74 is good; and .75 or more is excellent. The SIS internal consistency coefficients are very high, with alpha coefficients from .95 to .99 for all six domains and for the SNI score. Test–retest reliability with a 3-week retest delay is excellent (r = .79). The SIS User's Manual indicates fair to good interrater reliability, with correlation coefficients from .36 to .79 for each domain and a correlation of .54 for the index score. However, a second study (Tassé, Thompson, & McLaughlin, 2006) has indicated higher interrater reliability scores, ranging from .51 to .92 for each domain and a correlation of .90 for the index score. Interrater discrepancy might be explained by the procedure chosen to train interviewers. In the study published in the SIS User's Manual (Clay-Adkins, 2004), interviewers received informal instructions and a short description of the SIS. In Tassé, Thompson, and McLaughlin's study (2006), interviewers received a 1-day training session. The SIS User's Manual was also available. According to Tassé et al., higher interrater reliability coefficients clearly support the importance of formal training of SIS interviewers. Because of the nature of the SIS interview, determining SIS interrater reliability is not enough because this does not provide information about the scale's stability with different respondents. To permit separate analysis of interviewer and respondent concordance, Thompson, Tassé, and McLaughlin (2006) provided interrespondent reliability checks. SIS interrespondent reliability was good to excellent, with rs between .61 and .85 in each domain and .87 for the SNI score. Table 1 presents the SIS (original, English version; Thompson et al., 2004) correlation coefficients computed by Clay-Adkins (2004) as well as Tassé et al. (2006).

Table 1

SIS (Original Version) Correlations: Section 1 and SIS Index

SIS (Original Version) Correlations: Section 1 and SIS Index
SIS (Original Version) Correlations: Section 1 and SIS Index

A French translation and cultural adaptation for the population of Quebec (Canada) was proposed by Lamoureux (2006). Lamoureux used Tassé and Craig's (1999) translation and cultural adaptation method. First, six French speakers (Committee 1) individually translated the original version of the SIS and compared their translations to obtain a consensual preliminary version of the SIS French version (SIS-F). A second committee made up of three independent bilingual translators compared the preliminary version with the original SIS. The preliminary version was then revised by the two committees to obtain a consensual version of the SIS-F. Pilot field tests were then completed, which allowed the committees to develop a final SIS-F version. Exploratory factor analyses were computed using maximum-likelihood extraction with orthogonal rotation (Varimax). The extracted structure was similar to the structure of the original version. Eigen values indicated one main factor explaining a high percentage of variance and four or five others explaining smaller parts of the variance. The SIS-F demonstrated good internal consistency in each area, with coefficients equal to or higher than α = .89. These results were equivalent to those obtained with the original SIS. Internal consistency on the SNI score was excellent, with α = .98. The SIS-F convergent validity was also estimated using correlations with levels of intellectual disabilities. Strong correlations (r ≥ .50) that were significant, with p < .01, were demonstrated. These correlations were comparable with those obtained with the original SIS and suggested that support needs assessed with SIS-F are correlated with levels of intellectual disability. Furthermore, correlations with age were not significant, as demonstrated with the original SIS version, suggesting that SIS-F has good discriminate validity. These results indicate that SIS-F is an adequate translation and adaptation of the original SIS version. To determine if this instrument could be recommended for clinical purposes, its stability needed to be evaluated. Therefore, in this study, we assessed SIS-F's interinterviewer and interrespondent reliabilities.

Method

Participants

Participants were recruited through four Quebec local developmental disability agencies in which professionals were already trained to use SIS-F. Forty-two adults were evaluated to estimate the interinterviewer and interrespondent reliability of the SIS-F. Participants assessed on the SIS-F were randomly selected among the agencies' clientele. Participants had a mean age of 36.3 years, with a standard deviation of 11.93 years (range = 16–68 years). Twenty were females and 22 were males. Information concerning the participants' level of functioning was gathered to have a more accurate description of the sample. All the participants had a diagnosis of intellectual disability, as this is required to receive services from a developmental disability agency. However, recent diagnostic assessment had not always been conducted or data were not available. Consequently, IQ levels were unknown for 59% of the sample. Ten percent of the participants had an IQ higher than 70, 12% had an IQ between 51 and 69, 12% had an IQ between 36 and 50, and 7% had an IQ between 20 and 35. None were reported to have an IQ below 20. We also gathered information on adaptive functioning level, which is an assessment that is used more systematically in developmental disability agencies to plan services. Consequently, level of adaptive behavior functioning is missing for only 12% of the sample. Thirty-six percent had mild adaptive behavior deficits, 26% had moderate adaptive behavior deficits, 12% had severe deficits, and 14% had profound deficits of adaptive behavior. Because levels of adaptive behavior functioning tend to correlate with IQ levels (Kamphaus, 1987), we believe that the sample was representative of all the levels of functioning. Seven of the 42 participants had sensory impairments, and 8 had physical impairments. Ten of them were diagnosed with a psychi atric disorder, 4 were diagnosed with a developmental disability, and 14 had communication difficulties.

A total of 72 respondents participated in the reliability study. Respondents were required to have known the assessed person for at least 6 months and to have had the opportunity to observe him or her on a daily basis and in different settings. The person with the disability was excluded as a respondent for the purpose of the reliability study. Sixty-one percent of the respondents had known the assessed individual for at least 3 years. Thirty-four percent had known the person for 1 to 2 years, and only 5% of respondents had known the assessed person for less than one 1 year. All respondents reported knowing the person fairly well to very well. Thirty-nine individuals served as interviewers. The interviewers were recruited from among the professional staff of the local developmental disability agencies who had already been trained on the SIS-F. All interviewers met the minimal criteria recommended in the SIS User's Manual (Thompson et al., 2004). In addition, all interviewers attended a 1-day training session conducted by an AAIDD-recognized trainer on how to administer and score the SIS. All interviewers were trained by the same trainer. Seventy-six percent of interviewers had worked in the field of intellectual disability for at least 10 years.

Procedure

A resource person was identified in each agency. He or she was responsible for identifying persons who would be assessed, their respondents, and interviewers. This person checked each completed questionnaire to make sure all the items had been answered and the procedure had been followed. Two interviewers and two respondents were iden tified for each assessed person. Three SIS forms were completed for each individual. Interviewer 1 interviewed both respondents following the schedule described in Table 2 to compute interrespondent cor relations. Interviewer 2 interviewed Respondent 1 to determine interinterviewer correlation coefficients.

Table 2

Interviews Schedule

Interviews Schedule
Interviews Schedule

Results

Pearson correlations were computed for all six SIS domain scores and the SIS SNI score. Cicchetti and Sparrow's (1981) guidelines were used to interpret the reliability coefficients. The SIS-F SNI interinterviewer coefficient was .91, and its interrespondent correlation coefficient was .92. These coefficients fall in the excellent range. The SIS-F interrespondent and interinterviewer correlation coefficients for the six domains of Section 1 all fall in the excellent range (rs = .79–.92).

Section 2 distributions revealed ceiling effects on most of the measures. Means ranged from 5.64 (SD = 3.80) to 8.38 (SD = 4.06). Scores ranged from 0 to 12, but cumulative percentages of Scores 11 and 12 were higher than 33% for 16 measures on 24. Absolute value of distributions skewness was higher than 2 for 11 measures on 24. Due to this ceiling effect, scores obtained on the SIS-F Section 2, Protection and Advocacy Activities, were dichotomized, with 1 representing raw scores equal to the higher score that could be obtained on each item, and 0 representing other raw scores. The SIS-F Section 2 interrespondent reliability correlation coefficients ranged from .63 (good) to .90 (excellent). Interinterviewer correlation coefficients ranged from .35 (poor) to .72 (fair). Two interinterviewer coefficients were nonsignificant (see Table 4 ).

Table 4

SIS (French Version) Correlation Coefficients: Protection and Advocacy Activities

SIS (French Version) Correlation Coefficients: Protection and Advocacy Activities
SIS (French Version) Correlation Coefficients: Protection and Advocacy Activities

Due to important distribution abnormalities (skewness from 3.54 to 4.45 and kurtosis from 13.64 and 22.41), data obtained on the SIS-F Section 3, Exceptional Medical Support Needs, were also dichotomized, with 0 = total score equal to 0 and 1= score > 0. Thirty-nine percent of the Section 3 scores were equal to 0. As a consequence, no arithmetic transformations were possible. Interinterviewer correlation coefficient was .33 (poor) and interrespondent correlation was .74 (good; see Table 5 ). On the Exceptional Behavioral Support Needs variable, correlation coefficients were all good, with an interinterviewer correlation coefficient of .74 and an interrespondent correlation coefficient of .73.

Table 5

SIS (French Version) Correlation Coeffi cients: Exceptional Medical and Behavioral Sup port Needs

SIS (French Version) Correlation Coeffi cients: Exceptional Medical and Behavioral Sup port Needs
SIS (French Version) Correlation Coeffi cients: Exceptional Medical and Behavioral Sup port Needs

Discussion

Findings of the present study suggest that SIS-F has an overall good reliability. The first section of the instrument was reliable for planning supports needed by persons with intellectual disability. Results on this section were stable between different respondents and interviewers. Excellent correlations were also demonstrated on the SNI score. Consequently, SIS-F Section 1, Support Needs Scale, and the SNI score are recommended for clinical and administrative purposes. However, a test–retest reliability check should be conducted to study the SIS-F stability on different time measures.

On Section 1, most of the SIS-F reliability correlation coefficients were higher than those obtained with the SIS English version (Thompson et al., 2004) but were equal to those obtained by Tassé, Thompson, and McLaughlin (2006). As reported in Tassé et al. (2006), these findings suggest the importance of formal training of SIS interviewers to enhance SIS or SIS-F reliability. Such training was included in the procedure followed in the present study and in the Tassé et al. study, whereas reliability studies presented in the SIS User's Manual (Thompson et al., 2004) were conducted with interviewers who only received informal instructions and a short description of the SIS. The training offered to interviewers is particularly useful for explaining ambiguous SIS items and avoiding interpretation by the interviewer. Training must provide a common definition of each item and results in a more consistent use of the instrument.

The second section seemed less reliable. Correlation coefficients varied from poor to excellent, and some were nonsignificant. Interinterviewer correlations were particularly low. Results also demonstrated an obvious ceiling effect on these items. Consequently, the SIS-F Section 2, Protection and Advocacy Activities, should be carefully interpreted for clinical or administrative purposes. The SIS User's Manual (Thompson et al., 2004) also reported doubtful interinterviewer reliability on the protection and advocacy section. This section was originally included in the first section, but, considering its poor psychometric properties, it was split into a separate section. The SIS authors decided to keep these items to answer some important questions for support planning teams. The obvious ceiling effect strengthens the necessity of considering protection and advocacy support needs in the assessment, as it seems to be a significant support needs area for most of the evaluated persons. Additional research should investigate why interinterviewer correlations were not as good as interrespondent correlations or correlations obtained on the first section. The nature of the items may provide an explanation, as they are more subjective or ambiguous (Lamoureux, 2006). These results confirmed the need to train interviewers before using SIS or SIS-F but also indicated that such training may be insufficient. Supervision following the training could be useful to help interviewers master the skills and knowledge needed to use SIS or SIS-F.

The obvious ceiling effect highlights the need for future research to further describe protection and advocacy support needs for persons with intellectual or developmental disability. Research is useful for designing accurate interventions and services to develop protection and advocacy skills in persons with intellectual or developmental disabilities and, consequently, respect their fundamental rights. Moreover, it could be interesting to describe which individual characteristics are highly correlated or could explain such important protection and advocacy support needs. A systematic assessment of a large sample using SIS or SIS-F will help to answer these questions by allowing comparisons among groups of individuals.

Correlations obtained on the third section were good, except for the interinterviewer reliability on the Section 3A, Medical Support Needs, which was poor. The SIS-F Section 3 seemed to be reliable on assessing exceptional medical or behavioral support needs. The poor interinterviewer reliability on the medical support needs area is hard to understand considering the nature of the questions that do not require any interpretation to answer. No comparative data are available from previous studies on SIS Section 3. As a consequence, it is impossible to determine if the poor interinterviewer reliability was a translation effect. Again, interviewer training and supervision needs were confirmed by these results. Training should emphasize the importance of standard and consistent assessment and specific difficulties encountered while using SIS-F, especially with the second and the third sections.

Future research should investigate whether experience in using SIS or SIS-F has an impact on its reliability. Interviewers recruited in the present study were all using SIS-F for the first time. Because adequate training enhances the ability of interviewers to conduct SIS interviews, we hypothesize that additional experience would influence the assessment results. Furthermore, supervision while administering SIS or SIS-F for the first time may be useful to develop interviewer skills. Interviewer motivation and interest in conducting the assessment could be another important variable explaining the discrepancy in SIS and SIS-F reliability. In the pres ent study, interviewers demonstrated a high interest in participating. Future research should select interviewers randomly.

Table 3

SIS (French Version) Support Needs Scale and SIS Support Need Index Score Correlations

SIS (French Version) Support Needs Scale and SIS Support Need Index Score Correlations
SIS (French Version) Support Needs Scale and SIS Support Need Index Score Correlations

References

References
Cicchetti
,
D. V.
and
S. A.
Sparrow
.
1981
.
Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior.
American Journal on Mental Deficiency
86
:
127
137
.
Clay-Adkins
,
S. L.
2004
.
Reliability and validity of the Supports Intensity Scale.
Unpublished doctoral dissertation, Illinois State University, Normal, IL.
Kamphaus
,
R. W.
1987
.
Conceptual and psychometric issues in the assessment of adaptive behavior.
Journal of Special Education
21
:
27
35
.
Lamoureux
,
M.
2006
.
Les besoins de soutien des personnes présentant un retard mental manifestant des comportements problématiques [Support needs of individuals with intellectual disabilities and presenting challenging behaviours].
Unpublished doctoral dissertation, Université du Québec à Montréal, Montreal, Canada.
Luckasson
,
R.
,
S.
Borthwick-Duffy
,
W. G. E.
Buntix
,
D. L.
Coulter
,
E. M.
Craig
, and
A.
Reeve
.
et al.
2002
.
Mental retardation: Definition, classification, and systems of supports (10th ed.).
Washington, DC: American Association on Mental Retardation.
Luckasson
,
R.
,
D. L.
Coulter
,
E. A.
Polloway
,
S.
Reiss
,
R. L.
Schalock
, and
M. E.
Snell
.
et al.
(1992)
.
Mental retardation: Definition, Classification and systems of supports (9th ed.).
Washington, DC: American Association on Mental Retardation.
MacMillan
,
D. L.
,
F. M.
Gresham
, and
G. N.
Siperstein
.
1993
.
Conceptual and psychometric concerns about the 1992 AAMR definition of mental retardation.
American Journal on Mental Retardation
93
:
325
335
.
Tassé
,
M. J.
and
E. M.
Craig
.
1999
.
Critical issues in the cross-cultural assessment of adaptive behavior.
In R. L. Schalock (Ed.), Adaptive behavior and its measurement: Implications for the field of mental retardation (pp. 161–184). Washington, DC: American Association on Mental Retardation.
Tassé
,
M. J.
,
J. R.
Thompson
, and
C.
McLaughlin
.
2006
.
Inter-interviewer and inter-respondent concordance on the Support Intensity Scale.
Paper presented at the International Summit for the Alliance on Social Inclusion, Montreal, Canada.
Thompson
,
J. R.
,
B. R.
Bryant
,
E. M.
Campbell
,
E. M.
Craig
,
C.
Hughes
,
D. A.
Rotholz
,
R.
Schalock
,
W.
Silverman
,
M.
Tassé
, and
M.
Wehmeyer
.
2004
.
Supports Intensity Scale: User's manual.
Washington, DC: American Association on Mental Retardation.
Thompson
,
J. R.
,
C.
Hughes
,
R. L.
Schalock
,
W.
Silverman
,
M. J.
Tassé
, and
B.
Bryant
.
et al.
(2002)
.
Integrating supports in assessment and planning.
Mental Retardation
40
:
390
405
.
Thompson
,
J. R.
,
M. J.
Tassé
, and
C.
McLaughlin
.
2006
.
Interrater reliability of the Supports Intensity Scale (SIS).
American Journal on Mental Retardation
113
:
231
237
.

Author notes

Authors: Diane Morin, PhD (morin. diane@ uqam. ca), Associate Professor, and Virginie Cobigo, PhD (virginie. cobigo@ gmail. com), Department of Psychology, DS-2935, University of Quebec at Montreal, P.O. Box 8888, Downtown Station, Montreal (Quebec) Canada H3C 3P8