Abstract

The Learning Disability Screening Questionnaire (LDSQ), a brief screening tool for intellectual disability, was originally validated against the Weschler Adult Intelligence Scale, Third Edition (WAIS-III), which was superseded by the Weschler Adult Intelligence Scale, Fourth Edition (WAIS-IV) in the United Kingdom in 2010. This study examines the performance of the LDSQ using the WAIS-IV as the diagnostic intellectual assessment. Based on the original optimal cut-off score, the LDSQ sensitivity value was equivalent (91%) to that obtained in the original validation study, and the specificity value was higher at 92%. This suggests that the LDSQ remains valid when using the WAIS-IV as the comparative intellectual assessment.

There are a number of circumstances under which it may be helpful to have a quick and accurate indicator of whether an individual is likely to have an intellectual disability (ID). This can include the need to quickly identify people with ID in situations where they may be particularly vulnerable to exploitation, disadvantage, or abuse, such as criminal justice services (Talbot, 2008), but where limited access to an appropriately qualified psychologist precludes timely full diagnostic assessment (Crawford, Allan, & Jack, 1992). Screening assessments may also be helpful in instances where all that is required is an estimate of population characteristics to stratify or match participants in clinical research (Spinks et al., 2009). Screening may also occur in services where there are limited resources and/or high demands (Crawford, Anderson, Rankin, & MacDonald, 2010), for example, to prioritise those individuals most in need of full diagnostic assessment.

The Learning Disability Screening Questionnaire (LDSQ; McKenzie & Paxton, 2006) was designed as a brief screening tool to identify adults who are likely to have an ID. Developed in the United Kingdom (where “learning disability” is the commonly used terminology to refer to people with ID), the LDSQ is now used internationally in countries across Europe, as well as Canada, the United States, and Australia. The LDSQ has seven items relating to reading, writing, telling the time, current living situation, employment, previous support at school, and contact with ID services. Items are scored dichotomously and used to calculate a total percentage score. Those scoring below the cut-off score are considered likely to have ID.

Previous research has suggested that the LDSQ has good psychometric properties in groups of individuals at elevated risk of ID in both clinical and forensic settings (e.g., McKenzie, Michie, Murray, & Hales, 2012; Murray & McKenzie, 2013). In respect to the former setting, the scale has yielded sensitivity and specificity values of 91.2% and 87%, respectively, for ID in a sample referred to community ID health services (Paxton, McKenzie, & Murray, 2008). Sensitivity is the ability to correctly identify true positives (in this case those with an ID), and specificity is the ability to correctly identify true negatives (those who do not have an ID). This was based on a cut-off score determined by a receiver operating characteristic (ROC) curve analysis (Schoonjans, 1998), which provided the best balance between sensitivity and specificity. The LDSQ total percentage scores were found to correlate significantly with the Full Scale IQ measurement derived from the Weschler Adult Intelligence Scales (r = 0.751) and to discriminate between those with and without ID, with those with ID having a significantly lower score. In terms of inter-rater reliability, kappa values for each of the items indicated fair to excellent agreement, with the highest levels of agreement relating to current living situation, literacy, and employment (McKenzie & Paxton, 2006).

When considering forensic settings, the LDSQ was found to have sensitivity and specificity values of 82.3% and 87.5%, respectively, to correlate significantly with Full Scale IQ (r = 0.71) and to discriminate between those with and without ID, with those with ID again having a significantly lower score (McKenzie et al., 2012).

Research using Mokken scaling analysis was also conducted combining data from both community and forensic settings, (Murray & McKenzie, 2013). Mokken scaling analysis is a nonparametric item response theory technique that can be used to assess whether items in a scale form a difficulty hierarchy (Mokken, 1971). Applied to the LDSQ, it was used to evaluate whether knowing that one feature of intellectual disability (e.g., reading difficulty) applies to an individual can be used to infer whether other more or less severe difficulties (e.g., difficulty telling the time) are also likely to apply. This showed that, with the exception of the item relating to writing ability, the LDSQ items conformed to a hierarchy of difficulty such that information on individual items could provide clinically useful information about severity of impairment and other difficulties that an individual might experience. This allows preliminary inferences to be made about the severity of the difficulties even when the available information is incomplete. A subsequent analysis (Murray & McKenzie, 2014) used multicategory ROC analysis to assess whether the LDSQ could be used not only to identify an individual who should undergo full assessment for ID, but to also provide initial estimates of their appropriate diagnostic category in terms of “no ID,” “mild,” “moderate,” and ”severe” ID. They found that, although mean LDSQ scores increased across diagnostic categories from “severe impairment” to “no ID” as expected, the classification accuracy for those with ID was insufficient to categorise them according to severity of ID in a clinical context.

Recent research (Murray, Booth, & McKenzie, 2015) has suggested that the LDSQ shows measurement invariance in relation to gender. This suggests that it estimates level of ID in a manner that is not biased in favour of either males or females.

The LDSQ was validated against formal clinical diagnosis, given independently by health professionals working in the community or health setting. This was based on the assessment of the three criteria of ID (significant impairment of intellectual and adaptive functioning and childhood onset; British Psychological Society [BPS], 2001). As the BPS, the professional body in the United Kingdom, does not make a formal recommendation in relation to choosing a particular measure of adaptive functioning (BPS, 2001), a wide range of standardised and nonstandardised assessments were used by the independent diagnosing clinician. Unfortunately this meant that there were insufficient data from any one assessment to assess the relationship between LDSQ and adaptive functioning scores.

The Wechsler scales are commonly used to assess intellectual functioning (Reynolds, Ingram, Seeley, & Newby, 2013), and research has found that the LDSQ correlated significantly with the Wechsler Adult Intelligence Scales, Third Edition (WAIS-III; Wechsler, 1998) in both community (McKenzie & Paxton, 2006) and forensic clinical (McKenzie et al., 2012) populations.

It is recognised that factors such as the Flynn effect—that is, the observed rise in IQ scores over time (Flynn, 1984)—have relevance for the accuracy of assessments administered after the initial norming of the assessment and can make the norms of older assessments obsolete (Trahan, Stuebing, Fletcher, & Hiscock, 2014). For example, although the exact magnitude of the Flynn effect can vary according to the test used and age of the participant (Beaujean & Sheng, 2014), it has been found to have an effect at all levels of ability (Trahan et al., 2014). There is also some evidence that it may impact differentially according to intellectual ability, with people with lower levels of ability showing greater IQ gains over time (see Colom, Lluis-Font, & Andrs-Pueyo, 2005).

In recognition that the Wechsler scales are affected by these issues (Beaujean & Sheng, 2014), they are updated periodically. The WAIS-III was superseded by the fourth edition (WAIS-IV) in the United Kingdom in 2010 (Wechsler, 2010) and, although there remain concerns about the psychometric properties of the WAIS-IV in relation to people with ID (see Reynolds et al., 2013), this assessment is now commonly used to determine if an individual meets the criterion for ID of significant impairment in intellectual functioning. As such, it is important that the validity of the LDSQ is re-assessed in relation both to the WAIS-IV and clinical diagnoses based partly on the use of the WAIS-IV. The present study, therefore, reports on the performance of the LDSQ against diagnoses of ID, based on the WAIS-IV.

Method

Ethical Approval

Caldicott guardian approval was obtained to gather pre-existing assessment information. In Scotland, the Caldicott guardian acts to ensure the appropriate use of existing National Health Service (NHS) data in situations where obtaining individual patient consent would not be feasible.

Procedure

Data were gathered from a health board area in Scotland, which included community and forensic ID services. National Health Service provision in Scotland, with the exception of a few specialist services that are provided on a national basis, are organised by geographical areas, which are classified as health board areas. All service provision information was obtained from existing case notes and from the assessing clinician in relation to the following: basic demographic information, including age at assessment, gender, and diagnosis in relation to ID; WAIS-IV scores; and information to score the LDSQ items. All data were anonymised and no identifying information was gathered. The data relating to the LDSQ items had been gathered independently by the diagnosing clinician (an applied psychologist) during the assessment process, either directly from the client or by a third party respondent, such as a carer or relative, depending on the item. This is in keeping with the administration procedure of the LDSQ, which was designed to be completed with the individual him- or herself or by someone who knows him or her well.

Inclusion Criteria

Data were included for all adults who had been assessed by the ID services in the participating health board area using the WAIS-IV and for whom the LDSQ could be completed from existing information.

Participants

The total sample size was 103 and data were included for 83 individuals who met the inclusion criteria. Of these, 59 had a diagnosis of ID and 24 did not. Of those with ID, 37 were male and 22 were female, with a mean age of 32.1 years (SD = 15.4; range = 16 to 66) at assessment. The average WAIS-IV Full Scale IQ (FSIQ) of those with ID was 58.9 (SD = 6.5; range = 41 to 69); 44 participants had FSIQ scores that ranged from 55 to 69 and the remaining 15 participants had FSIQ scores below 55. Of those without a diagnosis of ID, 13 were male and 11 were female, with a mean age of 30.8 years at assessment (SD = 11.9; range = 16–55 years). The average FSIQ was 78.4 (SD = 10.9; range = 67 to 115). Two individuals had a FSIQ less than 70, but were considered by the assessing clinician not to have ID because they did not meet the other two diagnostic criteria. There were no significant differences between the two groups in terms of gender or age at assessment.

Measures

Learning disability screening questionnaire

This screening tool was designed to be used by a range of individuals without requiring a particular professional background or training. It comprises seven questions, as previously outlined, that are scored as “yes” or “no,” with each item receiving a score of “0” or “1.” These are then added to provide a total score, which is converted to a total percentage score based on the number of items completed (with a minimum requirement that 5 items are completed). The scoring of the LDSQ as a single dimension is supported by previous studies in which no evidence for multidimensionality has been found (Murray & McKenzie, 2013; Murray et al., 2015). The LDSQ can be completed with the individual suspected of having ID or someone who knows him or her well and takes approximately five minutes to complete. It has been found to have good psychometric properties including high sensitivity and specificity in both community (McKenzie & Paxton 2006; Paxton et al., 2008) and forensic (McKenzie et al., 2012) clinical samples, as outlined in the introduction.

Wechsler Adult Intelligence Scales, Fourth Edition, UK version

The WAIS-IV is considered among the foremost of intellectual assessments in terms of its psychometric properties across a range of demographic groups and ability levels (Wechsler, 2008). It is based on an extensive legacy of test development, although some concerns have been raised in relation to its use with people with ID. These include possible floor effects (Whitaker, 2012), the limited representation of people with ID in the original standardisation sample, and whether the factor structure underpinning the WAIS-IV is valid when applied to this group (see Reynolds et al., 2013). Notwithstanding these concerns, as the WAIS-IV is commonly used to measure intellectual functioning, the present study validated the LDSQ against this assessment. The WAIS-IV provides a FSIQ and four index scores: Verbal Comprehension Index, Perceptual Reasoning Index, Working Memory Index, and Processing Speed Index.

Statistical Procedure

We assessed the convergent validity of the LDSQ with the WAIS-IV by computing the Pearson correlation between WAIS-IV both FSIQ and index scores and LDSQ total percentage scores. With a range of 8 possible scores (the percentage scores corresponding to endorsing between 0 and 7 items), we judged it justifiable to treat the LDSQ as a continuous measure.

We used a ROC curve analysis (Schoonjans, 1998) to estimate the sensitivity and specificity of the LDSQ; that is, the ability to discriminate between those with and without ID. We also used the cut-point for ID defined in previous research and recommended in the LDSQ manual as a comparator to identify the positive and negative predictor values of the LDSQ.

Results

Descriptive Statistics

Convergent validity

The LDSQ total percentage scores were found to correlate significantly with WAIS-IV FSIQ and index scores, as shown in Table 1. In addition, we provide the correlations corrected for range restriction on FSIQ using an assumption of direct range restriction on FSIQ, specifically, Thorndike case 2 (see Sackett and Yang, 2000). Note that this correction assumes that there is no additional attenuation on LDSQ beyond that due to its association with FSIQ. It also assumes that the relation between FSIQ and LDSQ is linear and homoscedastic for all levels. Neither of these assumptions can be verified on the basis of empirical data because they concern unmeasured levels of LDSQ and FSIQ and their relation to each other and to underlying range restricting selection process in the population. These corrected correlations, therefore, represent only an approximate guide to the effect of range restriction in the current sample.

Table 1

Pearson's Correlations Between the LDSQ Percentage Scores and the WAIS-IV Full Scale IQ and Index Scores

Pearson's Correlations Between the LDSQ Percentage Scores and the WAIS-IV Full Scale IQ and Index Scores
Pearson's Correlations Between the LDSQ Percentage Scores and the WAIS-IV Full Scale IQ and Index Scores

Sensitivity and specificity

From the ROC analysis, the area under the curve was .945 (SE = .027), demonstrating that the LDSQ had a significant (p < 0.001) ability to discriminate between individuals with and without ID. Taking the cut-off point obtained from the original validation study of the LDSQ, which was also derived using ROC analysis (Paxton et al., 2008), as a comparative, the sensitivity and specificity values of the LDSQ were 91.5% and 91.7% respectively. This same cut-off point remained optimal in terms of a balance of sensitivity and specificity in the present sample.

In addition, based on the original cut-off score, determined by previous research, overall, 6 out of 83 (7.2%) individuals were incorrectly classified. Of these, 4 out of 59 (6.8%) of individuals with ID were classified by the LDSQ as not having an ID and 2 out of 24 (8.3) of those who did not have ID were classified as having ID.

Discussion

Receiving a diagnosis of ID can have significant implications for an individual in terms of accessing resources and support, particularly in settings where they may be vulnerable, such as criminal justice services (Scheyett, Vaughn, Taylor, & Parish, 2009). On the other hand, it may expose the individual to social stigma and reduced opportunities (see Paterson, McKenzie, & Lindsay, 2011). It is, therefore, crucial that the assessments used as part of the diagnostic process are valid and reliable. Screening tools, such as the LDSQ, are designed to give an indication of whether someone is likely to have ID or not in order to identify where full diagnostic assessment is appropriate. In this capacity, they may be used to identify potentially vulnerable individuals in a range of settings including health (McKenzie & Paxton, 2006), education (see McKenzie & Megson, 2011), and criminal justice services (Sndenaa, Palmstierna, & Iversen, 2010). As such, screening tools need to have strong psychometric properties, as well as high sensitivities and specificities with respect to a diagnosis of ID (Glascoe, 2005).

The present study, therefore, aimed to re-assess the performance of the LDSQ as compared against the gold standard criterion of formal clinical diagnosis of ID, when the criteria of significant impairment in intellectual functioning was assessed using the WAIS-IV. We found that the LDSQ showed good convergent validity with the WAIS-IV, as indicated by significant positive correlations between the LDSQ total percentage scores and the WAIS-IV FSIQ and index scores. A perfect association between WAIS-IV FSIQ and LDSQ scores would not be expected because the LDSQ also aims to measure adaptive functioning aspects of ID. Both adaptive functioning and intellectual functioning impairments are necessary for a diagnosis of ID, but these two aspects are to some degree dissociable (Murray, McKenzie, & Murray, 2013). Unfortunately, a limitation of the present study was that we were unable to assess the specific relationship between LDSQ scores and adaptive functioning as there were insufficient data from any single assessment to conduct this analysis.

The original optimal cut-off score for the LDSQ was used as a comparator to give an indication both of the sensitivity and specificity of the LDSQ based on clinical assessment using the WAIS-IV and the percentage of individuals with and without ID who were correctly and incorrectly classified using the LDSQ. It was found that this cut-off was also optimal for the present sample and yielded very similar sensitivity values to those found in the original validation study (91.2% versus 91.5% ), but higher specificity values (87% versus 91.7%), with only 6 out of the 83 individuals being incorrectly classified overall. This is encouraging and suggests that the LDSQ can continue to be used with some confidence in the context of the updated WAIS-IV being used by ID services to assess intellectual functioning.

The study did, however, have some limitations. Although our sample was representative of the composition of referrals to the participating clinical services, it was modest in size; future studies with larger samples and samples drawn from different populations in which there is elevated risk of ID would be beneficial. Another limitation concerns the gold standard of ID diagnosis itself, the WAIS-IV, which several authors (e.g. Whitaker, 2012) have noted is subject to misclassification errors due to factors such as the limited reliability of intellectual assessments at low levels of ability.

Conclusion

In summary, the LDSQ was found to have good convergent validity and to discriminate well between those with and without ID as assessed against clinical diagnosis of ID, with the WAIS-IV being used to assess intellectual functioning. It also showed good sensitivity and specificity based on the original cut-off scores, and indeed this score was also optimal for the current sample. This suggests that the LDSQ can continue to be used with confidence to help identify those who are likely to have ID and who may benefit from further diagnostic assessment.

Acknowledgments

The authors would like to thank Dr Amanda Michie for her support with this study.

References

References
Beaujean
,
A.
,
&
Sheng
,
Y.
(
2014
).
Assessing the Flynn effect in the Wechsler Scales
.
Journal of Individual Differences
,
35
(
2
),
63
78
.
British Psychological Society
. (
2001
).
Learning disability: Definitions and contexts
.
Leicester, England
:
Author
.
Colom
,
R.
,
Lluis-Font
,
J. M.
,
&
Andrs-Pueyo
,
A.
(
2005
).
The generational intelligence gains are caused by decreasing variance in the lower half of the distribution: Supporting evidence for the nutrition hypothesis
.
Intelligence
,
33
,
83
91
.
Crawford
,
J. R.
,
Allan
,
K. M.
,
&
Jack
,
A. M.
(
1992
).
Short-forms of the UK WAIS-R: regression equations and their predictive validity in a general population sample
.
British Journal of Clinical Psychology
,
31
,
191
202
.
Crawford
,
J. R.
,
Anderson
,
V.
,
Rankin
,
P. M.
,
&
MacDonald
,
J.
(
2010
).
An index-based short-form of the WISC-IV with accompanying analysis of the reliability and abnormality of differences
.
British Journal of Clinical Psychology
,
49
,
235
258
.
Flynn
,
J. R.
(
1984
).
The mean IQ of Americans: Massive gains 1932– 1978
.
Psychological Bulletin
,
95
,
29
51
.
Glascoe
,
F. P.
(
2005
).
Standards for screening test construction
. .
McKenzie
,
K.
,
&
Megson
,
P.
(
2011
).
Screening for intellectual disability in children: A review of the literature
.
Journal of Applied Research in Intellectual Disabilities
,
25
(
1
),
80
87
.
McKenzie
,
K.
,
Michie
,
A.
,
Murray
,
A. L.
,
&
Hales
,
C.
(
2012
).
Screening for offenders with an intellectual disability: the validity of the Learning Disability Screening Questionnaire
.
Research in Developmental Disabilities
,
33
,
791
795
.
McKenzie
,
K.
,
&
Paxton
,
D.
(
2006
).
Promoting access to services: The development of a new screening tool
.
Learning Disability Practice
,
9
(
6
),
17
21
.
Mokken
,
R. J.
(
1971
).
A theory and procedure of scale analysis: With applications in political research (Vol. 1)
.
Berlin, Germany
:
Walter de Gruyter
.
Murray
,
A. L.
,
Booth
,
T.
,
&
McKenzie
,
K.
(
2015
).
An analysis of differential item functioning by gender in the Learning Disability Screening Questionnaire (LDSQ)
.
Research in Developmental Disabilities
,
9
(
39C
),
76
82
.
Murray
,
A. L.
,
&
McKenzie
,
K.
(
2013
).
Estimating the severity of intellectual disability in adults: A Mokken scaling analysis of the Learning Disability Screening Questionnaire
.
Psychological Assessment
,
25
(
3
),
1002
1006
.
Murray
,
A. L.
,
&
McKenzie
,
K.
(
2014
).
The accuracy of the Learning Disability Screening Questionnaire (LDSQ) in classifying severity of impairment: A brief report
.
Journal of Intellectual and Developmental Disability
,
39
(
4
),
370
374
.
Murray
,
A. L.
,
McKenzie
,
K.
,
&
Murray
,
G. C.
(
2013
).
To what extent does g impact on conceptual, practical and social adaptive functioning in clinically referred children?
Journal of Intellectual Disability Research
,
58
(
8
),
777
785
.
Paterson
,
L.
,
McKenzie
,
K.
,
&
Lindsay
,
W. R.
(
2011
).
Stigma, social comparison and self-esteem in adults with an intellectual disability
.
Journal of Applied Research in Intellectual Disabilities
,
25
(
2
),
166
176
.
Paxton
,
D.
,
McKenzie
,
K.
,
&
Murray
,
G. C.
(
2008
).
Putting screening tools to the test
.
Learning Disability Practice
,
11
(
8
),
14
18
.
Reynolds
,
M. R.
,
Ingram
,
P. B.
,
Seeley
,
J. S.
,
&
Newby
,
K. D.
(
2013
).
Investigating the structure and invariance of the Wechsler Adult Intelligence Scales, Fourth Edition in a sample of adults with intellectual disabilities
.
Research in Developmental Disabilities
,
34
(
10
),
3235
3245
.
Sackett
,
P. R.
,
&
Yang
,
H.
(
2000
).
Correction for range restriction: An expanded typology. Journal of Applied Psychology, 85(1), 112
.
Scheyett
,
A.
,
Vaughn
,
J.
,
Taylor
,
M.
,
&
Parish
,
S.
(
2009
).
Are we there yet? Screening processes for intellectual and developmental disabilities in jail settings
.
Intellectual and Developmental Disabilities
,
47
(
1
),
13
23
.
Schoonjans
,
F.
(
1998
).
MedCalc statistics for biomedical research software manual
.
Belgium
:
MedCalc
.
Sndenaa
,
E.
,
Palmstierna
,
T.
,
&
Iversen
,
V. C.
(
2010
).
A step-wise approach to identifying intellectual disabilities in the criminal justice system
.
European Journal of Psychology Applied to Legal Context
,
2
(
2
),
183
198
.
Spinks
,
R.
,
McKirgan
,
L. W.
,
Arndt
,
S.
,
Caspers
,
K.
,
Yucuis
,
R.
,
&
Pfalzgraf
,
C. J.
(
2009
).
IQ estimate smackdown: comparing IQ proxy measures to the WAIS-III
.
Journal of the International Neuropsychological Society
,
15
(
4
),
590
596
.
Talbot
,
J.
(
2008
).
No one knows
.
London, England
:
Prison Reform Trust
.
Trahan
,
L. H.
,
Stuebing
,
K. K.
,
Fletcher
,
J. M.
,
&
Hiscock
,
M.
(
2014
).
The Flynn Effect: A Meta-Analysis
.
Psychological Bulletin
.
Advance online publication. http://dx.doi.org/10.1037/a0037173
Wechsler
,
D.
(
1998
).
Wechsler Adult Intelligence Scale–Third UK Edition: Administrative and scoring manual
.
London, England
:
The Psychology Corporation
.
Wechsler
,
D.
(
2008
).
Wechsler Adult Intelligence Scale–Fourth Edition: Technical and interactive manual
.
San Antonio, TX
:
The Psychology Corporation
.
Wechsler
,
D.
(
2010
).
Wechsler Adult Intelligence Scale–Fourth UK Edition: Administrative and scoring manual
.
London, England
:
The Psychology Corporation
.
Whitaker
,
S.
(
2012
).
The measurement of low IQ with the WAIS-IV: A critical review
.
Clinical Psychology Forum
,
231
,
45
48
.

Author notes

Karen McKenzie, Northumbria University; Phil Sharples, Institute of Mental Health and Wellbeing, University of Glasgow; and Aja L. Murray, Centre for Cognitive Ageing and Cognitive Epidemiology, University of Edinburgh