Abstract

Rett Syndrome (RTT) is characterized by severe impairment in fine motor (FM) and expressive language (EL) function, making accurate evaluations of development difficult with standardized assessm ents. In this study, the administration and scoring of the Mullen Scales of Early Learning (MSEL) were adapted to eliminate the confounding effects of FM and EL impairments in assessing development. Forty-seven girls with RTT were assessed with the Adapted-MSEL (MSEL-A), a subset (n = 30) was also assessed using the Vineland Adaptive Behavior Scales-Second Edition (Vineland-II) and a further subset (n = 17) was assessed using an eye-tracking version of the MSEL (MSEL-ET). Participants performed better on the visual reception (VR) and receptive language (RL) domains compared to the FM and EL domains on the MSEL-A. Individual performance on each domain was independent of other domains. Corresponding MSEL-A and Vineland-II domains were significantly correlated. The MSEL-ET was as accurate as the MSEL-A in assessing VR and RL, yet took a 44% less time. Results suggested that the MSEL-A and the MSEL-ET could be viable measures for accurately assessing developmental domains in children with RTT.

Rett syndrome (RTT; OMIM 312750) is a neurodevelopmental disorder that affects approximately 1 in 10,000 females by the age of 12. Although the diagnosis of RTT is made clinically (Neul et al., 2010), over 95% of individuals with RTT have mutations in methyl-CpG-binding protein (MECP2), which encodes a transcriptional regulator (Amir et al., 1999; Cuddapah et al., 2014). Girls with RTT demonstrate relatively typical development until approximately 6 months of age and then undergo developmental delay and regression (Cuddapah et al., 2014). The latter is characterized by loss of expressive language and purposeful hand use, as well as gait abnormalities and stereotyped hand movements. Impairment in other developmental domains is also a feature of the syndrome, although this has been difficult to assess directly and is often estimated from parent report questionnaires. Accurate developmental assessment adapted specifically for this population is imperative in order to better understand the needs of, develop effective therapies for, and improve the quality of life of individuals with RTT.

Previous studies assessing neurodevelopment in RTT have used traditional assessments such as the Vineland Adaptive Behavioral Scales II (Vineland-II; Sparrow, Cicchetti, & Balla, 2005) and the Bayley scales (Bayley, 1993) to estimate developmental ages ranging between 1 and 18 months for this population (Demeter, 2000). Standard administrations of cognitive and developmental assessments pose challenges for children with RTT because pointing, verbalizing, or holding/manipulating objects is frequently required to complete items across all domains. Therefore, the assessment may not accurately capture the child's cognitive abilities because the accepted responses are based on primary areas of skill loss for this population. To address some of these challenges, Leevers, Roesler, Flax, and Benasich, (2005) developed a neurocognitive assessment to evaluate more objectively the cognitive abilities in children with severe language and motor impairments. However, this assessment was still limited in estimating developmental age ranges beyond 24 months; was too underpowered (n = 8) to draw broad conclusions; and emphasized the use of facial expressions, breathing, skin color, and muscle tone to inform responses, most of which are known to be abnormal in children with RTT.

To our knowledge, previous research has not utilized preserved skills in RTT, such as communicative eye gaze (Baptista, Mercadante, Macedo, & Schwartzman, 2006; Didden et al., 2010; Djukic, Valicenti McDermott, Mavrommatis, & Martins, 2012; Hetzroni & Rubin, 2006) as a means to assess early cognitive and developmental domains. Currently, therapists, educators, and families are using preserved eye-gaze to facilitate communication via eye-tracking technology in RTT, and researchers have focused on attention and social cognition (Djukic, Rose, Jankowski, & Feldman, 2016; Djukic et al., 2012; Rose et al., 2013). The successes in these contexts suggest that eye-trackers could be a promising method for developing more accurate developmental assessments for RTT (Rose et al., 2013; von Tetzchner et al., 1996).

We chose to adapt the Mullen Scales of Early Learning (Mullen, 1989) based on the utility of the assessment to capture a wide age range (birth–68 months) and to evaluate skills across several domains of development easily mapped to skill loss associated with RTT: Visual Reception (VR), Fine Motor (FM), Receptive Language (RL), and Expressive Language (EL). Developmental age is calculated for each domain. The MSEL provides a separate developmental age for each domain, allowing us to track progress in specific areas. Assessment and scoring procedures from the original MSEL were adapted to reduce interdomain confounds due to the child's impairments in an effort to yield more accurate estimates of developmental skills. We introduced adaptations such as allowing more time for responses, enlarging testing items or using motivational objects, and accepting eye gaze as a valid response method, which resulted in two versions with (MSEL-ET) and without (MSEL-A) an eye tracking component. These adaptations were made in accordance with the Standards for Psychological and Educational Testing accommodations (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing, 2014).

The three specific goals of this study were (1) to test the feasibility of adapting a widely used developmental assessment to take advantage of the preserved forms of communication in the RTT population while minimizing the need for expressive language (EL) and fine motor (FM) skills; (2) to investigate the psychometric properties of the adapted measure such as reliability, sensitivity, and validity; and (3) to examine the performance of girls with RTT on the MSEL-A in relationship to the presentation of their disorder.

Method

Boston Children's Hospital's Office of Clinical Investigation and the Institutional Review Board approved this project. Written informed consent was obtained from each participant's guardian prior to testing. Written permission for adapting the copyrighted measure was granted from the Pearson Publishing Company (San Antonio, TX), publishers of the MSEL (Mullen, 1989).

Participants

Forty-seven girls with MECP2 mutations and classic RTT participated in the study, ranging in age from 22 to 131 months. Age cutoffs were determined based on the criteria of ongoing studies from which participants were recruited. Diagnosis was confirmed based upon parent interview, review of prior medical records, and physical assessment. Participants were recruited through the Boston Children's Hospital Rett Syndrome Program and the Rett Syndrome, MECP2 Duplication, and Rett-Related Disorders Natural History Study, which is a Research Consortium led by Dr. Alan Percy at the University of Alabama at Birmingham with 15 sites and co-investigators across the country, including Boston Children's Hospital.

Assessments and Measures

Feasibility: Adaptation of the MSEL (MSEL-A)

Before beginning administration of the MSEL-A, an extensive parent interview was completed to help the examiner understand what forms of communication or response methods the child used most frequently. Any physical limitations such as limited movement of the neck, visual side bias, and so forth, were noted and taken into consideration by the examiner during the assessment.

All modifications were categorized and tracked at the item-level to determine the most effective and commonly utilized adaptations. Modifications included (a) the replacement of standard objects with more motivating objects like favorite toys or snacks; (b) increased number of presses beyond the standard three prompts defined in our adapted protocol; (c) partial credit for multiple equivocal answers instead of full credit for one clear response; (d) equivocal responses in nonverbal subjects; (e) hand bracing during administration to prevent stereotypies and improve concentration; (f) providing motor planning for items involving body or hand movements; (g) involving parents when additional motivation is needed, particularly for items such as following commands; (h) accepting swiping or tapping as responses in place of pointing; (i) accepting large body movements with clear intent in place of precise movements (for example leaning in for a kiss); and (j) accepting eye gaze as a response method. Of note, the decision to award partial or full credit was made based on the team's extensive experience with infant populations and other standardized assessments such as the Autism Observation Schedule for Infants (Bryson & Zwaigenbaum, 2014) .

Girls with RTT often have delayed processing speed and test response (Belmonte et al., 2004). Therefore, we were flexible in the amount of time we allowed before requiring a response, sometimes waiting up to 1 min. Timing for a particular participant was based on the parent interview as well as the examiner's judgment of attention and processing speed throughout the assessment. Though there is not a strict time restriction for administering any items in the standard MSEL, waiting for longer than 20 s would be unusual when assessing typically developing children; therefore, time is noted as a specific administration guideline for the adapted version.

To standardize and track the number of opportunities each child was given, each item or question administered by the examiner was defined as one press. All items were delivered or pressed three times to ensure validity in response evaluation. Additional presses could be administered after an equivocal response in order for the examiner to confirm that the child made a clear choice, or to confirm that the gaze shift was a response rather than a nondeliberate shift of attention. The number of increased presses were tracked as a modification and considered in our adaptation analysis. In the standard MSEL there no restrictions on the number of times you can ask a subject an item, however asking numerous times would be considered atypical with typically developing children and therefore is specifically noted as an adaptation in our version.

Specific items in the VR and RL domains were adapted to lessen the confounding effects of any deficits in FM or EL skills. For example, in the receptive language domain, item 13, Understands Gesture and Command, calls for the child to give the examiner a block as the examiner holds out their hand and says “give the block to me.” Rather than requiring the child to pick up and hand the examiner a block, which necessitates a certain level of FM ability, another command is used. Example commands may include “give me a hug” (with arms out for a hug) or “come here” (with hand gesturing to come). These presses required gross motor movements in place of FM movements, which are possible for girls with RTT, but do not distort the RL component of the item. More broadly, any item that would require pointing or verbal confirmation to be awarded full points could be substituted for a response using eye-gaze, or taping/swiping or touching their nose to the correct answer. For more specific examples of how FM and EL confounds were reduced a full protocol is available upon request.

Overall, modifications allowed acceptable responses to be more flexible and utilize each child's preserved and individualized responses/communication styles to assess their full range of VR and RL abilities (i.e., eye-gaze rather than verbal responses). In addition, some testing materials in the MSEL kit were modified for ease of assessment. For example, an enlarged version of the stimulus book was created for the examiner to confidently judge responses using eye gaze responses due to the increased spacing of possible answers. Other items were also enlarged or replaced to increase clarity for the examiner and to reduce choking hazards, as girls with RTT may mouth objects. For example, paper swatches replaced crayons for the color identification item in the RL domain. Care was taken to ensure that none of the adaptations obfuscated the original intent of the item.

Standard methods of establishing a baseline and ceiling were used; however, partial credit was given at times if answers were consistently correct but not completely clear (equivocal) due to their response method or prolonged response time. This procedure may have artificially reduced a subjects' overall score; however, effort was taken to be conservative in estimating developmental levels when answers were not clear. Two trained examiners scored a subset of the assessments in order to ensure interrater reliability and validity of scoring. A complete MSEL-A manual and protocol are available upon request.

Feasibility: Adaptation of the Eye-Tracking (MSEL-ET)

Recent studies showing the promise of eye-tracking technology for cognitive assessments in RTT (Rose et al., 2013; von Tetzchner et al., 1996) coupled with our lab's expertise in eye-tracking technology lead us to develop an eye-tracking version of the MSEL. Not all items were suited for adaptation to eye-tracking, such as the FM and EL domains, or early parts of the RL domain. For example, item 6 where the examiner moves to the side of the subject and says, “hi baby” to see if their attention shifts to the examiner. Instead, we chose to adapt the RL and VR scale, when possible, onto the eye-tracker using PowerPoint slides displaying pictures of the standardized MSEL toys with animations on a Tobii Eye-Tracker monitor. For a subset of the sample (n = 17), we administered all adaptable items of the VR and RL scales (protocol available upon request) in accordance with the same guidelines of the MSEL-A. The eye-tracking system was individually calibrated to each participant based on a 5-point gaze determination repeated twice at the beginning of the assessment. Items were scored using visual inspection of live gaze trials during the animations of each item based on regions of interest and/or patterns of eye movements. For example, in order to receive credit for the VR puzzle item, the participant needed to first fixate on the shape and then look to the correct empty space on the puzzle. The stimuli presentation and resulting eye-gaze tracks were saved as video files for posttest scoring as needed.

The overall length of the MSEL-A (n = 47) averaged 121.04 minutes (s = 51.46) in one session. Consequently, one of the reasons for completing testing on an eye-tracker was to reduce administration time. Administration on the eye-tracker does this in two ways: It decreases time spent manipulating, putting away and finding objects, and it increases clarity of eye-gaze responses thereby speeding up scoring. Standard administration of the MSEL and MSEL-A results in differences in the number of administered items between subjects due to varied baseline starting points and ceilings, which limited our ability to directly compare total time of administration between the assessments for our sample. Furthermore, only a subset of the items on the MSEL-A could be administered on the eye-tracker.

Scoring

For both the MSEL-A and MSEL-ET assessments, certain items were unable to be adapted without distorting the intent of the original item due to the requirement of motor or language skills. These items were not administered and were not considered when establishing a baseline or a ceiling for scoring. An example of such an item is #10 on the VR scale: “turns cup right-side up.” In keeping with the published scoring procedure for the standard MSEL, such items were scored as full credit if they fell below or within the baseline and no credit if they fell above the baseline.

Many participants fell outside the chronological age range of the original MSEL precluding the use of the validated t scores; instead, raw scores were used to describe girls' developmental age across domains of the MSEL. Developmental quotients were calculated to compare results between different children and between the MSEL-A and Vineland-II[(age equivalent/chronological age)*100].

Psychometric properties: Interrater reliability

In order to assess interrater reliability, a subset of 15 MSEL-A assessments (32%) was independently scored by two trained examiners. Sensitivity of the measure was examined by comparing the distribution of scores within each domain. Preliminary convergent and discriminant validation of the adapted MSEL was performed using the Vineland-II, a standardized assessment of adaptive skills that would serve as a comparison measure for developmental domains such as EL, RL, and FM and a discriminant subdomain of Coping skills.

Psychometric properties: Convergent and discriminant validity

Vineland Adaptive Behavior Scales-Second Edition (Vineland-II; Sparrow, 2011) scores, which were used as a developmental comparison of the MSEL-A assessment, were completed within 2 months of the MSEL-A in order to eliminate possible developmental effects. The Vineland-II is a standardized parent interview questionnaire for individuals ranging from 3 years to 90 years of age. It evaluates adaptive behavior skills across four domains of development, with each domain containing two to three subdomains. The four domains are(1) Communication Skills, comprised of Expressive, Receptive and Written Language; (2) Daily Living Skills, comprised of Personal, Domestic, Community; (3) Social Skills, comprised of Interpersonal Relationships, Play and Leisure, Coping Skills; and (4) Motor Skills, comprised of Gross and Fine Motor. Parents are asked to respond to questions considering their child's behavior in the past 2 weeks in order to measure current functioning levels.

RTT profile: Phenotypic characterization

To investigate the contribution of various genotypic and phenotypic factors, the individuals' disorder stage, disease severity, and seizure status were determined at the time of their MSEL assessment. Participants' disorder stage was based on parent report of the proximity of gross motor, fine motor, receptive language, expressive language, and adaptive skill loss compared to the date of the MSEL-A assessment. Individuals who had lost any skills in these categories within the 12 months prior to MSEL-A were considered to be in active regression, and those whose last skill loss occurred before this time frame were categorized as postregression. We chose 12 months as a conservative limit to enable sufficiently powered comparisons between the two groups while ensuring that those determined as postregression were in a stable state of the disease. Overall disease severity was measured using the Clinical Severity Scale (Neul et al., 2008), a clinical rating metric developed specifically for RTT. The Clinical Severity Scale total score is a composite of 13 individual categories measuring common features of RTT. Each category is scored with either 4- or 5-point scale, with more severe symptoms coded as higher scores. Individuals were classified as having a history of seizures if at any point in their medical history they were diagnosed with a seizure disorder regardless of symptom control. Conversely, if an individual had never experienced a seizure disorder at any time in their life they were classified as having no history of seizures.

Genotypic characterization

MECP2 mutation status was also determined at the time of assessment. Consistent with previous classifications in the literature (Cuddapah et al., 2014), participants were sorted into two groups based on the severity profile of their mutations. Cuddapah and colleagues (2014) assessed mutation severity across 4,940 unique assessments of 1,052 participants by comparing mutation status to measures of growth, motor coordination, communicative abilities, respiratory function, autonomic symptoms, scoliosis, and seizures over time as a means to bisect into severe and mild (clinical severity) mutation groups.

Statistical analysis

Statistical analysis was performed using SPSS software version 21. The significance threshold was set to p = 0.05. Partial correlations were used while controlling for age in order to compare between MSEL-A domains and to look at the effects of phenotypic differences on the MSEL-A scores. Paired t tests were used to compare the performance between domains or assessments within individuals. Bivariate correlations were used to look at correlations across all domains on a group level. Two-way random intraclass correlations were conducted to measure absolute agreement between examiners and scores.

Results

Feasibility

All girls enrolled in the study (n = 47) completed the MSEL-A. Thirty of the girls who completed the MSEL-A also had Vineland-II data. Seventeen of the girls who completed the MSEL-A also attempted the MSEL-ET; of these, 14 girls were able to complete the VR MSEL-ET and 10 girls completed the RL domain. Missing data resulted from either (a) children whose abilities tested below the starting point for the RL MSEL-ET (the MSEL-ET and MSEL-A were used in combination to generate scores), or (b) children who fell asleep during the assessment.

Reliability

All examiners had a basic knowledge of the MSEL prior to MSEL-A training. Additional training on the MSEL-A was done via one 3-hour training session, a number of direct observations and co-scoring of the MSEL-A with RTT participants, and administration of the MSEL-A on RTT participants with a trainer present to co-score until consistency in the correct administration was evident. A high degree of reliability was found between examiners on all MSEL-A domain scores. The average measure of Inter-class correlation in the VR domain was 0.999 F(15,2) = 1801.98, p < 0.001, the RL domain was 1.000 F(14,2) = 2070.37.976, p < 0.001, the FM domain was 0.999 F(15,2) = 697.87, p < 0.001, and the EL domain was 0.998 F(15,2) = 423.57, p < 0.001.

Sensitivity: Distribution

As expected, scores were very low for the FM and EL domains with very little variability across the group. More individual variability was seen for the VR and RL domains on the MSEL-A. Of note, on the MSEL-A there were some girls who scored above average (> 100 Developmental Quotient) in the VR and RL domains, highlighting their preserved or acquired adaptive functioning in these areas. The Vineland-II showed global deficits, including the RL component of the Communication domain, in contrast to the RL measured by the MSEL-A. Detailed results of the distribution of participant scores can be found in Table 1 and Figure 1.

Table 1

Descriptive Statistics on MSEL-A, MSEL-ET, and Vineland-II Scores

Descriptive Statistics on MSEL-A, MSEL-ET, and Vineland-II Scores
Descriptive Statistics on MSEL-A, MSEL-ET, and Vineland-II Scores
Figure 1

Developmental quotients across overlapping domains for the MSEL-A and the Vineland-II. MSEL-A = Mullen Scales of Early Learning–Adapted and VABS = Vineland Adaptive Behavior Scales–Second Edition.

Figure 1

Developmental quotients across overlapping domains for the MSEL-A and the Vineland-II. MSEL-A = Mullen Scales of Early Learning–Adapted and VABS = Vineland Adaptive Behavior Scales–Second Edition.

Sensitivity: MSEL-A Interdomain Correlations

Group-level analysis

Partial correlations were used to investigate interdomain relations at a group level while controlling for age. FM raw scores positively correlated with EL raw scores (r = 0.316; p = 0.035; medium effect size); however, neither of these domains correlated with VR or RL raw scores. In contrast, VR raw scores strongly and positively correlated with RL raw scores (r = 0.730; p < 0.001; large effect size). Therefore, there were two sets of correlated skills: FM-EL and VR-RL skills.

Individual-level analysis

VR and RL skills were found to be significantly higher than both the FM, and EL (all d's > 1.04; large effect sizes) within the same participant. VR and RL raw scores were not significantly different from each other (d = 0.180; no effect). Additionally, FM raw scores were significantly higher than EL scores (d = 1.04, p < 0.001; large effect size). These findings indicated that the girls who preformed well on the VR domain were the same girls who also preformed well on the RL domain, regardless of their performance on the FM or EL domains. Girls that preformed well on FM also had better EL abilities, independent of their VR and RL skills, though the EL skills were most impaired. Detailed comparison results can be found in Table 2.

Table 2

Paired T Tests Comparisons of MSEL-A Inter-Domains and MSEL-A VINELAND-II Shared Domains Using Developmental Quotients

Paired T Tests Comparisons of MSEL-A Inter-Domains and MSEL-A VINELAND-II Shared Domains Using Developmental Quotients
Paired T Tests Comparisons of MSEL-A Inter-Domains and MSEL-A VINELAND-II Shared Domains Using Developmental Quotients

Validity: MSEL-A Correlations With Vineland-II

Domain correlations

Corresponding domain correlations between the MSEL-A and Vineland-II were computed first to examine the convergent validity of the MSEL-A for evaluating specific developmental domains assessed by both measures (i.e., FM, RL, and EL), using the Vineland-II as the gold standard. The MSEL-A FM domain significantly correlated with the Vineland-II FM domain as well as with the Vineland-II RL and Vineland-II EL domains (r's > 0.510; large effect sizes). The MSEL-A RL domain significantly correlated with the Vineland-II RL domain as well as with the Vineland-II FM and Vineland-II EL domains (r's > 0.363; small- large effect sizes). The MSEL-A EL domain significantly correlated only with the Vineland-II EL domain (r = 0.440; medium effect). In summary, individuals who scored better on a certain domain of the MSEL-A also scored better on the corresponding domain of the Vineland-II, demonstrating correspondence between the measures. Detailed correlation results can be found in Table 3.

Table 3

MSEL-A and Vineland-II Correlation Matrix Using Developmental Quotients

MSEL-A and Vineland-II Correlation Matrix Using Developmental Quotients
MSEL-A and Vineland-II Correlation Matrix Using Developmental Quotients

Individual-level analysis

We used paired t tests to determine how girls performed on the Vineland-II with respect to the corresponding MSEL-A domains. For the FM and RL domains, girls performed significantly better on the MSEL-A measure (d's = 1.37; large effect sizes) than on the Vineland-II. There was no significant difference in the EL domain between the MSEL-A and Vineland-II. EL skills are assessed at a comparable level by both measures.

Discriminant validity

Discriminate validity was explored using the coping skills domain within the Vineland-II. The Vineland-II domain did not correlate the MSEL-A VR, RL, or EL domains (r's < 0.304; no small effect sizes), but did moderately correlate with the FM domain (r = 0.532; medium effect). Detailed correlation results can be found in Table 3.

MSEL-A Compared to MSEL-ET

Administration time

Because the most utilized modification in the MSEL-A was eye gaze for a response, we tested the use of eye-tracking technology into the MSEL-A where possible (i.e., MSEL-ET). We compared the time of administration for the most commonly (maximum number of subjects) administered item across both the MSEL-A and MSEL-ET in order to maximize our power for analysis. Using a paired sample t test, we found that the matching items task, the most widely administered item across subjects, took significantly less time to administer on the MSEL-ET than on the MSEL-A (d = 3.61, p = 0.001; large effect size) within a 95% confidence interval of 1.49–5.72 (Figure 2).

Figure 2

Comparison of administration time for the Matching Objects task between the Adapted and Eye-tracking MSEL. ***p < .001.

Figure 2

Comparison of administration time for the Matching Objects task between the Adapted and Eye-tracking MSEL. ***p < .001.

Individual-level analysis

We tested the agreement (i.e., reliability) between MSEL-ET and MSEL-A scores in a subset of girls using paired t tests. No significant differences between raw scores were found for the VR or RL domains, d's < 0.17, p's > 0.711, 95% CI [−0.92–1.26]; no effects; Figure 3A–B. Additionally, there were no significant differences in terms of the highest item scored correctly between the MSEL-A and the MSEL-ET: VR and RL domains, (d's < 0.04, p's > 0.111; 95% CI [−1.05–1.94]; no effects; Figure 3B–C. This suggests that the use of an eye-tracker reduced the administration time and ease of scoring but did not alter the nature of the testing.

Figure 3

Raw scores and highest items scored correctly in Visual Reception and Receptive Language on the Adapted vs. Eye-tracking MSEL. MSEL-A = Mullen Scales of Early Learning–Adapted; MSEL-ET = Mullen Scales of Early Learning–Eye-Tracking Version.

Figure 3

Raw scores and highest items scored correctly in Visual Reception and Receptive Language on the Adapted vs. Eye-tracking MSEL. MSEL-A = Mullen Scales of Early Learning–Adapted; MSEL-ET = Mullen Scales of Early Learning–Eye-Tracking Version.

MSEL-A, Phenotype, and Genotype

To investigate the relation between performance within each developmental domain and the RTT phenotype, partial correlations between MSEL-A scores and Clinical Severity Scale were computed. On the Clinical Severity Scale, scores ranged from 12 to 31, with a mean score of 20. Clinical Severity Scale scores negatively correlated with FM raw scores (r = −0.336; small effect), meaning that increased clinical severity was associated with worse FM skills. One-way ANOVA were used to compare RTT disease stage, mutation severity, and seizure status with MSEL-A developmental quotients, which controls for age. Girls who were in stage 2 of RTT performed worse on all domains than girls in stage 1 of RTT (d's > 0.731, p's < 0.017; large effect sizes). In terms of mutation severity, girls with milder mutations had significantly higher developmental quotient scores on the VR and RL domains (d's > 0.892, p's < 0.003; large effect sizes) than girls with severe mutations, but no differences between groups were observed on the FM or EL domains (d's > 0.009, p's > 0.682; no effects). Girls with a history of seizure disorders performed worse globally compared to girls with no history of seizures (d's > 0.623, p's < 0.040; medium-large effect sizes).

Discussion

Summary and Interpretation

The primary goal of this study was to pilot adaptations of the Mullen Scales of Early Learning (MSEL) for examining developmental functioning in girls with RTT. We were able to complete assessments on all of the girls who enrolled in the study, and data suggested that, in addition to feasibility, there were promising levels of reliability, sensitivity, and convergent and discriminant validity with the Vineland-II for the pilot project. Moreover, there was good reliability between the two MSEL adaptations, the MSEL-A and MSEL-ET. Following is a summary and interpretation of the main findings.

Sensitivity: Developmental domain independence

We sought to determine if girls with RTT syndrome suffer from global development impairment, or if they have some preserved/acquired skills, as suggested by parent reports and numerous studies (e.g., see Fabio, Castelli, Marchetti, & Antonietti, 2013; Hetzroni & Rubin, 2006; Urbanowicz, Leonard, Girdler, Ciccone, & Downs, 2014). Based on our investigation of developmental domain relationships, we found that the VR and RL domains were highly correlated with each other, but they did not correlate with the FM and EL domains that were also highly correlated to each other. This shows that even in the presence of significant developmental impairment in FM and EL skills, there could be preservation of VR and RL skills. Further analysis showed that girls with RTT had significantly better VR and RL skills compared to FM and EL skills, and that FM skills were better than EL skills. These findings support previous work suggesting preserved or acquired developmental functioning in specific domains within girls with RTT, but that the most devastating impairment is in verbal communication area (Neul et al., 2010). Given that VR and RL domains appear to be less impacted by the disease, the development of therapies that take advantage of these preserved skills offer rich opportunities for improving functional skills and overall quality of life in girls with RTT. Overall, our data emphasize the importance of evaluating each domain separately in order to adequately estimate developmental abilities in RTT.

Validity: MSEL-A and Vineland-II overlapping domains

When we compared the MSEL-A with the Vineland-II, the developmental quotients on the Vineland-II were much lower and less variable across all overlapping domains. The increase in RL measures was as expected given the reduction of FM demand within this domain that may have been influencing the Vineland-II's assessment of RL. Surprisingly, the FM was also higher on the MSEL-A despite no adaptations within this domain. This could be due to differences in the psychometric properties of the two measures wherein the MSEL is standardized for a younger cohort and, therefore, might be more sensitive in measuring early FM skills compared to the Vineland-II. These data suggested that the MSEL-A might be more appropriate for assessing early development in RTT than the Vineland-II. Despite its apparent lower sensitivity, the Vineland-II is a well-established measure of adaptive skills that has been used by others and us in RTT (Kaufmann et al., 2012; Vignoli et al., 2010). Thus, it was adequate for determining convergent validity of our adapted MSEL. Indeed, we identified strong but broad correlations between the MSEL-A and Vineland-II overlapping domains. Nevertheless, the correlations also suggested that the domains of the Vineland-II are highly interdependent and that, for this reason, Vineland-II assessments may underestimate not only global but also specific developmental function in children with RTT. In terms of discriminant validity, the Vineland-II coping skills subdomain did not show any correlations to the VR, RL, or EL domain; however, it was correlated to FM skills, a MSEL domain that was not modified in our adapted version. In this line, our findings supported previous suggestions that available assessments are designed to rely on skills from other domains and when measured in populations with specific domain impairment, such as RTT, they confound the results of the assessment (Leevers et al., 2005; Rose et al., 2013). Our data underscored the need for measures that can better tease apart skills in each domain and determine the developmental potentials in populations with major impairment in FM and EL skills. The MSEL-A could begin to address this need based on the increased sensitivity and relative domain independence demonstrated in our study, though further validation is necessary.

MSEL-ET

The average length of the MSEL-A was approximately 2 hours in one session, which is long and fatiguing for girls with RTT, and time is often a limiting factor for clinical and research evaluations. In an effort to reduce this time, we adapted the VR domain and portions of the RL domain to an eye-tracker as many girls with RTT utilize this form of augmentative communication at home or at school and some studies have shown that this may be a promising avenue for assessment in girls with RTT (Baptista et al., 2006; Djukic et al., 2012; Rose et al., 2013). When comparing the MSEL-A and MSEL-ET versions, we found no differences in raw scores or the highest achieved item, indicating that the accuracy of the MSEL-A was not sacrificed by changing assessment modality. We did, however, find a significant reduction in administration time for representative items when using the MSEL-ET. As previously mentioned, we were not able to include all MSEL-A items in the MSEL-ET for all participants, as many of the items in the RL domain were not adaptable to the eye-trackers. The high baseline starting point of the RL MSEL-ET version caused a reduction in the number of participants that were able to complete the MSEL-ET RL domain. However, all subjects were able to complete some items of the RL domain with the MSEL-ET. Supplementing these items would still provide an advantage in reducing administration time during the MSEL-A assessments for these children. We conclude that eye-tracking technology could be incorporated into the MSEL-A, when possible, as a supplement to reduce administration time and improve accuracy of response evaluation for items where girls often use eye gaze to respond. Other potential uses of eye tracking for developmental assessments in children with RTT deserve further exploration but were beyond the scope of the present study.

Development, phenotype, and genotype

When examining how the performance on the MSEL-A related to phenotypic and genotypic features of our sample, we found that only the FM domain related to clinical severity. The VR and RL domains positively related to mutation severity: The more severe the mutation, the worse the child's VR and RL abilities. This relation may reflect some preservation of skills in these domains in milder forms of the disorder. As expected, a history of seizures and a more advanced stage of RTT were both related to globally poorer performance on the MSEL-A. Whether performance declines with stage or learning is protracted as the disorder progresses in girls with RTT remains unclear, however these factors have broad implications in terms of therapeutic potential and learning strategies for individuals with RTT.

Summary

Based on the results of this study, we conclude that the piloted MSEL-A offered a sensitive and initially valid approach to assessing development in girls with RTT as compared to the traditional methods, by utilizing preserved communications skills and eliminating confounds between domains. Additionally, the MSEL-ET proved to be a faster method for measuring the VR and RL domains in RTT without sacrificing accuracy. Altogether, our data suggested that the MSEL-A and MSEL-ET are promising instruments for determining developmental abilities in RTT. Accurate assessment is paramount in understanding the needs of individuals with RTT, which will ultimately result in the development of targeted therapies and interventions to improve their quality of life.

Future directions

Future researchers should investigate the potential of using the Standard MSEL, the MSEL-A and the MSEL-ET to not only measure current cognitive functioning, but also changes in cognitive functioning over time. In order to further investigate the psychometric properties of the MSEL-A and MSEL-ET, future studies should be done to measure the test-retest reliability, the sensitivity to interventions and/or interindividual variability in development. As fine motor and expressive language impairment is not exclusive to RTT, studies investigating the applicability of the MSEL-A to other populations with similar deficits would also be an important line of work to pursue, and would help with further validation and normalization of the MSEL. Investigating other developmental assessments using similar adaptations would also be helpful to compare and optimize adapted assessments for girls with RTT.

Limitations

Not all participants completed all portions of the study. This limitation in study design did not allow for correlations between the MSEL-A, MSEL-ET, and Vineland-II within the same participant. Additionally, this study does not have a comparison group to normalize the adapted versions of the MSEL. Finding an adequate comparison group is difficult because recruiting typically developing children, who are chronologically age-matched, would be developmentally inappropriate since they would fall outside the chronological age range of the MSEL. Moreover, a cognitively age-matched typically developing sample would be challenging to compare because our RTT adaptations may be unnatural or distressing to typically developing girls (e.g., using eye gaze to respond, arm bracing, or increasing wait times after presses). Therefore, we could anticipate a difference in standard scores within a typical population on the MSEL-A or MSEL-ET compared to a standard MSEL. Future studies examining populations with similar motor impairments to girls with RTT would help to more broadly assess the validity of the measure. Additionally, we were limited in options for measures of convergent validity due to the lack of direct assessment versus parent report (as has been historically examined by previous researchers interested in RTT).

References

References
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education; & Joint Committee on Standards for Educational and Psychological Testing
. (
2014
).
Standards for educational and psychological testing
.
Washington, DC
:
Authors
.
Amir,
R. E.,
Van den Veyver,
I. B.,
Wan,
M.,
Tran,
C. Q.,
Francke,
U.,
&
Zoghbi,
H. Y.
(
1999
).
Rett syndrome is caused by mutations in X-linked MECP2, encoding methyl-CpG-binding protein 2
.
Nature Genetics
,
23
(
2
),
185
188
.
Baptista,
P. M.,
Mercadante,
M. T.,
Macedo,
E. C.,
&
Schwartzman,
J. S.
(
2006
).
Cognitive performance in Rett syndrome girls: A pilot study using eyetracking technology
.
Journal of Intellectual Disability Research
,
50
(
9
),
662
666
.
Bayley,
N.
(
1993
).
Manual for the Bayley Scales of Infant Development II
.
San Antonio, TX
:
The Psychological Corporation
.
Belmonte,
M. K.,
Cook,
E. H.,
Anderson,
G. M.,
Rubenstein,
J. L. R.,
Greenough,
W. T.,
Beckel-Mitchener
, …
Tierney,
E.
(
2004
).
Autism as a disorder of neural information processing: Directions for research and targets for therapy
.
Molecular Psychiatry
,
9
(
7
),
646
663
.
Bryson,
S. E.,
&
Zwaigenbaum,
L.
(
2014
).
Autism observation scale for infants
.
In
V. B.
Patel,
V. R.
Preedy,
&
C. R.
Martin
(
Eds.
),
Comprehensive guide to autism
(pp
.
299
310
).
New York, NY
:
Springer
.
Cuddapah,
V. A.,
Pillai,
R. B.,
Shekar,
K. V.,
Lane,
J. B.,
Kathleen,
J.,
Skinner,
S. A.,
Olsen,
M. L.
(
2014
).
Associated with disease severity in Rett syndrome
.
Journal of Medical Genetics
,
51
(
3
),
152
158
.
Demeter,
K.
(
2000
).
Assessing the developmental level in Rett syndrome: An alternative approach?
European Child & Adolescent Psychiatry
,
9
(
3
),
227
233
.
Didden,
R.,
Korzilius,
H.,
Smeets,
E.,
Green,
V.,
Lang,
R.,
Lancioni,
G. E.,
&
Curfs,
L. M.
(
2010
).
Communication in individuals with Rett syndrome: An assessment of forms and functions
.
Journal of Developmental and Physical Disabilities
,
22
(
2
),
105
118
.
Djukic,
A.,
Rose,
S. A.,
Jankowski,
J. J.,
&
Feldman,
J. F.
(
2016
).
Rett syndrome: Recognition of facial expression and its relation to scanning patterns
.
Pediatric Neurology
,
51
(
5
),
650
656
.
Djukic,
A.,
Valicenti McDermott,
M.,
Mavrommatis,
K.,
&
Martins,
C. L.
(
2012
).
Rett syndrome: Basic features of visual processing—A pilot study of eye-tracking
.
Pediatric Neurology
,
47
(
1
),
25
29
.
Fabio,
R. A.,
Castelli,
I.,
Marchetti,
A.,
&
Antonietti,
A.
(
2013
).
Training communication abilities in Rett syndrome through reading and writing
.
Frontiers in Psychology
,
4
(
December
),
1
9
.
Hetzroni,
O. E.,
&
Rubin,
C.
(
2006
).
Identifying patterns of communicative behaviors in girls with Rett syndrome
.
Augmentative and Alternative Communication
,
22
(
1
),
48
61
.
Kaufmann,
W. E.,
Tierney,
E.,
Rohde,
C.,
Suarez-Pedraza,
M. C.,
Clarke,
M.,
Salorio,
C. F.,
Naidu,
S.
(
2012
).
Social impairments in Rett syndrome: Characteristics and relationship with clinical severity
.
Journal of Intellectual Disability Research
,
56
(
3
),
233
247
.
Leevers,
H. J.,
Roesler,
C. P.,
Flax,
J.,
&
Benasich,
A.
(
2005
).
The Carter neurocognitive assessment for children with severely compromised expressive language and motor skills
.
Journal of Child Psychology and Psychiatry and Allied Disciplines
,
46
(
3
),
287
303
.
Lord,
C.,
Rutter,
M.,
DiLavore,
P. C.,
Risi,
S.,
Gotham,
K.,
Bishop,
S.,
&
others
. (
2012
).
Autism diagnostic observation schedule: ADOS-2
.
Western Psychological Services
Los Angeles, CA
.
Mullen,
E. M.
(
1989
).
Infant MSEL manual: Infant Mullen Scales of Early Learning
.
Circle Pines, MN
:
American Guidance Service
.
Neul,
J. L.,
Fang,
P.,
Barrish,
J.,
Lane,
J.,
Caeg,
E. B.,
Smith,
E. O.,
Glaze,
D. G.
(
2008
).
Specific mutations in Methyl-CpG-Binding Protein 2 confer different severity in Rett syndrome
.
Neurology
,
70
(
16
),
1313
1321
.
Neul,
J. L.,
Kaufmann,
W. E.,
Glaze,
D. G.,
Clarke,
A. J.,
Leonard,
H.,
Bailey,
M. E. S.,
Renieri,
A.
(
2010
).
Rett syndrome: Revised diagnostic criteria and nomenlcature
.
Annual Neurology
,
68
(
6
),
944
950
.
Rose,
S.,
Djukic,
A.,
Jankowski,
J. J.,
Feldman,
J. F.,
Fishman,
I.,
&
Valicenti-Mcdermott,
M.
(
2013
).
Rett syndrome: An eye-tracking study of attention and recognition memory
.
Developmental Medicine and Child Neurology
,
55
(
4
),
364
371
.
Sparrow,
S. S.,
Cicchetti,
D. V,
&
Balla,
D. A.
(
2005
).
Vineland adaptive behavior scales: (Vineland II)
,
survey interview form/caregiver rating form
.
Livonia, MN
:
Pearson Assessments
.
Urbanowicz,
A.,
Leonard,
H.,
Girdler,
S.,
Ciccone,
N.,
&
Downs,
J.
(
2014
).
Parental perspectives on the communication abilities of their daughters with Rett syndrome
.
Developmental Neurorehabilitation
,
19
(
1
),
17
25
.
Vignoli,
A.,
Fabio,
R. A.,
La Briola,
F.,
Giannatiempo,
S.,
Antonietti,
A.,
Maggiolini,
S.,
&
Canevini,
M. P.
(
2010
).
Correlations between neurophysiological, behavioral, and cognitive function in Rett syndrome
.
Epilepsy and Behavior
,
17
(
4
),
489
496
.
von Tetzchner,
S.,
Jacobsen,
K. H.,
Smith,
L.,
Skjeldal,
O. H.,
Heiberg
, &
Fagan,
J. F.
(
1996
).
Vision, cognition and developmental characteristics of girls and women with Rett syndrome
.
Developmental Medicine and Child Neurology
,
38
(
3
),
212
225
.

Author notes

The authors would like to acknowledge and thank Frances Cooley, B.S., for her involvement in the MSEL-A data collection, Natalie Bruck, B.A., for her involvement in the Vineland-II data collection and medical record review for staging participants.