The purpose of this study was to examine self-determination outcome data in the year following a one-year cluster randomized controlled trial (C-RCT) comparing the impacts of a Self-Determined Learning Model of Instruction (SDLMI) only condition to a SDLMI + Whose Future Is It? (SDLMI + WF) condition. Using multilevel B-spline model analysis with Bayesian estimation, we examined ongoing patterns of growth after the trial ended and all students were exposed to SDLMI + WF. The findings suggest that the inclusion of an additional year of outcome data provided additional insight into the impact of more intensive intervention conditions over time. Specifically, after the initial year of implementation, the SDLMI + WF condition predicted greater annual gains than the SDLMI only condition, unlike findings in the first year which reflected the opposite pattern. This evidence suggests a nonlinear growth pattern over multiple years of intervention with more intensive interventions. Implications for future research and practice are discussed.
Researchers have consistently documented the positive impact of interventions to enhance self-determination on the in-school and postschool outcomes of students with disabilities (Burke et al., in press; Hagiwara et al. 2017). However, an array of contextual factors (i.e., personal and environmental factors; Shogren et al., 2014) have been found to influence the outcomes of self-determination interventions. For example, a variety of student-level factors have been shown to impact response to self-determination interventions, including age and disability label (Shogren et al., 2012; Wehmeyer et al., 2012). Further, teacher, school, and district-level factors are also critical to understand; for example, the duration of intervention implementation in a classroom or school (e.g., one semester vs. multiple years) as well as the supports available for implementation (e.g., technology, teacher training, school or district-level supports for coaching) have been shown to impact outcomes (Mazzotti et al., 2012; Shogren et al., 2014; Wehmeyer et al., 2013). Further research is needed to better understand the potential impact of such malleable factors on outcomes of self-determination interventions.
Two particularly salient issues that are deserving of more attention include: (a) the impact of longer term implementation of interventions on the development of self-determination, given research suggesting a nonlinear growth pattern particularly as intervention extends over multiple years (Wehmeyer et al., 2013); as well as (b) the impact of more intensive interventions (e.g., more time devoted to instruction, more scripted curricular practices; Shogren et al., 2016); and (c) the interaction of time and intensity on outcomes. There is currently a lack of clarity in the field on how to make decisions about the duration and intensity of self-determination interventions based on student needs (Shogren et al., 2016), particularly for students with more significant support needs (Shogren, Wehmeyer, Lane, & Quirk, 2017). For example, differing intervention approaches have been developed ranging from models of instruction that can be overlaid on ongoing curriculum and instruction requiring less instructional time but more teacher planning and problem solving (e.g., the Self-Determined Learning Model of Instruction [SDLMI]; Shogren, Raley, et al., 2018) to more intensive curricula that are scripted and require dedicated instructional time, typically in the context of transition planning (e.g., Whose Future Is It? [WF]; Wehmeyer & Palmer, 2011). There has been limited research on the impact of combining interventions, such as the SDLMI and WF, to deliver more intensive interventions. Further, differential growth in self-determination over time as a function of exposure to multiple interventions has never been examined. Addressing these issues is critical to providing direction not only for future research efforts and investments, but also to inform practice strategies particularly when contextualizing self-determination interventions in a tiered approach to service delivery in transition (Shogren et al., 2016).
The present study is part of an ongoing, multiyear project examining the implementation of interventions to promote self-determination in the context of transition planning for students with intellectual disability (ID) in the state of Rhode Island (RI). The project emerged after the state entered into a Consent Decree with the U.S. Department of Justice (DOJ) to address violations of Title II of the Americans with Disabilities Act (ADA, 1990) as interpreted by the Supreme Court in Olmstead v. L.C. (1999) in the context of employment supports and services (United States of America v. State of Rhode Island, 2014). The DOJ found “unnecessary and over-reliance upon segregated sheltered workshops and facility-based day programs” (p. 2) and emphasized the critical role of transition supports and services to enable transition to integrated, competitive employment. The Consent Decree named adolescents with ID eligible for transition services (aged 14 and older in state law) as a targeted population. In previous studies, we have reported on the impact of training and coaching supports to enable special education teachers to implement the SDLMI during the first year of the project (2015–2016), finding expected levels of student goal attainment as well as increases in teacher perceptions of student self-determination, but not student self-reported self-determination (Shogren et al., 2019). In the second year of the project (2016–2017), we implemented a cluster randomized controlled trial (C-RCT), randomly selecting half of the districts and the teachers within those districts to implement WF in addition to the SDLMI, comparing an SDLMI only to an SDLMI + WF condition (Shogren, Burke, et al., 2018). The pattern of results was slightly unexpected, with students in the SDLMI only condition (vs. the SDLMI + WF condition) showing greater gains in their self-reported self-determination. However, when those data were reported, only data from the 2016–2017 school year were available, and other studies have shown differential growth across multiple years of intervention with the SDLMI or other self-determination curricula, with relatively flat growth for the first year and more steep growth in subsequent years (Wehmeyer et al., 2013). Further, during the subsequent school year (2017–2018) teachers from the SDLMI only condition received training and access to WF (serving as a waitlist group) with all teachers in 2017–2018 implementing both the SDLMI and WF. As such, research examining the possible nonlinear impact of the SDLMI and WF interventions across implementation years is needed, particularly comparing growth patterns between more intensive (i.e., SDLMI + WF) and less intensive (i.e., SDLMI only) conditions. As such, our purpose in this article is to build on the work reported in Shogren, Burke, et al. (2018) and examine outcomes in the year following the C-RCT. Specifically, we followed students that participated in the C-RCT comparing the longer term outcomes of students who first participated in the SDLMI only condition to the SDLMI + WF group who received WF both years. We addressed the following research questions:
What is the differential impact of the SDLMI + WF versus SDLMI only conditions on student self-determination scores, after the initial year of intervention?
Does the impact of the SDLMI + WF condition on student self-determination scores increase after the initial year of intervention?
As described, the present study is part of a multiyear, ongoing project with data currently available for three school years (see Table 1 for intervention schedule). In the first year (2015–2016), teachers throughout the state of Rhode Island were trained on the SDLMI and supported (see Training and Coaching section) to implement the SDLMI (Shogren et al., 2019). In the second year (2016–2017), we added an additional component, randomly assigning half of the districts to a more intensive intervention condition that added WF to the SDLMI intervention to compare the impacts of the SDLMI only and SDLMI + WF conditions using a cluster randomized controlled trial design (C-RCT; Shadish et al., 2002; Shogren, Burke, et al., 2018). We chose to randomize school districts, rather than students, as coaching was delivered at the district level, meaning that there would be a high risk of contamination of effects if randomization occurred at student or teacher levels. In 2017–2018, all teachers who were in the SDLMI only condition in 2016–2017 received training on WF so that, by the start of 2017–2018, all teachers implemented the SDLMI and WF. Hereafter, for purposes of analyses, the group that transitioned from the SDLMI only to the SDLMI + WF condition in 2016–2017 is called the “immediate group” and the group that waited to transition to the SDLMI + WF condition until 2017–2018 is called the “waitlist group.”
Setting and Sample
The sample for the current analyses consists of students with ID and their special education teachers who participated in the study during the 2016–2017 and 2017–2018 school years. Specifically, the sample is comprised of 240 students with ID, 53 special education teachers, and 17 school districts. This sample includes a subset of the participants in the clustered randomized control trial (C-RCT) previously reported in Shogren, Burke, et al. (2018) that occurred during the 2016–2017 school year. To be included in the present analyses, student participants needed to (a) self-report their self-determination status on the Self-Determination Inventory: Student Report (SDI:SR; described subsequently) and (b) at least 75% of the SDI:SR had to be completed per measurement occasion (i.e., Fall 2016, Spring 2017, Fall 2017, and Spring 2018). There was a subset of students (n = 94) in the larger sample who did not have self-report data because of the significance of their support needs, and instead only had teacher-reported data on outcomes from the Self-Determination Inventory: Parent/Teacher Report (SDI:PTR). Given our focus on student-reported outcomes and previous research suggesting low correlations in ratings across teachers and students (Shogren, Anderson, et al., in press), we focused on the 240 students and their teachers in the present analyses. In the future, however, research should also explore changes in teacher perceptions on the SDI:PTR.
We should note that only data from the 2016–2017 and 2017–2018 school years was analyzed as the data collection procedures and systems differed in the 2015–2016 school year due to the rapid implementation of the project, and thus the data from this initial year cannot be meaningfully linked with subsequent years (see Limitations). It is also important to note when interpreting these results that most of the students in the sample as well as their teachers had exposure to the SDLMI in 2015–2016, the year immediately prior to the first data point reported in the present analyses. Outcomes from SDLMI implementation during the 2015–2016 school year are further described in Shogren et al. (2019)
Students and teachers reported demographic information, and we used teacher-reported data when demographic information differed across students and teachers or was missing from students. Table 2 provides available student demographic information. The sample had more male students (n = 156, 65%) than female (n = 84, 35%). The age range was 10 to 21 years (M = 16.57, SD = 2.15). In addition to ID, a secondary disability in one or more categories was reported for a total of 127 students (52%). The most commonly reported race/ethnicity was White/European American (n = 121, 50%). Of the 53 special education teachers in the current sample, 49 were female and 4 were male, with an average age of 40.75 years (SD = 12.51). Teachers reported that they knew the students they were supporting for an average of 2.43 years (SD = 2.02) and implemented the interventions with between 1 and 22 students.
Training and Coaching
During the 2015–2016 school year, teachers implemented the Self-Determined Learning Model of Instruction (SDLMI) with students following a one-and-one-half day training on the SDLMI led by University of Kansas (KU) researchers and staff at the Paul V. Sherlock Center at Rhode Island College (RIC; Shogren et al., 2019). During 2016–2017, teachers within districts randomly assigned to the SDLMI + WF condition (the “immediate” group in the present analyses) received an additional one-half day training on Whose Future Is It? (WF; Wehmeyer & Palmer, 2011) from KU researchers, and new teachers in both conditions received the same one-and-one-half day SDLMI training from 2015–2016 (Shogren, Burke, et al., 2018). During 2017–2018, all teachers in the SDLMI only condition (the “waitlist” group in the present analyses) received the same one-half day training on WF from 2016–2017 and began implementing WF.
To address challenges with promoting fidelity across a large number of teachers and school districts, trained district coaches provided coaching supports to teachers in both groups on the SDLMI. Coaches visited teachers' classrooms at least three times per year for approximately 45 to 60 min and conducted fidelity observations to identify strengths and areas of need and document fidelity of implementation using a systematic coaching process (see Hagiwara et al., in press for more information on the coaching process and implementation fidelity). Throughout 2016–2017 and 2017–2018, coaches attended monthly in-person problem-solving and professional development meetings with staff from KU and RIC that supported implementation of the Consent Decree throughout the state.
Fidelity of Implementation
Coaches in both groups in 2016–2017 and 2017–2018 completed The SDLMI Teacher Fidelity Measure (Shogren et al., 2015) at three time points during each school year for each of the teachers they were supporting to provide an objective measure of fidelity of implementation for the SDLMI. The SDLMI Teacher Fidelity Measure assesses quality of implementation indicators for all SDLMI Teacher Objectives rated on a scale of 0 (not at all) to 3 (completely). The measure includes four items linked to each Student Question in a given phase (e.g., “Did the student identify what they want to learn [i.e., identify strengths and needs, communicate preferences, interests, beliefs, and values, prioritize needs]?”), along with a fifth item addressing Educational Supports (i.e., “Were appropriate educational supports used?”). In 2016–2017, the average fidelity rating was 1.61 for the immediate group and 1.43 for the waitlist group. In 2017–2018, the average fidelity rating was 2.10 for the immediate group and 2.37 for the waitlist group, reflecting an increase in fidelity over time. These figures represent average to above average fidelity of implementation based on the metric of the scale.
In 2016–2017, coaches also reported procedural fidelity on implementation of WF for teachers in the SDLMI + WF condition. WF is a systematic curriculum with structured content and activities for each section, and thus an associated procedural checklist (e.g., “Did the teacher review the vocabulary words and comprehension questions with the student before they began using the reader?”) on a 0–4 scale (0 = not at all, 1 = a little, 2 = somewhat, 3 = mostly, 4 = definitely) was used. Procedural fidelity for WF in the 2016–2017 school year was 3.18, indicating above average (or high) fidelity. Given the systematic design of the WF curriculum and time constraints procedural fidelity data for WF could not be collected in 2017–2018.
The Self-Determined Learning Model of Instruction
The SDLMI (Shogren, Raley, et al., 2018) is a teaching model designed to enable educators to promote self-determined actions through a goal setting and attainment process that is directly taught to students as they set and work toward goals. The SDLMI can be applied to any goal domain, but in this project the focus was on transition-related employment goals. Teachers in the sample implemented the SDLMI throughout the entire project, which is ongoing, with a target of providing individualized supports for students to set and work toward three or more goals during each year. Students worked on goals related to career exploration, developing specific job-related skills, and identifying job shadowing or internship opportunities, in addition to other self-selected goals.
Teachers provided instruction and supported students to solve the problem in each of the three distinct phases of the SDLMI (Phase 1: What is my goal?, Phase 2: What is my plan?, Phase 3: What have I learned?). Each phase has four Student Questions that guide students through a problem-solving sequence to solve the problem posed in that phase. The Student Questions are linked to Teacher Objectives, which highlight specific objectives teachers want to achieve in supporting students to answer questions. Teachers supported students to interact with questions in a variety of methods, including verbally, through pictures, and through behavioral indicators and observations. Teachers utilized Educational Supports to provide direct instruction on skills embedded in the SDLMI (i.e., antecedent cue regulation, choice making, communication, decision making, goal attainment, goal setting, problem solving, self-advocacy, self-assessment, self-awareness, self-instruction, self-monitoring, and self-scheduling) at least twice per week as students worked through the problem-solving sequence, while also integrating students' goals and action plans into ongoing curriculum and instruction, as described in the SDLMI implementation protocols (Shogren, Raley, et al., 2018). Teachers made appropriate individualized modifications (e.g., alternate methods of communication or content presentation) based on their expertise and knowledge of student needs. For example, teachers of students with complex communication needs represented the Student Questions and potential response options with visuals with which students were familiar, and students used their preferred communication modes to indicate choices, preferences, and decisions related to their goals. More information on the SDLMI and its implementation is available in Shogren, Raley, et al. (2018) and at www.self-determination.org. Additionally, information on issues and challenges related to large-scale implementation of the SDLMI in the context of promoting positive transition outcomes for youth with ID is available in Burke et al. (2019).
Whose Future Is It?
During the 2016–2017 school year, teachers in the districts randomly assigned to the SDLMI + WF condition implemented Whose Future Is It? (WF) along with the SDLMI, and in the 2017–2018 school year, all teachers implemented both WF and the SDLMI. WF is an evidence-based transition-planning curriculum with 15 chapters organized into three sections: Getting to Know Your IEP, Decisions and Goals, and Your IEP Meeting (Wehmeyer & Palmer, 2011). Intervention materials included an Instructor's Guide, a Student Reader, and a Student Workbook, all of which were available in digital and print formats. Teachers supported students to engage with the Student Reader, through which content was delivered, and students completed activities linked to the curriculum using the Student Workbook. Accessibility features in the digital format included read-aloud narration and embedded vocabulary definitions. Teachers implemented WF one-on-one, in small groups, or with the whole class, with the target of progressing though all 15 chapters over the school year. WF implementation occurred for approximately 45 min three times per week, in addition to SDLMI implementation. For the immediate group, in which WF was implemented during both the 2016–2017 and 2017–2018 school years, the 15 chapters were repeated in the second year, with the goal of supporting students to build upon the skills related to transition planning developed in the previous year.
Student self-determination was measured using the Self-Determination Inventory: Student Report (SDI:SR; Shogren & Wehmeyer, 2016). Students completed the SDI:SR at the beginning and end of each school year, consistent with administration guidelines. The SDI:SR is a recently validated measure of self-determination based on Causal Agency Theory (Shogren et al., 2015) that defines self-determination as a dispositional characteristic defined as acting as a causal agent over one's life, which involves setting and going after goals by self-regulating one's actions over time. Students respond to items about their ability to be self-determined (i.e., to make choices, set and attain goals, and make decisions; Shogren, Little, et al., 2018). Items are worded as statements, such as “I have what it takes to reach my goals.” Students complete the assessment online, using a customized delivery platform where students respond to each item on a slider scale with anchors of agree and disagree, that are converted to scores between of 1 to 99 by the online system. The factor structure of the SDI:SR was tested with over 4,500 youth ages 13 to 22 with and without disabilities, including students with ID. Analyses suggest strong reliability of items and validity of scores in measuring self-determination, including differentiation of scores in youth with and without disabilities (Shogren, Little, et al., 2018). An overall self-determination score for each of the four measurement occasions (beginning and end of 2016–2017 and 2017–2018) was computed by averaging across all the marked items in the set of 21 items consistent with scoring protocols.
To explore the impact of SDLMI and WF implementation across the 2016–2017 and 2017–2018 school years, we engaged in a series of analyses. Because data came from four levels (i.e., measurement occasions [n = 4], students [n = 240], teachers [n = 53], and districts [n = 17]), we conducted multilevel B-spline model analysis to plot the growth in self-determination for students in the immediate group, who were in the SDLMI + WF condition in 2016–2017 and 2017–2018, and the waitlist group, who were in the SDLMI only condition in 2016–2017 but transitioned to the SDLMI + WF condition in 2017–2018. Multilevel model analysis allowed outcomes across occasions to be correlated within students, teachers, and districts, and the B-spline model analysis allowed curvilinear growth trajectories to be fit to data to allow the growth trajectories to be nonlinear (Zheng et al., 2011), given findings from previous research noted in the Introduction. We also recovered model parameters with Bayesian estimation, which is well suited for complex multilevel analyses with reduced samples (e.g., Gelman & Hill, 2007). Further detail model specifications, including prior distributions and Bayesian computation algorithms can be obtained by contacting the authors.
Although this study had missing outcome values in both groups across measurement occasions (see Table 3 for details), we addressed the missing data values with a full information likelihood approach to Bayesian estimation (which is the Bayesian equivalent to full information maximum likelihood [FIML]). That is, only the observed outcome data contributed to the calculation of the likelihood of the sample data. This approach to missing data analysis retains the desirable asymptotic properties of multiple imputation without needing as many modeling assumptions (Allison, 2012).
Before proceeding to the main analyses, we also conducted initial analyses to ensure that randomization of districts to immediate and waitlist groups yielded baseline equivalency in self-determination outcomes, as predicted. Without baseline equivalency, it is difficult to attribute the observed differences in self-determination outcomes between groups to intervention conditions. The observed self-determination outcomes at the initial measurement occasion for the immediate group (M = 67.03, SD = 22.52) were almost the same as for the waitlist group (M = 66.93, SD = 23.78; see Table 3), suggesting baseline equivalency. To formally test baseline equivalency, we fit a model () assuming baseline equivalency between groups (immediate vs. waitlist) and a model () allowing for baseline nonequivalency between groups. By examining the deviance information criterion (DIC; Ando, 2007), we found that (equivalency) fit the data slightly better than (nonequivalency) as it had the smaller DIC value, providing formal evidence for baseline equivalency and justifying moving forward with comparative analyses.
We took a Bayesian approach to address Research Questions 1 and 2. In Bayesian analysis, statistical hypotheses are judged by their probabilities in light of sample data (Howson & Urbach, 2006). Given weakly informed, regularization priors (which keep inferences in reasonable ranges, e.g., SDI:SR scores cannot exceed 99) and observed data, we examined the probability that (a) the SDLMI + WF condition predicted greater annual gains than the SDLMI only condition after the initial year of implementation and (b) the second year of SDLMI + WF implementation predicted greater annual gains than the initial year of SDLMI + WF implementation. In addition to reporting the distribution of conditional probabilities of effects given data, we also calculated and reported the most probable effect value given data and an 89% highest posterior density (HPD) interval estimate. In Bayesian analysis, an 89% HPD interval estimate for an effect indicates that there is an 89% probability that the true effect lies inside this interval, and no value outside of this interval has more probability than values enclosed inside it. Following McElreath (2016), we reported an 89% interval estimate as opposed to the more conventional 95% interval estimate for Bayesian analysis to avoid the appearance of frequentist null hypothesis testing.
Research Question 1—Impact of SDLMI + WF Versus SDLMI Only After Initial Year
Figure 1 plots the differential growth of self-determination scores for each randomization group (immediate vs. waitlist) over time, as obtained via the multilevel B-spline model analysis. As shown in Figure 1, in 2016–2017, the immediate group experienced lesser annual gains in self-determination scores than the waitlist group (as found in Shogren, Burke, et al., 2018), but this pattern changes in 2017–2018, with the immediate group showing the greatest annual gains in the study that year. In fact, after the first year of implementation, Bayesian analysis indicated that, given this data, there was a .848 probability that the SDLMI + WF condition yielded higher annual gains than the SDLMI only condition, which means the probability that the SDLMI only condition yielded higher outcomes is only .152. Further, the most probable expected value for the difference in annual gain between conditions, after the first year of implementation, is 5 units, 89% HPD (−2 units, 12 units). Using Cohen's d, this difference translates to .222 standard deviation units, a small effect size in favor of the SDLMI + WF condition.
Research Question 2—Impact of SDLMI + WF Condition Across Years
To further explore if the impact of SDLMI + WF exposure was significantly greater in the second year compared to the first year, we examined annual gains of the immediate group across years. Figure 1 shows that the immediate group experienced lesser annual gains in self-determination scores in 2016–2017 than in 2017–2018, and only when the immediate group was in the second year of the SDLMI + WF condition did it achieve the greatest amount of annual gain in the study. Bayesian analysis indicates that, within the SDLMI + WF condition, there is a .965 probability of obtaining greater annual gains in the second year of implementation than the first year of implementation. The most probable expected value for the difference in annual gain between years, given the data, is 7 units, 89% HPD (1 unit, 13 units). Using Cohen's d, this difference translates to .311 standard deviation units, a borderline small/medium effect size.
The purpose of this study was to explore the ongoing impact of self-determination interventions in the year following a C-RCT comparing an SDLMI only to SDLMI + WF condition. Such work is needed given that little is known about the long-term impact of sustained interventions to promote self-determination, with the limited work that does exist suggesting possible nonlinear growth patterns when intervention extends over multiple years (Wehmeyer et al., 2013). As noted in the Introduction, in a prior study reporting the outcomes of the one-year C-RCT, more positive student-reported self-determination outcomes were found in the SDLMI only condition (the less intensive condition) than in the more intensive intervention condition (i.e., combining the SDLMI + WF). Although this finding was not necessarily predicted, there were potential explanations for the finding, such as the SDLMI only condition allowing teachers more flexibility to individualize based on student needs as the SDLMI is overlaid on existing curriculum and instruction whereas WF is a scripted curriculum (Shogren, Burke, et al., 2018). However, by examining growth patterns over multiple years, which was the purpose of this complementary study, we were also able to explore the possibility of a nonlinear growth pattern as such a pattern would mean that examining the number of years of sustained implementation is important when estimating focal effects because the magnitude of gains might depend on the year of implementation.
Although the current findings are consistent with the prior research showing that, in 2016–2017, the SDLMI only condition had a greater than expected annual gain in student self-reported self-determination scores than the SDLMI + WF condition (Shogren, Burke, et al., 2018), the inclusion of an additional year of data provided additional insight into the impact of the intervention conditions over time. In the second year (2017–2018) the reverse pattern was seen, with those that were in the SDLMI + WF condition from the beginning (immediate group), showing greater gains than those in the SDLMI only condition. The findings suggest that after the initial year of implementation, the SDLMI + WF condition predicts greater annual gains than the SDLMI only condition. This evidence suggests a nonlinear growth pattern over multiple years of intervention with more intensive interventions. These analyses have significant implications for research, policy, and practice.
Implications for Research, Policy, and Practice
First, the findings suggest it may be critical to consider implementation of interventions to support the development of complex characteristics, like self-determination, over multiple years while youth are in secondary schools. Adolescents with ID may need longer term supports to develop and express the self-regulatory and behavioral flexibility skills taught through self-determination interventions (Shogren, Wehmeyer, & Palmer, 2017). Second, data collection on complex outcomes, like self-determination, needs to extend beyond single academic years in school settings and consider that there may be curvilinear growth trajectories. If sufficient data is not collected over time, these relationships cannot be explored and the nuanced impacts of interventions across multiple domains may be overlooked or poorly understood (National Academies of Sciences, 2018). Third, it suggests the need to consider policy and practices that promote an early focus on skills associated with self-determination prior to the initiation of transition planning, coordinating efforts across teachers, classrooms, schools, and districts to promote the supports needed for implementation. For example, transition planning is not required under IDEA until age 16 (although several states adopt a higher standard of age 14). Because promoting self-determination is often associated with transition planning, even though it is increasingly argued that these skills are relevant across the lifespan and should be targeted throughout K–12 education (Palmer, 2010; Papay et al., 2015), these skills are often not systematically targeted until secondary education. Fourth, further work is needed to explore how to make instructional decisions about the intensity and duration of self-determination interventions, particularly in the context of tiered systems of supports where students are exposed to more intensive interventions based on documented instructional need (Morningstar et al., 2017; Shogren et al., 2016). We did not examine individual growth trajectories and responses within intervention conditions. However, various contextual factors may impact not only response to intervention but the necessary intensity and duration of interventions for students with ID. Further research is needed in this area to allow for individualization and demonstration of factors that impact treatment response.
Limitations and Future Directions
Several limitations should be considered when interpreting the results of the current study. First, we only examined data from the 2016–2017 and 2017–2018 years of implementation. The project was initiated, however, in 2015–2016, and participating teachers during that year were trained and received coaching to implement the SDLMI. However, as has been described in other sources (Shogren, Burke, et al., 2018; Shogren et al., in press) implementation and data collection efforts occurred rapidly in the 2015–2016 school year, as a direct response to mandated change from the Consent Decree entered into by the state of Rhode Island. Given this, there were not the necessary data collection systems in place to collect and then link student and teacher data from 2015–2016 with subsequent years of data collection. However, this additional data would have provided critical information on the growth experienced by students during initial exposure to self-determination interventions. Although statistical analysis supports baseline equivalence prior to the C-RCT in 2016–2017, confirming it was valid to proceed with an examination of the differences across the intervention conditions, what is not known is the change in self-determination that occurred in 2015–2016 as a result of exposure to the SDLMI only for all student participants. As noted, ongoing work is needed exploring the even longer term impacts of intervention (e.g., throughout high school and into adulthood) as youth are exposed to varying types and intensities of intervention.
Second, it is important to note that a subset of the 2016–2017 sample of students described in Shogren, Burke, et al. (2018) did not complete the SDI:SR because of the intensity of their support needs. Data was also collected from teachers on the SDI:PTR for these students (as well as for the entire sample). As the focus of the present analysis was student-reported changes in self-determination, as this is generally considered the most valid way to assess this dispositional characteristic (Shogren, Anderson, et al., in press; Shogren, Little, et al., 2018), additional analyses should be undertaken to explore changes in teachers' perceptions. Other research has suggested differences in teacher and student perceptions of change, and changes in teacher perceptions may be meaningful to understand factors that influence teacher fidelity of implementation and perceptions of instructional needs (Shogren, Anderson, et al., in press; Shogren, Burke, et al., 2018). Further, ongoing work is needed to explore ways, in addition to proxy-report, to assess the self-determination of people with significant support needs who are unable to complete rating scales.
Another limitation of the analyses was the relatively limited sample size, including missing data resulting from the movement of students and teachers over the course of the study and how this data is treated in the model (see Table 3). A sample size of 17 school districts, which was the unit of randomization, is a relatively small sample in the context of multilevel analysis. However, with this limited sample size, Bayesian analysis allowed for comparison of the relative conditional probabilities of hypotheses given data. Although Bayesian analysis indicated that the null hypothesis is improbable relative to a positive effect hypothesis, ongoing work with larger sample sizes and additional, longitudinal data is needed to fully explore all possible alternative hypotheses. For example, we cannot completely rule out other factors (e.g., student development, exposure to other educational interventions and experiences) that could have impacted outcomes, and future research utilizing a business-as-usual control group should be considered. To engage in such research, however, ongoing work is needed to explore ways to enhance buy-in for both research activities as well as implementation of large-scale change initiatives (Cook & Odom, 2013; Fixsen et al., 2010). The ultimate goal of this project was to implement interventions statewide, supporting all transition-age students with ID in Rhode Island (the target of the Consent Decree). Buy-in and participation by districts and schools varied over the years of the project, often based on teacher and administrator knowledge of and buy-in related to the instructional value of self-determination. Overall there was an increase over time, from a sample of 184 students in 2015–2016 to total sample of 359 students in 2017–2018, but we did not reach full engagement with the approximately 700 transition-age students with ID identified in the Consent Decree. Ongoing efforts to explore how to manage systems change and promote buy-in for the implementation of evidence-based practices at all levels of the education systems are needed (Cook & Odom, 2013). Additionally, there needs to be a focus on interagency and interdisciplinary collaboration (Antosh et al., 2013), as the ultimate goal of transition is to support youth to transition from school-based services and supports to adult roles, which often requires supports from varying adult systems (i.e., long-term services and supports, vocational rehabilitation; Test et al., 2009). Further, as a primary goal of the Consent Decree and the efforts undertaken in Rhode Island is to promote the transition to integrated, competitive employment, there is a specific need to explore cross-system and cross-discipline training to promote seamlessness in the use of effective interventions to promote self-determination and competitive employment as youth move from school-based supports and services to adult supports and services (Wehman et al., 2018).
Overall, this study suggests that there are significant, and positive impacts of implementing self-determination interventions, specifically the SDLMI and WF for students with ID. Further, these interventions can be implemented by teachers, over time, with ongoing coaching and supports, leading to changes in student self-determination outcomes. The results suggest, however, that it is important to collect data over time to inform decisions about effective self-determination interventions and take a long-term, life course perspective on promoting self-determination.
The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R324L160002 to the University of Kansas. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.