Abstract

The purpose of this study was to examine the effect of supported employment intervention on the employment outcomes of transition-age youth with intellectual and developmental disabilities served by the public vocational rehabilitation system using a case-control study design. Data for this study were extracted from the Rehabilitation Services Administration Case Service Report (RSA-911) database for fiscal year 2009. The sample included 23,298 youth with intellectual and developmental disabilities aged between 16 and 25 years old at the time of application. The classification and regression tree (CART) method was used to estimate propensity scores and to adjust for selection bias on the basis of all prominent covariates relevant to the dependent variable (i.e., competitive employment). Results yielded six homogeneous subgroups, and receipt of supported employment was found to increase the employment rates across all of the groups. The effect of supported employment was especially strong for youth who were Social Security beneficiaries, special education students, and individuals with intellectual disabilities or autism who were high school graduates. These findings suggest that supported employment is an effective service for enhancing the vocational rehabilitation outcomes of young adults and provides valuable information for policy makers, health care providers, rehabilitation counselors, and educators.

The transition from school to adult life represents a critical and exciting phase of an individual's life. Yet, for many young adults with intellectual and developmental disabilities (IDD), the reality of securing fulfilling and meaningful postschool experiences seldom materializes as high unemployment rates continue to pose a significant challenge for people with disabilities (U.S. Bureau of Labor Statistics, 2013). This is particularly concerning for individuals with severe disabilities who overwhelmingly tend to be either unemployed or underemployed, despite their desire and ability to contribute to their communities through engagement in work (Brault, 2012; Roux et al., 2013). At the same time, young people with disabilities constitute a group with significant long-term potential and who, if they were to remain employed for the long term, would reduce expenses associated with the adult care system (Wehman, Schall et al., 2014). There is an urgent need to employ effective intervention strategies to mitigate barriers and facilitate successful employment outcomes for young adults with IDD. Additionally, there is increasing evidence that helping youth connect with early work experiences leads to better long-term vocational rehabilitation outcomes (Cimera, Burgess, & Wiley, 2013). For example, recent findings using the National Longitudinal Transition Study-2 dataset underscore the significance of early work experiences predicting later employment success (e.g., Carter, Austin, & Trainor, 2011, 2012; Wehman, Sima et al., 2014). Supported employment (SE) is one program that has been invaluable to many individuals with severe disabilities to secure employment in competitive, integrated work settings (cf. Wehman, 1981; Wehman, Inge, Revell, & Brooke, 2007). The SE interventions should have much application to transition-age youth with IDD given their comprehensive and ongoing nature, yet there is limited data documenting the efficacy of SE intervention with the younger IDD population.

Supported Employment

Supported employment was initially conceived as an employment support service for people with severe IDD, many of whom were spending their days in sheltered workshops and day activity centers. The primary components of SE include competitive work in an integrated setting with ongoing support services (66 Fed. Reg., 2001), and SE was specifically designed to embody the principles of consumer empowerment and individualized, community-based support (Wehman, Kregel & West, 1997). Since its inception, SE has assisted people with significant IDD achieve positive employment outcomes (Wehman et al., 2007).

There is also growing evidence that SE has been instrumental in assisting people with other disabilities, such as those with psychiatric impairments (Bond, 2004; Bond, Drake, & Becker, 2008; Campbell, Bond, & Drake, 2011), traumatic brain injury (Wehman, Targett, West, & Kregel, 2005), autism (Hendriks & Wehman, 2009; Howlin, Alcock, & Burkin, 2005; Wehman, Smith, & Schall, 2009; Wehman et al., 2012) and physical disabilities (Ottomanelli et al., 2012). Furthermore, SE is listed as an empirically validated intervention for people with severe mental illness by the Substance Abuse and Mental Health Services Administration (SAMHSA). Drake, Bond, and their associates have conducted a series of randomized controlled trials to evaluate the effectiveness of the Individual Placement and Support (IPS) model, a variant of SE, and found that the IPS intervention had a large effect size on improving the employment outcomes of people with severe mental illness (cf. Bond, 2004; Bond et al., 2008; Campbell et al., 2011). Although these studies offer strong support for the effectiveness of SE using randomized controlled trials, studies evaluating SE among transition-age youth with IDD remain relatively small in sample size.

Supported Employment and the State-Federal Vocational Rehabilitation Program

The U.S. state-federal vocational rehabilitation (VR) program is the oldest and most successful public program supporting the employment of individuals with disabilities (Martin, West-Evans, & Connelly, 2010). Serving approximately one million individuals per year and spending more than $2.5 billion annually, this program plays a large and instrumental role in assisting individuals with a wide range of disabilities, including transition-age youth, reach their employment goals (Martin et al., 2010; U.S. Government Accountability Office, 2005). Eligible applicants to state VR agencies include those who have a physical or mental impairment that presents a substantial impediment to employment and require VR services to facilitate employment acquisition or maintenance (Rehabilitation Act of 1973, §102). Currently there are state VR agencies in all 50 states and U.S. trust territories.

Because of limited financial resources, state VR agencies are required to develop “order of selection” plans to prioritize services for people with the most significant disabilities. Thus, the population served by VR has shifted significantly since the program's inception, moving away from a consumer base comprised primarily of those with physical restoration needs to one that serves a growing number of consumers with intellectual disability, brain injury, and mental illness (Research Triangle Institute International, 2002). Although VR counselors tailor services based on individual needs, in light of the federal reporting requirements, VR agencies within and across states follow a relatively standard rehabilitation process, which includes: eligibility determination, rehabilitation plan development, service provision, and job placement (Ditchman et al., 2013). The services that are offered, such as SE, can be provided directly by the VR counselor or may be contracted out to other professionals or agencies. Successful rehabilitation closure occurs when the services provided have led to at least 90 days in competitive employment in the most integrated setting possible that is consistent with the individual's interests, strengths, and abilities (Ditchman et al., 2013).

Supported Employment for Transition-Age Youth With IDD

As noted above, SE has been shown to be an empirically validated intervention for persons with psychiatric impairments (Luciano et al., 2014) using numerous randomized clinical trials. However, no SE studies have been found utilizing randomized clinical trials for people with IDD despite wide use of SE in community programs serving this population. The state-federal vocational rehabilitation program widely invests dollars in SE despite research based on small sample sizes that are not well controlled. Furthermore, the efficacy of SE for transition age youth with IDD has not been demonstrated empirically. Yet, hundreds of millions of dollars are being invested in this intervention without a well-controlled study.

For example, the extent to which demographic characteristics, disability type, or Social Security beneficiary status affect whether an individual receives SE or whether SE is more effective has not been thoroughly explored. This is due in part to an inherent limitation of research related to the VR system and its effectiveness—namely, the inability to conduct randomized controlled studies because, in the VR program, eligible consumers must be served immediately (Bolton, 2004). However, emerging advanced statistical techniques, such as propensity score matching methods, provide new avenues for investigating the effectiveness of interventions based on nonexperimental or observational data.

With a growing focus on competitive employment for people with severe disabilities within the VR system, SE has gained recognition as an important intervention. Over the years, SE has advanced from enclave and mobile work crew models (i.e., small group placements) to one that provides an individualized approach assisting one person at a time (Wehman et al., 2007), and state VR agencies have increasingly invested in SE as a service mechanism to enhance competitive employment closures. However, it remains unclear how effective SE has been as a VR service and what types of individuals benefit most.

Propensity Score Matching

Propensity score matching methods for nonexperimental causal studies originated from the medical field (Austin, 2011) and have generated considerable attention in educational, behavioral, and social sciences research due to the growing interest in using observational (or nonrandomized) studies and large secondary dataset analyses to estimate the effects of treatments on outcomes (Dehejia & Wahba, 2002; Fan & Nowell, 2011; Guo & Fraser, 2010; Luellen, Shadish, & Clark, 2005). In observational studies, because of the lack of random assignment of participants to treatment and control groups, baseline characteristics of participants who receive treatment often differ systematically from those who do not. Therefore, these systematic differences in baseline characteristics between these two groups must be controlled for when estimating the effect of the treatment on outcomes. When the number of characteristics is limited (e.g., two binary variables), matching is straightforward. However, when there are many variables, it is difficult to determine along which dimensions to match units or which weighting scheme should be adopted (Dehejia & Wahba, 2002). The propensity score is the probability of treatment assignment conditional on as many observed baseline subject characteristic variables as necessary (Austin, 2011). This natural weighting scheme allows one to design and analyze an observational study to correct for sample selection bias due to observable differences between the treatment and comparison groups (Austin, 2011; Dehejia & Wahba, 2002).

Study Purpose

Although there has been substantial data collected for SE as an intervention, the experimental control for IDD populations has been highly limited. Furthermore, the extent to which SE as a VR service influences successful rehabilitation outcomes for young adults with IDD has not been adequately explored. The purpose of this study was to examine whether SE is in fact an effective mechanism of change for young adults with IDD who were served by state VR agencies. The Rehabilitation Services Administration Case Report (RSA-911) dataset provides a rich source of information about all of the consumers served across state VR agencies, the services they receive, and their employment outcomes. Using this dataset, we conducted a matched case-control study using propensity score balancing to adjust for nonrandom assignment to evaluate the effect of SE on employment outcomes among young adults with IDD served by state VR agencies. Specifically, this study was designed to answer two key research questions:

  1. Is SE an effective intervention for enhancing VR outcomes for transition-age youth with IDD?

  2. What are the characteristics of youth with IDD who can benefit the most from SE intervention?

Method

Study Cohort

Data for this study were extracted from the U.S. Department of Education, Rehabilitation Services Administration Case Service Report (RSA-911) database. The RSA-911 data contain detailed information about demographics, disability, types of intervention services, and employment outcomes for all clients receiving state VR services in the United States, and the data are furnished annually to RSA by state VR agencies. Data from the RSA-911 for fiscal year 2009 were used for the analyses because it was the most current dataset available at the time of the study.

The sample in this study consisted of 23,298 youth with IDD aged between 16 and 25 years old at the time of application and whose cases were closed in the fiscal year 2009. Regarding the primary disabilities of the participants, 16,762 (71.9%) had intellectual disability; 3,131 participants (13.4%) were diagnosed with autism; 1,719 (7.4%) had cerebral palsy; and 1,686 (7.2%) had traumatic brain injury. There were 14,086 men (60.5%) and 9,212 women (39.5%). Racial and ethnic backgrounds were dominated primarily by White (62.4%), followed by 29.0% African American, and 8.6% Hispanics/Latinos. Asians and American Indians were excluded from this study because of small sample sizes. Mean age of the clients was 19.29 years (SD  =  2.45). In terms of education, 27.0% completed special education, 48.7% had less than a high school education, 20.1% completed high school, and 4.2% had some form of postsecondary education. About 43% of the participants were Social Security beneficiaries receiving Supplemental Security Income (SSI) and/or Social Security Disability Insurance (SSDI). Table 1 summarizes the demographic characteristics of the study sample.

Table 1 

Sample Characteristics (N  =  23,298)

Sample Characteristics (N  =  23,298)
Sample Characteristics (N  =  23,298)

Major Variables

Competitive employment was the primary outcome measure and SE intervention was the independent variable in this study. Competitive employment is defined in the RSA-911 manual as, “employment in an integrated setting, self-employment, or employment in a state-managed Business Enterprise Program (BEP) that is performed on a full-time or part-time basis for which an individual is compensated at or above the minimum wage.” (Note that BEP refers to vending facilities and small businesses operated by individuals with significant disabilities as well as home industry that fall under the management of the state VR agency.) Clients who were not working after completing their planned rehabilitation program were considered unsuccessful outcomes.

Five demographic covariates were used to adjust for selection bias, including: (1) gender (male, female), (2) race/ethnicity (White, African American, Hispanic/Latino), (3) education (special education [certificate of completion/diploma or in attendance]), less than high school education, completed high school (high school graduate or equivalency certificate [regular education students], associate degree, and bachelor degree), (4) type of IDD (autism, cerebral palsy, intellectual disability, traumatic brain injury), and (5) Social Security beneficiary status (yes, no).

Statistical Design and Analysis

Propensity score analysis was used to adjust for selection bias on the basis of all prominent covariates that were relevant to the outcome variable. In our study, a propensity score is the “probability” that a participant would receive SE intervention in VR. A classification tree was used to identify homogeneous subgroups of participants who have similar propensities of receiving SE interventions based on their demographic characteristics. The proportion of the participants receiving SE in each subset was then used as the estimate of the common propensity for that subgroup. To arrive at the propensity scores, first, demographic covariates were used in a classification tree model to classify participants into homogeneous subgroups and to estimate propensity scores (i.e., likelihood to receive SE). Homogeneous subgroups were then arranged in descending order based on their estimated propensity scores with comparisons made between participants who received SE and those who did not. A direct adjustment estimator was then applied to adjust for selection bias.

In this study, the classification and regression tree (CART) method with a 10-fold cross-validation was used to estimate propensity scores (Breiman, Friedman, Olshen, & Stone, 1984). The classification tree approach for propensity score matching has several advantages over the more commonly used logistic regression approach because (a) the classification algorithm automatically selects variables for the model; (b) the algorithm also automatically detects interactions in the data, so that those interactions do not have to be discovered or modeled explicitly as they do in logistic regression; (c) the tree's terminal nodes automatically supply the researcher with strata (homogeneous subgroups), eliminating the need to set stratification cut points; and (d) the classification tree approach is considered more intuitive to visualize and understand than other propensity score matching methods (Lee, Lessler, & Stuart, 2010; Luellen et al., 2005).

The CART method is an empirical, statistical technique based on recursive partitioning analysis. Recursive partitioning examines all available predictors and identifies a series of variables that are most related to the outcome measure (Zhang & Singer, 1999). It is a nonparametric statistical method and can handle data that are highly skewed or multimodal and categorical predictors with either an ordinal or non-ordinal structure (Fonarow, Adams, Abraham, Yancy, & Boscardin, 2005). Specifically, the CART method involves the segregation of different values of classification variables through a decision tree comprised of progressive binary splits. Every value of each predictor variable is considered as a potential split, and the optimal split is selected based on an impurity criterion (the reduction in the residual sum of squares due to a binary split of the data at that tree node). The Gini index is the splitting criterion for growing a CART tree. Each parent node in the decision tree produces two child nodes (subgroups), which in turn can become parent nodes producing additional child nodes. This process continues with both tree building and pruning until statistical analysis indicates that the tree fits without overfitting the information contained in the data set.

Cost-complexity pruning was used to prune the tree (Breiman et al., 1984). This process takes into account both the number of errors and the complexity of the tree. The size of the tree is used to represent the complexity of the tree. Data are divided into a learning (also called training) sample and a test sample. The complexity parameter (α: a measure of how much additional accuracy a split must add to the entire tree to warrant the additional complexity) would gradually increase during the pruning process. To calculate the misclassification error rate, the model is fitted to the learning sample to predict values in the test sample. The error rate is calculated for the largest tree as well as for every smaller tree. The 10-fold cross-validation involves splitting the data into 10 smaller samples with similar distributions of the response variable. Trees are then generated, excluding the data from each subsample in turn. For each tree, the error rate is estimated from the subsample excluded in generating it, and the cross-validated error for the overall tree is then calculated. Beginning at the last level (i.e., the terminal nodes), the child nodes are pruned away if the resulting change in the predicted misclassification cost is less than α times the change in tree complexity. As α increases, more and more nodes are pruned away, resulting in simpler trees. At the end, the smallest tree whose observed number of errors on the test set does not exceed the misclassification rate of the pruned tree plus one standard error (one standard error rule) is selected as the tree that optimally fits the true information in the dataset (Breiman et al., 1984; Lewis, 2000; Patil, Wadhai, & Gokhale, 2010). The Answer Tree statistical software package was used to conduct the CART analyses (Statistical Package for the Social Sciences [SPSS], 1999).

Results

Descriptive Statistics

For the overall sample, the proportion of transition-age youth with IDD in VR receiving SE was 36.38%. Individuals with autism (37%) or intellectual disabilities (40%) had a higher propensity to receive SE than young adults with cerebral palsy (23%) or traumatic brain injury (19%). Participants who had postsecondary education (associate degree [20%] and bachelor degree [17%]) were less likely to receive SE than youth with no postsecondary education (special education [55%], less than high school education [28%], and high school graduate [37%]). Social security beneficiaries (48%) had higher propensity to receive SE than nonbeneficiaries (28%).

Tree Classification

The CART analysis was used to examine the effect of gender, race/ethnicity, education, type of disability, and Social Security beneficiary status on the propensity to receive SE as a primary VR intervention (outcome variable). The decision tree initially grew to five levels with 20 homogeneous subgroups, but then the tree was pruned using the cost-complexity pruning method suggested by Breiman et al. (1984) to reduce overfitting. We employed the one standard error rule, which led to a final subtree with six homogeneous subgroups. In this optimally fitted tree, education was the most significant predictor of who would receive SE services, followed by Social Security beneficiary status and type of disability. Each subgroup is characterized by different combinations of the predictor variables, with estimated propensity scores in the subgroups ranging from a low of 24% to a high of 62%. Figure 1 provides a graphical presentation of the SE tree.

Figure 1 

Supported employment vs. no supported employment participation classification tree for transition-age youth with intellectual and developmental disabilities.

Figure 1 

Supported employment vs. no supported employment participation classification tree for transition-age youth with intellectual and developmental disabilities.

The following is a brief description of the six homogeneous subgroups that emerged from the original 20 groups after employing the cost-complexity pruning method, in the order of their propensity to receive SE.

Subgroup 4 (N  =  3,687). This group represents 3,687 transition-age youth who were Social Security beneficiaries and who had received special education in secondary school. The majority of the participants in this group were individuals with intellectual disabilities (85%), and 61% were male. The average age of the clients in this group at application was 20.69 years (SD  =  2.38). With a propensity score of .62 (62%), individuals in this group had the highest likelihood to receive SE services compared to individuals in the other subgroups. Of the individuals in this subgroup who received SE as a VR intervention, over half (58%) obtained successful employment closures, which was significantly higher than the employment rate of those who did not receive SE (37%), χ2(1, N = 3,687)  =  155.39, p < .001.

Subgroup 9 (N  =  1,760). This group represents 1,760 individuals with intellectual disability (82%) or autism (18%) who were Social Security beneficiaries. They were all high school graduates and over half (59%) of the youth in this group were men. Mean age of the clients at application was 21.35 years (SD  =  2.23). The propensity score estimate for this group was .54, indicating that 54% of the individuals in this subgroup received SE as part of their VR services. Individuals in this subgroup who received SE as a VR intervention had a significantly higher employment rate (63%) compared to those who did not received SE (43%), χ2(1, N = 1,760)  =  68.87, p < .001.

Subgroup 3 (N  =  2,604). This group represents 2,604 individuals who received special education in secondary school but were not Social Security beneficiaries. Sixty percent of the youth in this group were men and the majority of the participants in this group were individuals with intellectual disabilities (83%). Mean age of the clients was 19.24 years (SD  =  2.22). This group had a propensity score estimate of .44 (44%). Individuals in this subgroup who received SE as a VR intervention had an employment rate of 61%, compared to a 50% employment rate for those who did not receive SE, χ2(1, N = 2,604)  =  30.04, p < .001.

Subgroup 10 (N  =  3,427). This group represents 3,427 individuals with intellectual disabilities or autism who were Social Security beneficiaries. This group is distinguished from Subgroup 9 because it included high school dropouts and those who had completed some form of postsecondary education at application, as opposed to those who had graduated from high school but had not received further education at the time of application. Eighty-six percent of the youth in this group were individuals with intellectual disabilities and 14% were individuals with autism. Mean age of the clients was 18.74 years (SD  =  2.10). This group had a propensity score estimate of .37 (37%). Over half (54%) of those who received SE as a VR intervention were employed, compared to only 37% for those who did not receive SE, χ2(1, N = 3,427)  =  26.46, p < .001.

Subgroup 8 (N  = 1,228). This group represents 1,228 youth with cerebral palsy (42%) or traumatic brain injury (58%) who were Social Security beneficiaries and did not receive special education in secondary school. Mean age of the clients at application was 20.37 years (SD  =  2.56). This subgroup had a propensity score estimate of .24 (24%). Individuals in this subgroup who received SE as a VR intervention had an employment rate of 50%, compared to the 39% employment rate for those who did not receive SE, χ2(1, N = 1,228)  =  11.19, p < .001.

Subgroup 5 (N  =  10,592). This group was the largest, representing 10,592 individuals who did not receive special education in secondary school and were not Social Security beneficiaries. Mean age of the clients was 18.53 years (SD  =  2.19). This group had a propensity score estimate of .24 (24%). Individuals in this subgroup who received SE had an employment rate of 61%, compared to an employment rate of 54% for those who did not receive SE, χ2(1, N = 10,592)  =  28.41, p< .001.

Table 2 presents the homogeneous subgroups sorted by their propensity scores. Results indicate a positive effect of SE intervention for transition-age youth with IDD. This effect is strongest for Social Security beneficiaries who were special education graduates (21% difference in employment outcomes) and persons with intellectual disabilities or autism who graduated from regular high school (20% difference in employment outcomes). The propensity adjusted estimate of the overall effect of SE on VR outcomes showed that individuals who received SE had on average a 12.49% higher employment rate than individuals who did not receive SE.

Table 2 

Effect of Supported Employment on VR Outcomes of Transition-Age Youth With IDD

Effect of Supported Employment on VR Outcomes of Transition-Age Youth With IDD
Effect of Supported Employment on VR Outcomes of Transition-Age Youth With IDD

Discussion

In recent years, SE has emerged as the gold standard of vocational intervention for people with severe disabilities who have long histories of unemployment. However, the effectiveness of SE interventions on the employment outcomes of transition-age youth who are served by state VR agencies does not have good empirical data. The purpose of this study was to examine whether SE is in fact an effective mechanism of change for young adults with IDD who are served by this public program. Based on our results, the answer to this question is yes. The research findings clearly support SE as an effective use of funding that does indeed result in successful VR closures. This study contributes to the SE literature because it is the first to apply propensity score matching techniques to demonstrate the efficacy of SE based on the RSA-911 dataset, and our findings extend the literature on SE efficacy in several important ways.

First, our findings provide clear support for the effectiveness of SE as a VR service for promoting successful employment closures for young adults with IDD. Using propensity score matching to compare individuals who received SE and those who did not, by balancing them on several baseline demographic covariates using the CART method, yielded six homogeneous subgroups based on their propensity to receive SE. Across all of these subgroups, employment rates for individuals who received SE were consistently higher than for individuals who did not receive SE. This finding is consistent with the existing literature supporting the effectiveness of SE interventions (e.g., Campbell et al., 2011; Wehman et al., 2005, 2007, 2009) and indicates that providing consumers with severe disabilities individualized and ongoing supports leads to successful employment outcomes.

Our findings also suggest that this effect is especially prominent for Social Security beneficiaries who are special education graduates or young adults with intellectual disability or autism who graduated from high school. In fact, there was approximately a 20% difference in employment rates between those who received SE and those who did not for both of these subgroups. These findings call attention to the particular effectiveness of SE for those transition-age consumers receiving SSI or SSDI benefits. Beneficiary status has long been considered a major work disincentive in the rehabilitation literature. For instance, a study by Dutta, Gervey, Chan, Chou, and Ditchman (2008) found that VR adult consumers with mental impairments had a 39% reduction in the odds of being closed successfully if they received cash or medical benefits. This is also a concerning issue for young adults receiving benefits, and has resulted in the Social Security Administration (SSA) funding a number of Youth Transition Demonstration projects across the country designed to increase the likelihood that young adults will become employed, earn enough money to reduce their disability benefits, and eventually leave the disability rolls altogether (Frakera & Rangarajanb, 2009). Findings from our study lend further support to SE as an effective intervention for improving employment for this at-risk group, who generally represent those facing the most severe challenges.

Second, it is noteworthy that several groups emerged as having higher propensities to even receive SE services, such as those who are SSI/SSDI beneficiaries and also students who had received special education. In both cases, these are populations who might be viewed as less likely to succeed in competitive employment. Typically, persons receiving SSI/SSDI have already been documented as having a significant disability through a parallel service system, in this case the SSA. For those identified as special education students, they too have been recognized as having sufficiently unique challenges requiring funded services through the Individuals with Disabilities Act (IDEA) while in school. This finding is important because it indicates that rehabilitation counselors within the state-federal VR program are appropriately targeting SE for these groups. At the same time, it is likely that SE services, such as job coaching services, may be more labor intensive and long term to adequately serve this population with significant needs. Given that a successful employment outcome was based on 90 days of competitive employment, it is difficult to know for certain the extent to which the SE services provided met the long-term vocational goals of young adults with significant support needs.

On the other hand, it is regrettable that those clients who were SSI/SSDI beneficiaries and also high school dropouts were less likely to receive SE. Our findings suggest that educational status may impact propensity to receive SE. For instance, although Subgroups 9 and 10 were similar in terms of disability and Social Security beneficiary status, they were comprised of clients with different educational backgrounds and associated with varied propensity scores. Subgroup 9 consisted of high school graduates, whereas Subgroup 10 consisted of high school dropouts and those who had completed postsecondary education. Over half (54%) of the clients in Subgroup 9 received SE, compared to only 37% of those in Subgroup 10. This may be because VR counselors did not see those without high school degrees as sufficiently motivated to enroll into an SE program or it may be that there was a lack of transition planning or opportunities for self-determination while in high school. This is difficult to assess given the wide variability of when and why drop outs occur and the role of geographic or family influences.

Third, it is not surprising that individuals with significant intellectual disability and autism also had a higher propensity to receive SE services because this intervention was originally designed to serve these individuals who are at risk for sheltered and segregated settings (Rusch & Hughes, 1989). In 1998, the U.S. Supreme Court handed down the Olmstead decision, which promoted community integration for those with developmental disabilities. Since then, the U.S. Department of Justice has gone to numerous states and negotiated settlements that move in the direction of community integration and competitive employment.

Although it is important for these traditionally served disability populations to continue to receive SE, our findings suggest that SE may represent an underutilized service for young adults with cerebral palsy or traumatic brain injury, as these disability groups were associated with the lowest rates of receiving the intervention. Given that emerging research has documented the application of SE approaches with both brain injury (e.g., Wehman et al., 2005) and cerebral palsy (e.g., Wehman et al., 1991) populations, it is possible that some of these individuals would benefit from SE but do not receive it. It is also not surprising that individuals who completed postsecondary education were less likely to receive SE. This can be most likely explained by the fact that these persons had the cognitive abilities that when coupled with a college degree or certificate were less likely to require SE. It is also possible that VR counselors were less inclined to invest already limited case service funds in those who they saw had already completed college or trade school training. Additionally, most of these institutions of higher education have career centers that often help facilitate job placement.

Finally, because our findings were based on a large dataset, although we have on one hand a robust sample, we have only limited information on the specific details of the SE interventions provided to the consumers and the extent to which long-term supports were in place through other funding mechanisms after the 90-day case closure period. We know that many of these young people received services from a range of vendors contracted with VR counselors that vary in quality, size, and capital. On average, our findings suggest that those who receive SE have a 12.5% higher employment rate. However, it is possible our results could have been considerably more positive with better vendors or not as strong if the vendors were less than optimal, but this remains an unknown variable.

Limitations

Several limitations of this study suggest areas for further research. First, an important limitation is that the fidelity of the SE provided by agencies and vendors within and across states is unknown in this study. Although all state VR agencies offer SE, this intervention may look different depending on who provides the service and how well these providers adhere to SE principles. Additional research is needed to investigate the quality of the SE services being provided by VR agencies.

Second, although we employed a case control design, this study is not experimental and causation must be interpreted with caution. Although we have included all the relevant matching variables (i.e., gender, race/ethnicity, education, types of IDD, and SSI/SSDI status), there may be other demographic covariates that could be included in our case matching. For example, functional disability (severity) was not available in the RSA-911 dataset and might provide additional information that can improve the accuracy of the final tree solution even though SSI/SSDI status may already serve as a proxy variable for severity. Future research is needed to explore additional characteristics that could have an impact on the provision and effectiveness of SE interventions.

Third, our study is limited in that successful rehabilitation closure occurs after 90 days of employment, and does not provide longer-term tracking to monitor job retention. Moreover, employment closure does not take into consideration the consumer's satisfaction with the employment outcome. Future research should extend beyond this traditional 90-day mark and access more information about the long-term supports and outcomes of these youth who are employed.

Finally, the generalizability of our results may be compromised given that our study data were collected during the global financial crisis facing the United States in 2009. It is logical to assume that the recession and continued economic stagnation have an impact on VR outcomes. In fact, there is emerging evidence that the economic context can indeed affect the employment outcomes of VR consumers (Chan et al., in press; Cook et al., 2006). However, in this study we were not able to estimate the impact of high unemployment rates on the effectiveness of SE, which may have adversely affected the results. Future research investigating the effectiveness of SE in other fiscal years will be beneficial.

Implications for Policy and Practice

Findings from this study present important implications for policy and practice. This study represents one of the most expansive studies (including data from over 20,000 young adult consumers) demonstrating the effect of SE. Our results support the investment in SE at the national level and align with national- and state-level initiatives and legal mandates aimed at increasing the integration and economic independence of transition-age youth (e.g., Employment First initiative, IDEA mandates).

However, in order to maximize the benefit of SE for youth with IDD, counselors, VR providers, and transition specialists need to receive sufficient training to ensure the fidelity of the SE interventions used. Graham, Inge, Wehman, and Murphy (2013) showed that research on the efficacy of employment interventions was indeed valued by state and federal VR personnel and that data from a study such as this should be appropriately disseminated. Quality indicators, fidelity scales, external monitoring, and other forms of accountability are essential inasmuch as there is too wide a disparity in the quality of services being provided. Moreover, future research is needed to more systematically investigate the impact of factors related to SE delivery (e.g., length of time, fidelity, provider qualifications) and their role in affecting efficacy of the intervention.

At the same time, the SSA, the Center for Medical Services (CMS), and all state education agencies need to be aware of the empirical support for SE as an effective service, as it may be that early implementation of SE can directly and favorably impact their goals and missions. Clearly, schools want their students to gain employment in order to increase their Indicator 14 outcomes. The SSA is focused on reducing their cash benefit payouts, and CMS has a vested interest in the correct implementation of Medicaid waivers to achieve the best return on investment. Furthermore, these results have implications for the U.S. Department of Labor (DOL), and can inform One Stop Career Center staff in how to implement their performance indicators. Although the RSA-911 data system documents services provided and employment outcomes achieved by individuals with disabilities accessing the state-federal VR program, it does not systematically collect postclosure employment information. Data contained by the DOL and Internal Revenue Service may be useful to accessing the long-term employment outcome information in order to better assess the continuing efficacy of service interventions such as SE.

Finally, a major barrier to the employment of young adults is that there is no entitlement to services after the age of 21. The state-federal VR program can provide services such as SE to eligible consumers; however, this is a time-limited service. For those with severe disabilities who require support after the 90-day closure period, it is often excessively difficult for them to obtain the long-term support they need. There are obvious issues related to funding short-term employment services for people with severe IDD predicated on another funding source for long-term support. In other words, the very people who need SE also require support individualized to their unique needs for an indefinite nature to maintain employment, yet long-term funding is especially difficult for those with severe IDD to access (Certo et al., 2008; Migliore & Butterworth, 2008).

Conclusion

As previously noted, SE has been an important vocational service for people with severe disabilities, and this study provides large scale support for the effectiveness of SE interventions as provided through the state-federal VR system for young adults with IDD. Findings suggest that SE is particularly effective for individuals who are Social Security beneficiaries, special education students, and individuals with intellectual disabilities or autism who are high school graduates. These findings have implications for policy and service delivery, and draw attention to issues related to fidelity and long-term access to SE for those with severe disabilities.

Acknowledgments

The development of this paper was funded by #133A10007 with the National Institute on Disability and Rehabilitation Research and TransCen, Inc. Prior to conducting this research, the study received approval from the institutional review boards of the University of Wisconsin–Madison and Virginia Commonwealth University.

References

References
66 Fed. Reg. (January 17, 2001) to be codified at 34 C.F.R. pt. 361.
Austin
,
P. C
.
(
2011
).
An introduction to propensity score methods for reducing the effects of confounding in observational studies
.
Multivariate Behavioral Research
,
46
(
3
),
399
424
.
Bolton
,
B
.
(
2004
).
Counseling and rehabilitation outcomes
.
In
F
.
Chan
,
N. L
.
Berven
,
K. R
.
Thomas
(
Eds.
),
Counseling theories and techniques for rehabilitation health professionals
(pp.
444
465
).
New York, NY
:
Springer
.
Bond
,
G. R
.
(
2004
).
Supported employment: Evidence for an evidence-based practice
.
Psychiatric Rehabilitation Journal
,
27
(
4
),
345
359
.
Bond
,
G
.,
Drake
,
R
.,
&
Becker
,
D
.
(
2008
).
An update on randomized controlled trials of evidence-based supported employment
.
Psychiatric Rehabilitation Journal
,
31
,
280
290
.
Brault
,
M. W
.
(
2012
).
Americans with disabilities: 2010 (Current Population Reports No. P70-131)
.
Washington, DC
:
U.S. Census Bureau
.
Breiman
,
L
.,
Friedman
,
J. H
.
Olshen
,
R. A
.,
&
Stone
,
C. J
.
(
1984
).
Classification and regression trees
.
Pacific Grove, CA
:
Wadsworth
.
Campbell
,
K
.,
Bond
,
G. R
.,
&
Drake
,
R. E
.
(
2011
).
Who benefits from supported employment: A meta-analytic study
.
Schizophrenia Bulletin
,
37
(
2
),
370
380
.
Carter
,
E. W
.,
Austin
,
D
.,
&
Trainor
,
A
.
(
2011
).
Factors associated with the early work experiences of adolescents with severe disabilities
.
Intellectual and Developmental Disabilities
,
49
(
4
),
233
247
.
Carter
,
E. W
.,
Austin
,
D
.,
&
Trainor
,
A
.
(
2012
).
Predictors of postschool employment outcomes for young adults with severe disabilities
.
Journal of Disability Policy Studies
,
23
(
1
),
50
63
.
Certo
,
N. J
.,
Luecking
,
R. G
.,
Murphy
,
S
.,
Brown
,
L
.,
Courey
,
S
.,
&
Belanger
,
D
.
(
2008
).
Seamless transition and long-term support for individuals with severe disabilities
.
Research & Practice for Persons with Severe Disabilities
,
33
(
3
),
85
95
.
Chan
,
Y. C
.,
Wang
,
C. C
.,
Ditchman
,
N
.,
Kim
,
J. H
.,
Pete
,
J
.,
Chan
,
F
.,
&
Dries
,
B
.
(
in press
).
State unemployment rates and vocational rehabilitation outcomes: A multi- level analysis
.
Rehabilitation Counseling Bulletin (Special Edition).
Cimera
,
R. E
.,
Burgess
,
S
.,
&
Wiley
,
A
.
(
2013
).
Does providing transition services early enable students with ASD to achieve better vocational outcomes as adults
?
Research and Practice for Persons with Severe Disabilities
,
38
(
2
),
88
93
.
Cook
,
J. A
.,
Mulkern
,
V
.,
Grey
,
D. D
.,
Burke-Miller
,
J
.,
Blyler
,
C. R
.,
Razzano
,
L. A
.,
&
Steigman
,
P. A
.
(
2006
).
Effects of local unemployment rate on vocational outcomes in a randomized trial of supported employment for individuals with psychiatric disabilities
.
Journal of Vocational Rehabilitation
,
25
,
71
84
.
Dehejia
,
R. H
.,
&
Wahba
,
S
.
(
2002
).
Propensity score matching methods for non-experimental causal studies
.
Review of Economics and Statistics
,
84
,
151
161
.
Ditchman
,
N
.,
Wu
,
M
.,
Chan
,
F
.,
Fitzgerald
,
S
.,
Lin
,
C
.,
&
Tu
,
W
.
(
2013
).
Vocational rehabilitation
.
In
D
.
Strauser
(
Ed.
),
Career development, employment, and disability in rehabilitation
(pp.
343
360
).
New York, NY
:
Springer
.
Dutta
,
A
.,
Gervey
,
R
.,
Chan
,
F
.,
Chou
,
C. C
,
&
Ditchman
,
N
.
(
2008
).
Vocational rehabilitation services and employment outcomes of people with disabilities: A United States study
.
Journal of Occupational Rehabilitation
,
18
(
4
),
326
334
.
Fan
,
X
.,
&
Nowell
,
D. L
.
(
2011
).
Using propensity score matching in educational research
.
Gifted Child Quarterly
,
55
(
1
),
74
79
.
Fonarow
,
G. C
.,
Adams
,
K. F
.,
Abraham
,
W. T
.,
Yancy
,
C. W
.,
&
Boscardin
,
W. J
.
(
2005
).
Risk stratification for in-hospital mortality in acutely decompensated heart failure: Classification and regression tree analysis
.
Journal of the American Medical Association
,
293
(
5
),
572
580
.
Frakera
,
T
.,
&
Rangarajanb
,
A
.
(
2009
).
The Social Security Administration's youth transition demonstration projects
.
Journal of Vocational Rehabilitation
,
30
,
223
240
.
Graham
,
C
.,
Inge
,
K
.,
Wehman
,
P
.,
&
Murphy
,
K
.
(
2013
).
Moving employment research into practice: Knowledge and application of evidence-based practices by state vocational rehabilitation agency staff
.
Journal of Vocational Rehabilitation
,
39
,
75
81
.
Guo
,
S
.,
&
Fraser
,
M. W
.
(
2010
).
Propensity score analysis: Statistical methods and analysis
.
Thousand Oaks, CA
:
Sage
.
Hendricks
,
D. R
.,
&
Wehman
,
P
.
(
2009
).
Transition from school to adulthood for youth with autism spectrum disorders: Review and Recommendations
.
Focus on Autism and Developmental Disability
,
24
(
2
),
77
88
.
Howlin
,
P
.,
Alcock
,
J
.,
&
Burkin
,
C
.
(
2005
).
An 8 year follow-up of a specialist supported employment service for high-ability adults with autism or Asperger syndrome
.
Autism: The International Journal of Research & Practice
,
9
(
5
),
533
549
.
Lee
,
B. K
.,
Lessler
,
J
.,
&
Stuart
,
E. A
.
(
2010
).
Improving propensity score weighting using machine learning
.
Statistics in Medicine
,
29
(
3
),
337
346
.
Lewis
,
R. J
.
(2000
,
May
).
An introduction to classification and regression tree (CART) analysis
.
Lecture presented at the annual meeting of the Society for Academic Emergency Medicine, San Francisco, CA.
Luciano
,
A
.,
Drake
,
R. E
.,
Bond
,
G. R
.,
Becker
,
D. R
.,
Carpenter-Song
,
E
.,
Lord
,
S
.,
Swanson
,
S. J
.
(
2014
).
Evidence-based supported employment for people with severe mental illness: Past, current, and future research
.
Journal of Vocational Rehabilitation
,
40
(
1
),
1
13
.
Luellen
,
J. K
.,
Shadish
,
W. R
.,
&
Clark
,
M. H
.
(
2005
).
Propensity scores: An introduction and experimental test
.
Evaluation Review
,
29
(
6
),
530
558
.
Martin
,
R
.,
West-Evans
,
K
.,
&
Connelly
,
J
.
(
2010
).
Vocational rehabilitation: Celebrating 90 years of careers and independence
.
American Rehabilitation
,
15
18
.
Migliore
,
A
.,
&
Butterworth
,
J
.
(
2008
).
Trends in outcomes of the vocational rehabilitation program for adults with developmental disabilities: 1995–2005
.
Rehabilitation Counseling Bulletin
,
52
,
35
44
.
Ottomanelli
,
L
.,
Goetz
,
L. L
.,
Suris
,
A
.,
McGeough
,
C
.,
Sinnott
,
P. L
.,
Toscano
,
R
.,
Thomas
,
F. P
.
(
2012
).
Effectiveness of supported employment for veterans with spinal cord injuries: Results from a randomized multisite study
.
Archives of Physical Medicine and Rehabilitation
,
93
(
5
),
740
747
.
Patil
,
D. D
.,
Wadhai
,
V. M
.,
&
Gokhale
,
J. A
.
(
2010
).
Evaluation of decision tree pruning algorithms for complexity and classification accuracy
.
International Journal of Computer Applications
,
11
,
23
30
.
Rehabilitation Act of 1973, as amended by Pub. L. 99-506, to be codified at 29 U.S.C. § 701 (1986).
Research Triangle Institute International
. (
2002
).
Longitudinal study of the vocational rehabilitation program. Report 1: How consumer characteristics affect access to, receipt of, and outcomes of VR services
.
Raleigh, NC
:
Author
.
Roux
,
A
.,
Shattuck
,
P
.,
Cooper
,
B
.,
Anderson
,
K
.,
Wagner
,
M
.,
&
Narendorf
,
S
.
(
2013
).
Postecondary employment experiences among young adults with an autism spectrum disorder
.
Journal of American Academy of Child & Adolescent Psychiatry
,
52
,
931
939
.
Rusch
,
F. R
.,
&
Hughes
,
C
.
(
1989
).
Overview of supported employment
.
Journal of Applied Behavior Analysis
,
22
(
4
),
351
363
.
Statistical Package for the Social Sciences
. (
1999
).
SPSS AnswerTree algorithm summary
.
Chicago, IL
:
Author
.
U.S. Bureau of Labor Statistics
. (
2013
).
Persons with a disability: Labor force characteristics - 2012 (USDL-13-1141)
.
U.S. Government Accountability Office
. (
2005
).
Vocational rehabilitation—Better measures and monitoring could improve the performance of the VR Program (GAO-05-865)
.
Washington, DC
:
Author
.
Wehman
,
P
.
(
1981
).
Competitive employment: New horizons for severely disabled individuals
.
Baltimore, MD
:
Paul H. Brookes Publishing
.
Wehman
,
P
.,
Inge
,
K. J
.,
Revell
,
W. G
.,
&
Brooke
,
V. A
.
(
2007
).
Real work for real pay: Inclusive employment for people with disabilities
.
Baltimore, MD
:
Paul H. Brookes Publishing
.
Wehman
,
P
.,
Kregel
,
J
.,
&
West
,
M
.
(
Eds.
). (
1997
).
Supported employment research: Expanding competitive employment opportunities for persons with significant disabilities
.
Richmond, VA
:
Virginia Commonwealth University, Rehabilitation Research & Training Center on Supported Employment
.
Wehman
,
P
.,
Lau
,
S
.,
Molinelli
,
A
.,
Brooke
,
V
.,
Thompson
,
K
.,
Moore
,
C
.,
&
West
,
M
.
(
2012
).
Supported employment for young adults with autism spectrum disorder: Preliminary data
.
Research and Practice for Persons with Severe Disabilities
,
37
(
3
),
1
10
.
Wehman
,
P. H
.,
Revell
,
W. G
.,
Kregel
,
J
.,
Kreutzer
,
J. S
.,
Callahan
,
M
.,
&
Banks
,
P. D
.
(
1991
).
Supported employment: An alternative model for vocational rehabilitation of persons with severe neurologic, psychiatric, or physical disability
.
Archives of Physical Medicine and Rehabilitation
,
72
,
101
105
.
Wehman
,
P
.,
Schall
,
C
.,
Carr
,
S
.,
Targett
,
P
.,
West
,
M
.,
&
Cifu
,
G
.
(
2014
).
Transition from school to adulthood for youth with autism spectrum disorder: What we know and what we need to know
.
Journal of Disability Policy Studies
.
Advance online publication. http://dx.doi.org/10.1177/1044207313518071
Wehman
,
P
.,
Smith
,
M
.,
&
Schall
,
C
.
(
2009
).
Transition from school to adulthood for youth and young adults with autism: Growing up in the real world
.
Baltimore, MD
:
Paul H. Brookes Publishing Co
.
Wehman
,
P
.,
Sima
,
A. P
.,
Ketchum
,
J. M
.,
West
,
M
.,
Chan
,
F
.,
&
Luecking
,
R
.
(
2014
).
Predictors of successful transition from school to employment for youth with disabilities
.
Manuscript submitted for publication.
Wehman
,
P
.,
Targett
,
P
.,
West
,
M
.,
&
Kregel
,
J
.
(
2005
).
Productive work and employment for persons with traumatic brain injury: What have we learned after 20 years
?
Journal of Head Trauma Rehabilitation
,
20
(
2
),
115
127
.
Zhang
,
H
.,
&
Singer
,
B. H
.
(
1999
).
Recursive partitioning in the health sciences
.
New York, NY
:
Springer
.

Author notes

Paul Wehman, Department of Physical Medicine and Rehabilitation, Virginia Commonwealth University; Fong Chan, Rehabilitation Research and Training Center on Evidence-Based Vocational Rehabilitation Practices, University of Wisconsin-Madison; Nicole Ditchman, Department of Psychology, Illinois Institute of Technology; and Hyun-Ju Kang, Department of Rehabilitation Psychology and Special Education, University of Wisconsin-Madison.

Correspondence concerning this article should be addressed to Paul Wehman, Virginia Commonwealth University, Medical College of Virginia, 1314 W. Main Street, Richmond, VA, 23284 (e-mail: pwehman@vcu.org).