Information was gathered via a mailed questionnaire from approximately 85% of Maryland service providers offering vocational services funded by the Maryland Developmental Disabilities Administration. The weekly earnings of persons placed in supported employment were 3.5 times the earnings of persons employed on the premises of the service provider. Nevertheless, there was strong evidence that substantial improvements are needed, and possible, in supported employment. Among urgently needed improvements are (a) the ability to recruit and retain qualified vocational workers, (b) enhanced procedures for locating more and higher quality jobs, (c) expanded transportation arrangements, (d) improved training for vocational workers, and (e) an information system to measure performance and identify problems.
During 1995, an estimated 140,000 persons in the United States were receiving supported employment services (Wehman, Revell, & Kregel, 1997) and were earning a mean hourly wage of $4.70. During that same year, Braddock, Hemp, Rizzolo, Parish, and Pomeranz (2002) reported that slightly over 83,000 persons were in supported employment financed by state developmental disabilities agencies, by far the largest funder of supported employment services. The higher number reported by Wehman et al. is largely due (even after allowing for measurement error and duplication) to the fact that many people are in supported employment financed by state mental health agencies, state vocational rehabilitation agencies, local educational agencies, and miscellaneous other sources. Since 1995, the number of supported employees funded by state developmental agencies has grown almost 30% (Braddock et al., 2002). Although the more comprehensive data collected at the Virginia Commonwealth University by Wehman et al. ended in 1995, it is reasonable to presume that supported employment services funded by agencies other than developmental disabilities agencies also grew significantly.
These numbers are particularly impressive when one recalls that only a little over 20 years ago, most of these supported employees would have either been deemed unable to work or would have been relegated to low-paying workshop jobs. One could easily conclude that supported employment programs throughout the country are doing a credible job. However, many people—parents and professionals alike—believe that major improvements are needed (Degeneffe et al., 2001). The improvement most often urged in the literature is the need to place greatly increased numbers of persons with severe limitations in supported employment and other integrated employment positions (Olney & Kennedy, 2001; Kregel & Wehman, 1997). Less often emphasized, but just as important, is the need to improve the quality of supported jobs (i.e., hours worked per week, pay scales, and other job aspects).
In the past, the development of employment programs for persons with severe disabilities has been primarily stimulated by advocates on the basis of anecdotal information and extensive personal experience. Their efforts have been highly successful not only in promoting the growth of these programs, but in changing the ways in which legislators, program managers, parents, consumers, and others view the vocational abilities of persons with severe disabilities.
We expect that in the future we will see continued expansion and improvement of supported employment programs. In this growth, however, the need is less for advocacy than for solid and reliable data that will guide and justify changes in these programs. In particular, data are needed that will identify the most successful providers of supported employment and the practices and characteristics of these providers that contribute to their success.
Unfortunately, data suitable for this purpose are very limited. As noted by West, Johnson, Cone, Hernandez, and Revell (1998) “supported employment research has not specifically addressed the types of services provided. . . . the types of funding methods employed, types of services provided, or other implementation issues” (p. 358). Instead, most of the surveys reported in the literature have been focused on the characteristics of supported employees (i.e., level of mental retardation, gender, ethnicity) that are associated with different levels of vocational success (Mank, Cioffi, & Yovanoff, 1998, McDermott, Martin, & Butkus, 1999; Rusch, Heal, & Cimera, 1997).
A survey of supported employment providers sponsored by the Maryland Association of Community Services and the Maryland Supported Employment Leadership Team was conducted during 2000. The Supported Employment Leadership Team was subsequently reorganized into a Maryland Chapter of the Association for Persons in Supported Employment. In this paper, we report on the conclusions of this survey.
The Maryland Survey had four purposes: (a) to describe staffing characteristics of service providers who received funding for vocational services from the Maryland Developmental Disabilities Administration (DDA); (b) to identify the vocational accomplishments (hours worked, hourly earnings) of all individuals receiving vocational services funded by the Maryland DDA, which included persons in both supported employment and in facility-based employment; (c) to identify variations in vocational outcomes among service providers; and (d) to identify areas where the supported employment program could be changed to enhance the level of vocational accomplishments.
The survey was intended to collect information from all service providers that offered vocational services funded, in part or in total, by the Maryland DDA. In order to receive funding for vocational services from the Maryland DDA, an individual must have a severe and long-lasting disability (or combination of disabling conditions) that began before age 22 (other than the sole diagnosis of mental illness). In addition, the disability must result in the inability to live independently without continuing and regular assistance.
All data were collected at the level of the service provider. Thus, we collected information on the total number of consumers who were provided with vocational assistance by each service provider, the number in supported employment, and their average wage and number of hours of work. Because some providers did not have information on wages, hours of work, or transportation arrangements readily available, respondents were asked to estimate these values.
Persons funded by DDA for vocational services were divided into four groups: those (a) placed in community-based paid employment, (b) placed in community-based volunteer employment, (c) placed in facility-based employment, and (d) waiting to be placed on a job. All persons who were in community-based employment funded by DDA were regarded as being placed in supported employment, and the two terms, community-based employment and supported employment are used interchangeably.
Development of Questionnaire
A questionnaire addressing these issues was drafted and pretested with 4 Maryland service providers (In Maryland, the term service provider rather than rehabilitation facility or community rehabilitation program is used to describe these organizations). The pretesting was sequential among the 4 service providers. After each service provider completed the draft questionnaire, we met with that individual and reviewed the answers for each question for accuracy, clarity, difficulty, and relevance. The questionnaire was then revised before being pretested with the next service provider. Each iteration resulted in significant improvements to the questionnaire.
In almost all cases, both in the pretest and the survey itself, the person who completed the questionnaire had some responsibility for supervising other individuals who provided vocational services to consumers. Although there was a wide range of job titles that were given for the person submitting the questionnaire, they were usually identified as a director, supervisor, acting director, or assistant director of programs identified as providing employment services or vocational services. In a few cases, the respondent was listed as the director or chief executive officer of the service provider and, in one case, his or her administrative assistant.
The questionnaire was initially submitted to 76 service providers (one of which was subsequently identified as no longer in operation) identified from lists maintained by the Maryland DDA and the Maryland Association of Community Services. Fifty questionnaires were returned. One respondent operated in three locations for which separate data were provided. Thirty-three were completely filled out. Seventeen respondents did not answer one or more questions. However, we emphasize that the response rates per question were unusually high—for most questions, the response rate was 95% or higher among the completed questionnaires. Seventeen service providers on the initial lists indicated that they did not provide vocational services and were dropped from the survey.
If we assume that all of the nonrespondents provided vocational services funded by the Maryland DDA, then there are 59 such organizations on the lists that we used. Of these, 85% submitted questionnaires. This unusually high response rate was due to frequent follow-ups to remind the service providers of the importance of completing the questionnaires and to strong support from the sponsoring organizations. In addition, when there was inconsistency or ambiguity in the responses, we contacted the respondent and obtained clarification.
The questionnaires were completed over approximately a 6-month period. The data can be best interpreted as showing the values that existed on any average day during the study period (2000).
Forty-nine service providers gave information on the number of direct service positions for employees who provided vocational services to individuals with significant disabilities who were funded by the Maryland DDA. These service providers reported a total of 879 positions, although over 15% of these positions were reported as not being filled. Roughly, adjusting for nonreporting service providers, we estimate that there were about 1,060 direct service vocational worker positions, filled and unfilled, that provided support to DDA-funded individuals. We emphasize that these data exclude employees in supervisory positions, unless they also provided direct services. Considering only filled positions, this comes to a little over seven supported employees for each direct service worker
There was a surprisingly large number of different job titles: 50 job titles were reported for direct service vocational workers. The three most frequently used were job coach, employment specialist, and vocational counselor.
On average, there were a little over 17 positions per service provider. However, over one fourth of the service providers had 5 or fewer full-time equivalent (FTE) vocational workers and an equivalent percentage had over 20 FTE positions providing these services (Table 1). Approximately two thirds of the total number of direct service vocational positions in Maryland were reported by service providers with 20 or more positions.
Number of Supported Employees
Forty-eight service providers reported that 7,637 individuals were receiving vocational services at the time of the survey. Of these, almost 70% were partially or totally funded by DDA. Of these, fewer than half (44.6%) were participating in community-based paid employment (supported employment) and a small number (3.3%) were participating in community-based volunteer work. A little fewer than half of the DDA-funded individuals were employed in paid employment located solely on the premises of the service provider. In addition, a significant number of individuals receiving vocational supports from all sources (5.7%) were not employed at the time of the survey.
Assuming that the average number of persons funded by DDA for supported employment was the same in the nonreporting facilities as in those that returned questionnaires, we estimate that around 6,200 individuals with disabilities were being funded for vocational services by DDA, and about 2,800 of these people were placed in community-based supported employment in Maryland. In addition, some individuals receiving vocational funding from other agencies (e.g., Department of Mental Hygiene, the Department of Rehabilitation Services, or local school districts) were also in supported employment.
The number of DDA-funded individuals receiving vocational services varied widely among service providers. Just slightly over 30% of the service providers had 40 or fewer individuals funded for vocational services by DDA (see Table 1). This included persons in both supported and facility-based employment and those waiting for jobs. Fourteen service providers served over 100 individuals funded for vocational services by DDA.
Hours and Wages
The survey collected information from each service provider on average hourly earnings and average hours worked per week of persons in the different types of employment. From this data, we estimated average hours of work and average wages by employment status in Maryland by (a) calculating total hours of work and total earnings for each employment status and for each service provider, (b) summing these data for all service providers, and (c) dividing these totals by the number of workers in each employment status.
Average hours worked per week among DDA-funded individuals employed in the community were estimated to be 30% higher than those employed on the premises of service providers (Table 2). Even more important, average hourly earnings were estimated to be twice as high for those working in the community ($5.53) compared to facility-based employment ($2.27). Consequently, average weekly earnings were about 3.5 times higher for those in community employment than for those employed entirely on the premises of service providers ($134 as compared to only $40). Volunteer work lagged far behind in terms of hours worked. This estimate for supported employment is consistent with the hourly earnings reported by Wehman et al. (1995) and is very close to the amount reported by Mank et al. (2000).
Amount of Support Provided
Most persons in supported employment funded by DDA, approximately two thirds, required some support each day. However, only 4% required a vocational worker to devote exclusive attention to them throughout the day. Over one third of the supported employees in community-based employment and over half of the individuals in facility-based or community volunteer employment needed someone on the work premises who was prepared to provide the necessary supports most of the day, although the person providing the supports might assist more than one worker.
About one quarter of the supported employees in both community- and facility-based employment required several prompts or interventions during the work day, but a vocational worker was not required to be there at all times. Over one third of the supported employees in community-based employment required minimal intervention, defined as no more than one intervention per day; occasionally, these supported employees required an intervention only once every week or two.
Variation in Supported Employment Outcomes
One of the striking observations from survey data was the great variation among service providers in the success they had in placing persons with developmental disabilities in supported employment. Less than 5% of the service providers were able to locate jobs in the community for 100% of DDA-funded individuals receiving vocational supports (Table 3). About two thirds found jobs for fewer than 50% and about 15% located community-based jobs for fewer than 10%.
The variation in hours of work and hourly wages was also significant. Only a fifth of the service providers reporting community-based employment reported an average of over 25 hours of work per week per employee. In no case did a service provider report that the average hours worked per week among individuals funded by DDA that they placed on supported jobs in the community exceeded 30 hours. Slightly over half reported an average of fewer than 20 hours per week, and 1 in 10 reported an average of fewer than 10 hours per week.
Of those reporting hourly earning for community-based supported employees, a little over half reported hourly wages that, on average, equaled or exceeded $5.50, the federal minimum wage. However, only 3 service providers reported that average earnings among community-based supported employees fell below $4 an hour. In contrast, only 15% of service providers reporting on facility-based employment indicated that average earning exceeded $4 an hour, and none reported that average earnings exceeding $5.50 an hour. Over three fifths of the service providers reporting on community-based paid employment indicated that these supported employees averaged over $100 per week in contrast to slightly over 10% of service providers reporting on facility-based employment.
There are, of course, numerous possible factors contributing to these wide variations in outcomes among service providers, including differences in the severity of disability and other characteristics of supported employees (Cimera, 1998; Rusch et al., 1997), differences in availability or types of jobs in the local job market, and differences in availability of transportation to jobs (West, Revell, & Wehman, 1997). Of great importance is that the large variations identified in the Maryland survey lead directly to the inference that differences in the skills of vocational staff and approaches to the provision of supported employment services are a major factor causing differences in vocational outcomes. This is consistent with the conclusions drawn by Mank et al. (1998, 2000) that improved vocational outcomes were associated with the amount of coworker training and the number of hours of direct and indirect support per week, although many other features associated with the provision of services were undoubtedly also operating.
It is sometimes assumed that service providers in rural areas are more disadvantaged, particularly in the availability of jobs and local public transportation than service providers in urban areas. The Maryland survey did not obtain data that would permit the measurement of meaningful differences among service providers on this variable. Although we attempted to distinguish between rural and nonrural service providers, we found that a large majority of service providers were either in urban areas or in large counties contiguous to urban areas. These contiguous counties had large parts that could be considered urban or rural (and many variations in between), and without collecting additional information from the service providers, we did not feel comfortable attempting to distinguish between rural and nonrural service providers.
One aspect of these service providers, however, was unambiguous, and that was size, as measured by the number of persons funded by DDA for vocational services. We initially hypothesized that larger service providers would have superior vocational outcomes compared to smaller ones. This was because we thought that larger service providers would be more likely to be in urban areas than small providers and that they would have more flexibility in organizing their vocational workers (e.g., staff could be more specialized).
Service providers were divided into three groups, those with up to 40 persons funded for vocational services by DDA, those with 41 to 100 persons in this category, and those with over 100 persons. It was somewhat surprising to find few differences between large and small service providers with respect to hours and earnings among consumers placed in community-based paid employment. Persons placed by large providers worked more hours per week, but had lower earnings, and net weekly wages between the three size groups were almost identical (Table 4).
In the case of facility-based employment, there was little difference between the three groups in terms of average hours worked per week, but average earnings per hour was significantly lower for large service providers. Consequently, weekly earnings for persons receiving services from providers with 40 or fewer persons funded for vocational services by DDA were twice as high for persons in community-based employment as for those in facility-based employment. In the case of service providers with 100 or more persons funded for vocational services, however, because of low facility earnings, weekly earnings were four times greater for individuals in community-based employment as for persons in facility-based employment.
Enormous differences were noted in the types of employment found for consumers within the three categories of service providers (Table 5). Those that served 40 or fewer persons placed only 13% of persons funded for vocational services by DDA in facility-based employment, whereas those serving over 100 persons placed slightly over half of the individuals in facility-based employment. Of course, these relationships were reversed when considering community-based employment; small service providers were far more likely than large providers to place persons in integrated community employment. There are numerous possible reasons that could account for these large differences. For example, these differences could be due to: variations in the ability levels of the persons to whom they provide vocational support, the greater ease with which large service providers can operate in-house production activities, and a greater commitment to integrated employment by small service providers. Almost certainly, further research will point to multiple causative factors.
Undoubtedly, the reliance on supported employment by smaller service providers explains why the percentage not working was twice as high for service providers with 40 or fewer employees as compared to those with over 100 employees. It is often easier to place individuals with significant disabilities in a workshop environment than to locate a community job.
Needed Improvements in Supported Employment Services
Based on the survey findings, we identified five areas where improvements are needed in the provision of supported employment services. Of course, we recognize that improvements are undoubtedly needed in many other areas, such as the establishment of incentive-based funding and reduction of staff resistance to community placements (West, Revell, & Wehman, 1998). Moreover, we must also emphasize that improvements in any one area are usually much more effective if improvements are simultaneously made in other areas (Marrone, Hoff, & Gold, 1999). Given that the percentage of people in supported employment was approximately twice as high in Maryland as nationally in 1995 (as reported by Wehman et al., 1997), the clear conclusion is that most, probably all, supported employment programs throughout the United States are equally in need of improvement
Recruitment and Retention of Direct Service Workers
Problems in hiring and retaining qualified vocational personnel have numerous adverse effects, including inappropriately high case loads; inability to locate a service provider for all persons in need of supported employment, and too many inexperienced staff persons. Unfortunately, there was strong evidence that many service providers are encountering major difficulties in this area.
As reported above, about 15% of the positions reported by service providers are vacant.
Over 40% of the service providers reporting in the Maryland Survey indicated that they usually or sometimes had difficulty hiring direct service vocational workers that met their qualifications.
Almost all service providers, 86%, reported that they were encountering some difficulties in retaining staff employed to provide vocational services. Forty percent reported that they faced such difficulties either frequently or constantly.
Over half of the service providers reported that the average length of employment for direct service personnel providing vocational services was 3 years or less.
Over 90% of the service providers reported that low salaries were the most important factor that interfered with their ability to recruit or retain employees providing vocational services; in fact, almost two thirds of the service providers indicated that noncompetitive wages constituted a critical difficulty. Similarly, half of the service providers responding to the survey indicated that low salaries or inadequate benefits were a critical issue adversely affecting their ability to retain employees.
As an indication of the salaries that are paid, the Maryland Survey contained a question in which service providers were asked to estimate the beginning salaries for new hires for direct service vocational workers. The estimated median starting salary was slightly less than $18,000 per year in 2000. Only one position in eight would be filled at a salary exceeding $22,000 (Table 6). Several service providers noted that the starting salary would be adjusted upward if the applicant had experience or formal training.
One consequence of low starting salaries is that some service providers adjust their desired educational expectations for new direct service employees downward because of the low salaries that they are able to offer. We need to distinguish between applicants with experience in providing vocational services and those without experience because many service providers are willing to reduce educational requirements if the applicant has been working in the field. If applicants lacked experience, service providers reported that they anticipated hiring employees with only a high school education or its equivalent for the demanding and complicated tasks that vocational employees must perform in almost 40% of the positions. If prospective employees had some experience, service providers looked for no more than 12 years of school in over two thirds of the positions.
In the Maryland survey, service providers were asked to identify the percentage of individuals funded by DDA for vocational services affected by each of four different employment barriers. As expected, transportation was the most important barrier. In fact, 40% of the service providers reported that transportation issues created employment difficulties for 75% or more of the individuals needing vocational supports (Table 7). Because over half of the persons funded for vocational service by DDA work on the premises of the service provider, it is likely that these estimates of transportation difficulties, high as they are, underestimate the number that would be adversely affected by transportation problems if service providers attempted to place all DDA-funded persons in community-based employment.
Transportation problems vary widely, depending upon the needs of the individual and the availability of local bus systems, paratransit, taxis, etc. In some cases, particularly in rural areas, public bus service, taxis, and paratransit may be nonexistent or otherwise severely limited in availability. Even in more developed areas, bus service may not have stops in convenient locations or may be hard to get to. Most people enjoy the easy convenience of automobiles to get them to and from a subway or bus station, but few supported employees with significant disabilities are able to drive.
For the most part, the service providers partially solved transportation problems by transporting workers in agency-owned vehicles (65% of supported employees) or personal vehicles owned by agency staff (8%). Public transportation was only used for little over one fifth of the supported employees, paratransit for about 15%, and parents and other family members in about 12% of the cases. These percentages sum to over 100% because some supported employees use more than one type of transportation (e.g., a parent drives the supported employee to a bus station; a supported employee must use taxis at times when public transportation is not available, for example, on weekends or evenings).
Almost all service providers indicated a need for improvements in public transportation (i.e., lower costs, more frequent service, service on weekends and evenings, and travel to more distant communities). It is noteworthy that the transportation problems for consumers in supported employment differ in important ways from the transportation problems for consumers who receive traditional time-limited services from vocational rehabilitation programs. In supported employment, the service provider usually makes arrangements to provide transportation for consumers if public transportation is not available. This option does not exist for persons in time-limited services. Moreover, although transportation cost is often an important issue for both groups of consumers, those in time-limited services must bear most of these costs themselves, whereas for some individuals in supported employment, most or all of these costs are borne by the service provider.
Training of Direct Support Employment Service Staff
Clearly, low rates of pay and minimal educational requirements create a necessity to provide high quality training programs for vocational workers. For the most part, service providers train new employees through apprenticeship/mentoring activities. However, training for the multiple and complex tasks that vocational workers must perform is hindered by the relatively small number of new hires at any point in time.
It is not surprising that service providers responded overwhelmingly that direct service employees needed additional training in numerous areas (Table 8). The three most important areas were training in job development, building natural supports, and consumer support/training techniques. Almost as important was training in communications/negotiations, assistive technology at work, and Social Security benefits.
The content that would be covered by these areas should be familiar to most readers. It merits mention, however, that we believe most respondents interpreted Social Security benefits to include the Supplemental Security Program (SSI); the Social Security Disability Program (SSDI), which includes the Childhood Disability Benefits Program (paid to adult children whose disabilities began in childhood and whose parent is either disabled, retired, or deceased and qualified for Social Security Benefits) and Medicare, Medicaid, Food Stamp, Maternal and Child Health, and other state and locally operated programs. Individuals with significant disabilities and their families frequently receive benefits from two or more of these programs. In fact, recipients of SSDI are usually eligible for Medicare, and recipients of SSI are usually eligible for Medicaid. Sometimes individuals with disabilities receive both SSDI and SSI. Unfortunately, these programs typically have differing eligibility requirements, differing work incentive provisions, and interact in complicated ways, which few people fully comprehend.
After transportation, the need for improving employment opportunities was identified as the second largest employment obstacle by the Maryland Survey. Improving employment opportunities includes both the development of additional jobs and improving the pay, hours of work, and other aspects of supported employment opportunities. The need for increasing employment opportunities was demonstrated by the following. (a) The great majority of persons in supported employment in Maryland did not work full-time, and many earned less than the minimum wage. Rogan, Banks, and Howard (2000) reported that many supported employees indicated a desire to work more hours. (b) Service providers reported that they were looking for community-based employment for over one third of the individuals in facility-based employment. (c) Service providers were seeking paid employment for over half of the DDA-funded persons who were in volunteer work only. (d) Almost 800 individuals with disabilities funded for vocational services were reported as not working. (e) The great variation in the percentage of DDA-funded persons placed in supported employment among service providers, discussed above, would indicate that some providers could greatly improve their performance in this area.
Although survey data demonstrated the need for locating additional community-based jobs, and increasing the number of hours of work of work and pay rates on these job, information was not obtained on many other critical aspects of supported jobs (e.g., the permanency of jobs, the fringe benefits provided by existing jobs, or the opportunities for career advancement that these jobs offer). Pumpian, Fisher, Certo, and Kimberly (1997) emphasized the importance of enabling supported employees to have the opportunity to change jobs during their work career. In several recent focus groups held with relatives of supported employees in Maryland, deficiencies in these areas were sources of major discontent.
One fact was clear from conducting this survey. The availability of public information on the overall performance of supported employment, the performance of individual service providers, or the performance of selected demographic groupings of individuals receiving vocational services, or even the costs of providing services was scant to nonexistent in Maryland (see also Kimmich & Becker-Green, 2000). Consequently, the state legislature, state funding agencies, and service providers have little information with which to measure success or identify changes in the system needed to improve performance. Individuals needing vocational supports have little information to enable them to judge which service providers are most likely to be most helpful to them.
Despite the rapid growth and evolution of supported employment programs, there was strong evidence in Maryland that the vocational accomplishments of supported employees could be substantially enhanced. Efforts to improve the provision of vocational services have, by and large, relied on transferring increasing amounts of control over the types of vocational goals that are selected, and the services that are provided, to those individuals who are receiving the services. These efforts are usually subsumed under closely related and encompassing terms, such as self-determination, empowerment, and choice. Underlying each of these approaches is the belief that giving individuals with disabilities greater control over their vocational (and other) services will improve outcomes. The soon to be implemented “Ticket to Work” program by the Social Security Administration also embodies this philosophy.
Although some individuals with disabilities are perfectly capable of making their own decisions with respect to jobs and services, many others, particularly among persons with mental disabilities, will require assistance, usually from trained vocational workers. However, as shown by the Maryland Survey, many improvements in the service system will need to be implemented before programs to empower persons with disabilities to make their own decisions will be successful.
Clearly, both the number and quality of jobs available to individuals with disabilities will need to be expanded if the ability to choose vocational goals is to be meaningful. Even if more aggressive campaigns open up more jobs, the choices available to individuals with disabilities will remain limited unless greatly improved systems of transporting them are developed. In fact, one way to expand the number and quality of jobs is to improve transportation systems for individuals with severe disabilities.
The ability of individuals with disabilities to exercise choice effectively will often depend upon the skill of a vocational worker to assist them to make a meaningful appraisal of potential jobs and the services and assistance that will be needed to carry out the jobs. Service providers must be given the resources with which to recruit and retain motivated and skilled vocational employees. As shown by the Maryland Survey, they will invariably require training in a wide variety of areas, including many in which available training is currently either inadequate or nonexistent. To assure the relevance and usefulness of training, all planners of training programs should be subject to follow-up field-based evaluation to determine whether the training actually improved the way in which the vocational employees assist individuals with disabilities. Asking training session participants to complete a short questionnaire evaluating the session immediately after its completion falls far short of the rigorous evaluation that is needed.
The Maryland Survey has provided important insights into the effectiveness of supported employment services and the issues that impede the most efficient provision of these services. There are, however, many vital questions about the issues raised in this survey that could not be fully addressed, partly because (a) of the limitations imposed by summary data by service providers and (b) the data that would be needed to explore these issues further were not collected. In particular, the reasons for the wide variations among service providers in effectiveness in providing supported employment need to be thoroughly explored. These reasons will almost certainly involve all of the issues that have been raised above and many more (e.g., the costs of providing services).
The identification of the reasons for variations in effectiveness must be evidence-based. To do this, all state and local programs funding supported employment should develop and maintain rigorous information systems that not only contain indices of performance (e.g., hours worked, earnings per hour, wait lists), but also data needed to identify program deficiencies (turnover rates, differences among providers, consumer complaints). In order to be maximally useful for meaningful research into the effectiveness of efficiency of supported employment, four conditions must be met: (a) the database must contain information on each individual being assisted (summary data have too many limitations); (b) a sampling procedure for obtaining additional supplementary information for specific research purposes should be in place; (c) the system should include not only supported employment, but also facility-based and nonsupported employment; and (d) different funding agencies should use uniform definitions of variables, whenever possible, with the goal of developing a unified information system. In addition, performance data collected by the system should be broken down by service provider and published to assist individuals with disabilities to make informed choices as to which one(s) they should obtain services from.
Maintaining a public information system and identifying those best practices that cause some service providers to be more effective than others will provide ongoing stimulus for improving the provision of services. Lacking this information, improvement will be greatly delayed; in fact, we will not know whether improvement is being made at all.
We do not believe that much money and time should be invested in benefit–cost studies. Most such studies have been controversial and of dubious accuracy, and resources would be better used in seeking ways of improving programs rather than conducting benefit–cost studies of programs that themselves may have a long way to go before achieving their maximum effectiveness.
NOTE: The author thanks the Maryland Association for Persons in Supported Employment and the Maryland Association of Community Services for sponsoring and supporting this survey and the executive board of the Maryland Association of Community Services for the insightful and helpful critiques of material presented in this paper. Special thanks go to Mitzi Francis and Joyce Lippman of the Montgomery County (MD) Arc, Colleen Garuder of the MD Administration on Developmental Disabilities, Mark Wurzbacher of Wurzbacher and Associates, and the anonymous reviewers who greatly improved the paper with their suggestions.
Author:Ronald W. Conley, PhD, Independent Consultant, 22820 Peach Tree Rd., Boyds, MD 20841. firstname.lastname@example.org.