Abstract

The application of scientific data in the development and implementation of sound public policy is a well-established practice, but there appears to be less consensus on the nature of the strategies that can and should be used to incorporate research data into policy decisions. This paper describes the promise and the challenges of using research evidence to inform public policy. Most specifically, we demonstrate how the application of a large-scale data set, the National Core Indicators (NCI), can be systematically used to drive state-level policy decisions, and we describe a case example of one state's application of NCI data to make significant changes to its Intellectual and Developmental Disabilities waiver. The need for continued research in this area is highlighted.

In the United States and many other countries, science and technology are widely seen as essential to the nation's ability to compete economically and to achieve national social, educational, and health care goals. The application of scientific data in the development and implementation of sound public policy is a well-established practice. Policy makers, like elected officials, are expected to be accountable, to explain the motives for their actions, and to base public policy decisions on information that can be understood and validated by others. Ideally, such information is based on an impartial assessment of conditions and not designed to favor one interest group over another. Scientific evidence differs from an unexamined “common sense” understanding of particular events or issues and from knowledge based on untested “faith” in that it is generated through a process that invites critical review and external scrutiny. Scientific evidence derives from a specific approach or methodology that is transparent and involves identifying an issue or question, accumulating empirical and measurable data, evaluating the issue in light of the information gathered, and drawing conclusions on the implications of the findings for policy and practice, utilizing “systematic and rule-governed efforts that guard against self-deception—against believing something is true because one wants it to be” (National Research Council, 2012, p. 3).

The use of scientific evidence as a basis for determining the effectiveness of national policy has been a mainstay in the federal government for decades. The Study of Equality in Educational Opportunity, commissioned in 1966 by the United States Department of Health, Education, and Welfare to assess the impact of the Civil Rights Act of 1964, is frequently referenced as the point at which both state and national policy makers began to actively turn to social science as a basis for public policy development (National Research Council, 2012). The trend toward data-informed policy has only increased since that time. A memorandum issued by the President's Office of Management and Budget (Zients, 2012), titled Use of Evidence and Evaluation in the 2014 Budget, instructed federal agency department heads to include in their fiscal year 2014 budget submissions reference to the use of evidence and evaluation with respect to administrative data, new technology, resource allocation, grant making, and other activities, noting that, “Where evidence is strong we should act on it. Where evidence is suggestive, we should consider it. Where evidence is weak, we should build the knowledge to support better decisions in the future” (p. 1).

Few dispute the role that scientific evidence can play in policy development, but there appears to be less consensus on the nature of the strategies that can and should be used to incorporate research data into policy decisions. Many authors describe the process through which information and data are used to develop and implement public policy not as a logical progression or series of steps, but rather as a series of nonlinear, recursive activities that take place in a dynamic environment influenced by several individual, institutional, and systemic factors all interacting at the same time (Davies, 2005; Weiss, 1977). Part of the problem is that the production of evidence through scientific research and the development of public policy are conducted in two very separate environments, utilizing different methodologies to achieve their ends, and frequently are structured to accomplish separate purposes, goals, and outcomes (Brownson, 2009; Brownson, Royer, Ewing, & McBride, 2006; Choi et al., 2005; National Research Council, 2012). As a result of this disparity, highly relevant and persuasive data gathered and analyzed through a structured methodology may not find their way into policy deliberations at the state or federal levels. While the concept of evidence-based policy is robust, it is important to note that the dynamic nature of the policy-making environment is such that even the best, most persuasive data can be ignored in the face of political or even personal priorities.

The focus of this report is on the description and application of evidence-based policy in the design and delivery of services and supports to individuals with intellectual and developmental disabilities (IDD). We will review the nature and use of evidence in policy context, descriptions of the meaning of evidence-based policy, and considerations regarding the use of data and evidence in state policy and practice and will offer a case example of the application of large-scale data sets to inform state policy and practice.

Nature and Use of Evidence in Policy

Several authors have written on the nature and use of evidence in public policy and in disability and rehabilitation policy in particular. Evidence is defined broadly, acknowledging the dynamics of the environment within which policy is conceived and developed. Johnston et al. (2009), for example, have described evidence as, “the knowledge that connects research to practice,” involving the use of the “best available evidence”—integrated with clinical expertise and the values and experiences of people with disabilities and other stakeholders—to guide decisions about clinical and community practices (p. 2). The focus on making use of the best available evidence rather than the best “possible” information or the most exacting experimental design as a basis for policy decisions recognizes that within the public policy context a strong case frequently can be made based on information gathered through methods that are less rigorous than those relying on the use of randomized control group trials (Dijkers, 2009). The broader definition of evidence for policy use also reflects an understanding that the politics of decision making force a nonlinear approach that is characterized by communication and interaction among a diverse group of stakeholders, each of whom may bring their own evidence and perspectives to the process (Black & Donald, 2001; Brownson, 2009; Davies, 2005).

Dunworth (2008), in a paper published by the Urban Institute, wrote that evidence-based policy seeks to address three questions: “What exactly is the problem? What are the possible ways to address the problem? And what are the probable impacts and costs of each?” (p. 1). The author cautions that important scientific evidence is but one of many factors that enter into policy-related discussions. And furthermore, the comparative effectiveness of alternative response strategies may have less impact on policy than political and social values held by individual policy makers who might argue toward one course of action over another. Research evidence may not be used in policy because it is seen as too narrowly focused or irrelevant and not addressing the policy issue at hand because the information is complex and requires interpretation, addresses aspects of a separate population group that may not appear to be generalizable, or reveals findings that are contentious and open to interpretation (Black & Donald, 2001). Also, policy makers may not be operating in an environment that is open to change regardless of the nature of data that are presented. A recent report on the application of government databases in public policy making underscored these barriers and the need for both researchers and policy makers to fully understand the implications of the political and social context within which the data will be used and the dynamics of that use (Helbig, 2012).

Other authors have noted that the use of evidence in policy can be understood in terms of a continuum across three domains: process, focusing on approaches to increase the likelihood of policy adoption; content, identifying specific policy elements that are likely to be effective; and outcomes, understanding and communicating the potential policy impact of the information (Brownson, 2009). At an operational level, steps related to the actual use of scientific information in policy include identifying the problem or specific issues to be addressed, selecting target groups, assessing the magnitude or importance of the issue to be addressed, reviewing alternative policy responses, estimating anticipated consequences and alternatives, explaining the relevance of the issue and need for change, and evaluating the results after implementation (European Commission Directorate General for Research, 2010; National Research Council, 2012).

Using Data and Evidence in State Policy

The Importance of the Policy Context

State agencies with statutory responsibility for delivering services and supports to people with IDD, like all public human services agencies, are confronted with a daunting array of economic, regulatory, financial, operational, and workforce challenges. Although current indicators of the health of the national economy suggest that the nation is slowly rebounding from the Great Recession, revenues in a majority of states remain at or below 2008 levels. In fiscal year 2012, for example, according to the Center for Budget and Policy Priorities, 42 states closed budget shortfalls in excess of $103 billion, primarily (76%) through cuts in state spending on human services and programs rather than through increased revenues (12%). It is important to note that such reductions were made at a time when demand for services from significant numbers of individuals who were unemployed and in need of public assistance was still high. Accumulated state revenue losses since the beginning of the recession in 2007 are so large that even if state economies rebounded at a sustained growth of 8.3% states would not reach the rate of economic expansion experienced between 1998–2008 until 2019 (Oliff, 2012).

At the federal level, fiscal imbalances created by a combination of tax cuts and expanding government obligations to pay for the national defense expenditures, public entitlements to health care, and interest payments on the national debt are producing growing annual expenditure shortfalls and obligations. The economic picture is complicated by the changing demographical picture in the United States as members of the “baby boom” generation reach age 65 years and beyond. According to estimates from the U.S. Census, the number of Americans 65 years of age and older will more than double from 40.2 million in 2010 to 88.5 million in 2050 (Greyson & Velkoff, 2010). According to the 13th Annual Transamerica Retirement Survey of Workers published in 2013, 10,000 baby boomers qualify every day for Social Security benefits. The challenge of supporting increasing numbers of older citizens is exacerbated by data projecting that the number of workers supporting each Social Security beneficiary will decrease from approximately 2.9 workers per enrollee in 2010 to 2.1 workers per enrollee in 2035 (Board of Trustees, Federal Old-Age and Survivors Insurance and Federal Disability Insurance Trust Funds, 2011). These data have significant implications for human services in general and IDD services in particular. According to figures released by the U.S. Census Bureau Populations Division Interim State Population Projections (2005), the growing numbers of adults 65 years of age and older who can be expected to require assistance in the years ahead surpassed the number of workers who traditionally are employed to provide such care (females aged 25–44) in 2010. This gap is projected to worsen in the decades ahead.

The national context within which state IDD policy is developed and implemented is further complicated by significant changes in the array of services and supports available to people with disabilities through the Medicaid program. Medicaid is operated as a state-federal partnership in which state expenditures for qualifying programs and beneficiaries are matched by federal medical assistance funding at a rate that varies between 50%–75% as determined by a formula based on economic conditions and other variables within each state. A federal medical assistance percentage (FMAP) rate of 55%, for example, means that the federal government covers 55% of the total qualifying program costs under the Medicaid program. The state is responsible for the remaining 45% of the costs. Because the preponderance of publicly financed services and supports furnished to people with IDD are offered through state Medicaid programs, changes in Medicaid policy and practice can profoundly influence state policy. Increasingly, states are utilizing managed-care strategies to organize the financing and delivery of long-term supports and services to elders and people with disabilities. Many are turning to large health care corporations with limited experience in the design, management, and provision of lifelong supports to people with intellectual, developmental, and other disabilities to administer, fund, and provide services.

The Affordable Care Act (ACA) includes several provisions to increase the number of Medicaid-funded options for states interested in expanding support for home- and community-based services (HCBS) for people with IDD, older Americans, and individuals with disabilities. The Money Follows the Person Demonstration sets aside $2.25 billion to fund projects in 47 states, resulting in the transition of individuals residing in institutional programs to community settings through 2016. The State Balancing Incentive Payment Program is designed to support community service development by providing enhanced federal matching funds of up to 5% to states that spent less than half of their total Medicaid expenditures on noninstitutional services in 2009. The Community First Choice Option under Section 1915(k) of the Social Security Act qualifies participating states to receive a 6% increase in their FMAP funding rates for HCBS services. Another Medicaid state plan option under Section 1915(i) of the Social Security Act allows states to target benefits to particular groups of individuals with disabilities, modify income eligibility criteria, and furnish services statewide on an entitlement basis. Additional federal initiatives under the ACA provide three-year planning and implementation grants to states to support the development of innovative approaches for integrating care for individuals who are dually eligible for Medicaid and Medicare. This population is thought to comprise the most vulnerable, highest need, and highest cost beneficiaries in the U.S. health care system. Finally, the Health Home State Medicaid Plan Option provides funding for states to develop comprehensive systems of care coordination for Medicaid beneficiaries with two or more chronic illnesses or disabilities.

State Challenges in the Use of Evidence-Based Policy

The current highly dynamic political, regulatory, and increasingly commercial environment presents significant challenges to state policy makers charged with responsibility for administering publicly funded services for people with intellectual and developmental disabilities. Randomized control group trials, the “gold standard” of evidence-based practice in the medical health care field, rarely produce results that reflect or factor in the economic, political, and social conditions within which public policy is implemented (Black & Donald, 2001; Lakin 2012). Indeed, in the real-world environment within which state IDD policy operates, it is seldom possible to create and fund program and service alternatives that furnish specific supports to one group of eligible individuals (treatment) and not to another group of similarly eligible individuals (control).

Additional challenges stem from the nature of social research findings, which seldom produce unequivocal results, with evidence that can be complex, ambiguous, and contradictory (Dunworth, 2008) without a clear link between inputs and outcomes. As a result, policy makers must make judgments based on the preponderance of good evidence currently available, utilizing a wide range of research methodologies, including detailed qualitative data, comprehensive summaries, and authoritative interpretations of research findings (Lakin, 2012).

The parameters of evidence necessary for policy development are much broader than those required for narrowly defined clinical practice, and empirical evidence typically plays a small role in policy formulation (Lareau & Walters, 2010). Research data constitute but one more variable that must be taken into consideration in the political context within which policy is formed (McDonnell, 1988). Policy entrepreneurs with vested interests in promoting particular service delivery approaches, such as managed care, as well as advocates, business interest groups, labor unions, and other stakeholders play key roles in the development and direction of public programs and initiatives. Policy makers must weigh the perspectives of each, separating unexamined opinion against conclusions drawn from evidence that has been carefully gathered, evaluated, and validated. Evidence can inform the decisions that policy makers are called upon to make and assist them in achieving the outcomes that are intended if mechanisms are established to ensure that research has a place at the table.

NASDDDS–AUCD Evidence-Based Policy Initiative

In 2009, the National Association of State Directors of Developmental Disabilities Services (NASDDDS) and the Association of University Centers on Disabilities (AUCD) entered into a partnership to foster research on key topical areas, resulting in the production of evidence to support policy making. NASDDDS defines evidence-based policy as “the responsible application of the best available evidence in the design, administration and reform of programs, services and supports in a manner consistent with achieving independence, productivity, inclusion and self-determination for individuals with developmental disabilities” (National Association of State Directors of Developmental Disabilities Services, 2009, p. 1). By establishing an active working relationship with AUCD and its network of university researchers and scholars, NASDDDS has outlined a process for building a solid basis of knowledge to inform the development of policy that is consistent with national goals for people with IDD articulated in public commitments in federal laws, court cases, and regulations (Developmental Disabilities Assistance and Bill of Rights Act Amendments of 2000), including

  • Increasing self-determination and personal control in decisions affecting people with disabilities and their families;

  • Providing opportunities to people with disabilities to live and participate in their own communities

  • Improving quality of life for individuals and families as they define it for themselves

  • Supporting families as the most important and permanent unit of development, protection, and lifelong assistance to persons with disabilities

  • Investing in each individual's developmental potential and capacity to contribute in age-related roles as productive, respected community members

  • Assuring access to sufficient, high-quality health and social supports to protect each person's health, safety, rights, and well-being.

The NASDDDS–AUCD notion of “evidence-based policy” is drawn from the broadly accepted concept of “evidence-based practice” as applied in medical and clinical settings. The evidence-based policy initiative reflects the understanding that policy makers have a professional responsibility to acquire and apply the best available knowledge in execution of their professional roles and underscores the duty of policy makers to use sound evidence in their efforts to create, implement, and improve publicly financed services and supports for persons with IDD. NASDDDS and AUCD acknowledge the fluid environment within which public policy is crafted and the reality that political and social factors must be considered along with evidence on best or promising practices (Lakin, 2012).

The Role of Evidence-Based Policy in Creating System Change

Evidence is an essential element in creating and sustaining policy change, but evidence alone is not sufficient. Systems change also requires clear and visionary leadership with the commitment to introduce and sustain practices with evidence demonstrating their effectiveness (Hall, Butterworth, Winsor, Gilmore, & Meltzel, 2007). Also required are appropriate incentives to induce providers and other stakeholders to adopt necessary changes, training and technical assistance to enable practitioners to operationalize new strategies, demonstration projects to validate and publicize new evidence-based approaches, and information diffusion mechanisms to enable stakeholders to gain the necessary knowledge and skills to implement evidence-based changes (Hall et al., 2007). Evidence-based policy requires the direct participation of policy makers, researchers, subject matter experts, and others who care about the issue in an open dialogue throughout the research, data analysis, and dissemination process (Tyden, 2000).

National Core Indicators

The National Core Indicators (NCI) assessment tool documents the outcomes and performance of state IDD service-delivery systems. Developed in 1997 by NASDDDS and the Human Services Research Institute (HSRI), the program has expanded to include 35 states and 22 substate entities, such as counties and designated regions (NASDDDS, 2007). Participating states use a common set of surveys and data-collection protocols to gather information from a representative sample of program participants each year. The NCI survey gathers descriptive information on individuals receiving support, including demographics, level of functioning, diagnosis, etc. as well as data on the types of services received, setting size and funding, individual outcomes, personal choice, friendships, community involvement, employment, individual experience with services, and access to needed supports, among others.

To ensure that the NCI instrument validly reflected national and state goals for IDD services and the needs of state policy makers and other stakeholders, the NCI survey was constructed with detailed input from a Program Advisory Committee, including state agency staff and technical advisors. A pretest was conducted with a group of individuals with intellectual disability to establish face validity that highlighted problematic questions, confusing terminology, and phrases that required further explanation or definition. Alternative wording and additional changes were included within the final survey. Field-testing occurred in three states with inter-rater reliability tests yielding agreement of 92%–93%, and a lone test-retest of reliability resulting in 80% agreement (Smith, 2001). The NCI is used in a majority of states to inform policy development and improve the quality and responsiveness of services furnished to people with IDD.

Using Data for Policy: Kentucky Example

The following case example describes the effective application of evidence generated through the NCI program in the development and implementation of IDD policy in the State of Kentucky. The examination of the use of NCI data in policy was initiated as a part of the NASDDDS–AUCD Evidence-Based Policy Initiative. An earlier version of this case example was published in the Community Services Reporter (Sheppard-Jones, Hall, & Kleinert, 2011).

Kentucky's involvement with NCI began in 1998 with the Kentucky Department of Behavioral Health, Developmental and Intellectual Disabilities contracting with the Human Development Institute (HDI), Kentucky's University Center for Excellence in Developmental Disabilities, to oversee all project activities. HDI has provided a cadre of trained interviewers, the majority of whom have a disability, to conduct a yearly minimum of 400 interviews with adults receiving at least one service in addition to case management. Participants are randomly selected each year from the universe of individuals receiving state-funded services in Kentucky with intentional oversampling in some years to ensure greater representation of particular subgroups of consumers (e.g., individuals supported by the Supports for Community Living [SCL] waiver, the state's residential IDD waiver, or individuals transitioning from facility-based to community placements). In 2007–2008, we purposely sampled only individuals receiving SCL waiver services to get a more complete picture of both their life satisfaction and the quality of their life outcomes.

Establishing a Quality Improvement Committee

In 2010, the Kentucky National Core Indicators Quality Improvement Committee (QIC) was convened at the request of the department. The purpose of the group was to review NCI data and other relevant information in order to make recommendations related to the SCL waiver, and it included 13 committee members. Group members represented Kentucky NCI staff and self-advocate NCI interviewers, family members, university researchers, and Kentucky Division of Developmental and Intellectual Disabilities staff. The committee focused on Kentucky's NCI Consumer Survey data (2007–2008) and the Kentucky Core Indicators General Population Survey data (2007–2008) as the basis for recommendations.

The Kentucky General Population Survey was a telephone survey administered to a random sample of 515 Kentuckians and, as such, provided a unique comparison on many of the NCI indicators. The General Population Survey, administered via telephone biannually, mirrors the quality-of-life items that are asked through the NCI Consumer Survey, minus those items related to case management, support staff, or service delivery. Some items on the General Population Survey are identical in wording to the Consumer Survey, and others have been slightly reworded based upon pilot testing and their appropriateness for a telephone administration. This General Population Survey yielded an important measure in judging how good is “good enough.” This consideration is especially important in the case when being in the “average” range with other NCI states might simply reflect a paucity of life opportunities for individuals with IDD across all of the participating states and not be a true comparison of similar life outcomes for individuals without disabilities. The work of the QIC subsequently resulted in four recommendations, tied to specific areas in which Kentucky lagged behind national averages for participating NCI states as well as with the results of our General Population Survey.

Recommendations and Policy Steps Incorporated

Specifically, the committee noted these four areas: 1) employment, 2) health and exercise, 3) use of psychotropic medications, and 4) loneliness and friendships, in which Kentucky residential waiver recipients with intellectual disability fared significantly poorer than the national average for persons with IDD in NCI participating states. Whereas the committee's priority recommendations of improvement were based upon the 2007–2008 NCI data, for purposes of this article, we have chosen to use the most recently available NCI data (2010–2011) (NASDDDS and HSRI, 2012). We have used the most current available data for two reasons: 1) to give the reader a sense that the basis for our findings was not simply a year one “blip” but rather these patterns have held across years and 2) to emphasize the importance of carefully reviewing large-scale data sets with each cycle, always taking into account the most current information we have.

In the area of employment, Kentucky data for 2010–2011 indicated that only 8% of SCL waiver recipients had an integrated job (at least 20 hours per week at typical wages) in the community, compared to a national average of 16% for all NCI respondents. While both percentages are absolutely dismal, it was clear that for the vast majority of Kentucky waiver recipients, “active treatment” typically represented participation in day activity programs or some similar arrangement. Second, only 17% of Kentucky's SCL waiver recipients reported engaging in moderate physical exercise for at least 30 minutes three times per week, and the NCI average was 26% for 2010–2011. Third, 63% of the individuals sampled on Kentucky's residential IDD waiver were reported to take at least one mood, anxiety, behavioral, or psychotic medication, compared to a national average of 54% in the NCI sample for that year. Finally, there were substantial unmet needs in the area of loneliness and friendships. A full 60% of all individuals on our state SCL waiver reported feeling lonely, either often or sometimes, compared to a NCI average of 40% for the 2010–2011 data cycle. This compares to just 29.4% of Kentuckians without disabilities on this same loneliness measure in our General Population Survey. These data are further accentuated by the small percentage of Kentucky waiver recipients who reported having friends who were not staff or family (just 53% compared to an overall NCI average of 76%) in 2010–2011.

Kentucky State Policy Responses and Programmatic Initiatives

In this section, we provide the measures initiated by the Kentucky Department of Behavioral Health, Developmental and Intellectual Disabilities (KY DBHDID) to address each of the four priority areas targeted the Kentucky National Core Indicator Quality Committee.

Recommendation 1: Increasing integrated, community employment for Kentucky citizens receiving IDD waiver services

In response to this recommendation from the Kentucky NCI Quality Committee, the KY DBHDID has taken the following steps:

  1. Making employment a priority in all spoken or written communication and strengthening its collaboration with the state vocational rehabilitation agency with a memorandum of agreement on an identical hourly rate for supported employment. Trainings conducted by the department must consistently and clearly emphasize employment outcomes.

  2. Revising both of Kentucky's DD Medicaid waivers, including the state's residential (SCL) and nonresidential (Michelle P.) waivers, increasing the rate of hourly payment for supported employment to qualified service providers by nearly 100% to ensure high-quality outcomes of integrated employment. The department held over 60 focus groups and community forums statewide, inviting key stakeholders that included members of the legislature to support this policy change.

  3. Decreasing a more than adequate rate for day activity services by 11%, which may have actually served as a disincentive to the development of high-quality supported employment services. Although this part of the change was resisted, the department showed day activity service providers how the increase in funding rates for supported employment and community access would more than make up for decreased day activity service revenue if day activity service providers were willing to expand community access and supported employment opportunities.

Recommendation 2: Increasing opportunities for regular participation in health and wellness activities

We recognize that obesity rates for adults with developmental and intellectual disabilities are higher than the general population with the 2010–2011 Core Indicator data indicating that 67.3% of Kentuckians with IDD participating in the NCI Survey for that year were either overweight or obese. Responses by KY DBHDID to this recommendation have included

  1. Creating a new service, Community Access, to encourage citizens with IDD to become part of community life, not just during the day, but in the evenings and on weekends. This change was most welcomed by community providers as many were already attempting community participation with inadequate funding rates.

  2. Increasing both the rate of payment and the availability of funds for this new service.

  3. Implementing the Health Matters curriculum (Marks, Sisirak, & Heller, 2010)—the KY DBHDID and HDI provided, in collaboration with the University of Illinois University for Excellence in Developmental Disabilities, a train-the-trainer workshop with a combination of 25 disability-specific and generic agencies along with university researchers to promote health and wellness opportunities. Currently, a 19-agency pilot, using the Peer-to-Peer Health Messages curriculum (Marks, Sisirak, & Heller, 2010), is underway with results being used to build strategies for a subsequent statewide rollout.

  4. The funding, provided by our Kentucky UCEDD, for a statewide request for proposals to demonstrate successful models with 33 applications received and eight funded. These mini-grants were intended to help organizations bolster their particular interest and expertise areas and focused on a broad range of activities (e.g., early childhood fitness, inclusive summer camps, community gardening).

Recommendation 3: Decreasing the use of psychotropic medications

In order to decrease the percentage of individuals on our SCL waiver who are prescribed psychotropic medications, the following steps have been taken by the KY DBHDID:

  1. Implementing the Supports Intensity Scale (Thompson et al., 2004), Conflict Free Case Management, and a Community Guide option for those who choose to consumer direct as further means for increasing supports, insuring proper monitoring, and a greater voice for the individual. A key point of compromise was implemented to address significant resistance from providers to Conflict Free Case Management. This change allowed families to keep a current conflicted case manager (i.e., a case manager who works for the same agency that provides their loved one's services) while mandating Conflict Free Case Management when that particular case manager leaves the employment of the provider.

  2. Implementing a Health Risk Screening Tool and the KY Board of Nursing approved Medication Administration training curriculum to insure the correct administration of medication and training more than 120 provider agency nurses in the first year.

  3. Developing an “exceptional rate protocol” based on severity of need to ensure that adequate personal and community supports are provided individuals with challenging behaviors.

  4. Redefining Behavior Supports as clinical consultation with significant rate adjustments to encourage providers to use behavioral support as a less intrusive intervention whenever possible and appropriate.

  5. Adding a new waiver service Person Centered Coach to increase “life connections.”

Recommendation 4: Loneliness and friendships

Kentuckians with IDD reported having fewer friends (other than family and staff) than any other NCI surveyed state and reported being lonely more frequently than any other. The following steps have been taken by the KY DBHDID to address this lack of relationships outside of family and staff:

  1. Enhancing every payment rate and service that support individuals to be alongside others without significant disabilities in the community while decreasing payment to those services that isolate people.

  2. In collaboration with the University of Minnesota University Center for Excellence in Developmental Disabilities, launched a “Community Belonging” initiative. This project is starting with 10 agencies, which will, in turn, serve as mentors for other provider agencies. The intent of this initiative is to ensure that people are connected to their communities via relationships other than paid supports.

Assessing the Effectiveness of the Changes: Impact Upon Life Outcomes

The KY NCI QIC continues to meet to address the state's status on these four targeted indicators as well as Kentucky's status on the full set of NCIs. As this article goes to press, most of the changes we have outlined are just going into effect (or are taking effect over the course of the next year), and their impact should become visible in subsequent years' NCI reports. Without the clear charge for a group like Kentucky's QIC to regularly review and report on progress (to the State DD Agency, self-advocates, and families) as well as to make “mid-course” recommendations, it is likely that evidence-based policy change will remain elusive even with the best of intentions. Given the importance of these indicators not just for DD state policy leaders, but for families and self-advocates, it also becomes essential that reports be created in accessible language and charts; see http://www.belongingky.org/?page_id=19 for an example of how Kentucky is creating just such a report. It is noteworthy that policy makers often remark that they too prefer the use of accessible reporting formats in understanding how the data collected clearly relate to peoples' lives.

The Research Agenda Ahead

It is too early to determine if the broad policy changes Kentucky has mandated will result in improved outcomes for individuals with intellectual and developmental disabilities receiving state DD waiver services in that state. However, it is clear that true systems change is more than policy change. Rather, creating meaningful changes in life outcomes requires a broad set of strategies—evidence-based policy (albeit an essential tool) is but one of those strategies. As we have noted, Hall et al. (2007) identified the following elements as all playing an essential part: a) Clear and visionary leadership that takes into account evidence-based practices, b) the establishment of appropriate incentives for change, c) the training and technical assistance necessary to effect that change (based on evidenced-based practice), d) the demonstration of what is possible through the creation of pilot sites demonstrating evidence-based approaches, and e) a diffusion mechanism that enables agencies that have a stake in that change to gain the necessary knowledge and skill sets to sustain the change. Within the action steps we have noted above, Kentucky is addressing each of these five elements; we will use our ongoing NCI data cycles as a significant piece of our evidence evaluating the success of these efforts.

A second lesson learned is that evidence-based policy requires ongoing partnerships of trust between universities and state developmental disability agencies. Universities have, as a key part of their mission, the generation of new knowledge that can be directly applied to improving life conditions of their constituents. State developmental disability agencies have as their primary focus the use of the best available evidence to fashion and create service structures that enhance community presence, participation, and opportunity for the individuals they serve. Universities can play a significant role in creating credibility for policy makers (who are the lead architects for the systems) and with legislators who must fund those services. State agency and university partnerships must be designed to rigorously and systematically collect outcome data, analyze those data, and report them in sound and publicly accessible formats if the promise of evidence-based policy is to be realized.

Continuing challenges

The challenges to systematically use EBP to inform public policy are very real. First are political constraints, including vested interests in retaining the status quo (e.g., sheltered employment, adult activity programs). For example, the number of individuals with IDD receiving supported employment services grew only slightly between 1999 and 2010 (5.2%), and those participating in facility-based day programs increased by 23.4% in the same period (State Data.info, 2012) despite what we know about the uniformly positive life outcomes associated with a job in the community.

Even when there is clear leadership to create change, flat and declining state revenues as well as the growing pressures on state budgets to insure an adequate “safety net” of services for its citizens create barriers to change. These are further accentuated by the “start-up” costs of new services in the community or of the conversion of outdated services (e.g., sheltered workshops to integrated employment). Finally, advocacy groups, including families who may have been told by professionals in the past that congregate services really were the best choice for their son or daughter, may resist new service options despite even clear evidence that supports those options.

As states attempt to maximize the resources they do have, they are seeking to understand what constitutes the criterion of best available evidence—that is, “When is good, good enough”? As we have noted earlier, research in the social sciences seldom produced unequivocal results, and rather, we must accept a preponderance of good evidence. While the gold standard of research, clinical trials with random assignment, is, even under the best of circumstances, very difficult in real life, much of our nation's policy in other spheres of public policy (e.g., the use of seat belts, smoking bans in response to the impact of second-hand smoke) is based on just such a preponderance of good evidence. Evidence-based policy is a systematic and rigorous strategy to apply the best available evidence we have to improve the lives of the people we serve.

Acknowledgments

The preparation of this article was supported in part by the State of KY Department for Behavioral Health, Developmental and Intellectual Disabilities through a contract with the Human Development Institute.

References

References
Black
,
N
.,
&
Donald
,
A
. (
2001
).
Evidence based policy: Proceed with care
.
Brisitsh Medical Journal
,
323
(
7307
),
275
279
.
doi:10.1136/bmj.323.7307.275
Board of Trustees, Federal Old-Age and Survivors Insurance and Federal Disability Insurance Trust Funds
. (
2011
).
2011 Annual report of the the board of trustees, federal old-age and survisors insurance and federal disability insurance trust funds
.
Washington, DC
:
U.S. Government Printing Office
.
Brownson
,
R. C
. (
2009
).
Policy, politics and collective action: Understanding evidence-based public health policy
.
American Journal of Public Health
,
99
(
9
),
1576
1583
.
Brownson
,
R. C
.,
Royer
,
C
.,
Ewing
,
R
.,
&
McBride
,
T. D
. (
2006
).
Researchers and policymakers: Travelers in parallel universes
.
American Journal of Preventive Medicine
,
30
(
2
),
164
172
.
doi:10.1016/j.amepre.2005.10.004
Choi
,
B. C
.,
Pang
,
T
.,
Lin
,
V
.,
Puska
,
P
.,
Sherman
,
G
.,
Goddard
,
M
.,
Clottey
,
C
. (
2005
).
Can scientists and policy makers work together
?
Journal of Epidemiological Community Health
,
59
(
8
),
632
637
.
Davies
,
H
.,
Nutley
,
S
.,
&
Walter
,
I
. (
2005
).
Assessing the impact of social science research: conceptual, methodological and practical issues. A background discussion paper for ESRC Symposium on Assessing Non-Academic Impact of Research, May 2005,
24
p. .
Dijkers
,
M
. (
2009
).
When the best is the enemy of the good: The nature of research evidence used in systematic reviews and guidelines. NCDDR Task Force on Systematic Review and Guidelines
.
Austin, TX
:
SEDL
.
Dunworth
,
T. H
. (
2008
).
Beyond ideology, politics and guesswork: The case for evidence based policy. The Urban Institute
.
European Commission Directorate General for Research
. (
2010
).
Communicating research for evidence-based policymaking: A practical guide for researchers in socio-economic sciences and humanities. Communication Unit
.
Brussels
:
European Commission
.
Greyson
,
K. V
.,
&
Velkoff
,
V. A
. (
2010
).
The next four decades: The older population in the United States: Population estimates and projections. Current Population Reports
.
Washington, DC
:
U.S. Department of Commerce Economics and Statistics Administration, U.S. Census Bureau
.
Hall
,
A. C
.,
Butterworth
,
J
.,
Winsor
,
J
.,
Gilmore
,
D
.,
&
Meltzel
,
J
. (
2007
).
Pushing the employment agenda: Case study research of high performing states in integrated employment
.
Intellectual and Developmental Disabilities
,
45
,
182
198
.
doi:10.1352/1934-9556(2007)45[182:PTEACS]2.0.CO;2
Helbig
,
N. C
. (
2012
).
The dynamics of opening government data. University at Albany SUNY, Center for Technology in Government
.
Albany, NY
:
Research Foundation of State University of New York
.
Human Services Research Institute and National Association of State Directors of Developmental Disabilities Services
. (
2012
).
Consumer outcomes: Final report. 2010–2011 NCI adult consumer survey data
.
Johnston
,
M. V
.,
Vanderheiden
,
G. C
.,
Farkas
,
M. D
.,
Rogers
,
E. S
.,
Summers
,
J. A
.,
&
Westbrook
,
J. D
. (
2009
).
The challenge of evidence in disability and rehabilitation research and practice: A position paper. National Center for the Dissemination of Disability Research
.
Kentucky National Core Indicators
. (
2010
).
2010 recommendations report
.
Lakin
,
K
. (
2012
).
Overview. NASDDDS AUCD Evidence Based Policy Initiative
. .
Lareau
,
A
.,
&
Walters
,
P. B
. (
2010
).
What counts as credible research? Teachers College Record ID, (15915)
.
Marks
,
B
.,
Sisirak
,
J
.,
&
Heller
,
T
. (
2010
).
Health matters for people with developmental disabilities: Creating a sustainable health promotion program
.
Baltimore, MD
:
Paul Brookes
.
McDonnell
,
L
. (
1988
).
The politics of education: Influencing policy and beyond
.
In
D. C. S. H
. (pp.
Furman
,
19
39
).
Mahwah, NJ
:
Lawrence Erlbaum Associates
.
National Association of State Directors of Developmental Disabilities Services
. (
2009
).
NASDDDS Proposal to AUCD on Evidence Based Policy
.
Alexandria VA
:
NASDDDS Research Committee
.
National Association of State Directors of Developmental Disabilities Services & Human Services Research Institute
. (
2007
).
National core indicators: Ten years, twenty-six states and countless achievements
.
National Association of State Directors of Developmental Disabilities Services & Human Services Research Institute
. (
2012
).
NCI state report: Kentucky
.
National Research Council
. (
2012
).
Using science as public policy
.
In
K
.
Prewitt
,
T. A
.
Schwandt
,
&
M. L
.
Straf
(
Eds.
),
Committee on the Use of Social Science Knowledge in Public Policy, Division of Behavioral and Social Sciences and Education
.
Washington DC
:
National Academies Press
.
Oliff
,
P. M
. (
2012
).
States continue to feel recession's impact. Center for Budget and Policy Priorities
.
Available at www.cbpp.org
Research Committee
. (
2009
).
National Association of State Directors of Developmental Disabilities Services
.
Alexandria, VA
:
NASDDDS
.
Sheppard-Jones
,
K
.,
Hall
,
S
.,
&
Kleinert
,
H
. (
2011
).
Using large-scale data sets to inform state DD policy
.
Community Services Reporter
,
18
(
9
),
4
10
.
StateData.info
. (
2012
).
State intellectual and developmental disability agencies' service trends. Data Note No. 38, 2012
.
Smith
,
G
. (
2001
).
National core indicators project: Phase II consumer survey technical report
.
Boston, MA
:
Human Services Research Institute
.
Thompson
,
J
.,
Bryant
,
B
.,
Campbell
,
E
.,
Craig
,
E
.,
Hughes
,
C
.,
Rotholz
,
D
.,
Wehmeyer
,
M. L
. (
2004
).
Supports intensity scale: User's manual
.
Washington, DC
:
American Association on Intellectual and Developmental Disabilities
.
Tyden
,
T
. (
2000
).
Researchers learn research dissemination by doing
.
Science Communication
,
21
(
23
),
296
308
.
United States Census Bureau
. (
2005
).
Interim state population projections
.
Weiss
,
C
. (
1977
).
Research for policy's sake: The elightenment function of social research
.
Policy Analysis
,
3
(
4
),
531
546
.
Zients
,
J. D
. (
2012
).
Memorandum to the heads of executive departments and agencies: Use of evidence and evaluation in the 2014 Budget
.
Washington DC
:
Office of Management and Budget
.

Author notes

Authors:

Charles Moseley (e-mail: cmoseley@nasddds.org), National Association of State Directors of Developmental Disabilities Services, 113 Oronoco St., Alexandria, VA 22314, USA; Harold L. Kleinert, and Kathy Sheppard-Jones, Human Development Institute, University of Kentucky Lexington; Stephen Hall, Kentucky Department for Behavioral Health, Developmental and Intellectual Disabilities.