Background 

Many efforts over the past decade have focused on developing quality improvement and safety curricula for residents. Sponsoring institutions have encountered challenges aligning resident projects with institutional quality and safety priorities, engaging faculty mentors, and securing support for resident initiatives from executive leadership.

Objective 

We developed a small grants program to support resident-led change projects intended to improve the clinical learning environment. We assessed program acceptability to residents and faculty, impact of program structure in supporting successful change projects, and program feasibility and financial sustainability.

Methods 

Program acceptability was assessed through a review of resident participation. Three aspects of resident change project success were considered: (1) accomplishment of stated aims; (2) institutional change beyond the end of grant funding; and (3) academic publication or presentation. The impact of program structure on project success was assessed through a review of submitted end-of-year narrative reports.

Results 

The Award Selection Committee has given 41 awards to 44 residents over 4 years, engaging 21% (44 of 213) of residents. Seventy-one percent of projects (29 of 41) produced changes that continued beyond the grant year, and 46% (19 of 41) produced an academic publication or presentation. At the end of the grant period that funded the program's initial 3 years, the chief executive officer elected to continue program funding.

Conclusions 

A small grants program supporting resident-led change projects intended to improve the clinical learning environment is acceptable to residents and faculty, feasible to administer, and sustainable with support from institutional senior leaders.

What was known and gap

Many sponsoring institutions have focused on developing quality improvement and safety curricula for residents, but have had difficulty aligning resident projects with institutional priorities and getting support from executive leadership.

What is new

A small grants program to support resident-led change projects intended to improve the clinical learning environment in 1 sponsoring institution.

Limitations

Success and sustainability of projects measured with qualitative assessments. Residents self-reporting of learning may have led to social desirability bias.

Bottom line

The program was acceptable to residents, faculty, and institutional senior leaders, and presents a feasible strategy for alignment between resident and institutional priorities.

The Institute of Medicine first highlighted challenges with patient safety and quality in US hospitals nearly 2 decades ago.1,2  The Accreditation Council for Graduate Medical Education (ACGME) introduced the Clinical Learning Environment Review (CLER) program in 2013 as a part of its response to these reports.3 

The ACGME accreditation processes hold graduate medical education (GME) leadership accountable for compliance. For the CLER program, the ACGME made the strategic decision that improvements in CLER focus areas (patient safety, health care quality, care transitions, supervision, well-being, and professionalism) necessitate engagement of institutional executive leadership.4 

Over the past decade, many efforts have focused on developing quality improvement (QI) and safety curricula for residents.5,6  Many sponsoring institutions have encountered challenges aligning resident projects with institutional quality and safety priorities, engaging faculty mentors, and securing support for resident initiatives from executive leadership. Although other institutions have developed small grants programs to provide resources for resident-led QI projects, published reports focus almost exclusively on small grants initiatives to engage faculty.7,8 

We developed a small grants program to support resident-led change projects aligned with institutional goals. We assessed (1) the feasibility and acceptability of the program; (2) the impact of the program's structure in supporting successful resident-led institutional change projects; and (3) the sustainability of the initiative through ongoing financial support from institutional leaders.

Setting and Participants

The Cambridge Health Alliance (CHA) is an integrated public health care system in the greater Boston, Massachusetts, area with academic affiliations to Harvard Medical School and Tufts University. The institution houses 7 ACGME-accredited programs that train 101 residents in internal medicine, family medicine, adult psychiatry, child psychiatry, consultation-liaison psychiatry, geriatric psychiatry, and a transitional year internship. Efforts to assess and improve the clinical learning environment at CHA are coordinated by the associate director of GME for quality and safety and the CLER Program Steering Committee, which comprises core quality and safety faculty from each training program.

Logic Model

The figure illustrates the aims, key activities, and proposed mechanisms that the small grants program contributes to creating intended outcomes. Through leading change projects, residents develop an appreciation for the complexity of the systems in which they work and emerge with the competencies necessary to engage in health system improvement.

figure

CEO-CLER Innovation Grant Program Logic Model

Abbreviations: CEO, chief executive officer; CHA, Cambridge Health Alliance; QI, quality improvement.

figure

CEO-CLER Innovation Grant Program Logic Model

Abbreviations: CEO, chief executive officer; CHA, Cambridge Health Alliance; QI, quality improvement.

Close modal

We intended a similar transformation for faculty, who develop their own knowledge and skills through relationships with residents and their change projects. Ideally, experience with resident-led projects empowers and equips faculty as change agents in their own practice environments.

In keeping with the CLER mandate, we also endeavor to influence institutional leaders. The chief executive officer (CEO) decision to allocate funds to the program raises his or her level of engagement with residents. Seeing positive change initiated by residents helps leaders view those residents as institutional assets. The investment may create a virtuous cycle that motivates leaders to create and sustain a healthy clinical learning environment.

Intervention

In academic year 2014–2015, CHA's CEO authorized an annual commitment of $20,000 to fund a small grants program enabling residents to lead change projects intended to improve the clinical learning environment. The annual grant cycle begins in July with a request for proposals to all graduate-level clinical residents. Submissions can include a request for up to $2,000 of financial support. Members of the CEO-CLER Innovation Grants Committee review initial proposals and provide detailed formative feedback. Revised final proposals include a project charter with a timeline and budget and a signed faculty mentor agreement. The committee reviews submissions and makes final funding decisions in September using an evaluation rubric described in the box.

box CLER Innovation Grants Program Application Evaluation Criteria
  • Likely positive impact on patients, families, and staff

  • Alignment with grants program priorities and institutional strategic direction

  • Feasibility of successful implementation within constraints of time and money

  • Engagement of and support from relevant stakeholders

  • Appropriateness of budget request and likelihood of project sustainability

  • Diversity of projects and represented graduate medical education programs within the annual grant program's overall portfolio

Grant recipients receive invitations to 2 QI methods and program evaluation workshops and access to a fixed number of hours of individual technical assistance from a community-based research institute affiliated with the health system. They prepare posters describing their work for an internal academic poster session and final reports summarizing key accomplishments and lessons learned. The request for proposals, the application form, and the template for the final report are available as online supplemental material.

Evaluation

We assessed program acceptability to residents and faculty through a review of the number of participating residents, self-reported resident learning, resident use of provided technical assistance, and resident suggestions for program improvement. We noted 3 characteristics of project success: (1) accomplishment of originally stated aims; (2) sustainability of the change effort beyond the grant year; and (3) academic publication or presentation. Data for the assessment are obtained from submitted end-of-year narrative reports and interviews with program directors and resident project faculty mentors.

The CHA Institutional Review Board declared this project exempt.

Over 4 years, the small grants program has given 41 awards to 44 different resident principal or co-principal investigators, engaging 21% of residents (44 of 213). Awards have been well distributed among our core training programs: 14 grants with resident principal investigators from internal medicine (34%), 11 from adult psychiatry (27%), 8 from child psychiatry (20%), 6 from family medicine (15%), and 3 (7%) from our transitional year internship and psychology training program. The program has grown annually over the 4-year period, from 14 submissions and 9 funded grants in year 1 to 21 submissions and 13 funded grants in year 4. Table 1 describes the funded projects mapped to CLER domains.

table 1

Innovation Awards Categorized by Clinical Learning Environment (CLER) Domain

Innovation Awards Categorized by Clinical Learning Environment (CLER) Domain
Innovation Awards Categorized by Clinical Learning Environment (CLER) Domain

Although assessing success of individual projects is subjective, the review of final reports revealed that all but 2 funded projects accomplished stated aims to a meaningful degree. Seventy-one percent of projects (29 of 41) produced change that continued beyond the grant year. Nearly half of projects (46%, 19 of 41) produced academic publications or external presentations; approximately the same number (49%, 20 of 41) were successful in leveraging additional funds from the institution or outside sources.

Themes from final reports support the program's logic model and suggest that resident participants are developing QI competencies and building a sense of their role as change agents. Many residents reflect on the complexity of creating institutional change and the need to adapt to unanticipated external circumstances. Nearly half of the final reports (46%, 19 of 41) describe the importance of relationship-building and engaging key stakeholders. Several reports express gratitude for the program's investment in residents and enthusiasm to start creating positive institutional change.

Two themes emerged in response to queries about opportunities for improvement in the CEO-CLER Innovation Grants Program. First, change projects invariably take longer than anticipated; many projects requested an earlier timeline for award determination and the opportunity to extend funding beyond the grant year. Second, many responses noted that logistics for dispersal and flow of funds within the institution was cumbersome.

Two-thirds of participants have availed themselves of technical assistance, most commonly seeking help for project evaluation, data collection and analysis, and poster preparation. Several responses reported difficulty engaging technical assistance secondary or limited understanding about available assistance and demanding resident schedules.

Resident reports describe appreciation for faculty investment and participation. Over the 4-year period, a core faculty group has emerged who served as mentors for multiple projects. Faculty support for the program is also evidenced by a growing number of faculty using the program to recruit resident participants for their own change projects. Faculty mentors have also elected to participate with their resident mentees in technical assistance workshops.

Discretionary funds authorized by our CEO for this program came initially from an undesignated Arnold P. Gold Foundation grant. After 4 years, when external funding was no longer available, our institutional leaders were sufficiently convinced of the program's value to continue funding.

Discussion

Over the course of 4 years, 21% (44 of 213) of residents at CHA participated in the small grants program to support resident-led change projects. The increasing number of submissions suggests the program is acceptable to residents and faculty. Because the program has led to project success and alignment with institutional priorities, institutional leaders have elected to provide ongoing financial support.

Researchers have noted the challenge of creating bidirectionality in resident engagement in systems improvement—linking “bottom-up” change projects prioritized and initiated by residents and “top-down” institution-mandated agendas.9,10  Some institutions have introduced pay-for-performance incentives into resident contracts to encourage participation in institutional change agendas.11  Our small grants program provides a pragmatic mechanism for accomplishing this bidirectionality. Although residents identify and shape proposals in their own areas of concern, the formal request for proposals permits us to name specific areas of strategic importance to the institution. Feedback on first iterations of proposals encourages applicants to revise drafts in partnership with key institutional leaders and stakeholders. Alignment with institutional priorities is heavily weighted in the selection criteria.

While others have reviewed alternative approaches to engage residents in quality and safety projects more generally,12  our small grants program is framed with the aim of improving the clinical learning environment. This framing has enabled a broad focus on areas of resident concern, including attention to resident well-being and has invited residents to assume relevant leadership responsibility in their own practice settings and training programs.

Principles of emergent design have proven critical to the program's success. The ways in which our program has evolved over its 4 years may inform others who wish to embark on a similar effort (see table 2 for enumerated adaptations).

table 2

Adaptations in Clinical Learning Environment Review (CLER) Innovation Awards Program Design

Adaptations in Clinical Learning Environment Review (CLER) Innovation Awards Program Design
Adaptations in Clinical Learning Environment Review (CLER) Innovation Awards Program Design

The program is feasible to implement in a variety of contexts. We were fortunate to have initial external grant support, which permitted time to develop and demonstrate the program's value to institutional leaders. In addition to the financial investment, the program has benefited from administrative support to organize the application process, dispense funds, and manage communication. The program also requires time and effort from the associate GME director, the CEO-CLER Innovation Awards Committee, faculty project mentors, and other staff.

Financial support for resident initiatives is crucial to the program, but not necessarily through purchasing power. The money—and the discipline associated with applying for funding and budgeting a project—engages residents and faculty, focuses resident plans, and communicates symbolic institutional support for resident endeavors. Often, substantial proportions of budgeted money remain unspent even though projects achieved their stated goals.

This program evaluation is limited to review of program participation, descriptive estimates of program acceptability and feasibility, qualitative assessments of success and sustainability of individual projects, academic output, and resident self-reports of learning. A more rigorous evaluation design would be needed to determine the global impact of the program on patient outcomes or resident competencies. We have not sought to study the proposed mechanisms for change outlined in the logic model and do not have data describing the longer-term impact of these change projects on the culture or career trajectories of award recipients. The small grants program is an elective opportunity for interested residents and serves as a complement to other required didactic and experiential curricula in quality and safety. Efforts to understand the perspectives of residents and faculty who choose not to participate may be instructive.

A small grants program supporting resident-led change projects to improve the clinical learning environment is acceptable to residents, faculty, and institutional senior leaders. The program presents a feasible strategy for creating engagement and alignment between resident and institutional priorities and provides a scaffold for enhancing the likelihood of change project success.

1
Kohn
LT,
Corrigan
JM,
Donaldson
MS,
eds
.
To Err is Human: Building a Safer Health System
.
Washington, DC
:
Committee on Quality of Health Care in America, Institute of Medicine, National Academies Press;
1999
.
2
Institute of Medicine
.
Crossing the Quality Chasm: A New Health System for the 21st Century
.
Washington, DC
:
Committee on Quality of Health Care in America, National Academies Press;
2001
.
3
Nasca
TJ,
Weiss
KB,
Bagian
JP.
Improving clinical learning environments for tomorrow's physicians
.
N Engl J Med
.
2014
;
370
(
11
):
991
993
. doi:.
4
Weiss
KB,
Bagian
JP,
Nasca
TJ.
The clinical learning environment: the foundation of graduate medical education
.
JAMA
.
2013
;
309
(
16
):
1687
1688
. doi:.
5
Wong
BM,
Levinson
W,
Shojania
KG.
Quality improvement in medical education: current state and future directions
.
Med Educ
.
2012
;
46
(
1
):
107
119
. doi:.
6
Patow
CA,
Karpovich
K,
Riesenberg
LA,
Jaeger
J,
Rosenfeld
JC,
Wittenbreer
M,
et al.
Residents' engagement in quality improvement: a systematic review of the literature
.
Acad Med
.
2009
;
84
(
12
):
1757
1764
. doi:.
7
Nieman
LJ,
Kelliher
GJ.
Stimulating medical education research through small grants
.
Acad Med
.
1991
;
66
(
10
):
601
602
.
8
El-Sawi
NM,
Sharp
GF,
Gruppen
LD.
A small grants program improves medical education research productivity
.
Acad Med
.
2009
;
84
(
suppl 10
):
105
108
. doi:.
9
Johl
K,
Grigsby
RK.
Engaging learners in health system quality improvement efforts
.
Acad Med
.
2017
;
92
(
5
):
593
597
. doi:.
10
Philbert
I.
The competencies: the ACGME and the community in 2008 and beyond
.
ACGME Bull
.
September 2008.
.
11
Bischoff
K,
Goel
A,
Hollander
H,
Ranji
SR,
Mourad
M.
The Housestaff Incentive Program: improving the timeliness and quality of discharge summaries by engaging residents in quality improvement
.
BMJ Qual Saf
.
2013
;
22
(
9
):
768
774
. doi:.
12
Johnson Faherty
L,
Mate
KS,
Moses
JM.
Leveraging trainees to improve quality and safety at the point of care: three models for engagement
.
Acad Med
.
2016
;
91
(
4
):
503
509
. doi:.

Author notes

Editor's Note: The online version of this article contains the request for proposals, application form, and final report template.

Competing Interests

* These authors are considered co–senior authors.

Funding: The authors received funding from the Arnold P. Gold Foundation.

Conflict of interest: The authors declare they have no competing interests.

Supplementary data