Swimming With Sharks: Teaching Residents Value-Based Medicine and Quality Improvement Through Resident-Pitched Projects

As the cost of health care continues to skyrocket, academic medical centers are increasingly investing in value-based medicine (VBM) as a means of increasing quality while decreasing costs.^1,2  Meanwhile, the Accreditation Council for Graduate Medical Education mandates that residents receive training in systems-based practice and practice-based learning and improvement.^3,4  The Clinical Learning Environment Review expects experiential learning in health care quality and patient safety (PS) and gives feedback to institutions to improve graduate medical education (GME) training.^5,6  Although national efforts, most notably the Veterans Affairs (VA) Chief Residents in Quality and Patient Safety program and the High Value Practice Academic Alliance, have made strides to bridge the gap between hospital investment in VBM and GME quality improvement (QI) training, curricular innovation is still needed.^7,8 

While didactic-based rotations are a common setting to teach health care value and cost-effectiveness, a particularly engaging and lasting way to teach QI and VBM may be through creating and implementing projects to reinforce and cultivate proficiency.^9,10  Inspired by the television show Shark Tank, several institutions have undertaken hosting venture capital–style pitch competitions to generate QI projects and educational innovation.^11–13  The format of the show lends itself to problem identification and value proposition articulation, which translates naturally into teaching about health care quality and value. None of these efforts at adapting a Shark Tank–style competition to hospital QI or GME education, to our knowledge, address the gap between GME training and resident involvement in project idea generation as front-line workers.^7,14 

We hypothesized that creating a new project-based curriculum focused on quality and value that culminates in a project pitch to hospital leadership would (1) increase resident confidence using essential QI tools and institutional data to solve systems-based value challenges, and (2) increase the likelihood of resident participation in future quality and value projects.

New York University School of Medicine's internal medicine residency program is a large university-based academic program that participates in the VA Chief Residents in Quality and Patient Safety program. Trainees rotate at 3 sites: a university hospital, a municipal referral hospital, and a VA hospital. Our program uses a “4 + 2” schedule (4 weeks inpatient + 2 weeks outpatient clinic). During postgraduate year (PGY) 2, residents have a mandatory 2-week quality improvement/patient safety (QI/PS) rotation during one of their outpatient blocks with 40 hours of dedicated nonclinical time over 2 weeks along with required ambulatory clinic. We repeated the curriculum in 3 consecutive blocks (over 6 weeks) each year with one-third of the residents participating in each block. During the first year (2017), only categorical residents participated in this curriculum due to scheduling limitations. During the second year (2018), all categorical and research/fast-track residents participated, as well as half of our primary care residents who chose to use elective time to participate. Faculty advisors were associate program directors or site directors within our residency program and did not receive any compensation for participation.

After identifying learning objectives and performing a gap analysis to determine educational needs, we developed the final curriculum, which included 3 hours of interactive introductory Lean training, with a focus on charter construction, process mapping, problem identification, and solution design; and 6 hours of in-person lectures delivered by quality, safety, and value leaders from our institution, emphasizing institutional priorities and case studies to illustrate concepts (table 1). In addition, residents applied these principles in a transitions-of-care exercise that included reviewing 3 of their own readmissions to determine contributing systems causes. The curriculum aligns trainee educational goals with existing institutional VBM and QI/PS initiatives as identified by residency program leadership.

For the remainder of the time (15 to 20 hours), residents designed a value-based QI project pitch (sample schedule provided as online supplemental material). Residents were divided into teams of 4 to 5 residents per group with 3 or 4 teams per 2-week block. Supported by the chief resident and a faculty advisor, each team was tasked with picking a process they identified as inefficient, unsafe, or low value. Each resident team submitted initial ideas and received feedback (idea form provided as online supplemental material).

Residents then worked with hospital VBM project managers (nonclinical project management staff) to use local data to analyze the opportunity for improvement, design potential solutions, and pitch a compelling argument for their idea. In the first year, VBM managers helped residents identify and request specific data, including financial data, to be extracted by data analysts. In the second year, residents had access to Slicer Dicer, a proprietary Epic module (Epic Systems, Verona, WI), which allowed rich exploration of our own data. We also shifted the focus away from specific monetary savings to generalizations about the value proposition. Residents still had the option of requesting additional data from the VBM managers, and several teams did. In both years, we had a “VBM team office hours” session (nicknamed the “Nurse Shark Tank”), in which VBM team members helped each team frame their problem, identify areas where they needed to understand the process better, or collect/request data to support their identification of the problem.

Teams were encouraged to observe the process, interview stakeholders (eg, nurses, laboratory staff, physical therapists, pharmacists), and create a process map and Ishikawa diagram of the current state.¹⁵  They identified potential solutions primarily from published literature and then worked with multidisciplinary stakeholders to choose the ones most likely to be successful. Teams were given specific deadlines for deliverables and guidance about how much time to spend on each activity during protected team time, with flexibility allowed for each team to use the time as needed to advance their project under the guidance of the chief resident and their faculty advisor.

The curriculum culminated in a 2-hour formal “Shark Tank” event on the final day of the 2-week rotation. Each resident team had 15 minutes to present their pitch followed by 15 minutes of questions and comments from the “Sharks.” The “Sharks” were senior clinical and administrative leadership, including the chief medical officer, chief quality officer, chief of medicine, and clinical lead of value-based management. They scored pitches on projected value, impact, feasibility, creativity, and alignment with departmental and institutional goals (table 2). Each team received focused oral feedback on their ideas, methods, and presentations. At the end of the 2-week curriculum, all teams were given the opportunity to work with a faculty advisor to implement their project during the subsequent academic year. The highest-priority projects as identified by senior leadership received additional administrative and project management support. Some residents worked on implementation of their project ideas during protected time in their mandatory PGY-3 QI/PS rotation; others received coverage for clinical activities to participate in project design meetings and executive-level presentations. For successfully implemented projects, residents handed off project management to clinical and operational staff.

We conducted retrospective, pre-post, perception-based paper surveys at the end of the 2-week curriculum and follow-up surveys among participants 1 year after completion of the curriculum. The 15-item survey of 4-point Likert-type scales to self-assess knowledge, attitudes, and skills was developed by the authors (clinician educators with QI/PS expertise) without further testing. Along with that survey, we asked graduating residents about their involvement in quality, safety, and value projects during residency and their career intentions. We also tracked resident presentations of QI/VBM projects and how many of the project proposals were actually completed.

The New York University School of Medicine deemed this project exempt from Institutional Review Board review.

Likert scales were compressed to positive (very likely/likely) and negative (very unlikely/unlikely) for pre-post comparison (provided as online supplemental material). Pre-post results were compared using the related-sample sign test as the surveys were paired by participant. Precurriculum survey results were compared with 1-year follow-up survey results using Pearson's chi-square test (unpaired analysis). Effect sizes were estimated using Cohen's d for the precurriculum to postcurriculum comparison. Correction for multiple testing was not performed. There were no significant differences when data were analyzed separately by year (not shown), so results from the 2 years were combined. All analyses were conducted using SPSS Statistics 24 (IBM Corp, Armonk, NY).

In the first 2 years of the curriculum (2017–2018), 92 residents participated in 19 teams, resulting in 19 project pitches (table 3). Of all second-year residents, 81% (43 of 53) participated in 2017 and 92% (49 of 53) participated in 2018. Among the participants, 83% (76 of 92) consented and responded to the survey. Overall, 94% (60 of 64) recommended the course. There were significant improvements in resident self-assessed knowledge, confidence levels, and comfort with QI and value learning objectives after the curriculum (provided as online supplemental material). For example, after the curriculum, participants reported an increase in their ability to identify unsafe or inefficient processes in the hospital (56% to 96%, P < .001, Cohen's d = 0.85). They felt more comfortable in their abilities to use process mapping (18% to 86%; P < .001; Cohen's d = 1.20) and principles of Lean management to propose solutions (16% to 64%; P < .001; Cohen's d = 1.14). More participants reported being likely or highly likely to participate in quality, safety, and value projects (25% to 70%, P < .001, Cohen's d = 1.08) and to suggest quality, safety, and value proposals to hospital leadership (12% to 65%; P < .001; Cohen's d = 1.25).

Of the 43 participants from the initial year of the curriculum, 39 (91%) completed a 1-year follow-up survey. Improvements were sustained at 1-year follow-up for all survey questions (provided as online supplemental material). For example, at 1-year follow-up, the percentage of residents who felt comfortable or very comfortable creating a solution to a process in the hospital or clinic that was unsafe, inefficient, or costly was sustained (25% pre; 78% post; 78% follow-up; P < .001). Similarly, the percentage of residents who reported they were likely or very likely to collaborate with nonclinical personnel to analyze costs and opportunities of potential interventions was also sustained (17% pre; 79% post; 78% follow-up; P < .001). The same percentage of residents saw quality, safety, and value projects as part of their scholarly activity after the curriculum and at 1-year follow-up (42% pre; 84% post; 84% follow-up; P < .001 for pre/post; P = .001 for 1-year follow-up).

Of the participants, 95% (41 of 43) had at least 1 poster presentation accepted to our internal Quality and Safety Day. Seventeen of 39 (44%) survey respondents reported that they were involved in QI/VBM projects beyond their required rotations, of which 10 had resulted in presentations at national meetings prior to completion of residency training. After this curriculum, several residents applied and were selected for the High Value Practice Academic Alliance Future Leaders Program for advanced VBM training. Ten of 39 residents (26%) responded that they were very likely to consider a career that focused on improving quality, safety, and value. This curriculum has helped the institution identify leaders within the residency class, several of whom have been hired to stay on as faculty.

To facilitate dissemination, chief residents leading the curriculum identified several key elements (derived from participant feedback) that are necessary to successfully implement this type of curriculum. These include putting residents in charge, providing mentorship/advising, encouraging vertical alignment within the institution, supporting opportunity identification with local data and published best practices, and providing the tools for success through skills-based workshops (figure).

Four of the 9 pitches from the first iteration in 2017 received institutional support, resulting in hospital improvements under the guidance of faculty advisors (table 3). In 2018, following the second iteration, 2 of 10 projects were categorized as high priority and are in various stages of implementation.

In this QI/VBM curriculum, in which internal medicine residents developed specific project pitches delivered to institutional leaders, self-assessment surveys showed that experiences were highly acceptable and increased perceived knowledge, skills, and comfort in these topics. These perceptions were sustained at 1-year follow-up. Successful pitches resulted in actual institutional changes.

Others have shown that experiential participation in QI projects during residency improves knowledge and engagement.^16,17  One institution published their efforts to incorporate games into teaching QI with success improving resident QI knowledge and confidence.¹⁰  Our curriculum, which culminates in a gameshow-style event, builds on this by having residents develop real project pitches with the potential to improve value for patients and the hospital. Our sustained results at 1 year suggest that it may not be essential for residents themselves to complete projects for the curriculum to change their attitudes and perceived skills in these areas.

The excitement generated by this endeavor resulted in an overwhelming interest among our hospital leadership. We had more leaders who wanted to be “Sharks” than we could accommodate, and the institution began exploring how to expand this program to other residency programs and beyond GME to include advanced practice providers. We have tested the feasibility at other sites by piloting an expansion at an affiliated community hospital with support from a chief resident and hospitalist advisor with promising initial results.

Our findings may not generalize to other settings, particularly those without institutional support and faculty leaders in quality and value education. As our survey was developed without additional validity evidence, respondents may not have interpreted questions as intended. In addition, the surveys measured self-assessed knowledge and skills, and may not reflect actual competencies. Although the survey response rates were generally high, some residents may find the curriculum less useful or unattractive.

Further research might examine whether this curriculum can be replicated in other settings, perhaps through the VA Chief Residents in Quality and Patient Safety program, with other specialties, or with nonphysicians such as advanced care practitioners, nurses, and pharmacists. Other considerations include whether the curriculum is more effective in a 2-week intensive format or spread out over half-day sessions throughout the academic year.¹⁶ 

This 2-week curriculum to teach QI/VBM to internal medicine residents through didactics, exercise, and project pitches to institutional leaders resulted in improvements in self-assessed knowledge and skills and comfort with the topics. These improvements were sustained at 1 year. Successful resident-pitched projects were implemented and produced institutional improvements as well as academic presentations. The curriculum appears to improve resident attitudes toward future participation in quality, safety, and value projects.