There is growing awareness of a high prevalence of work-related stress and burnout in physicians,1,2 the toll it takes on physicians,3 its cost to institutions,4,5 and its consequences for patient care.6,7 Recognizing the need to make well-being a principal component of its activities, the Accreditation Council for Graduate Medical Education (ACGME) established new standards,8,9 and education and outreach activities to promote well-being in accredited programs and institutions.10,11 The multiple dimensions of the well-being standards and diverse needs of specialties make it important for programs and institutions to make reasoned choices about well-being interventions, well-being programs (sets of interventions targeted to groups such as residents or faculty), and methods for program evaluation.
To assist in these decisions, we introduce (1) a Well-Being Interventions Typology (WIT) of various approaches, and (2) a Well-Being Evaluation Schema (WES) to facilitate robust evaluation of well-being programs.
The Well-Being Interventions Typology (WIT)
While there is no widely accepted definition of physician well-being, the construct is broader than ameliorating burnout and encompasses physical, emotional, and professional components.12,13 The ACGME well-being standards use this broader conceptualization.8,9 We aggregated components of the ACGME standards8 and published interventions to reduce burnout,14,15 enhance well-being,16 promote professional satisfaction,17 expand mentoring18 and peer support,19 and increase physician engagement20 into a single Table, the Well-being Interventions Typology (WIT). Like prior publications,14,15,20 we characterized interventions as targeting individuals (residents or faculty), work units, programs, or institutions.
The Well-Being Evaluation Schema (WES)
To facilitate evaluation of well-being interventions, we adapted 2 existing frameworks with utility evidence—Kirkpatrick's assessment of training outcomes21 and the Best Evidence in Medical Education (BEME) scale22 —to create the Well-Being Evaluation Schema (WES) shown in the Figure. Both frameworks have been used extensively to evaluate education interventions. The current version of Kirkpatrick's model adds consideration of the relevance of interventions to the intended group, participants' engagement, required drivers to reinforce desired performance,21 leading indicators21 that offer early information about implementation success, and return on expectations to show how interventions meet key stakeholder expectations. Kaufman's expansion of Kirkpatrick's model promotes a focus on return on investment (ROI) and societal outcomes.23
To create the WES, we added 3 dimensions to the existing frameworks: (1) Level Zero (baseline or preintervention) data on distress, burnout, engagement, and other relevant metrics; (2) SMART24 goals to promote actionable and measurable interventions; and (3) an approach for evaluating non-education interventions (such as access to mental health services or reducing non-physician work in clinical settings), with Level 1 utilization data having a direct impact on outcomes of interest.
Using the WES
Evaluating outcomes is essential to understanding what interventions work in specific contexts, identifying best practices for adoption or adaptation, and negotiating for resources with institutional leaders. A key attribute of the WES framework is the inclusion of process and early outcome metrics as leading indicators21,25 of implementation effectiveness. Lack of participation in an intervention by targeted groups is an early leading indicator that offers critical information about attributes that may create barriers, such as the public location of a mental health clinic or the inability to attend medical appointments or scheduled wellness events due to work demands. Leading indicators can compensate for the fact that outcomes of interest often have significant time lags. In a case vignette (see the Box) and online supplemental material we highlight use of the WES to design and evaluate an institutional well-being program.
Box Using the WIT and WES to Evaluate and Improve an Institution's Well-Being Programing
Sharon and Richard, the designated institutional official (DIO) and chief well-being officer of a midsized sponsoring institution, discuss the current and planned well-being programs prior to an annual graduate medical education (GME) retreat. Areas for improvement include high burnout scores in residents and information suggesting local culture favors “toughing it out.” Richard and Sharon consulted the WIT and selected 2 new interventions intended to address the problems with local culture: enhancing resident access to medical and mental health services and having all residents develop a personal well-being plan.
Sharon suggests an existing intervention adding scribes and allowing for additional time for faculty debriefings in a family medicine continuity clinic should be expanded to 2 additional specialties to increase sample size and generalizability. She notes that studying interventions to address workflow problems and learning challenges in ambulatory settings is an opportunity for scholarship and could add to a limited body of literature.
At the GME retreat the WES is used to assess the impact of existing and planned programs. Attendees develop SMART (specific, measurable, actionable, relevant and time-bound) goalsa to track progress and projecting the return on expectationsb for the new interventions. This highlights that some existing well-being programs lack the resources needed for full implementation and sustainability. Attendees confirm use of the WES is helpful in selecting data to clarify expectations for implementation success.
Additional considerations by institutional leaders and a sample completed WES are provided as online information.
Kirkpatrick D, Kirkpatrick J, Kirkpatrick W. The New World Kirkpatrick Model. https://www.kirkpatrickpartners.com/Our-Philosophy/The-New-World-Kirkpatrick-Model.
Use of the WIT and WES and Opportunities for Scholarship
Together, the WIT and the WES are expected to facilitate improved design and holistic evaluation of well-being programs. Important components are return on expectation (ROE),21 which estimates the organizational value of an intervention, and ROI,24 the rate of return (in improved well-being and associated benefits) for the funds invested in well-being programs to evaluate whether the investment is financially and conceptually sound. Analyses using ROE and ROI are important given the resources needed for well-being programming, including time and opportunity costs and prioritizing finite financial resources to well-being over other programs and activities.
We acknowledge limitations. While the overall approach is rooted in established assessment approaches and evidence-informed interventions, there is no primary validity evidence for these tools in selecting and evaluating well-being interventions. Further study is needed, and use of the WIT and WES will offer opportunities for scholarship, including identifying, describing, and disseminating information on effective sustainable interventions.
Burnout in physicians is high, and interventions are needed to promote well-being during their training and throughout their career. Given the broad implementation of well-being programs, it is critical to identify the interventions with positive returns on expectations and on investment. The WIT and the WES can assist in planning interventions with clarity around purpose and outcomes and in evaluating outcomes. This is critical for institutional decision-making and for scholarship to identify and disseminate best practices in this relatively young field.
