ABSTRACT

Background 

To optimize resident learning, programs need to readily assess resident well-being. There is a lack of easy-to-use, acceptable instruments for this task.

Objective 

We created a well-being “fuel gauge,” and assessed the acceptability and feasibility of this weekly electronic communication pipeline for residents to report and discuss their well-being.

Methods 

A well-being fuel gauge assessment was administered weekly over the course of 1 academic year (July 2016 to June 2017) in a large internal medicine residency program. The well-being gauge asked residents to report their fuel levels using a 1 to 5 Likert-type scale (1, empty; 3, half tank; and 5, full tank). Residents who provided low scores (1 or 2) were contacted by program leadership, and the program director sent weekly e-mail updates that addressed residents' comments on their well-being fuel gauge.

Results 

Of 163 residents, 149 (91%) provided data on their well-being fuel gauge, with a 53% average weekly response rate. Fifty-four percent of residents (80 of 149) reported a low score over the course of the year, and 4 residents only used the assessment to report a low score. Comments on average consisted of 280 characters (SD = 357) and were lengthier and more prevalent with lower fuel gauge scores. We analyzed the relationship between scores and comments.

Conclusions 

The well-being fuel gauge was well accepted by most residents and was easy to administer and to oversee by program directors. It facilitated ongoing monitoring of well-being and follow-up to address factors contributing to low well-being.

What was known and gap

Given high rates of resident burnout and distress, there is heightened interest in regular monitoring of resident well-being.

What is new

A well-being “fuel gauge” is a quick and easy-to-administer weekly survey tool in an internal medicine residency program.

Limitations

Data were collected in a single institution and single specialty, limiting generalizability.

Bottom line

The well-being fuel gauge facilitated ongoing monitoring and follow-up to address factors contributing to low well-being.

Introduction

“Readiness for learning”1  is a simple, yet undebatable, requisite condition for trainees to truly benefit from their learning experiences. Learners must have the required foundational knowledge and experience, they must have the capacity to learn, and they must be in an appropriate affective state. In regard to the latter point, extensive research has demonstrated the effects of stress- and well-being–related factors on cognitive performance and learning.24  Numerous studies have identified various well-being–related components and their effect on resident learning. For example, sleep deprivation, perceptions of psychological safety, being humiliated, and conflicts with professors/staff are contributors.57  Programs must maintain high levels of resident well-being to maximize residents' learning and ultimate future successes as postgraduate physicians. However, programs may currently be limited in their ability to identify well-being issues and facilitate resident well-being communications to the program.

There is a wide range of well-being assessments currently used,8  with great variation across the available instruments. Concerns about negative stigma and underreporting for self-report instruments, assessment costs, and time for completion are all considerations.813  Many of the available assessments, such as the Maslach Burnout Inventory (which takes between 10 and 15 minutes to complete), require a substantial investment of time, which might limit frequent administration.13  Furthermore, many assessments incur low response rates, with many studies reporting rates in the 20% to 40% range.12,14 

Given the lack of easy-to-use, acceptable instruments to allow program directors to follow residents' perceptions of well-being in real time, we created a well-being “fuel gauge” assessment. We examined its acceptability to residents, its feasibility, and the preliminary evidence of its validity.

Methods

Setting and Participants

The well-being fuel gauge was examined in a large internal medicine residency program at the University of Texas Southwestern. The first administration was e-mailed the first week of July 2016, and data for this effort were collected over 1 academic year (until June 31, 2017).

Intervention

We elected to assess the broader construct of well-being, rather than focus on depression or burnout. This assessment captures these components, but our efforts needed to be directed at preventing these issues, which required more sensitivity. In an effort to reduce negative stigma that is oftentimes associated with these assessments, we reframed our perspective. Well-being management is akin to taking care of a high-performance sports car. You must put forth effort to maintain the vehicle if you want it to perform optimally, and you need a dashboard to check on system levels. We asked residents to report their fuel levels with the item, “Overall, my well-being fuel tank is:” with a 5-point Likert-type scale (1, empty; 3, half tank; 5, full tank). We also provided a section for optional comments, with the item “If you have recommendations on how we could increase well-being (for you, or the residency in general), please share here.” This assessment was distributed weekly and took only a brief time for residents to complete.

Automatic alerts were sent to the program director and coordinator when low scores were submitted. The low-scoring residents were contacted, typically within 24 hours, by the associate director or program director. Unless other extenuating circumstances existed (eg, a resident reported being upset, in which we would immediately call or visit), a “checking in” e-mail was sent (asking them to let us know how we could help and offering recovery time, mental health resources, or whatever else might be needed). If a comment was provided, responses were tailored accordingly. In cases when a second low score submission was obtained within a short period from a particular resident, then a call or visit was made.

Many of the comments were publicly addressed (the identity of the respondent was kept confidential) in an e-mail from the program director to all residents. Please see the online supplemental materials for the implementation process, a step-by-step guide, and a workflow diagram.

Outcomes

We examined the utilization of the well-being fuel gauge via completion rates, the scores provided, whether or not comments were provided, and the association of specific scores with comments. To obtain additional insight into the nature of the information provided by the comments (table 1), we utilized natural language processing software, the Linguistic Inquiry and Word Count (LIWC) program. This quantitative content analysis software analyzes the frequency of word usage related to a large number of content areas (see example words in table 1) and has evidence of validity.

table 1

Relations of Comment Content With Fuel Gauge Scores

Relations of Comment Content With Fuel Gauge Scores
Relations of Comment Content With Fuel Gauge Scores

The study received exemption as a quality improvement initiative by the University of Texas Southwestern Institutional Review Board.

Analysis

Descriptive statistics and letter counts (using the “LEN” function) were computed in Microsoft Excel (Redmond, Washington), and correlations were conducted in SPSS version 24 (IBM Corp, Armonk, NY). Comments provided were dichotomized as 0 (comment not provided) or 1 (comment provided), and the well-being fuel gauge score was also dichotomized as 0 (high, scores of 3, 4, and 5) or 1 (low, scores of 1 and 2).

For the LIWC data, relevant content areas from the psychological process categories were identified, and all personal concern content areas were retained. This resulted in 28 analyses. To reduce a familywise error, the Bonferroni correction (α = .05/28) was applied, which resulted in a statistical significance criteria of α = .002 for these analyses. Communication content categories from the LIWC were correlated to well-being fuel gauge scores.

Results

Our overall participation rate was 149 of 163 residents (91%), with a 52% (SD = 8.34) average weekly completion rate over the course of administration. Response rates increased slightly over the course of the year (r = 0.279, P = .045). All scale points were adequately utilized (table 1), with 193 low score (scores of 1 or 2) submissions. Of the 163, there were 80 of 149 unique residents (54%) sending in a low score. We had 4 residents who completed the well-being fuel gauge exclusively to submit a low score. Weekly average scores (low of 3.79 at end of July) and response rates by week are shown in figure 1.

figure 1

Average Weekly Fuel Gauge Scores and Response Rates by Week

Note: The index for the average weekly score (bars) is represented by the left number column, whereas the response rates (line) is represented by the right number column. Week 1 represents the first week of July 2016, week 9 represents the last week of August, and week 37 represents the second week of March 2017.

figure 1

Average Weekly Fuel Gauge Scores and Response Rates by Week

Note: The index for the average weekly score (bars) is represented by the left number column, whereas the response rates (line) is represented by the right number column. Week 1 represents the first week of July 2016, week 9 represents the last week of August, and week 37 represents the second week of March 2017.

Of 4303 well-being fuel gauge submissions, 681 (16%) included comments (see figure 2 for specific examples). On average, comments had 279.67 (SD = 356.9) letters. We also examined the percentage of comments provided for various scores (table 2), and as scores became lower, a higher number of submissions contained comments (r = –0.18, P < .001). Longer comments were also associated with lower scores (r = –0.30, P < .001). Examining low versus high dichotomized scores, whether or not comments were present (r = –0.21, P < .001), and length was further supported (r = –0.25, P < .001).

figure 2

Representative Resident Comments From the Fuel Gauge

Note: <> represents information removed due to space and/or sensitive content.

figure 2

Representative Resident Comments From the Fuel Gauge

Note: <> represents information removed due to space and/or sensitive content.

table 2

Frequency of Scores and Comment Descriptives

Frequency of Scores and Comment Descriptives
Frequency of Scores and Comment Descriptives

Using the natural language processing software, individuals reporting higher well-being fuel gauge scores had comments that were higher in words expressing positive emotion, affective processes, leisure, reward drive, and drives in general. Individuals reporting low fuel gauge scores had comments higher in words that reflected negative emotion, sadness, tentativeness, health content, and cognitive processes (table 1).

Discussion

Over a full year, the well-being fuel gauge intervention appeared acceptable to residents and the program director, with sustained weekly responses. This resident well-being instrument required little time or resources to execute and exhibited early evidence of validity in the content of comments tracked with high or low scores.

Regarding practicality, this assessment may take as little as a few seconds for residents to complete, relative to more comprehensive tools that may take more than 10 minutes.13  It also is a free assessment tool, while many other instruments have costs that may be prohibitive to some programs (especially if seeking frequent administration).12,13 

In regard to program administration and oversight, the well-being fuel gauge required approximately 30 minutes at the beginning of the year for a coordinator to set up the automatic distribution of weekly assessments. The associate program director and program director monitored the weekly assessments, which required collectively less than 1 hour a week on average. Other assessments necessitate calculations of raw data and for inferences to be derived regarding individual well-being levels.1517  The results of the fuel gauge are immediately interpretable. Moreover, actionable, qualitative information is provided to help guide program director actions from the comments (which no other assessment, to our knowledge, provides directly). The weekly e-mail addressing the comments required approximately 1 h/wk from the program director.

Many residents have come forward and shared their excitement regarding the well-being fuel gauge and the program director's communications addressing their well-being comments. Program directors, associate program directors, and faculty have received many positive comments and e-mails in this capacity. Surprisingly, we learned from numerous applicants that our residents have been using the well-being fuel gauge as a positive aspect of our program—and, in turn, applicants are recognizing the program as a “well-being friendly” residency.

There are several limitations to this study. First, the data were collected in a single institution and single specialty, which may limit generalizability. While we have assessments over the course of a year, it is possible that novelty effects may have played a role and that results may not generalize across years. Furthermore, we relied solely on self-reporting and did not compare this assessment to other indicators of well-being.

Future research should study the effectiveness of the instrument in other resident populations and examine the relations of this assessment with other indicators of well-being.

Conclusion

The well-being fuel gauge for assessing resident well-being was easy to administer, relatively simple to oversee, and well accepted by residents. The tool has facilitated assessment and monitoring of resident well-being by the program.

References

1
Schindler
AW.
Readiness for learning
.
J Child Educ
.
1948
;
24
(
7
):
301
304
.
2
Schwabe
L.
Wolf
OT.
Stress impairs the reconsolidation of autobiographical memories
.
Neurobiol Learn Mem
.
2010
;
94
(
2
):
153
157
.
3
Dobkin
PL.
Hutchinson
TA.
Teaching mindfulness in medical school: where are we now and where are we going?
Med Educ
.
2013
;
47
(
8
):
768
779
.
4
de Quervain
DJ.
Stress and glucocorticoids impair retrieval of long-term spatial memory
.
Nature
.
1998
;
394
(
6695
):
787
790
.
5
Papp
KK.
Stoller
EP.
Sage
P.
et al.
The effects of sleep loss and fatigue on resident-physicians: a multi-institutional, mixed methods study
.
Acad Med
.
2004
;
79
(
5
):
194
200
.
6
Baldwin
DC.
Daugherty
SR.
How residents say they learn: a national, multi-specialty survey of first- and second-year residents
.
J Grad Med Educ
.
2016
;
8
(
4
):
631
639
.
7
Torrala
KD.
Koo
LK.
Byrne
JM.
et al.
Does psychological safety impact the clinical learning environment for resident physicians? results from the VA's Learners' Perceptions Survey
.
J Grad Med Educ
.
2016
;
8
(
5
):
699
707
.
8
Raj
KS.
Well-being in residency: a systematic review
.
J Grad Med Educ
.
2016
;
8
(
5
):
674
684
.
9
Baer
L.
Blais
MA.
Handbook of Clinical Rating Scales and Assessment in Psychiatry and Mental Health
.
New York, NY
:
Humana Press;
2009
.
10
Hunt
MG.
Auriemma
J.
Cashaw
ACA.
Self-report bias and underreporting of depression on the BDI-II
.
J Pers Assess
.
2003
;
80
(
1
):
26
30
.
11
Guille
C.
Zhao
Z.
Krystal
J.
et al.
Web-based cognitive behavioral therapy intervention for the prevention of suicidal ideation in medical interns: a randomized clinical trial
.
JAMA Psychiatry
.
2015
;
72
(
12
):
1192
1198
.
12
Dyrbye
LN.
Satele
D.
Sloan
J.
et al.
Ability of the Physician Well-Being Index to identify residents in distress
.
J Grad Med Educ
.
2014
;
6
(
1
):
78
84
.
13
Maslach
C.
Jackson
SE.
Leiter
MP.
Maslach Burnout Inventory Manual. 3rd ed
.
Palo Alto, CA
:
Consulting Psychologists Press;
1996
.
14
Kassam
A.
Horton
J.
Shoimer
I.
et al.
Predictors of well-being in resident physicians: a descriptive and psychometric study
.
J Grad Med Educ
.
2015
;
7
(
1
):
70
74
.
15
Diener
E.
Emmons
RA.
Larson
RJ.
et al.
The Satisfaction With Life Scale
.
J Pers Assess
.
1985
;
49
(
1
):
71
75
.
16
Bech
P.
Olsen
LR.
Kjoller
M.
et al.
Measuring well-being rather than the absence of distress symptoms: a comparison of the SF-36 Mental Health subscale and the WHO-Five Well-Being Scale
.
Int J Methods Psychiatr Res
.
2003
;
12
(
2
):
85
91
.
17
Maslach
C.
Jackson
SE.
Leiter
MP.
Maslach burnout inventory manual
.
In
:
Zalaquett
CP.
Wood
RJ.
eds
.
Evaluating Stress: A Book of Resources, 3rd ed
.
Lanham, MD
:
Scarecrow Press;
1997
:
191
218
.

Author notes

Editor's Note: The online version of this article contains more detail regarding implementation processes, a step-by-step guide, and a workflow diagram.

Funding: This research was funded by the Alliance for Academic Internal Medicine Innovation grant program and by a grant from the Southwestern Academy of Teachers.

Competing Interests

Conflict of interest: The authors declare they have no competing interests.

This research was presented in part at the Alliance for Academic Internal Medicine's Academic Internal Medicine Week, Baltimore, Maryland, March 19–22, 2017, and at the Innovations in Medical Education Conference, Los Angeles, California, February 23–24, 2018.

Supplementary data