Resident Fuel Levels: Reframing, Assessing, and Addressing Well-Being

“Readiness for learning”¹  is a simple, yet undebatable, requisite condition for trainees to truly benefit from their learning experiences. Learners must have the required foundational knowledge and experience, they must have the capacity to learn, and they must be in an appropriate affective state. In regard to the latter point, extensive research has demonstrated the effects of stress- and well-being–related factors on cognitive performance and learning.^2–4  Numerous studies have identified various well-being–related components and their effect on resident learning. For example, sleep deprivation, perceptions of psychological safety, being humiliated, and conflicts with professors/staff are contributors.^5–7  Programs must maintain high levels of resident well-being to maximize residents' learning and ultimate future successes as postgraduate physicians. However, programs may currently be limited in their ability to identify well-being issues and facilitate resident well-being communications to the program.

There is a wide range of well-being assessments currently used,⁸  with great variation across the available instruments. Concerns about negative stigma and underreporting for self-report instruments, assessment costs, and time for completion are all considerations.^8–13  Many of the available assessments, such as the Maslach Burnout Inventory (which takes between 10 and 15 minutes to complete), require a substantial investment of time, which might limit frequent administration.¹³  Furthermore, many assessments incur low response rates, with many studies reporting rates in the 20% to 40% range.^12,14 

Given the lack of easy-to-use, acceptable instruments to allow program directors to follow residents' perceptions of well-being in real time, we created a well-being “fuel gauge” assessment. We examined its acceptability to residents, its feasibility, and the preliminary evidence of its validity.

The well-being fuel gauge was examined in a large internal medicine residency program at the University of Texas Southwestern. The first administration was e-mailed the first week of July 2016, and data for this effort were collected over 1 academic year (until June 31, 2017).

We elected to assess the broader construct of well-being, rather than focus on depression or burnout. This assessment captures these components, but our efforts needed to be directed at preventing these issues, which required more sensitivity. In an effort to reduce negative stigma that is oftentimes associated with these assessments, we reframed our perspective. Well-being management is akin to taking care of a high-performance sports car. You must put forth effort to maintain the vehicle if you want it to perform optimally, and you need a dashboard to check on system levels. We asked residents to report their fuel levels with the item, “Overall, my well-being fuel tank is:” with a 5-point Likert-type scale (1, empty; 3, half tank; 5, full tank). We also provided a section for optional comments, with the item “If you have recommendations on how we could increase well-being (for you, or the residency in general), please share here.” This assessment was distributed weekly and took only a brief time for residents to complete.

Automatic alerts were sent to the program director and coordinator when low scores were submitted. The low-scoring residents were contacted, typically within 24 hours, by the associate director or program director. Unless other extenuating circumstances existed (eg, a resident reported being upset, in which we would immediately call or visit), a “checking in” e-mail was sent (asking them to let us know how we could help and offering recovery time, mental health resources, or whatever else might be needed). If a comment was provided, responses were tailored accordingly. In cases when a second low score submission was obtained within a short period from a particular resident, then a call or visit was made.

Many of the comments were publicly addressed (the identity of the respondent was kept confidential) in an e-mail from the program director to all residents. Please see the online supplemental materials for the implementation process, a step-by-step guide, and a workflow diagram.

We examined the utilization of the well-being fuel gauge via completion rates, the scores provided, whether or not comments were provided, and the association of specific scores with comments. To obtain additional insight into the nature of the information provided by the comments (table 1), we utilized natural language processing software, the Linguistic Inquiry and Word Count (LIWC) program. This quantitative content analysis software analyzes the frequency of word usage related to a large number of content areas (see example words in table 1) and has evidence of validity.

The study received exemption as a quality improvement initiative by the University of Texas Southwestern Institutional Review Board.

Descriptive statistics and letter counts (using the “LEN” function) were computed in Microsoft Excel (Redmond, Washington), and correlations were conducted in SPSS version 24 (IBM Corp, Armonk, NY). Comments provided were dichotomized as 0 (comment not provided) or 1 (comment provided), and the well-being fuel gauge score was also dichotomized as 0 (high, scores of 3, 4, and 5) or 1 (low, scores of 1 and 2).

For the LIWC data, relevant content areas from the psychological process categories were identified, and all personal concern content areas were retained. This resulted in 28 analyses. To reduce a familywise error, the Bonferroni correction (α = .05/28) was applied, which resulted in a statistical significance criteria of α = .002 for these analyses. Communication content categories from the LIWC were correlated to well-being fuel gauge scores.

Our overall participation rate was 149 of 163 residents (91%), with a 52% (SD = 8.34) average weekly completion rate over the course of administration. Response rates increased slightly over the course of the year (r = 0.279, P = .045). All scale points were adequately utilized (table 1), with 193 low score (scores of 1 or 2) submissions. Of the 163, there were 80 of 149 unique residents (54%) sending in a low score. We had 4 residents who completed the well-being fuel gauge exclusively to submit a low score. Weekly average scores (low of 3.79 at end of July) and response rates by week are shown in figure 1.

Of 4303 well-being fuel gauge submissions, 681 (16%) included comments (see figure 2 for specific examples). On average, comments had 279.67 (SD = 356.9) letters. We also examined the percentage of comments provided for various scores (table 2), and as scores became lower, a higher number of submissions contained comments (r = –0.18, P < .001). Longer comments were also associated with lower scores (r = –0.30, P < .001). Examining low versus high dichotomized scores, whether or not comments were present (r = –0.21, P < .001), and length was further supported (r = –0.25, P < .001).

Using the natural language processing software, individuals reporting higher well-being fuel gauge scores had comments that were higher in words expressing positive emotion, affective processes, leisure, reward drive, and drives in general. Individuals reporting low fuel gauge scores had comments higher in words that reflected negative emotion, sadness, tentativeness, health content, and cognitive processes (table 1).

Over a full year, the well-being fuel gauge intervention appeared acceptable to residents and the program director, with sustained weekly responses. This resident well-being instrument required little time or resources to execute and exhibited early evidence of validity in the content of comments tracked with high or low scores.

Regarding practicality, this assessment may take as little as a few seconds for residents to complete, relative to more comprehensive tools that may take more than 10 minutes.¹³  It also is a free assessment tool, while many other instruments have costs that may be prohibitive to some programs (especially if seeking frequent administration).^12,13 

In regard to program administration and oversight, the well-being fuel gauge required approximately 30 minutes at the beginning of the year for a coordinator to set up the automatic distribution of weekly assessments. The associate program director and program director monitored the weekly assessments, which required collectively less than 1 hour a week on average. Other assessments necessitate calculations of raw data and for inferences to be derived regarding individual well-being levels.^15–17  The results of the fuel gauge are immediately interpretable. Moreover, actionable, qualitative information is provided to help guide program director actions from the comments (which no other assessment, to our knowledge, provides directly). The weekly e-mail addressing the comments required approximately 1 h/wk from the program director.

Many residents have come forward and shared their excitement regarding the well-being fuel gauge and the program director's communications addressing their well-being comments. Program directors, associate program directors, and faculty have received many positive comments and e-mails in this capacity. Surprisingly, we learned from numerous applicants that our residents have been using the well-being fuel gauge as a positive aspect of our program—and, in turn, applicants are recognizing the program as a “well-being friendly” residency.

There are several limitations to this study. First, the data were collected in a single institution and single specialty, which may limit generalizability. While we have assessments over the course of a year, it is possible that novelty effects may have played a role and that results may not generalize across years. Furthermore, we relied solely on self-reporting and did not compare this assessment to other indicators of well-being.

Future research should study the effectiveness of the instrument in other resident populations and examine the relations of this assessment with other indicators of well-being.

The well-being fuel gauge for assessing resident well-being was easy to administer, relatively simple to oversee, and well accepted by residents. The tool has facilitated assessment and monitoring of resident well-being by the program.