Does Psychological Safety Impact the Clinical Learning Environment for Resident Physicians? Results From the VA's Learners' Perceptions Survey

The provisions of a learning-oriented culture and a productive work environment are important to the achievement of resident competencies.¹  The Accreditation Council for Graduate Medical Education (ACGME) requires graduate medical education (GME) programs to provide a clinical environment where residents learn competencies for independent practice, and patients receive supervised, safe, and high-quality care.² 

A critical element often overlooked in assessing clinical learning environments is whether residents believe their clinical environment is psychologically safe.³  Psychological safety (PS) refers to the perception that there are no negative consequences to self, status, or career for taking interpersonal risks, such as reporting mistakes or problems.⁴  PS has been shown to be highly related to increased team engagement, increased rates of medical error reporting, personal work engagement, and quality system improvement.^5–10 

While qualitative research has shown associations between communication and PS,⁹  few studies have measured resident perceptions of PS at their teaching hospitals and made correlations with attributes of their learning environment. This study applied advanced statistical methods to nationwide survey data to investigate the predictors of PS and its effects on resident satisfaction with their clinical facilities.

Since 1946, health professions education has been a statutory mission for the Department of Veterans Affairs (VA) health care facilities.¹¹  Resident physicians train at VA medical centers where interprofessional team and patient-centered care are emphasized through the VA's Patient Aligned Care Team models of care.¹² 

Data came from the VA's Learners' Perceptions Survey (LPS),^13,14  which is an anonymous, voluntary, computer-assisted, web-based survey of all students and residents engaged in supervised patient care at VA medical facilities as part of an accredited health professions education program. The purpose of the LPS is to assess VA performance in providing clinical training environments for health professions trainees. Since 2001, the survey has been administered annually by the Office of Academic Affiliations (OAA) using a data accounting system to ensure anonymity.¹⁵  The OAA works with local education leadership to send direct e-mails to potential eligible respondents to complete the survey.

In addition to trainee background information, the LPS contains 84 satisfaction-based questions across 9 domains describing a trainee's VA teaching, working, and clinical experiences for a given academic year. Respondents are asked a summary question at the end of each domain to describe their overall satisfaction with the domain. Satisfaction is reported on 5-point Likert scales (1, very dissatisfied, to 5, very satisfied). PS was measured using a 5-point Likert scale that asked whether a respondent agreed with the statement, “Members of the clinical team of which I was a part are able to bring up problems and tough issues.”⁹ 

Survey responses show overall good internal consistency and 1-dimensional ratings by satisfaction domain,¹⁴  across disciplines,^16,17  with good predictive,¹⁸  and construct validity.¹⁹ 

In this study, LPS survey data were limited to resident physicians who completed the survey from July 1, 2010, through June 30, 2014. Domain satisfaction was scored for each respondent as a 5-point ordinal scale computed from responses to the domain summary question, and as a continuous measure calculated as z scores computed from responses to 4 domain element questions. Domain z scores are sufficient statistics that represent satisfaction intensities as latent factors.²⁰  The 4 domains were learning, faculty and preceptors, personal, and systems and processes for handling medical errors. These domains were selected to represent the broad range of learning experiences.¹⁴ 

Covariates to predict PS included the respondent's self-reported sex; academic level (eg, postgraduate year 1 [PGY-1], PGY-2, PGY-3, etc); specialty as defined by ACGME²¹  (table 1 footnote); medical school status (US versus non-US graduate); characteristics of patients seen based on percentage of patients who are older than 65 years, chronic medical illness, multiple medical illnesses, mental illness, alcohol/substance dependence, low socioeconomic status, and lack of social support; and hospital service complexity. The VA classifies service complexity on a 5-point Likert scale based on the facility's patient volume, types of services offered, patient complexity, teaching status, number of training programs, and research activities (table 1 footnote).

Respondents often have different thresholds when mapping the intensity of their satisfaction on a 5-point Likert scale. Therefore, we computed a response bias index to measure the extent to which responders give higher or lower satisfaction ratings than their colleagues. The index was computed for each respondent as a z score; this was calculated from satisfaction responses to 3 element questions describing the facility's computerized medical record system, location convenience, and parking availability, minus the mean z score across all respondents at the same facility and academic year. Response bias indices are positive, zero, or negative depending on whether a respondent gave a higher, equivalent, or lower satisfaction rating, respectively, than their peers.

The collection of LPS survey data, processing, and analyses were conducted under license from the Office of Management and Budget (Information Collection No. 2900-0691, VA Form No. 10-0439). Data use was covered under the US Office of Management and Budget (No. 2900-0691) and is exempt from human subjects' research.

Analyses to compute associations between PS, its predictors, and its satisfaction outcomes were done for both ordinal and continuous satisfaction scores, and to account for observed and unobserved confounding covariates. For part 1, observed covariates were examined as independent predictors of PS by fitting generalized linear models²²  (GLMs) with cumulative logit link functions and multinomial distributions. For part 2, associations between PS and domain satisfaction were computed using a GLM for 5-point ordinal domain, and for continuous z score domain using a GLM with identity link function and Gaussian or gamma distribution. To control for confounding when computing associations between PS and domain satisfaction, we used a control function technique commonly utilized by social scientists.²³  This allows estimates of the association between PS and domain satisfaction to be (1) adjusted for known, observed covariates that can confound the association; (2) corrected for the response bias index to account for respondents having different reporting thresholds; and (3) calibrated using the residual inclusion method²⁴  to control for unobserved confounders that were not available in the study data set.

Of a total of 43 000 residents with known e-mail addresses, 13 044 completed the LPS (30% completion rate). The LPS resident physician sample has been shown to be comparable to the distribution of ACGME residents (exclusive of obstetrics-gynecology and pediatrics) with regard to specialty choices, academic level, US graduate status, geographic area, and sex.²⁵ 

Description of the study sample is provided in table 1. Most respondents came from facilities within the top 3 of the 5 VA complexity levels (n = 11 480, 88%), and 48% (n = 6316) of respondents were from internal medicine and its subspecialties. Most residents were in their first 3 years of training (n = 9087, 70%), US graduates (n = 9713, 75%), and male (n = 7418, 57%). Most respondents reported seeing 25% or more patients who were 65 years or older (91%, 11 754 of 12 910), had a medical illness (95%, 12 310 of 12 909), or had multiple illnesses (96%, 12 326 of 12 910). On the other hand, only 67% reported seeing 25% or more patients with mental illness (8675 of 12 909) or who lacked social support (8629 of 12 909). Approximately 89% (11 599 of 13 044) of the sample agreed their VA experience was psychologically safe, with approximately half (48%, 6204) strongly agreeing.

Element responses by domain were shown to be reliable (intraclass correlations between 0.57 and 0.87), consistent (Cronbach's alpha from 0.93 to 0.98), and 1-dimensional (second-largest eigenvalue from 0.26 to 0.92). The 80% intraclass correlation for all 41 elements across the 4 domains indicates responses clustered around their respective domain relative to the total variance across all element questions. PS responses did not cluster by facility, with between-facility responses accounting for only 12% of the total variance among respondents. This underscores the importance of analyzing PS at the respondent level, rather than at a facility-aggregated level. Responses to the single PS question explained 88.4% of the total variance to the 2 related questions: “I feel free to question the decisions or actions of those with more authority” (asked in 2011), and “It is safe to take a risk in this clinical team” (asked in 2012–2014).

Overall, 11 599 respondents (89%) agreed that members of the clinical team could “bring up problems and tough issues,” including 6204 (48%) who had strongly agreed. Table 2 reports how factors independently were associated with PS. Responders who reported PS tended to be in less complex facilities, in other medicine and psychiatry specialties, male, and saw more patients who were aged, had multiple illnesses, with positive social supports, and had no co-occurring mental illness. PS was also associated with responders who tended to over report satisfaction (ie, positive response bias). No association was found between PS and academic level.

Estimates of the association between PS and domain satisfaction are reported in tables 3 and 4, by analytic strategy (adjusted, corrected, and calibrated) and domain scale (5-point Likert ordinal and continuous z scores). Overall, PS was strongly associated with residents' satisfaction of their clinical learning experiences, with respondents who reported 1 higher level of PS being 5.7 times more likely to report a higher level of satisfaction for their clinical learning environment (95% confidence interval [CI] 5.4–6.1, P < .001). The odds ratio drops to 5.6 (95% CI 5.3–6.0) when adjusting for observed covariates, to 4.2 (95% CI 4.0–4.5) when also correcting for response biases, and to 3.0 (95% CI 2.8–3.3, P < .001) when calibrating for unobserved confounding factors. For z score satisfaction ratings, 1 higher level of PS was associated with a Cohen's d of 1.3 standard deviates (95% CI 1.2–1.5, P < .001). Final PS effect size estimates varied only slightly by domain.

This study applied advanced analytic methods to a national data set to provide quantitative evidence that trainees overall gave high ratings for psychological safety in the VA health care system. The study also showed that PS ratings varied by specialty, clinical setting, and resident, even within the same facility and time period. Finally, despite different analytic (adjusted, corrected, calibrated) and scoring (ordinal, continuous) methods, PS had a strong, statistically significant, and robust association with how residents rated satisfaction with their VA learning, personal, and clinical experiences.

Specifically, our study showed that psychiatry and specialties falling under the “other medicine” category (eg, allergy and immunology, dermatology, family medicine, neurology, physical medicine and rehabilitation) had higher levels of PS (table 2). It is possible that this greater sense of PS among these specialties depends not only on the residents' interaction with the clinical environment but may also reflect an inherent predisposition to PS, as noted by a higher tolerance to ambiguity for those choosing the field of psychiatry.^26,27 

The large PS effect on satisfaction with personal experiences is consistent with qualitative studies that have linked PS to workplace communications in the VA.⁸  The ACGME Resident Survey revealed that resident-faculty interactions, procedural volume, variety, and didactic offerings were among the attributes residents used to gauge training quality.²⁸  These findings underscore the importance of psychologically safe clinical settings if residents are to achieve ACGME milestones,²⁹  provide high-quality patient care,³⁰  and foster professionalism and accountability. These, in turn, are associated with resident competence³¹  and allow clinical obligations to be resolved without fear of intimidation.³² 

Our finding that PS is more prevalent in less complex facilities supports the theory that complex settings present higher levels of stress by requiring navigation through more difficult systems. It has been suggested that in complex systems, education should focus on skills learned at the time when mistakes are made.³³  GME-adapted traditional apprenticeship models are based on “legitimate peripheral participation,”³⁴  which focuses on a resident's evolution of professional identity—from being a largely passive participant in low-risk situations when a master practitioner is nearby to an active one who has integrated and is demonstrating the knowledge, skills, and attitudes of the community. Migration theory³⁵  contextualizes this further by assuming that demanding work systems tolerate margins of acceptable performance (eg, clinical teams under pressure to provide expeditious care may practice at a “normal-illegal stage” where minor infringements of safety are implicitly accepted).³⁶ 

Contrary to popular notions that junior residents perceive less PS than their senior counterparts, we found no association between PS and academic level after adjusting for confounding factors and utilizing a response bias index. Our finding that men report greater PS is consistent with women reporting greater anxiety, less self-confidence, and more apprehension to communicate with their supervising physicians.^37,38 

More research is needed to understand where boundaries lie between safety and harm when residents ask for help. For instance, attending physicians describe “strong” residents as being able to handle heavy workloads without supervision.³⁹  Residents' failure to ask for help may stem from concerns about appearing “weak,” or a perceived erosion of their autonomy and professional identity. Conversely, supervisor approachability and scope of the clinical question are also determinants of trainee decisions to seek supervision.^34,40  Similar attributes are drivers of PS in other industries, leading to job satisfaction, organizational commitment, and quality improvement work.⁴¹ 

The study has limitations. First, data were based on self-reports. Second, PS was measured based on a single question that was available across all 4 study years. Third, surveys may not be the best vehicle to assess PS, as what residents say they will do and what they will actually do are not always congruent, especially given the external pressures of medicine.^42–44  A fourth limitation is the low response rate of 30%. However, our purpose was not to infer the prevalence of psychological safety, but rather to make more robust comparisons in how PS, PS predictors, and satisfaction outcomes co-vary. There were no significant differential response rates among respondents in psychologically safe and unsafe environments that could bias association estimates. A fifth limitation is that the same resident physician may have responded to the LPS survey in different years. Finally, we used an observational data set to study PS, drawing inferences from nonrandomized concurrent comparison groups with blinded participants' study, where issues of confounders and selection biases can be addressed analytically.^45,46 

Future studies should assess organizational structure and PS in medical education and health care settings, and could use more direct methods of program evaluation.

Overall, this study provides strong evidence that PS is important to residents' perceptions of their clinical learning experience. Organizations that promote and measure psychologically safe learning environments may increase resident satisfaction with their GME experiences.