Learning Through Experience: Influence of Formal and Informal Training on Medical Error Disclosure Skills in Residents

When patients experience medical errors, physicians have an ethical and professional duty to disclose them in a transparent and empathic manner. Physicians-in-training need to acquire the necessary competencies to communicate effectively with patients and families in these situations. The Accreditation Council for Graduate Medical Education (ACGME) Clinical Learning Environment Review (CLER) program seeks to improve engagement of residents and faculty in enhancing quality, safety, and professionalism within the academic medical center,¹  explicitly identifying training in error disclosure as an element of the “CLER Pathways to Excellence.” Medical schools and residency programs increasingly educate their trainees in error disclosure.²  While trainee willingness to disclose mistakes has improved over time,³  it is unclear whether their disclosure skills have improved.

We assessed error disclosure skills in internal medicine (IM) residents in 2005, and found residents often omitted important facts surrounding the error, did not explicitly apologize, and failed to discuss future prevention of errors.⁴  Since that time, the local landscape relevant to quality and safety of care has changed significantly. In addition to increased formal training on patient safety at our institution, the Canadian Patient Safety Institute⁵  published national guidelines for error disclosure in 2008. These are fully endorsed by regulatory bodies⁶  and malpractice agencies.⁷  The guidelines have been translated into open disclosure policies at our institutions.⁸ 

Such widespread change may influence resident experiences with error disclosure, and have a positive impact on their disclosure skills. Our study sought to measure error disclosure skills in residents within training programs involving 3 specialties; we then compared this with our 2005 data for IM residents to determine whether disclosure skills have changed over time. We also explored resident perspectives on elements in the clinical learning environment that influence their learning about and experiences with error disclosure.

We conducted this study in 3 residency programs (IM, paediatrics, and orthopaedic surgery [OS]) at the University of Toronto in Ontario, Canada. As part of the formal academic half-day curriculum for these programs, we delivered a 3-hour error disclosure workshop, during which we taught residents a structured communication framework for error disclosure. Residents practiced disclosing errors to standardized patients (SPs) and received feedback from peers and SPs. Although mandatory, some residents were unable to attend the workshops due to scheduled vacations or call responsibilities. We taught this workshop yearly between 2009 and 2013 for IM residents, and once for paediatrics and OS residents in 2012. Informal discussions related to error disclosure also likely occurred at hospital-based morbidity and mortality rounds, other patient safety educational sessions, or in the clinical setting; training programs did not formally track or document these occurrences.

The structured assessment took place during a formative objective structured clinical examination (OSCE) organized by each program. Scheduling of the OSCE, which determines which residents are assigned to the error disclosure station, and resident availability dictated which residents were eligible for inclusion in our study. We only included residents assigned to the error disclosure station who consented to having their data used, which were 86 eligible postgraduate year (PGY) 2 IM residents, 20 PGY-2 and PGY-3 paediatrics residents, and 13 PGY-3 OS residents in 2012–2013. We also included 48 eligible PGY-2 IM residents who had their error disclosure skills assessed using the same structured assessment in 2005.⁴ 

The Research Ethics Boards of the University of Toronto and the Hospital for Sick Children in Toronto approved this study. Consent was obtained for the use of resident data derived from structured assessments and interviews.

Each resident read a vignette describing a medical error and then disclosed the error to an SP. SPs rated each resident's performance and provided immediate feedback. Each resident also completed a questionnaire about his or her prior experience with and training in error disclosure, and whether he or she had received feedback on disclosure skills in practice.

Although it would have been preferable to use the same error scenario of unintended insulin overdose as was used in 2005 (provided as online supplemental material), because we subsequently used this scenario in teaching, we worried that residents might be familiar with it. Therefore, in 2012 we created a new scenario—administration of medication despite a well-documented allergy (provided as online supplemental material)—and trained SPs on it. We then piloted the new scenario on residents and faculty from all 3 programs. To examine for the potential effect of the case (insulin versus allergy) on resident disclosure skills, we randomly assigned IM residents in 2013 to undergo error disclosure skills assessment using either the “insulin case” or the “allergy case.” In anticipation of this analysis, we did not use either case for our disclosure training that year.

For all error disclosure skills assessments, SPs were trained to use a previously developed rating scale (provided as online supplemental material).^4,9  The rating scale includes 5 subdomains focused on what patients expect from an error disclosure conversation¹⁰: (1) explanation of the medical facts regarding the error; (2) honesty and truthfulness; (3) empathy; (4) prevention of future errors; (5) and general communication skills. The SPs graded overall performance on each subdomain using a 5-point Likert scale and scored individual items within each subdomain using a 3-point scale (1, not done at all; 2, attempted but either not complete or not effective; and 3, excellent). As our primary outcome measure, we calculated the mean of the overall ratings for each of the 5 subdomains. In the 2005 analysis,⁴  interrater reliability coefficients between SP ratings and an independent physician observer were fair to excellent for the 5 subdomains (0.51–0.80) and the overall score (0.74); therefore, in 2012 and 2013, we only collected and reported individual SP ratings.

For our first analysis, we conducted a 1-way analysis of variance (ANOVA) to test the specific relationship between a training program (IM versus paediatrics versus OS) and SP rating of overall performance, as well as performance on individual subdomains in the 2012 cohort. Second, we conducted a 1-way ANOVA with the case (insulin versus allergy) as the between-subjects factor to assess the performance implications of the 2 different cases (2013 cohort only). Third, we conducted a multiple linear regression analysis to investigate the influence of predictor variables on overall performance, including study cohort (2005 versus 2012 versus 2013); training program (IM versus paediatrics versus OS); prior experience with error disclosure; attendance at a disclosure workshop; and any self-reported prior disclosure training.

We performed all analyses using SPSS version 22 (IBM Corp, Armonk, NY). We considered P < .05 as statistically significant for all analyses except for the multiple comparisons of subdomain performance across programs, when we applied the Bonferonni correction (dividing P = .05 by 5, the number of comparisons) and considered P < .01 to be statistically significant.

To explore resident experiences with error disclosure in the real-world setting, we invited IM, paediatrics, and OS residents who attended our disclosure workshop to participate in individual semistructured interviews 6 to 12 months after attending the workshop. We sampled residents who attended the workshop to explore how residents' perception of formal training had prepared them for error disclosure in practice. We attempted to sample purposively using a snowball approach, but we were unable to recruit additional residents (despite offering a modest incentive to encourage participation).

We interviewed each participant for 45 to 60 minutes. We audiotaped and transcribed interviews verbatim. We based our interview guide (provided as online supplemental material) on the literature about teaching and learning error disclosure.

We conducted a qualitative descriptive analysis¹¹  to explore residents' experiences with disclosing errors in the clinical setting, and to characterize the role of formal versus informal learning. We chose this methodology because our aim was to provide an in-depth description of participants' experiences, and we did not start with a preexisting theoretical framework on which to base our data collection and analysis. Two investigators (B.M.W. and L.S.) independently read each transcript, met and carried out constant comparative analysis, iteratively adjusted the interview guide, and identified key themes and representative excerpts from the transcripts.

Table 1 summarizes trainee characteristics across the 3 cohorts of data collection (2005, 2012, and 2013). In 2012, we included 23 of 29 (79%) eligible IM, 16 of 20 (80%) paediatrics, and 10 of 13 (77%) OS residents, all of whom were assessed using the allergy case. In 2013, we included 53 of 57 (93%) additional eligible IM residents to evaluate the effect of case difficulty on performance. For historical comparison, we included 42 of 48 (88%) eligible IM residents from 2005, who were assessed using the insulin case.

A similar proportion of residents reported that they had disclosed an error in the clinical setting in all 3 cohorts (P = .39). In 2012–2013, a higher proportion of residents reported having received feedback on their disclosure skills (39% [19 of 49] and 58% [31 of 53] in 2012–2013 versus 17% [7 of 42] in 2005, P < .001) and having received formal training on disclosure (88% [43 of 49] and 94% [50 of 53] in 2012–2013 versus 50% [21 of 42] in 2005, P < .001) compared to residents in 2005.

We compared residents' skills in error disclosure for the 49 residents in the 2012 cohort. We found no significant difference in overall performance for IM, paediatrics, and OS residents, as represented by mean scores across subdomains of error disclosure skill (F_2,48 = 1.73, P = .19), with scores ranging from 4.1 ± 0.5 to 4.4 ± 0.5. We did find significant cohort differences on the subdomains of “explanation of facts regarding the error” (F_2,48 = 10.91, P < .001) and “general communication skills” (F_2,48 = 5.29, P = .009), with IM and paediatrics residents performing better than OS residents.

To address the concern that differences in case difficulty might account for differences between the 2012–2013 cohorts (with the use of allergy and insulin cases) and the 2005 cohort (insulin case only), we compared the 2013 cohort of IM residents' performance on the 2 different cases. Residents performed better overall (P  =  .002) on the insulin case. Since only the 2005 cohort encountered the “easier” insulin case, this finding would disfavor detecting any improvement in error disclosure skill from the 2005 to the 2012–2013 cohorts.

We used the average overall disclosure performance scores as the dependent variable and pooled data from the 3 cohorts to conduct the multiple linear regression analysis. Stepwise inclusion of all variables (study cohort, training program, prior experience with error disclosure, attendance at our disclosure workshop, and any self-reported prior disclosure training; total model R² = 0.24) revealed that only cohort was significantly associated with overall error disclosure performance (2012 versus non-2012: standardized regression coefficient β = 0.54; t = 3.78; P < .001; for 2013 versus non-2013: β = 0.31; t = 2.63; P = .004); all other variables are P > .12. As a post hoc test, we conducted a 1-way ANOVA with cohort as the independent variable (F_2,137 = 18.88, P < .001), which showed that residents in 2012 (4.4 ± 0.5) performed significantly better (P = .05) than residents in 2013 (4.1 ± 0.8), and both 2012 and 2013 resident cohorts performed significantly better (P < .001) than residents in 2005 (3.5 ± 0.8).

We interviewed 9 residents (4 IM, 4 paediatrics, and 1 OS) to explore their experiences with error disclosure and the role of formal versus informal learning in the clinical setting.

Residents spoke candidly about how their involvement with errors during training motivated them to learn about error disclosure. Residents who had significant involvement with an error described feeling added ownership for that patient and family and took personal responsibility to ensure an optimal patient outcome: “Even though I wasn't supposed to be in the hospital, I stayed for the rest of the night [after an error] until he [the patient] got his procedure, and I accompanied him to the procedure and just basically made sure that everything went as smoothly as possible” (IM 3).

This personal ownership drove their desire to be prepared for the error disclosure process and participate in the actual disclosure conversation: “I felt particularly bad about my part . . . I think it would have been nice to be there for the disclosure, but I think it's often not feasible” (Paediatrics 1). Residents felt that disclosure is now a core skill expected of all residents. Encouragingly, although not universal, some residents noted that the current culture of medicine is generally more supportive: “I think it's getting to the point where it's ‘something that is done.' It's not something new or questioned. It's just what you do, which is good” (IM 2). “I think the culture is becoming more open to disclosure of errors that may have even been considered too minor to disclose years before” (IM 1).

Residents felt that faculty from all 3 programs were supportive during disclosure conversations, and that this support was essential: “I do feel like we have the support of our staff surgeons. And that's a necessary component when you're disclosing medical errors as a trainee” (OS 1). “They [the staff] are always there to discuss it. They ask, ‘Are you okay to do it yourself, or do you want me to come?' The staff is there to support, guide you, and ensure that you're doing everything correctly” (IM 2).

Residents described preparing for and learning about error disclosure both formally and informally (table 2). Formal mechanisms included (1) stand-alone teaching sessions outside of the clinical context, including didactic presentations and workshops, and (2) hospital-based clinical rounds, such as morbidity and mortality case rounds. Residents perceived that formal sessions had variable utility, and individuals with fewer real-world experiences of error disclosure found that the workshop helped to prepare them: “I personally have not had any major experiences with error disclosure . . . Things which I remember [from the workshop] . . . How do you approach them, what should you do, who should actually go and speak to the patient, when should someone go and speak to the patient, who goes in and explains to the patient about the error which took place. And, of course then, the plan we have for minimizing such errors in the future” (IM 4).

The informal mechanisms that prepared residents for error disclosure occurred in the clinical setting and involved (1) observing others disclose errors, and (2) predisclosure planning with more senior residents or faculty physicians. Role modeling, when it occurred, was noted to be especially effective, with residents describing learning a lot by watching others. With respect to senior resident or faculty support, residents described involvement that balanced appropriate supervision with resident autonomy. This was usually well handled, but there were instances when faculty took over the disclosure and thus excluded the resident from gaining experience: “The staff ended up doing the disclosure. I felt prepared enough to do the disclosure myself based on our discussion, but she ended up doing the disclosure” (Paediatrics 1).

No residents reported being expected to disclose errors independently if they did not feel comfortable. The informal learning about error disclosure seemed particularly relevant in situations where residents worked in teams. Importantly, residents noted the significance of having both formal and informal learning opportunities, as they complemented each other: “The 2 or 3 didactic sessions gave that framework and then, seeing kind of practical examples of those sorts of things on an ongoing basis, when I'm on certain rotations, like on the medicine rotation, gives kind of practical examples to see it again and again” (IM 1).

One resident identified variability in faculty error disclosure practices, while recognizing the need for disclosure despite the negative faculty role modeling. This suggests that this resident was able to distinguish appropriate responses and behaviors from suboptimal ones, but this may not be true for all trainees, and may represent an ongoing threat of the hidden curriculum to appropriate error disclosure teaching and learning: “It's very physician-specific, in terms of what kind of error disclosure you get . . . there are [staff] physicians who do not take error disclosure as seriously or as aggressively as other physicians” (Paediatrics 3).

We found that current IM, paediatrics, and OS residents performed well on a structured assessment of their error disclosure skills, and they performed significantly better than a historical cohort of IM residents 7 years earlier. Neither self-reported prior training in error disclosure nor attendance at our formal error disclosure workshop correlated significantly with performance on the skills assessment. These findings prompted exploratory qualitative interviews, which revealed the importance of informal learning and the impact of these types of learning activities and experiences on their future error disclosure practices.

These results suggest that the observed improvement in resident disclosure skills may reflect a number of structural and cultural changes that have taken place in our local and broader context over the past 7 years, some of which may have influenced the clinical learning environment positively to support informal resident learning about error disclosure. This hypothesis is supported by the fact that some residents spoke candidly about the strong and supportive culture surrounding error disclosure, and the important role faculty members and peers played in supporting them in the disclosure process. They also remarked on the importance of observing others role model disclosure behaviors.

Our study findings do not allow us to determine which local changes or external factors (or a combination of the 2) contributed to the observed improvements in resident error disclosure skills. Given the critical link between the safety culture of the learning environment and the downstream patient safety knowledge, skills, and attitudes of physicians-in-training, future research could focus on institutions to identify strategies for creating a supportive environment. This supportive environment could foster the development of core patient safety attitudes and behaviors such as skills in error disclosure. Furthermore, given the ACGME CLER program's focus on patient safety,¹²  data from site visits could identify environments that support error disclosure, and make available key strategies that other institutions could emulate.

Our study has several limitations, including that we studied 3 residency programs at a single institution, and our findings may not be fully generalizable to other settings or disciplines. One of the 2 cases used was more difficult than the other. However, this case was only used in 2012 and 2013, disfavoring our finding of overall skills improvement. While several of our SPs were the same in 2005 and 2012–2013, several SPs were also new. Previous reliability testing showed moderate to good interrater reliability among SPs,⁴  and we did not repeat interrater reliability testing with new SP raters. We had difficulty recruiting residents for interviews and did not fully achieve thematic saturation. In addition, residents who volunteered may have had more positive experiences with error disclosure than the overall population of interest.

At a single institution, residents in IM, paediatrics, and OS have comparable skills in error disclosure, as judged by an OSCE with SPs, and performed better than a historical cohort of IM residents. Additional findings suggest that formal training alone, such as workshops, may not be sufficient to improve error disclosure skills. We identified key activities, such as role modeling and a strong local patient safety culture, that create a sense among residents that error disclosure is “just something you do.” These findings support the recent emphasis on the clinical learning environment as a critical target for advancing patient safety education.