Creating Patient Safety Team Members Through a Simulation-Based Interprofessional Root Cause Analysis Course Open Access

Patient safety events resulting from medical errors interfere with the delivery of safe and high-quality health care, negatively affect physician well-being, and represent the third leading cause of death in the United States.^1-3  It appears that historically few graduate medical education (GME) programs have incorporated formal training regarding medical errors into their curricula. Root cause analysis (RCA), recognized as a powerful tool for achieving safer health care, has been increasingly used as an educational method for residents.⁴ 

In 2017, the Accreditation Council for Graduate Medical Education updated their Common Program Requirements to include that “residents must participate as team members in real and/or simulated interprofessional clinical patient safety activities, such as root cause analyses.”⁵  However, participating effectively in an RCA may require intentional training or prior experience. Most facilities lack standardized RCA processes and have insufficient opportunities for all residents to paricipate.⁶ 

Patient safety literature suggests that simulation is useful for teaching complex skills, including communicating effectively within multidisciplinary care teams,⁶  and for improving participants' familiarity with the RCA process.⁷  Some facilities have developed mock RCAs to train residents, leading to increased confidence in RCA participation.⁷  However, these mock RCAs have been small in scale and involve only one department.

Our institution developed an RCA training program using didactic lectures and small group activities to replicate an RCA using a real sentinel event case. The framework was built on RCA²  (RCA Squared) guidelines published by the National Patient Safety Foundation in 2016⁸ and RCA Tools from the VHA National Center for Patient Safety.⁹  Our objective was to use simulation to improve RCA knowledge and confidence for a diverse interprofessional group of participants, while simultaneously increasing the pool of trained personnel, particularly within GME, to participate in RCAs. We describe the development and implementation of our course, RCA W3 (What happened? What should have happened? What are you going to do to fix it?), and discuss the outcomes and feasibility to encourage adoption of similar programs elsewhere.

Eligible participants for this voluntary course during academic years 2018-2019 (AY18) and 2019-2020 (AY19) included all residents and fellows, as well as GME faculty and non-physicians interested in patient safety. Participants were from all departments at our institution, an urban 425-bed military health care facility with 36 total GME and graduate allied health programs. The course was advertised across our health care market and during Graduate Medical Education Council meetings to encourage programs to support resident attendance. The course utilized a flipped classroom approach.

Participants attended an 8.5-hour in-person course, for which continuing medical education credit was available. Content taught by patient safety leaders included didactic sessions on fundamentals of patient safety science, RCA principles, fact gathering, identifying causal factors, and developing corrective action plans. Didactic sessions were followed by facilitator-led small groups (see online supplementary data). There were 12 (AY18) and 17 (AY19) small groups of 5 to 6 members with 1 to 2 facilitators per group. A real sentinel event from the institution was used to ensure high fidelity in the simulated environment. Course participants practiced interviewing with 10 volunteer “actors” portraying the roles of the personnel involved in the sentinel event. Each had access to the redacted medical record and personal statements of the individuals interviewed during the real-world sentinel event. Volunteer facilitators included trained patient safety specialists and clinical staff with experience leading an RCA. All facilitators attended a 1-hour train-the-trainer session prior to the course.

Each small group simulation experience was followed by a large group where findings were presented and discussed. Redacted RCA components from the case were released sequentially to allow participants to compare their ideas to those generated during the actual RCA. The large group sessions were facilitated by the course director (R.I.M.) as per the course agenda (online supplementary data).

Prior to the course, participants completed an 11-item pre-course survey, which collected data on prior participation or education about the RCA process (online supplementary data). All surveys used for the course were developed by the authors and no validity data was collected prior to their use. Four questions used a Likert scale to explore participant comfort interviewing for an RCA and confidence in participating or leading an RCA. After initial survey submission, participants completed 5 hours of pre-course content focusing on knowledge to maximize meaningful engagement (online supplementary data).

Following pre-course work completion, a 27-item pre-course assessment evaluated knowledge of topics covered in the pre-course material. The pre-course assignments, evaluations, and redacted charts were released iteratively, requiring participants to finish one task before gaining access to the next.

Immediately following the course, and again 9 to 10 months after the course, participants completed a post-course survey and follow-up survey to examine confidence levels with the RCA process (online supplementary data). All surveys contained questions 7 to 10 of the follow-up survey (confidence participating in an RCA, leading an RCA, interviewing people involved in sentinel events, and RCAs leading to patient safety improvements). Results of these surveys were reported as percentages and as means (confidence intervals) based on a standard Likert scale of 1 to 5 (Table 1). Likert scale data was analyzed in a repeated measures analysis of variance comparing any difference in the initial 3 surveys collected: pre-course survey, pre-course assessment, and post-course survey, then in comparison to the 10-month follow-up results. Additionally, the authors collected information on subsequent participation of course attendees in real-world RCAs by reviewing RCA attendance records, which are maintained by hospital patient safety, at 6-month intervals following implementation of the course, an ongoing practice at our institution to determine the level of GME participation in patient safety at the institutional level. SurveyMonkey was used for surveys; statistics were performed using JMP 13.2 (SAS Institute Inc., Cary, North Carolina).

Institutional Review Board screening determined the study to be exempt as non-research.

In AY18 and AY19, 162 course participants were trained, including 59 (36.4%) residents and fellows from 23 different GME programs. Response rates were 96.9% (157 of 162) for pre-course survey, 92.6% (150 of 162) for pre-course assessment, 97.5% (158 of 162) for post-course survey, and 81.5% (132 of 162) for follow-up surveys (Table 1). The pre-course survey was used to establish participants' baseline experiences and formal education about RCAs. It demonstrated that most course attendees had never participated in an RCA and had no prior training in conducting an RCA. Additionally, the majority had no prior exposure to basic tools used in the RCA process, including RCA² : Improving Root Cause Analyses and Actions to Prevent Harm, the VA Corrective Action Hierarchy, and Safety Assessment Codes (Table 1).

A comparison of the 4 questions: confidence participating in an RCA, confidence that an institutional RCA will lead to improved patient safety, comfort interviewing staff members involved in a sentinel event, and confidence leading an RCA showed statistically significant improvements in all but confidence in institutional RCAs when comparing the pre-course survey to the post-course and follow-up surveys (Table 2). The follow-up survey demonstrated a slight decline in confidence compared to immediately after the course; however, confidence remained higher than before the in-person course (statistically significant compared to before the pre-course survey). Confidence in RCAs leading to improved patient safety were not significant. Otherwise, respondents' values for this question remained constant without any statistically significant changes. The greatest improvements in confidence for participation, interviewing, and leading an RCA were by residents, followed by faculty across all categories, which was sustained on the follow-up survey. In particular, nurses and “other” personnel only reported a statistically significant improvement in confidence leading an RCA immediately following the pre-course assessment and at the follow-up survey.

RCA participation collected from RCA attendance records demonstrated that 90% (9 of 10) of institutional RCAs had resident/fellow team members in AY19 and 92.3% (12 of 13) in AY20, compared to 63% (5 of 8) in the year preceding course implementation. In the 3 years following course implementation, 45 residents/fellows participated in institutional RCAs, of whom 32 were course graduates. As of April 2022, 44.0% (26 of 59) residents/fellows who attended the course have participated in institutional RCAs, higher than any other professional group of attendees.

The RCA W3 course is a feasible model for implementation at other institutions. It requires minimal funding but does need significant person-hours to complete the lengthy process of identifying an appropriate case for RCA simulation, redacting medical records, recruiting and training course facilitators (of which more than 30 hours were needed to allow for small group ratios of 1-2 facilitators:5-6 participants), and the logistical demands of finding appropriate space. Subsequent iterations of the course required substantially less person-hours because redacted case materials were completed and the facilitator pool was established. Prior course participants who had subsequent institutional RCA experience were also recruited as facilitators.

A total of 162 interprofessionals from 48 departments participated in the RCA W3 courses. Participants improved and retained confidence in their ability to conduct an RCA. The course helped to create a pool of RCA-trained residents/fellows for institutional RCAs. The resident/fellow group had the greatest improvement in confidence for participation, interviewing, and leading an RCA, which may be secondary to the higher rate of RCA participation in this group. The pool of residents/fellows and faculty with additional RCA training now helps to ensure GME participation for future RCAs.

The course design is informed by experiential learning theory as proposed by Kolb and Kolb.¹⁰  By using small group simulation, discussion, and feedback, the course allows for concrete experience followed by reflection discussions that promote the application of the new perspectives and ideas gained. These are critical when considering the 4 principles of andragogy (adult education) from Knowles et al.¹¹  Additionally, while prior studies have examined the use of simulation to teach RCA competencies to trainees, these interventions were not interprofessional.^12,13  The use of multidisciplinary small groups and selection of a sentinel event case promotes interprofessional collaboration, encourages emotional/experiential learning, and creates an environment of higher fidelity in comparison to resident-only simulated interventions.^14,15  Additionally, it capitalizes on the capacity of simulation to effectively practice communication within a multidisciplinary team.⁶ 

Previous studies demonstrate that resident practice patterns reflect those of their training institution, which remains true years later, suggesting that professional development is highly influenced by exposures during GME training.^16,17  This is of particular significance in the military health system where we retain 100% of our GME graduates.

Compared to other RCA training platforms, this course was developed as a 1-day model to facilitate resident and faculty participation with minimal competition with occupational and program demands. Following course completion, we learned that participation in institutional RCAs was often challenging for residents due to the short lead time, which was mitigated through development of a computer-based “RCA On-Call Calendar” so participants could identify their availability in advance. There was a nonsignificant increase in the percentage of RCAs with resident/fellow participation following the course.

The study has several limitations. While the residents and fellows who participated had the greatest retention in confidence regarding RCA participation, it is unknown whether this increase was due to course participation or the higher rate of institutional RCA participation. Data collected focused on participants' self-perceived knowledge and confidence (a relatively weak outcome measure) and did not include objective assessment of competence performing an RCA or in retention of RCA knowledge at follow-up. Finally, the response rate for the follow-up survey was 81.5%, which was lower than for the other 3 surveys.

Despite these limitations, this initiative benefits from its potential for expansion in number of courses, participants, or exportability to other facilities. Course development and material assembly has been accomplished, thus the work involved for future iterations has substantially decreased. Our hope is to explore virtual formats and to increase the course frequency as there was notable increase in popularity from one year to the next with positive feedback.

The interprofessional curriculum designed to train participants to conduct an RCA in response to a patient safety event demonstrated a sustained increase in confidence and participation in institutional RCAs, especially among residents.