An Assessment of an Educational Intervention on Resident Physician Attitudes, Knowledge, and Skills Related to Adverse Event Reporting

The reporting of adverse patient events including near misses and unsafe conditions is essential for patient safety, patient care, improvements in physician-patient communication, and the education of health care workers—all of which contribute to safer systems-based practices. The Institute of Medicine reported in 1999 that the deaths of up to 100 000 patients per year may be due to preventable adverse events.1 More than 10 years ago, the authors of the report1 asked health care organizations to create voluntary reporting systems to improve the understanding of factors that contribute to medical errors. The Joint Commission requires the establishment of a reporting system by accredited organizations for adverse events.2 Despite these mandates and perceived benefits of reporting,1,2 a survey of physicians in teaching hospitals revealed that only 54.8% of the physicians surveyed knew how to report medical errors and only 39.5% knew what errors to report.3 In our own institution, fewer than 1% (<30 total reports) of event reports come from the more than 500 resident physicians who rotate through the University of Illinois Medical Center at Chicago (UIMCC). In a questionnaire survey, White et al4 found that only 31% of interns or residents reported receiving instruction in error disclosure techniques. Kaldjian et al5 identified factors that may facilitate (eg, responsibility to the patient and profession) or impede (eg, attitude, fears, and anxieties) reporting of adverse events. To investigate whether attitudes toward reporting and reporting skills could be improved through education, a patient safety and medical fallibility curriculum was developed by Madigosky et al.6 The researchers found that this curriculum improved some attitudes and skills towards error reporting in the short-term, but improvements could not be sustained after 1 year.

Our study is the first of its kind. It pairs an educational training program (intervention) focused on the importance of adverse event reporting with an expectation of reporting as a means of engaging anesthesiology resident physicians in the Accreditation Council for Graduate Medical Education (ACGME) 6 core competencies. The objective was to evaluate attitudes, knowledge, and skills of anesthesiology residents before and after an educational intervention focusing on the medical center's comprehensive response to adverse event reporting and follow-up.

Participants in this prospective assessor study included anesthesiology resident physicians in a clinical rotation at UIMCC. The residents were exempt from informed consent since this study was considered an educational practice in error reporting, which is mandatory training for residents. The anesthesiology residents served as their own historical controls. The survey was administered at the residents' scheduled educational conference. The survey questions are found in table 1. To ensure anonymity, anesthesiology residents were assigned a numerical identification, with the key being held by the honest broker. The honest broker was an individual not involved in the study design or proceedings who had access to a list corresponding to individual subject names with their numerical designation. With this system, data per resident could be tracked to compare serial survey results and quantity of reports. The data were maintained in a password-secure database. All data were managed by the honest broker. All data were “deidentified” before release to the independent review panel. The study protocol was reviewed and deemed exempt by the University of Illinois at Chicago Institutional Review Board.

For the online reporting system, access to the database for data retrieval and analysis was restricted to the Department of Safety and Risk Management. However, for concerns of patient safety and the well-being of residents involved in adverse events, and for the fulfillment of the ACGME-identified competencies, the adverse event reports were reviewed daily by the Department of Safety and Risk Management and the Department of Anesthesiology's residency program director. Confidential support services were provided as part of a departmentally approved “second victim”7 program, as indicated and identified by the Department of Safety and Risk Management and the program director.

At t  =  0, anesthesiology residents were given a questionnaire survey evaluating individual attitudes, experience, and knowledge regarding adverse event reporting. For this study, questions were aggregated into the following domains: fear and barriers to reporting, institutional support, and the mechanics of reporting. This survey provided a baseline attitude for each subject before the educational intervention. Anesthesiology faculty assisted with the survey administration.

After the survey completion, residents participated in the educational intervention that consisted of an interactive case-based lecture focusing on the importance and mechanics of adverse event reporting. An adverse event reporting expectation was implemented for anesthesiology residents. The residents were also informed that after the report of some selected events they would be engaged in root-cause analysis and performance improvement follow-up efforts. The goals of the program were to improve knowledge, attitudes, and skills related to patient safety and adverse patient events. The educational program was taught in a case-based lecture format with hands-on interaction with the online reporting system. The intervention included the following.

I. A 90-Minute Educational Program (Including the Following Topics)

1. A. The definitions of adverse event, medical error, serious error, minor error, and near miss. Also, the ways in which event reporting addresses all 6 ACGME core competencies8 were emphasized.

2. B. UIMCC's comprehensive response to adverse events in relation to the ACGME core competencies: patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice.8 

3. C. The mechanics of reporting and responding to adverse events,9 are as follows:

   1. Reporting: Residents were instructed on the methods of reporting and the process of online reporting. The completed report is sent to the Department of Safety and Risk Management, a report copy is sent to the program director's work list, a portfolio is created for the resident, and the report is entered into a database.

   2. Investigation/process improvements: After reporting, the degree of harm is assessed by the Department of Safety and Risk Management. If harm occurred, an investigation ensues, a root-cause analysis is completed, a “second victim” is considered, and bills are held.

   3. Communication: The UIMCC Patient Communication Consult Service is a service at UIMCC available 24 hours a day, 7 days a week, to respond to all serious unexpected adverse events linked to patient harm. It involves a multidisciplinary team approach to the rapid investigation of adverse events and assistance in assuring effective and honest communication with patients and families.

   4. Apology and remedy: Residents are educated to understand the power of effective, honest communication and apology, when indicated, for patients and providers; to acquire knowledge and skills in the element of “full disclosure”; to develop skills necessary for apology; to understand the impact of “no pay for bad performance”; and to appreciate the financial impact of harm caused by error.

1. A. Distributed to residents during educational program.

2. B. Included references to the following: National Quality Forum Serious Reportable Adverse Events in Healthcare, The Joint Commission Sentinel Events, and Centers for Medicare and Medicaid Services.

3. C. Provided connection to the application of the 6 ACGME core competencies.

4. D. Included Occurrence Reporting Quick Reference Guide.

1. A. Reviewed material from educational program.

2. B. Provided feedback to anesthesiology resident physicians.

3. C. Discussed type and quantity of adverse event reports.

4. D. Discussed adverse event–associated process improvements.

1. A. Provided near-immediate feedback to residents upon receipt of adverse event reports. In high-yield specific situations, residents were engaged in root-cause analyses and the subsequent process improvement efforts. Those process improvements were then communicated to medical staff leadership and University of Illinois Board of Trustees, with subsequent feedback to the resident physicians.

2. B. Met regularly with residency program directors to answer clarifying questions and provide educational support.

The aforementioned formal educational lecture program began in July 2008 and was repeated in July 2009 for the educational benefit of new anesthesiology residents. The additional elements of learning and education discussed above are ongoing and have become an integral part of the formal resident curriculum.

After the education session, all residents were informed that they were expected to report adverse events via the online reporting application or any other methodology while rotating at UIMCC and that these reports would serve as one basis for assessing their progress in the core competencies.

Quarterly adverse event reports were analyzed retrospectively for the 2 years before the intervention and then prospectively on a quarterly basis by the independent review panel. Analysis of the adverse event reports by the departmental quality officer and other departmental content experts included an assessment as to whether or not the absence of an attending anesthesiologist played a role in the event. In addition, the category (eg, equipment, medication, procedure related) and degree of harm or potential harm were identified. At t  =  12 months, residents responded to the same survey given at t  =  0 month. Survey responses were compared between t  =  0 and t  =  12 months. The data were compiled by the independent review panel and analyzed.

Statistical analysis was performed with Statistical Package for the Social Sciences (SPSS, Chicago, IL), version 15.1. One-way analysis of variance (ANOVA) was used to test the difference between preintervention and postintervention data. Normality assumption was checked for the total scores.

Our results demonstrated that a comprehensive educational intervention beyond a single lecture, coupled with an expectation of reporting, increased the number of adverse event reports submitted by anesthesiology resident physicians, improved their attitudes about the importance of reporting, and produced a source for learning opportunities and process improvements in the delivery of anesthesia care.

Anesthesiology resident physicians' attitudes about error disclosure improved significantly after an educational intervention with an error-reporting requirement. We compared the preintervention and postintervention survey results. There was a statistically significant improvement in resident attitudes regarding the value of error reporting and the available emotional support for error reporting between the preintervention and postintervention surveys. Furthermore, after the intervention, residents expressed less concern about factors that can impede error reporting. For example, resident worries about disciplinary action, litigation, lack of collegial support, or concerns about what incidents to report decreased (table 1). Furthermore, after the intervention, resident attitudes about the adequacy of the current system for reporting patient safety problems and about the availability of support services for providers coping with stress also improved (table 1).

The number of adverse event reports increased from 0 per quarter in the 2 years preintervention to 28 per quarter for the 7 quarters (21 months) postintervention, with no sign or evidence of decay (figure).

An analysis of the anesthesiology resident physicians' reports showed several categories of harm events, near misses, and unsafe conditions, including reports of disruptive providers. Of the harm events associated with complications related to procedures (intubation, nerve block, central line placement), 9 of 17 were associated with the inadequate supervision by anesthesiology attending physicians (table 2).

This study showed that a comprehensive educational curriculum focusing on the importance of adverse event reporting, coupled with an expectation of reporting as a means of engaging residents in the 6 ACGME core competencies, increased the number of anesthesiology resident physicians' adverse event disclosures, improved attitudes about medical error disclosure, and produced a sustainable source for potential learning opportunities and process improvements in the delivery of anesthesia care.

Other studies have evaluated an online reporting system10 as well as an educational curriculum's effect on attitudes toward adverse event reporting.6 Unlike the study of Madigosky et al,6 our study of anesthesiology resident physicians also used an online reporting system, coupled with an expectation of reporting as a means of engaging anesthesiology resident physicians in the root-cause analysis and follow-up, thereby increasing the exposure and opportunity for assessment in the 6 ACGME core competencies. All factors in our methodology contributed to the sustained improvement in resident attitude and behavior.

The improvement in attitude toward and increase in frequency of adverse event reporting can be explained by the educational intervention and the decreased expression of concern about factors that impede error reporting. Furthermore, anecdotal resident comments revealed that, in the residents' opinion, adverse event reporting into the online database emphasized a systems-related approach as opposed to a “shame and blame” environment. Several residents commented that they felt “safe” with the reporting methodologies and follow-up. Residents also expressed that they felt empowered by opportunities to make an impact in the care of patients and felt the support of their colleagues, as reflected in their improvement in attitude.

Are these results specific to anesthesiology resident physicians? A study by Jagsi et al11 revealed that surgical residents were more likely to report adverse events than medical or hospital-based (pathology, anesthesiology, radiology, emergency medicine) residents. In investigating factors affecting error reporting by physicians, Kaldjian et al3 found that responses from pediatricians were not significantly different from those of physicians in family medicine and internal medicine.

With regard to the specific adverse events reported, 9 of 17 complications related to procedures (intubation, nerve block, central line placement) were associated with inadequate supervision by anesthesiology attending physicians (table 2). In a survey by Jagsi et al,11 residents reported procedural and medication-related events as the most common adverse events and felt that one of the causes of their most recent error was inadequate supervision. Measures to address the potential systems-level failures identified by our residents underwent root-cause analysis and process and performance improvement identification via UIMCC's comprehensive process for responding to adverse events.

Our study has several limitations. First, residents' answers may have been influenced by social desirability, leading us to overestimate positive attitudes. Since the attitudes that impede error reporting were indeed expressed preintervention, with an improvement in attitude postintervention, we feel the residents' responses were not greatly influenced by perceived desired behaviors. Second, we assumed computer proficiency in the online reporting system. Our training program explained how to report adverse events in the current system; furthermore, the residents reflected their proficiency, as shown by the increase in the quantity of reports during the study period.

Third, we did not confirm whether the adverse event reporting rates were comparable to adverse event data found in patients' medical chart reviews. O'Neil et al12 compared resident reports (89) with a medical record review (85) and found that both methods uncovered similar numbers of adverse events; however, the 2 methods converged on the case related to the same patient for only 41 of the cases of error. Perhaps the reviewers were unable to identify events owing to failure of documentation. Also, the residents were more likely to report preventable adverse events and errors of commission than were physician reviewers during medical record review. Therefore, resident reporting may identify adverse events or near misses that other mechanisms may miss. O'Neil et al12 reported further that the physician reporting mechanism was less costly than the medical chart review.

Residents are our front-line group of future physicians. It is essential that these future attending physicians have appropriate training, mentoring, and support in adverse event reporting. This approach can succeed in residency programs with the visible commitment of the Department of Safety and Risk Management and a nonpunitive approach to adverse event reporting. Furthermore, simulation may play a role in providing residents with an arena in which to practice disclosure activities in a controlled environment.