Background

Following the Accreditation Council for Graduate Medical Education recommendations in 1999 to foster education in the systems-based practice (SBP) competency by examining adverse clinical events, institutions have modified the morbidity and mortality conference (MMC) to increase SBP-related discussion. We sought to examine the extent to which SBP-related content has increased in our department's MMCs compared with MMCs 10 years prior.

Method

We qualitatively analyzed audio recordings of our MMCs during 2 academic years, 1999–2000 (n  =  30) and 2010–2011 (n  =  30). We categorized comments and questions from moderators and faculty as SBP or non-SBP and characterized conferences by whether adverse events were presented and which systems issues were discussed.

Results

Compared with MMCs in 1999–2000, present-day MMCs included a greater average percentage of SBP comments stated (69% versus 12%; P ≤ .001) and questions asked (13% versus 1%; P  =  .001) by the moderator, SBP comments stated (44% versus 4%; P ≤ .001) and questions asked (19% versus 1%; P ≤ .001) by faculty, and were more likely to present adverse events (87% versus 13%; P < .001). Interrater reliability for the distinction between SBP and non-SBP content was good (κ  =  0.647). Most common categories of systems issues discussed in 2010–2011 were critical laboratory value processing and reporting, institutional policies, and hospital-based factors.

Conclusions

Over the past decade, our MMC has transformed to include more discussion of SBP-related content and adverse events. The MMC can be used to educate residents in SBP and can also serve as a cornerstone for departmental quality and safety initiatives.

What was known

Increased awareness of causes of adverse clinical events has resulted in enhanced focus on systems-based practice (SPB) in the past decade, extending to the morbidity and mortality conference (MMC).

What is new

MMCs today included a greater percentage of SBP comments stated compared with a decade ago, with the most common systems issues being critical laboratory value reporting, institutional policies, and hospital-based factors.

Limitations

Single-site study limits generalizability, analysis focused on quantity rather than quality of SPB content.

Bottom line

Growing focus on SBP can help identify systems problems and enhance residents' interest in improving the system in which they provide care. Programs should consider the MMC as a venue for resident education in SBP.

Since its inception in the mid-20th century, the morbidity and mortality conference (MMC) has played a pivotal role in resident education.1,2 The initial MMCs focused on systems failures and improving patient outcomes, but during the late 20th century internal medicine and surgery residency programs used MMCs to publicly address the fallibility of individual physicians.1,37 By the 1990s, nearly 90% of programs had longitudinal MMCs with varied formats, and most cases did not include morbidity or mortality and rarely discussed unexpected adverse outcomes or processes that contributed to systems-based failures.2,8,9 

Because it provides a forum to discuss adverse events and medical errors, the MMC enables residency programs to address the systems-based practice (SBP) competency as defined by the Accreditation Council for Graduate Medical Education (ACGME) and to fulfill the Institute of Medicine (IOM) recommendations to educate physicians in the examination of systems issues.1012 Nevertheless, programs have had difficulty teaching and assessing the SBP competency.1315 Internal medicine programs have adopted or modified MMCs to focus more on identifying systems issues and implementing improvements from select cases.12,1518 Investigations of MMCs have characterized participant attitudes, frequency of adverse events discussed, and hospital changes resulting from events discussed during the conference. However, no studies have specifically assessed the content of the MMC to verify the prevalence of systems issues discussion in the years following the IOM and ACGME recommendations.

The purpose of our study was to examine the extent to which SBP-related content and presentation of adverse events have increased in our department's MMCs compared with MMCs 10 years prior. We hypothesized that we would detect an increased focus on SBP-related issues. We also depict the function of our MMC in 1999 compared with its current placement within our hospital's quality improvement (QI) processes.

Study Design

We conducted a retrospective cohort study of MMCs at the Beth Israel Deaconess Department of Medicine comparing academic years 1999–2000 to 2010–2011. The overall format of our institution's MMC during the years studied was a large, multidisciplinary conference that was mandatory for residents and was well attended by department leadership (eg, residency program directors, chair) and other members of the medical faculty. For each conference, participants from other specialties (eg, radiology, pathology) and other professions (eg, nursing, pharmacy) were invited as consultants. Conferences were 1 hour long and held once a week in the morning during most weeks in the academic year. Chief medicine residents (CMRs) selected cases for discussion and prepared projector or PowerPoint slides narrating the clinical course for a resident to read verbatim during the conference. CMRs moderated the conferences, allowing audience and participant questions and comments. Of the 38 MMCs during the 1999–2000 year, 33 were recovered in audiocassette format and converted to digital audio format for the purposes of this study. All MMCs from the 2010–2011 academic year were digitally recorded for educational purposes. The Institutional Review Board declared this study exempt on the grounds that it assessed the quality of standard educational activities.

Preliminary review of MMC recordings confirmed that the content of MMCs was evident in 3 forms: (1) scripted language based on the projector or PowerPoint slides (usually limited to clinical information), (2) questions from the CMR to invited faculty and resident consultants, and (3) spontaneous comments and questions from the audience. We observed that SBP-related content was best reflected in the latter two categories. We then categorized all nonscripted language as (1) questions or comments; (2) arising from CMRs, residents, or faculty; and (3) SBP related or not. We defined questions as nonrhetorical requests for information or elucidation directed toward specific individuals with the explicit expectation of a response, whether or not it was worded as a question. For example, a request to the invited pathologist to review pathology slides for the audience would be considered a question. All other statements were considered comments. We defined content as SBP if it related to resources or services within the infrastructural design of the health care setting both inside and outside of our own institution (eg, online medical record, sign-out process, hospital policies, etc). All other content was considered non-SBP by default and tended to pertain to patient care, medical management, or medical knowledge. If a participant asked two or more questions in series, they were counted as one question if the contents of the questions were related. Direct responses to questions were not counted as additional content. However, follow-up comments from other individuals were considered new content. Questions or comments intended to seek clarification about historical or clinical information (eg, “Was a phosphorus checked at this point?”) and prescripted case information presented by the reader or CMR were not classified or analyzed.

We characterized each MMC case by whether it involved a death, a respiratory or cardiac arrest, a longer hospitalization, and/or a readmission. We determined whether the case presentation included an adverse event and/or a medical error. We defined an adverse event as an injury resulting from a medical intervention and a medical error as a failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim. The discussion of adverse events was further characterized as none, implicit, or explicit. For each conference, we sought agreement with the following statement: “Blame was attributed to an individual during this conference,” with the responses agree, neutral, and disagree. We also identified the SBP topics addressed during all MMCs analyzed from the 2010–2011 year.

Reliability

We (J.D.G. and G.C.H.) compared and discussed pilot results of counts and ratings of a single MMC prior to embarking on the formal analysis. After analyzing half of the MMCs, we performed a midpoint calibration and discussed cases where our counts differed by greater than 3, making adjustments where applicable (which did not always result in identical counts). We repeated this process after all MMCs were reviewed.

Data Analysis

We tabulated the number of comments stated and questions asked by speaker (CMR, residents, and faculty) and by category (SBP versus non-SBP). We compared results, as well as the number of adverse events and medical errors, by academic year studied. We used χ2 tests for frequencies (eg, question and comment counts, adverse events) and Wilcoxon rank sum for ordinal variables (eg, blame of an individual). We used a weighted Cohen κ to determine interrater reliability. The analyses were performed using Stata/IC-8 (College Park, Texas).

We analyzed 30 MMCs from 1999–2000 (excluding 3 because of technical issues) and 30 MMCs from 2010–2011. The types of cases discussed did not differ by involvement of codes, readmission, or prolonged hospitalizations; more cases in 2010–2011 included a patient death (table 1). Compared with MMCs in 1999–2000, recent MMCs included a greater percentage of SBP comments stated (69% versus 12%; P < .001) and SBP questions asked (13% versus 1%; P < .001) by the CMR moderator, and a greater percentage of SBP comments stated (44% versus 4%; P < .001) and SBP questions asked (19% versus 1%; P < .001) by faculty (table 2). Interrater reliability for the differentiation of SBP from non-SBP content was good at 0.647.

Table 1

Characteristics of Cases Presented and Discussed in Morbidity and Mortality Conferences in 1999–2000 (n  =  30) and 2010–2011 (n  =  30)

Characteristics of Cases Presented and Discussed in Morbidity and Mortality Conferences in 1999–2000 (n  =  30) and 2010–2011 (n  =  30)
Characteristics of Cases Presented and Discussed in Morbidity and Mortality Conferences in 1999–2000 (n  =  30) and 2010–2011 (n  =  30)
Table 2

Percentage of Systems-Based Practice (SBP) and Non-SBP Comments and Questions per Morbidity and Mortality Conference in 1999–2000 and 2010–2011

Percentage of Systems-Based Practice (SBP) and Non-SBP Comments and Questions per Morbidity and Mortality Conference in 1999–2000 and 2010–2011
Percentage of Systems-Based Practice (SBP) and Non-SBP Comments and Questions per Morbidity and Mortality Conference in 1999–2000 and 2010–2011

The percentage of MMC cases that involved an adverse event was greater in 2010–2011 than in 1999–2000 (88% versus 13%; P  =  .001). Additionally, the percentage of MMC cases involving medical errors was greater in 2010–2011 than in 1999–2000 (43% versus 7%; P  =  .002). The allocation of blame to an individual or persons occurred infrequently and was not statistically different between 1999–2000 and 2010–2011.

A total of 97% (29 of 30) of MMCs in 2010–2011 included discussion of SBP topics compared with 10% (3 of 30) of MMCs in 1999–2000. A total of 25 of 30 MMCs (83%) discussed at least 2 SBP topics, whereas 1 conference did not discuss SBP-related issues, but focused primarily on medical management. Five cases with systems issues discussed are presented in table 3; a table describing all MMCs studied is provided as online supplemental material.

Table 3

Characteristics and Systems Issues Discussed at 5 Selected Morbidity and Mortality Conferences (MMCs) in 2010–2011

Characteristics and Systems Issues Discussed at 5 Selected Morbidity and Mortality Conferences (MMCs) in 2010–2011
Characteristics and Systems Issues Discussed at 5 Selected Morbidity and Mortality Conferences (MMCs) in 2010–2011

We found that our institution's MMCs incorporated a significantly increased amount of SBP-related content compared with MMCs one decade ago. Similarly, the proportion of MMCs focused on adverse events and medical errors and the degree to which adverse events were discussed also increased. To our knowledge, this work represents the first time the content of MMCs has been examined to confirm an increased emphasis on systems improvement. Our findings are consistent with our observation that systems improvement has become integral to our residency program, spurred by IOM and ACGME recommendations. The MMC provides an ideal venue for discussing events resulting in adverse patient outcomes, system failures related to events, and system improvements occurring in the QI/patient safety process. Working with the QI process, the MMC allows for nonjudgmental evaluation of adverse events and medical errors, with the primary goals of improving patient care and educating all participants through increased awareness about issues related to the SBP competency.

Our results also are consistent with prior studies showing a shift in MMC content to explicitly include systems-related issues. Bechtold et al16 and Deis et al17 described revised MMCs with increased emphasis on systems-based failures and interventions, identifying areas for possible improvement. Kravet et al15 reported an MMC format that sought to teach the 6 ACGME competencies. Additionally, Szostek et al12 implemented a “systems audit” into their MMC, providing residents opportunities to perform root-cause analyses of adverse events, which resulted in several institutional improvements and improved resident perceived awareness of systems issues. However, no study has reported the extent of SBP issues discussed at each conference, the degree of blame in each conference, or factors related to adverse events longitudinally during 2 time periods.

Our results also reflect a prior study regarding adverse events and medical errors discussed in MMCs. Pierluissi et al8 showed adverse events, and medical errors were presented during a minority of cases (37% and 18%, respectively). For our 1999–2000 period, the prevalence of adverse events and medical errors presented was similar (13% and 7%), but it showed a marked increase after the format changes, now more in accord with the ACGME's intended goal to discuss cases focused on adverse clinical events. We hypothesize the low prevalence of adverse events and medical errors presented in 1999 may reflect the underdeveloped review process and perceived limited educational benefit from presenting these cases.

The shift in MMC content over the past decade is concordant with an increasing emphasis on the role of SBP in achieving safe and reliable patient care delivery. Adverse events occurring in complex medical settings often highlight vulnerabilities within systems, rather than independent individual errors. The purpose of reviewing adverse events lies not in ascribing individual blame, but rather in learning how the system of care can be improved to prevent similar events. To this end, our Department of Medicine has embedded the blame-free MMC in our quality improvement efforts, linking it to resident education (figure). Once an adverse event has been identified from voluntary reporting or standard monthly outcomes data review, the case is assigned for review by a nurse patient safety coordinator from the Department of Health Care Quality (DHCQ) and a physician reviewer from the Department of Medicine; in approximately two-thirds of cases, an internal medicine resident serves as physician reviewer (with mentorship from patient safety core faculty). The findings are then presented at a monthly departmental Medical Peer Review Committee (MPRC) meeting. Based on the committee's recommendations, cases with opportunities for systems improvement are referred to the departmental Medical Patient Care Committee for potential practice redesign, and cases from which wider learning might be important are referred to the CMRs to present in the MMC. Additionally, cases not yet presented at MPRC but identified and presented at MMCs by CMRs are referred for formal review. Consistent with this process, the content presented by the CMRs is tailored to highlight vulnerable systems issues and identify potential strategies for improvement.

FIGURE

Relationship between the Medicine Residency Curriculum and Hospital's Quality Improvement Process, 2010–2011

Abbreviations: PCAC, Patient Care Assessment and Quality Committee; QI, quality improvement; CMR, chief medicine resident; D. of HCQ, department of health care quality.

FIGURE

Relationship between the Medicine Residency Curriculum and Hospital's Quality Improvement Process, 2010–2011

Abbreviations: PCAC, Patient Care Assessment and Quality Committee; QI, quality improvement; CMR, chief medicine resident; D. of HCQ, department of health care quality.

Close modal

Our MMC serves multiple functions in disseminating QI education and promoting patient safety objectives. Members of the DHCQ and MPRC regularly attend MMCs, both as participants and invited guests to discuss systems issues. We believe our MMC reflects a culture where medical residents as frontline caregivers are actively identifying and referring adverse events, further reinforcing the importance of real-time systems-failure identification and improvement cycles on our medicine services. As further evidence of our safety culture, global curricular objectives have incorporated fundamental training in QI and patient safety principles and resulted in a QI rotation, allowing residents to work alongside patient safety core faculty, perform root-cause analyses of adverse events, present at QI and patient safety meetings, and play an instrumental role in systems redesign.1921 The MMC works synergistically with the QI process to increase awareness of systems issues for both residents and faculty within our department, which likely improves the care delivery in our hospital.

Study limitations include the assessment of a single site, which limits generalizability due to differences in MMC format, QI culture, or dynamics of participation in MMCs. The questions and comments served as a proxy for SBP content but were analyzed by quantity rather than quality. We also discounted scripted language from slides, which included some SBP content, but these data were more prevalent in 2010–2011 MMCs. Approximately 20% of MMCs from 1999–2000 were not analyzed, which may have affected our data. We analyzed 2 time points rather than MMCs in all interim years; therefore, we can only make conclusions about the overall change, rather than interval trends. Finally, our study does not demonstrate an improvement in resident SBP knowledge or skills.

In addition to providing a platform for residents and faculty to discuss and address quality problems and SBP issues, we believe the MMC likely enhances resident knowledge about QI that can lead to increased identification and improvement of the systems in which we provide care. Finally, a hospital QI infrastructure in which the MMCs are meaningfully integrated may encourage residents to become “champions of change” in the QI process. Programs should consider using the MMC as a venue for resident education in SBP.

1
Orlander
JD
,
Barber
TW
,
Fincke
BG
.
The morbidity and mortality conference: the delicate nature of learning from error
.
Acad Med
.
2002
;
77
(
10
) :
1001
1006
.
2
Orlander
JD
,
Fincke
BG
.
Morbidity and mortality conference: a survey of academic internal medicine departments
.
J Gen Intern Med
.
2003
;
18
(
8
) :
656
658
.
3
Campbell
WB
.
Surgical morbidity and mortality meetings
.
Ann R Coll Surg Engl
.
1988
;
70
(
6
) :
363
365
.
4
Carmichael
DH
.
Learning medical fallibility
.
South Med J
.
1985
;
78
(
1
) :
1
3
.
5
Bosk
CL
.
Forgive and Remember: Managing Medical Failure
.
Chicago, IL
:
University of Chicago Press
;
1979
.
6
Ruth HS
HF
,
Grove
DD
Anesthesia Study Commission; findings of 11 years' activity
.
J Am Med Assoc
.
1947
;
135
(
14
) :
881
884
.
7
Prasad
V
.
Reclaiming the morbidity and mortality conference: between Codman and Kundera
.
Med Humanit
.
2010
;
36
(
2
) :
108
111
.
8
Pierluissi
E
,
Fischer
MA
,
Campbell
AR
,
Landefeld
CS
.
Discussion of medical errors in morbidity and mortality conferences
.
JAMA
.
2003
;
290
(
21
) :
2838
2842
.
9
Seigel
TA
,
McGillicuddy
DC
,
Barkin
AZ
,
Rosen
CL
.
Morbidity and Mortality conference in Emergency Medicine
.
J Emerg Med
.
2010
;
38
(
4
) :
507
511
.
10
Accreditation Council for Graduate Medical Education
.
Common Program Requirements
. .
11
Kohn
LT
,
Corrigan
J
,
Donaldson
MS
.
To Err Is Human: Building a Safer Health System
.
Washington, DC
:
National Academy Press
;
2000
.
12
Szostek
JH
,
Wieland
ML
,
Loertscher
LL
,
Nelson
DR
,
Wittich
CM
,
McDonald
FS
.
et al . et al
A systems approach to morbidity and mortality conference
.
Am J Med
.
2010
;
123
(
7
) :
663
668
.
13
Ziegelstein
RC
,
Fiebach
NH
.
“The mirror” and “the village”: a new method for teaching practice-based learning and improvement and systems-based practice
.
Acad Med
.
2004
;
79
(
1
) :
83
88
.
14
Heard
JK
,
Allen
RM
,
Clardy
J
.
Assessing the needs of residency program directors to meet the ACGME general competencies
.
Acad Med
.
2002
;
77
(
7
) :
750
.
15
Kravet
SJ
,
Howell
E
,
Wright
SM
.
Morbidity and mortality conference, grand rounds, and the ACGME's core competencies
.
J Gen Intern Med
.
2006
;
21
(
11
) :
1192
1194
.
16
Bechtold
ML
,
Scott
S
,
Dellsperger
KC
,
Hall
LW
,
Nelson
K
,
Cox
KR
.
Educational quality improvement report: outcomes from a revised morbidity and mortality format that emphasised patient safety
.
Postgrad Med J
.
2008
;
84
(
990
) :
211
216
.
17
Deis
JN
,
Smith
KM
,
Warren
MD
,
Throop
PG
,
Hickson
GB
,
Joers
BJ
.
et al . et al
Transforming the morbidity and mortality conference into an instrument for systemwide improvement
. .
18
Fussell
JJ
,
Farrar
HC
,
Blaszak
RT
,
Sisterhen
LL
.
Incorporating the ACGME educational competencies into morbidity and mortality review conferences
.
Teach Learn Med
.
2009
;
21
(
3
) :
233
239
.
19
Tess
AV
,
Yang
JJ
,
Smith
CC
,
Fawcett
CM
,
Bates
CK
,
Reynolds
EE
.
Combining clinical microsystems and an experiential quality improvement curriculum to improve residency education in internal medicine
.
Acad Med
.
2009
;
84
(
3
) :
326
334
.
20
Aronson
MD
,
Neeman
N
,
Carbo
A
,
Tess
AV
,
Yang
JJ
,
Folcarelli
P
.
et al . et al
A model for quality improvement programs in academic departments of medicine
.
Am J Med
.
2008
;
121
(
10
) :
922
929
.
21
Weingart
SN
,
Tess
A
,
Driver
J
,
Aronson
MD
,
Sands
K
.
Creating a quality improvement elective for medical house officers
.
J Gen Intern Med
.
2004
;
19
(
8
) :
861
867
.

Author notes

Jed D. Gonzalo, MD, was Chief Medicine Resident at Beth Israel Deaconess Medical Center, a Clinical Fellow in Medicine at Harvard Medical School, and a General Internal Medicine Medical Education Fellow and Clinical Fellow in Medicine at the University of Pittsburgh Medical School; Julius J. Yang, MD, PhD, is Director of Inpatient Quality, Silverman Institute for Healthcare Quality and Safety at the Beth Israel Deaconess Medical Center, and an Assistant Professor at Harvard Medical School; and Grace C. Huang, MD, is Director of Assessment at the Shapiro Institute for Education and Research, Beth Israel Deaconess Medical Center, and an Assistant Professor at Harvard Medical School.

Funding: The authors report no external funding source for this study.

The authors would like to thank the 2010–2011 Chief Medicine Residents David Baker, Alex Carbo, and Mark Aronson from the Beth Israel Deaconess Medical Center for their contributions to this study.

The authors would like to thank the 2010–2011 Chief Medicine Residents David Baker, Alex Carbo, and Mark Aronson from the Beth Israel Deaconess Medical Center for their contributions to this study.

Supplementary data