I have had the opportunity to review numerous manuscripts reporting on efforts to teach patient safety and quality improvement (PS/QI) to others, initially as a reviewer, and now as an associate editor for the Journal of Graduate Medical Education. The most likely reason that I have been asked to review many of these papers is that I published a systematic literature review in 2010 that summarized the different ways to teach PS/QI and the effects of these various formats on learning outcomes.1
Writing this next paragraph saddens me, as I will now summarize what amounts to 2 years of blood, sweat, and tears into 5 sentences. Our systematic review revealed a number of different approaches taken to teach PS/QI and emphasized the importance of experiential learning. PS/QI curricula improved learner knowledge and attitudes, as well as some clinical processes (particularly when teaching methods included learner QI projects). However, there were insufficient numbers of studies evaluating the impact on behavior change. Consequently, it is not known whether trainees who undertook PS/QI training were more likely to engage in PS/QI activities in their future practice. Our review also identified a number of factors that limited successful implementation, which included the fact that lack of faculty capacity was often an important barrier.
Since we published that review, more articles summarizing education in PS/QI have been published. At a certain point, I tried to keep track of these to see how the field was evolving and even updated our systematic review in an attempt to describe new and emerging trends.2 In the end I stopped for 2 reasons. First, there were simply too many new studies for one to keep up. Second, many of these studies tended simply to confirm the findings of prior ones, rather than offer suggestions for novel insights or approaches. Not surprisingly, 2 recent reviews (a realist review3 and an updated systematic review4) came to similar conclusions as we did 5 years ago, both with respect to the impact of PS/QI training on learner outcomes, and the facilitators and barriers to successful implementation.
Despite this proliferation of articles describing examples of PS/QI training, the early findings of the Accreditation Council for Graduate Medical Education (ACGME) Clinical Learning Environment Review program have been sobering and have identified numerous examples of PS/QI practices that are not integrated or appropriately role-modeled by faculty.5 There are also reports of significant variations in practice attributable to residents with respect to core patient safety practices such as appropriate prescribing of venous thromboembolism prophylaxis.6 These findings contribute to significant concern about the lack of preparedness of recent graduates for future PS/QI practice.7
Clearly, there remains a critical need to develop and disseminate new and innovative approaches to deliver PS/QI training. However, if we are to advance the field of PS/QI education, and, more importantly, create optimal PS/QI training to better prepare physicians for future practice, we will need to shift our focus toward those aspects that have been explored to a lesser extent. In an attempt to provide guidance to prospective authors wishing to disseminate their PS/QI education research in the form of a peer-reviewed publication, I offer the following 3 suggestions (box).
Studies that describe and evaluate effective faculty development programs for patient safety and/or quality improvement (PS/QI) educators
Studies that place a greater focus on assessment tools for competency in PS/QI
Studies that demonstrate successful integration of PS/QI training into the clinical learning environment
More Evaluative Studies Needed That Describe Effective Faculty Development Programs for PS/QI Educators
The lack of faculty capacity to teach PS/QI has been identified as such an important barrier to curricular implementation and sustainability that national organizations, like the Association of American Medical Colleges,8 the Society for Hospital Medicine,9 and the Royal College of Physicians and Surgeons of Canada,10 have invested in developing large-scale train-the-trainer programs to build capacity for teaching PS/QI. The presence of such programs is insufficient, and what is lacking are published examples of robust evaluations of faculty development programs in PS/QI education.11 Much of the faculty development literature is underdeveloped, not only as it relates to PS/QI education, but also as it relates to the range of relevant topics.
Articles that more critically explore PS/QI faculty development efforts are needed. Potential questions of interest could include:
Which existing or innovative faculty development models train faculty who continue to teach PS/QI to residents at 1 and 5 years?
What incentives most effectively encourage faculty (particularly clinical teachers) to participate in PS/QI faculty development activities?
Are there innovative approaches to faculty development that have a greater likelihood of changing trainee behaviors as they relate to PS/QI?
The added advantage that authors may derive from disseminating their evaluation of faculty development efforts is that they might also extend and expand the evidence base for faculty development in general, particularly for those topics in medical education where there is an imperative to teach, but faculty capacity is generally lacking.
A Greater Focus on Assessment Tools for Competency in PS/QI
Many training programs are actively implementing competency-based training models, with an increased emphasis on competencies, milestones, and entrustable professional activities. One of the greatest challenges facing the successful implementation of competency-based medical education is the availability of assessment tools with evidence of validity.
This is particularly true for PS/QI competencies. As compared to reports on PS/QI teaching, there are relatively few published examples of assessment tools to assess PS/QI competencies.12 The most frequently cited tool is the Quality Improvement Knowledge Assessment Tool Revised, which can be used to assess QI knowledge.13 Additional examples include observed structured clinical examination stations that assess PS skills such as error disclosure.14,15 Other groups have piloted Mini-CEX (Mini–Clinical Evaluation Exercise)–type direct observation tools to provide learners feedback on their patient handovers.16 Given the limited menu of options, novel approaches to learner assessment with respect to PS/QI would certainly make meaningful contributions to the literature.
Despite the emergence of PS/QI training in numerous graduate medical education programs over the past decade, many of which directly engage residents in experiential QI projects, little is known about whether residents change their future behaviors with respect to PS/QI. There may be modest improvements in self-reported changes to specific behaviors, such as incident reporting and error disclosure, although more objective measures of behavior change have been far less promising.17 Future studies that describe approaches to PS/QI training that result in meaningful, longstanding changes in behavior would be extremely helpful to medical educators at large.
Studies That Demonstrate Successful Integration of PS/QI Training Into the Clinical Learning Environment
Patient safety and quality improvement training often occurs outside of the clinical context, with time carved out from daily work to execute QI projects rather than having QI work integrated into everyday clinical activities. This separation of PS/QI learning from clinical learning creates tensions with respect to competing priorities, which often presents itself as a concern over having a lack of time to dedicate to QI project work. Separating out QI learning makes training in these curricula less generalizable to future practice settings. Not surprisingly, a study of recent family medicine graduates showed that QI training in residency did not change future practices with respect to participating in QI.18
This is why studies describing successful examples of PS/QI learning that is fully integrated into day-to-day clinical work would be so helpful in advancing the field. To date, there are only a few examples where educators have achieved such meaningful levels of integration. For example, the White River Junction Veterans Affairs Medical Center in Vermont, which is affiliated with the Geisel School of Medicine at Dartmouth College, made systems improvement a core resident responsibility when they rotated onto the internal medicine ward.19 This resulted in important improvements in actual patient care outcomes: more patients received appropriate pneumococcal vaccination and venous thromboembolism prophylaxis, and more physicians washed their hands and counseled patients to stop smoking.
We need many more examples of how to do this well. Such reports could help to improve our understanding of the optimal pedagogical approaches, the full breadth of logistic issues, resource requirements, and implications for faculty development. Perhaps most important, evaluative studies would be critical in helping to determine whether this integrated approach achieves better outcomes and greater impact than the existing approaches to teaching PS/QI.
I hope that these 3 suggestions will act as a helpful guide to those who wish to disseminate their PS/QI educational initiative. Taken together, future studies on these 3 important areas will truly advance how we teach and assess PS/QI in graduate medical education. I urge those already innovating in these areas to accelerate their efforts to disseminate the results of their work and, in particular, their lessons learned. These would play a critical role in helping advance toward the ultimate goal of graduating the types of physicians that are urgently needed to build and repair our health care system so as to provide the best possible care for our patients and populations.