In the past decade, national attention has increasingly focused on issues of patient safety and health care quality1,2—and health care organizations and the institutions responsible for graduate medical education (GME) have responded. Executive leaders of our nation's hospitals and medical centers are increasingly using the concepts of high reliability systems3,4 for the purpose of making their health care environments a safer place for patients. These leaders are setting goals for improving the quality of care and implementing systems-based approaches to reach those goals.
At the same time, GME has been undergoing a major evolution, adopting competencies and introducing milestones that include a focus on interprofessional as well as patient and family communication and systems-based practice. The ACGME has incorporated elements of patient safety, quality improvement, and professionalism in accreditation requirements; these elements served as the impetus for selecting the six focus areas of the Clinical Learning Environment Review (CLER) Program.
Sponsoring Institutions, hospitals, and medical centers are paying increasing attention to educating residents and fellows about patient safety, health care quality, and professionalism as is reflected in the results from the first round of CLER visits. For example, among the residents and fellows interviewed in the group sessions:
Approximately three-quarters reported they knew the clinical site's priorities in the area of patient safety.
Nearly 95% reported knowing the clinical site's process for reporting an adverse event, near miss/close call, or unsafe conditions.
Sixty-three percent reported they had ready access to organized systems for collecting and analyzing data for the purpose of quality improvement.
Nearly all reported they have attended institutional training on topics of professionalism and ethics.
These findings suggest that clinical learning environments (CLEs) are engaged in efforts to increase awareness of the issues surrounding patient safety and quality through education. The next challenge, for both GME and CLEs, will be to move from the current state of heightened awareness and knowledge to a collaboration that promotes enhanced experiential learning and demonstrated competence in the practices of patient safety and health care quality, and the other CLER focus areas.
As in the section reporting on the overarching themes,5 the members of the CLER Evaluation Committee selected the findings shown below after reviewing aggregated data from the site visits presented by the CLER staff.6 The highlighted text presents selected quantitative and qualitative data drawn from the CLER site visits. (For more information on the specific terminology used in describing the results, please refer to the section on detailed findings.)7 The non-highlighted sections that follow present a discussion by the Evaluation Committee of the significance of those findings and the challenges and opportunities within each focus area.
While many CLEs provided didactic training in patient safety, it was uncommon for CLEs to provide residents, fellows, and faculty members with opportunities for experiential learning.
In general, residents and fellows lacked clarity and awareness of the range of conditions that define patient safety events and were unaware of how CLEs use the reporting of adverse events and near misses/close calls to improve systems of care, both broadly and at the individual departmental level.
Though most residents and fellows were aware of their CLE's process for reporting patient safety events, fewer of them appeared to have used it themselves to report events. When trainees did file a report, or have others file it for them, many received little or no feedback from the CLE.
Across CLEs, a limited number of residents, fellows, and faculty members participated in interprofessional, interdisciplinary, systems-based improvement efforts such as patient safety event reviews and analyses.
Residents, fellows, and faculty members are essential members of the health care team and, as such, their participation is important to achieving the CLE's goals for patient safety. Critical to resident, fellow, and faculty physician participation is their ability to clearly understand and recognize what defines a patient safety event, near miss/close call, or unsafe condition and their responsibility to report such occurrences.
Didactic approaches such as presentations during resident orientation and web-based modules on patient safety may be helpful but are insufficient. Experiential learning, through participation in activities such as interprofessional, interdisciplinary reviews of patient safety events, enables residents, fellows, and faculty members to apply a systems approach to identify and address potential causes of harm. In addition, the experience of receiving feedback encourages reporting and helps residents, fellows, and faculty members understand how patient safety can be improved in individual departments and across the organization. When a resident, fellow, or faculty member receives no response to his/her report, it deters future reporting. Each report is a potential learning experience as well as an opportunity to improve patient care.
Broadly communicating the key learnings and improvement plans from patient safety event reviews to residents, fellows, faculty members, and other staff will help promote the value of CLE efforts to improve patient safety. Through this, CLEs and GME can join together to create a culture of safety and teach residents and fellows about the role of systems thinking in forging sustainable improvements in health care.
Leaders in the CLEs and the GME community must also ensure that faculty members have the skills to educate and train residents and fellows to become competent in risk identification, harm reduction, and creating a culture of safety. Without good role models, residents and fellows may receive messages from the faculty members (either implicit or explicit) that contradict the CLE's efforts to create a culture of quality and safety. That could leave the next generation of physicians ill prepared to advance the teaching and practice of safe care.
Health Care Quality
Across CLEs, most residents, fellows, and faculty members indicated they were aware of the organization's priorities for health care quality improvement (QI); occasionally they could accurately identify them.
While most residents and fellows indicated they participate in QI projects, many interviewed appeared to have a limited knowledge of QI concepts and the specific methods and approaches to QI employed by the CLE.
Many residents and fellows seemed to view QI engagement as implementing solutions prescribed by the CLE or their department.
In most CLEs, residents and fellows appeared to have limited participation in interprofessional QI teams.
Organizational success in quality improvement depends on a well-functioning QI system and the ability to focus on prioritized goals. Health care organizations often set broad QI objectives and let their workforce (as well as their residents and fellows) identify specific goals that are meaningful at the local level. Residents and fellows should be encouraged to design and implement interprofessional QI efforts that align with the CLE's overall goals. This will increase the chance of attracting organizational resources and enhance the probability of sustained improvement.
If residents and fellows are to learn to improve the health of the populations they serve, they need to be aware of quality goals such as those set by regulators, payers, and others outside the CLE (e.g., use of universal protocol, or reducing central line-associated blood stream infections, catheter-associated urinary tract infections, or potentially avoidable 30-day readmissions). They also must learn to critically evaluate the CLE's own processes of patient care and how those affect patient outcomes.
Didactic approaches are helpful but insufficient, and data from the CLER site visits suggests that residents and fellows' exposure to quality improvement is often fragmented. Learners rarely have the opportunity to work through the full scope of an improvement effort. Instead, they may plan an intervention they never get to test, or implement a change with limited knowledge of the background evidence and no opportunity for follow-up evaluation. Experiential training in all phases of QI is necessary to develop the skills essential to improving health care quality.
Optimal QI strategies should include formal, reliable, and regular structural links between the efforts generated by the residents, fellows, and faculty members and the CLE's staff-led efforts to improve care. Coordinating the resident and fellow QI efforts with those of the organization would benefit patients, tap into a rich resource of innovation, and provide the foundation for life-long QI success.
Health Care Disparities
Few CLEs appeared to have a formal strategy for addressing health care disparities or a systematic approach to identifying variability in the care provided to or clinical outcomes of their known vulnerable patient populations.
In addressing health care disparities, many CLEs were focused on specific issues such as improving access to care for low-income patients, or meeting regulatory requirements such as interpreter services or community needs assessments. When the CLEs involved residents and fellows in health care disparities, it was most often at the level of providing direct service to select patients (such as those at low-income community-based clinics) or providing care in the context of short-term community outreach projects (e.g., health fairs).
Across most CLEs, education and training on health care disparities and cultural competency was largely generic, and often did not address the specific populations served by the institution. Generally across CLEs, residents and fellows reported that learning about health care disparities and cultural competency was happening in an ad-hoc manner.
There are differences in health care and health outcomes among various populations and sub-populations within the US.8 Residents and fellows should be aware of these disparities and participate in efforts to eliminate them.
While access to care—limited by financial constraints, workforce shortages, and geographic challenges—is an important contributing factor to health care disparities, it is not the only factor. There are many socio-cultural and economic barriers that affect patient care. The CLER site visits found that few CLEs have a well-defined or easily articulated formal strategy to routinely monitor and address health care disparities among their patients. As a result, residents and fellows may not learn how to best manage these issues in their later careers.
CLEs need to assure that their residents and fellows learn to recognize health care disparities and strive for optimal outcomes for all patients, especially those in potentially vulnerable populations. As front-line caregivers, residents and fellows are a valuable resource for formulating strategy on these matters. They can assist the CLEs in addressing not only low income populations, but also those that experience differences in access or outcome based on gender, race, ethnicity, sexual orientation, health literacy, primary language, disability, geography, and other factors.
The diverse, often vulnerable, patient populations served by CLEs also provide an important opportunity for teaching residents and fellows to be respectful of patient cultural differences and beliefs, and the social determinants of health. In learning cultural competence, residents and fellows would benefit from moving beyond one-time educational activities to a more formal, longitudinal curricular-based program of progressive educational activities that continues throughout training. Similarly, they may benefit from experiential learning within a community context for some of the culturally unique groups in the local environment. That would prepare them not only to address the disparities they face today but other unknown challenges that will arise in their future careers.
In general, CLEs were working to standardize and improve their processes for transitioning patients from the acute hospital setting to post-acute care (e.g., ambulatory, intermediate, or long-term care). Residents and fellows were occasionally engaged in the CLE's efforts to design these strategies.
Occasionally CLEs indicated they were working towards a standardized organization-wide approach to managing inter-departmental transfer of patients assigned to resident and fellow teams (e.g., ED to inpatient, OR to ICU, ICU to floor, or medicine to surgery).
Most CLEs did not appear to have a standardized approach to facilitating resident and fellow hand-offs at change of duty that included the essential elements of safe, reliable transitions of care.
Across CLEs, a limited number of programs appeared to use formal criteria to assess residents and fellows' skills in change-of-duty hand-offs. It was uncommon for programs to consistently engage faculty members in observing resident and fellow hand-offs.
As front-line care providers, residents and fellows often see firsthand the consequences of ineffective and inefficient care transitions.
CLEs would benefit from including residents and fellows in strategic planning around transitions of care. When the resident and fellow role in such strategies is limited to implementing changes designed by others, they lose the opportunity to gain experience in developing systems-based approaches to quality improvement.
In many CLEs, faculty members and program directors confuse standardizing hand-offs with a request to create a single and uniform “one size fits all” solution. CLEs and the GME community should be encouraged to find solutions that standardize essential properties of the hand-off process while allowing for additional specialty- or unit-specific components as needed.
Moreover, patient hand-offs are an important communication skill that transcends any individual training program. With their increasing reliance on electronic communication, CLEs would benefit from greater diligence in assuring that residents and fellows develop the verbal communication skills that assure good hand-offs. Resident and fellow hand-offs should be supervised and evaluated by faculty members in a fashion similar to evaluation of other clinical care and communication competencies. By calling attention to the importance of good hand-offs, such supervision could promote better care transitions throughout the CLE.
Across most CLEs, residents, fellows, and faculty members reported an overall culture of close supervision within the GME community. CLEs also faced challenges of under- and over-supervision. Many faculty members and program directors perceived that external factors were contributing to a culture of over-supervision that impeded resident and fellow readiness for clinical practice after training.
Across most CLEs, there were residents and fellows who reported that they have personally experienced—or have witnessed peers in—clinical situations in which they felt there was inadequate supervision.
Few CLEs provided nursing and other clinical staff members with systematic resources that allowed them to check an individual resident's required level of supervision for performing a patient procedure.
Some program directors reported having managed issues related to resident supervision within the past year, some of which were related to patient safety events. In general, the CLEs' patient safety and quality leaders indicated that they did not actively monitor the supervision of residents and fellows except retrospectively, after a patient safety event had occurred.
The findings illustrate the inherent difficulty that CLEs have in achieving the appropriate level of supervision, an issue that is necessarily complicated by the need for different degrees of supervision at different stages of training.
Over-supervision was identified as a challenge in many CLEs. Over-supervision of residents and fellows can have the negative consequence of producing physicians who are unprepared for independent practice. Billing requirements, payment policies, and regulatory and accreditation rules may be influencing CLEs and residency programs to place significant restrictions on the amount of patient care that residents and fellows can perform without direct supervision.
With regard to under-supervision, the findings suggest that often the only interface between GME and the hospital or medical center's patient safety and health care quality department happens after a major patient safety event has occurred. Patient safety and quality departments could benefit from working with GME to develop proactive monitoring of physician learners. Any proactive monitoring system would be likely to be underutilized unless the CLE clearly communicates expectations regarding use of this information in the daily workflow of clinical care for all members of the clinical team.
Comprehensive solutions for appropriate supervision of, and delegation of authority and responsibility to, residents and fellows within CLEs require ongoing attention and monitoring. This responsibility, while centered within GME, needs to include regular review by the executive leadership of the CLEs.
Issues of appropriate supervision are seldom simple and require careful oversight by faculty members. Due to the complex nature of clinical care, other members of the clinical team need to be involved in assuring adequate supervision in a role that is supportive to faculty members and their residents and fellows. Such support requires information on the need for and type of supervision for each resident and fellow to be available to faculty members, supervising residents and fellows, and non-physician clinical staff (e.g., nursing) in an accessible and timely manner. It also requires that such information is reliably being used to support clinical care.
Fatigue Management, Mitigation, and Duty Hours
In general, CLEs had developed and implemented some form of fatigue management for residents and fellows. Strategies included those required by accreditation standards (e.g., adherence to duty hour restrictions, availability of call rooms, and education on fatigue management), as well as other strategies (such as offering taxi rides when the resident was too tired to drive home).
In many CLEs, residents, fellows, faculty members, and nurses reported observing resident fatigue that was related to factors other than the number of hours worked (e.g., periods of high patient volume or high-acuity patient care).
In many CLEs, faculty members reported a significant increase in their own fatigue.
Many GME programs enforced duty hour limits so strictly that they, in effect, discouraged using the exceptions permitted by the ACGME Common Program Requirements due to concerns this would trigger added scrutiny and/or citations.
Many faculty members and program directors perceived that there could be increased risk to patients due to frequent hand-offs prompted by institutional efforts to comply with duty hour requirements.
Most CLEs have met their responsibilities to follow duty hour guidelines and implemented the basic strategies required for ACGME accreditation. Nevertheless, residents, fellows, faculty members, and nurses still report instances of resident and fellow fatigue. Fatigued providers can place patients at risk for medical errors and also jeopardize their own health (e.g., car accidents, burn out). Fatigue management is about both patient safety and provider well-being.
Across most CLEs, assessment of resident fatigue appears to be largely limited to monitoring the number of hours worked. Yet there are many other factors that can cause fatigue, including task or mental overload due to high-volume or high-acuity patient activity, circadian rhythm disruption, chronic sleep deficit, and non-work related activities. Moreover, “fatigue” can also be a precursor to burnout or a marker for depression.9 CLEs should be encouraged to train residents, fellows, faculty members, and other clinical staff to consider such factors—and not only work hours—in determining a provider's “fitness for duty.”
This paradox of CLEs meeting the requirements while still having reports of fatigue suggests that the most common strategies for fatigue management may be insufficient. CLEs need to implement more advanced strategies, such as scheduling to maximize rest and re-set circadian rhythms, strategic naps, batching calls, and systems to relieve tired providers. For meaningful change to occur, CLEs need to progress from individual tactics toward system-wide strategies that are routinely monitored to ensure their efficacy.
These strategies should also include faculty members. A number of factors related to financial productivity, patient complexity, and regulatory requirements have increased faculty workload over the past decade. The ACGME, through accreditation requirements and attention to duty hours, has encouraged better fatigue management for residents and fellows. However, there are no widespread comparable guidelines for faculty members, resulting in faculty fatigue, increased patient safety vulnerability, and physician burnout. Patient care would benefit from CLEs taking a more systematic approach to fatigue management that includes all health care professionals.
By instituting and enforcing duty hour requirements, the ACGME has responded to public concerns and established standards to mitigate risks to patient safety. However, within the GME community there appears to be some reluctance to use the flexibility built into the ACGME requirements—that is, the exceptions in the Common Program Requirements. The CLER site visit data suggests that more work is needed to communicate circumstances in which making an exception is not only acceptable but necessary in order to assure safe, high quality patient care.
Based on faculty concerns raised during the group interviews, the current ACGME duty hour requirements appear to have amplified the importance of achieving good hand-offs at every change of care. As noted above, many faculty members and program directors perceived increased risk to patients due to more frequent hand-offs. While this concern is worth noting, it also should be stressed that when hand-offs are performed in an accurate and reliable manner, more frequent hand-offs should not, in and of themselves, increase patient risk. However, when hand-off processes are not accurate or reliable, increasing the frequency of hand-offs could increase vulnerabilities in patient care.
This first set of CLER visits focused on selected topics within professionalism, including disrespectful and disruptive behavior, honesty and integrity, and mistreatment of residents and fellows.
Across nearly all CLEs, residents, fellows, and faculty members reported that they had received education about professionalism. For residents and fellows, this education most frequently occurred at orientation, and through subsequent annual online modules.
Across some CLEs, residents, fellows, and clinical staff described witnessing or experiencing incidents of disruptive or disrespectful behavior on the part of attending physicians, residents, nurses, or other clinical staff. These ranged from descriptions of isolated incidents to allegations of disruptive behavior that was chronic, persistent, and pervasive throughout the organization.
Some residents and fellows reported they have had to compromise their integrity to satisfy an authority figure. In many CLEs, leadership was unaware of this perception.
In most CLEs, residents, fellows, faculty members, and program directors appeared to lack a shared understanding of the process residents and fellows would follow to resolve perceived mistreatment if seeking assistance outside of the mechanisms offered by GME.
While all CLEs have mechanisms to address disruptive and disrespectful behaviors, the findings suggest that existing mechanisms are not always effective. Even if a CLE has an aggressive system to deal with unprofessional behavior, that system is undermined if standards are applied inconsistently. Similarly, no CLE wants its personnel to compromise their integrity but the finding that some residents and fellows feel that way suggests the need for further exploration. Absent robust attention to these issues, resident, fellow, and faculty physicians and other staff may conclude that the system lacks integrity or makes exceptions capriciously.
The findings suggest that CLEs are committed to providing a clinical and educational environment where professionalism is manifest. However, the findings also suggest that didactic and computer-based education, while necessary, are not sufficient to guide residents and fellows in important aspects of professionalism. CLEs are encouraged to improve resident and fellow learning through planned experiential activities. Such training could include role modeling and procedures to address a diverse set of topics, such as safety culture, team-work, patient interaction, cultural sensitivity, and diversity.
With regard to issues of mistreatment, there may be situations in which residents or fellows are uncomfortable pursuing resolution within the GME administrative structure. In these situations the resident or fellow and the CLE are at high risk for an unfavorable outcome. While all CLEs have established confidential, non-punitive reporting mechanisms for handling unprofessional behavior, this does not ensure residents and fellows are familiar and comfortable with the reporting mechanisms outside of GME.
CLER Evaluation Committee: James P. Bagian, MD, PE, Co-Chair, University of Michigan, Ann Arbor, MI; Kevin B. Weiss, MD, Co-Chair, ACGME, Chicago, IL; William M. Barron, MD, MMM, FACP, Loyola University Health System, Maywood, IL; Terry L. Cline, PhD, Oklahoma State Department of Health, Oklahoma City, OK; John Patrick T. Co, MD, MPH, Partners Healthcare, Boston, MA; Anjali Dogra, MD, Johns Hopkins University School of Medicine, Baltimore, MD; John F. Duval, MBA,* Virginia Commonwealth University, Midlothian, VA; David Entwistle, MHSA, University of Utah Hospitals and Clinics, Salt Lake City, UT; Rosemary Gibson, MSc, Author, “Wall of Silence,” “The Treatment Trap,” Arlington, VA; Diane M. Hartmann, MD,* University of Rochester Strong Memorial Hospital, Rochester, NY; Linda Headrick, MD, University of Missouri–Columbia School of Medicine, Columbia, MO; Marcia Hutchinson, MD, Mercer University School of Medicine, Macon, GA; LCDR Dinchen Jardine, MD, Naval Medical Center–Portsmouth, Portsmouth, VA; Zachary Lopater, MD,* Central Georgia Radiation Oncology, Macon, GA; Douglas E. Paull, MD, MS, VHA National Center for Patient Safety, Ann Arbor, MI; Russell Postier, MD,* University of Oklahoma College of Medicine, Oklahoma City, OK; Andrew M. Thomas, MD, MBA, The Ohio State University, Columbus, OH
* Past CLER Evaluation Committee member