CLER Program Future Directions: Optimizing the Clinical Learning Environment

Since 2012 the Clinical Learning Environment Review (CLER) Program has provided Accreditation Council for Graduate Medical Education (ACGME)-accredited Sponsoring Institutions with formative feedback to clinical sites that serve as clinical learning environments (CLEs) for resident and fellow physicians. The program’s goal has been to provide information that will help CLE executive leaders and graduate medical education (GME) leaders identify opportunities for improvement in the CLER Focus Areas to optimize both learning and patient care. The CLER National Reports of Findings^1-3  have demonstrated that the process of providing formative feedback has led to improvements in the Focus Areas (such as improved resident physician reporting of patient safety events) and identified challenges in the nation’s CLEs that were not previously evident prior to initiating the CLER Program’s structured, purposeful assessment of the CLE.

To date, the ACGME’s CLER Program has achieved several important outcomes. The program has: (1) stimulated new connections between GME leaders and the executive leaders of their CLEs and health systems; (2) improved resident and fellow engagement in the CLER Focus Areas, most notably in patient safety; (3) published periodic reports on CLER findings and trends that provide a national perspective; and (4) informed ACGME accreditation relevant to the CLER Focus Areas through inputs to the Common Program Requirements and the Institutional Requirements.

The CLER Program is now well positioned to expand its efforts to accelerate improvement and respond to the needs of the GME/CLE community. Throughout the 4-plus cycles of CLER visits to date, many designated institutional officials (DIOs) have noted that the CLER Program is designed to visit a single clinical site and have expressed a desire to engage the executive leaders of other CLEs within their Sponsoring Institution. DIOs have also expressed interest in having a way to conduct their own assessments of their CLEs. Additionally, CLE executive leaders and GME leaders continue to ask the CLER Program to share best practices to address the CLER Focus Areas as they seek to learn from the successes of others.

DIOs frequently note the CLER Program’s site visits and outreach initiatives have prompted closer interactions with their CLE’s executive leaders and resulted in better alignment with the CLE’s priorities. The CLER Pursuing Excellence initiative⁴  highlighted the importance and substantive value of enhancing CLE-GME integration. The GME leaders participating in this initiative moved beyond their usual scope of overseeing medical education to contribute to key aspects of their organization’s health care delivery strategy. The initiative demonstrated that purposeful efforts to integrate CLE and GME strategic goals advance the CLE’s overall mission to grow value-based care, drive clinical quality, and improve patient experience.

Considering the experience and growing needs of the GME community, there is opportunity to evolve and metamorphize the CLER Program into a more comprehensive program with multiple approaches to improve the learning environment and accelerate CLE-GME integration.

The redesign of the CLER Program into a new CLE Program will build upon the successful efforts of the CLER site visit program, the CLER Pursuing Excellence initiative, and other models of collaborative engagement and learning. In its redesign, the CLER Program will expand to offer GME and CLE executive leaders new tools, resources, and opportunities to accelerate their efforts to optimize the CLE.

The new CLE Program is designed to have 4 main components: (1) a national CLER site visit program; (2) a CLE Toolkit for Formative Assessment; (3) a CLE Quadruple Aim initiative; and (4) other CLE resources. The new CLE Program will address the need for innovative efforts that can expand the program’s reach. This will require a broadening of the branding and terminology formerly attached to “CLER” to recognize that “Reviews” conducted via CLER site visits will be one of several components of the expanded CLE activities and resources. Therefore, the program’s name will change from CLER Program to CLE Program.

CLER site visits will continue to be the core component of the program, delivering high-fidelity formative assessment and feedback to CLE and GME leaders and aggregate national data to inform future ACGME accreditation requirements.

Since the inception of the CLER Program, the number of ACGME-accredited Sponsoring Institutions has grown considerably, from several hundred in 2012 to more than 870 in 2023. To address this growth, the CLER site visit program has moved to a sampling approach to identify Sponsoring Institutions to visit during each cycle—targeting approximately 300 Sponsoring Institutions per 2-year cycle. Samples are randomly generated based on certain strata (eg, geographic location, number of ACGME-accredited programs) at the start of each cycle. The CLER Program will continue to ensure the samples are proportionally representative of all Sponsoring Institutions each cycle.

The CLER site visit program will also continue to develop and implement subprotocols to augment CLER site visits.

The second component of the new CLE Program is a resource called the CLE Toolkit for Formative Assessment, and use of the toolkit is voluntary. The toolkit is a direct response to the GME community’s requests for the CLER Program to expand its reach to other CLEs. The CLER Program often receives requests to visit sites other than the site traditionally designated. The ACGME Data Resource Book⁵  noted a total of 8640 unique participating sites to which residents and fellows rotated during the 2021-2022 academic year—a volume that far exceeds the CLER Program’s capacity to address. A CLE Toolkit for Formative Assessment can provide the GME community with a structured and standardized approach to self-administered assessment that can be used across all CLEs within a Sponsoring Institution.

Also, while CLER site visits provide a comprehensive and in-depth assessment of the CLE, these visits are periodic snapshots. The toolkit can help CLE and GME leaders maintain the momentum for improvement and innovation in their CLEs between CLER site visits.

The toolkit will consist of modular components designed to: provide a process to identify existing gaps and areas for improvement, present comparative data within and across CLEs (eg, benchmarking), including multiple participating sites that are part of a single Sponsoring Institution, and allow for assessment of existing CLE improvement efforts and innovations.

The information gathered can promote reflection and conversations among CLE executive leaders, GME leaders, and clinical care team members to facilitate ongoing learning and improvements in the CLER Focus Areas.

The CLE Program will employ a systematic approach to the toolkit’s development and dissemination to ensure each module yields valid and reliable information. The first module will address the CLER Focus Area of patient safety. Key steps in the developmental process include: (1) conducting focus groups with key stakeholders; (2) recruiting a small group of Sponsoring Institutions to test the toolkit; (3) conducting multiple rounds of testing to gather input on content, usability, and feasibility; and (4) designing educational sessions to provide guidance and support on how to locally administer toolkit modules.

While the first module will focus on patient safety, the same process for development and dissemination will be employed when developing modules in the other CLER Focus Areas.

Additionally, the CLE Toolkit for Formative Assessment will provide comparative data across CLEs for benchmarking. The ACGME will collect de-identified data on selected measures from each Sponsoring Institution using the toolkit.

The third component of the new CLE Program speaks to the CLE and GME community’s desire to identify successful approaches to address challenges and opportunities for improvement by inviting them to join a learning collaborative to focus on the Quadruple Aim. This new initiative will be directed at strengthening partnerships between GME and CLE executive leaders to advance the Quadruple Aim of simultaneously improving patient outcomes, enhancing patient experience, improving health care practitioner well-being, and reducing cost of care (Sponsoring Institutions may expand to address the Quintuple Aim, which includes advancing health equity).⁶ 

The CLE Quadruple Aim initiative will tap into the National Learning Community of Sponsoring Institutions to identify and work with GME and CLE executive leaders who are motivated to address the Sponsoring Institution 2025 pillars of Demonstrating Commitment to Excellence in Patient Care and Optimizing Health Systems for Learning (see Figure) and jointly solve for CLE systems-based challenges.

This initiative builds upon a number of successful models from within and outside the ACGME.^7,8  Between 2016 and 2021, for example, the CLER Program’s Pursuing Excellence initiative conducted a series of collaboratives designed to promote transformative improvement in the CLEs of ACGME-accredited Sponsoring Institutions.⁹ 

This initiative is different from many GME initiatives in that the challenges the participants choose to address will originate from the executive leaders of the CLEs and subsequently leverage GME insights to jointly design and test new approaches to solving for them.

For each team participating in the CLE Quadruple Aim initiative, the CLE’s executive leaders will be asked to identify a current challenge that:

• can benefit from GME insights and collaboration;

• identifies specific, measurable, achievable, relevant, and time-bound (SMART) goals that incorporate each of the elements of the Quadruple Aim, including how the intervention will impact the well-being of the clinical care team;

• is scoped such that the improvement activities fit a 12-month time frame; and

• is linked to 1 or more CLER Pathways properties.

The CLE Quadruple Aim initiative will start with a small cohort of 8 to 10 teams that volunteer through an ACGME call for interested participants. The initial group will start what will become a continuous series of collaborative learning cycles in a 2 + 12 + 1-month time frame, with new cohorts of CLE teams entering each cycle. At the launch of each cycle, the new cohort will meet in person. Subsequent meetings will occur remotely via Zoom frequently enough to keep the work on track.

The 2 + 12 + 1 time frame includes 2 months of prework (eg, teams refining the scope and goals of their activity) and 12 months of “run time” (rapid-cycle tests of change and approximately one month to consolidate knowledge gained and widely share lessons learned). At the conclusion of each cycle, the teams will post a summary of their efforts that details successful approaches and lessons learned on a designated site at which the information will be publicly accessible to the entire CLE/GME community. The initiative will provide a mechanism to sustain the community of past participants for regular peer-to-peer support through remote group check-in meetings 2 to 4 times a year.

CLE Program staff will serve as conveners, facilitators, and coaches. In these roles, they will assist teams to identify and engage new stakeholders, foster peer-to-peer dialogue and problem solving, and guide team member engagement.

Each team will designate a local lead “owner” (eg, DIO or designee such as associate DIO, CLER director, etc) and will include a mix of GME and CLE executive leaders plus residents, fellows, and other members of the clinical care team as appropriate for their topic.

In particular, the initiative will seek to foster CLE/GME relationships that elevate GME leaders so they are increasingly engaged as key assets in design, testing, implementation, and evaluation of CLE initiatives.

In the fourth component of the new CLE Program, staff will curate a collection of resources to be housed by the ACGME and made available to the CLE/GME community. Resources may include but are not limited to:

• summaries of successful practices and lessons learned from using the CLE Toolkit for Formative Assessment;

• summaries of successful practices and lessons learned from participating in the CLE Quadruple Aim initiative;

• resources developed by the ACGME as part of outreach efforts resulting from the National Learning Community of Sponsoring Institutions; and

• resources, tools, and successful practices crowdsourced from the CLE/GME community.

CLE Program staff will also continue to provide other opportunities for collaboration such as webinars and workshops.

The goal of redesigning the CLER Program is to accelerate efforts to optimize CLEs for both learners and patients. This redesign involves 4 components as outlined above:

• Component 1 continues to give clinical sites, the nation, and the ACGME a high-fidelity assessment of progress in the CLER Focus Areas.

• Component 2 allows the ACGME to build upon the evidence base created and maintained by CLER site visits by increasing capacity among GME and CLE executive leaders to assess their own CLEs.

• Component 3 strengthens the partnerships between GME and CLE executive leaders such that GME takes an active lead role in CLE efforts to design, test, implement, and evaluate systems-based changes to patient care that further the Quadruple Aim.

• Component 4 provides resources to all members of the CLE/GME community, serving as a vehicle to disseminate successful practices and lessons learned.

With more than 4 cycles of experience and a decade of insightful feedback from DIOs, chief executive officers, and other CLE executives to draw upon, the CLER Program is poised to implement its innovative redesign—and, in the process, foster new partnerships to optimize patient care.

The authors would like to acknowledge all of the members of the CLER Program staff both for their contributions to the working groups that shaped the various components of the CLER Program redesign and for their dedication to serving the GME and CLE communities in their efforts to optimize learning and patient care: Isabelle Bourgeois, MPA; Robert Casanova, MD, MHPE; Marian D. Damewood, MD, FACOG; Kevin Dellsperger, MD, PhD; Robin Dibner, MD; Brenda Moss Feinberg, ELS; Staci A. Fischer, MD, FACP, FIDSA; Paula Hensley, MPH; Joshua Miron, MA; Wardah Mohammad, MA; Douglas E. Paull, MD, MS, FACS, FCCP, CHSE, CPPS; Sandra Rials, MS; Ana L. Sainz; Melissa Schori, MD, FACP, MBA, CPPS; Jordan Stein, PhD; Hongling Sun, PhD; Elizabeth Wedemeyer, MD; Esther Woods; Martha S. Wright, MD, Med; James R. Zaidan, MD, MBA; and Jose Zayas, DO, FAAP.

The ACGME News and Views section of JGME includes data reports, updates, and perspectives from the ACGME and its Review Committees. The decision to publish the article is made by the ACGME.