The CLER National Report of Findings 2018: Trends in the CLER Focus Areas

The Clinical Learning Environment Review (CLER) Program assessed and monitored a selected set of observations in each of the 6 CLER Focus Areas1 over the last 2 sets of visits. This section offers the CLER Program's first look at changes over time in each of the Focus Areas. Of note, findings presented in this section of the CLER National Report of Findings 2018 reflect a 2-point analysis; future iterations of this report will present trends. The measures examined are not comprehensive and do not summarize the full scope of resident and fellow engagement in the CLER Focus Areas. Instead, they offer a snapshot that paints a multidimensional picture of the clinical learning environment (CLE). These findings are intended to further stimulate new discussions on continuously improving the CLE. Appendix B provides additional information on selected changes from the first set of CLER Site Visits.

The results are based on matched cases (ie, CLEs) and a combination of quantitative (eg, resident and fellow responses to closed-ended questions in group interviews) and qualitative information (eg, observations and interviews on walking rounds). Details on data sources and the methods for analysis are described elsewhere in this report.2 

The reported changes on selected measures are not designed to imply plausible explanations of effects or to establish causal relationships. Additionally, statistical significance does not necessarily imply practical significance as the differences may not be large enough to have practical implication. Many factors may influence change, such as awareness and understanding of the CLER Focus Areas, opportunities for engagement (eg, participation in patient safety event investigations), and attention to improvements in selected CLER Focus Areas (eg, patient safety and health care quality). These factors may vary across CLEs and change over time; thus, such factors should be considered in interpreting these findings. In the future, the CLER Program will further explore the reasons for such changes.

Between-cycle changes on selected measures in patient safety are presented in FIGURE 1. In both sets of visits, a high percentage of residents and fellows reported that the clinical site provided a supportive and nonpunitive environment for reporting errors, with a median (interquartile range [IQR]) finding of 96.7% (93.6%–100%) in Cycle 1 and 100% (96.4%–100%) in Cycle 2.

Across CLEs, the percentage of residents and fellows who reported experiencing an adverse event, near miss/close call, or unsafe condition was higher in Cycle 2 than in Cycle 1. In Cycles 1 and 2, the median (IQR) findings were 68.1% (58.0%–76.1%) and 72.7% (63.8%–80.9%), respectively (P < .001). In addition, the percentage of residents and fellows who reported these events into their CLE's patient safety event reporting system increased significantly, with a median (IQR) finding of 46.5% (33.8%–59.9%) in Cycle 1 and 50.0% (37.5%–66.7%) in Cycle 2 (P < .01). Little change was noted from Cycle 1 to Cycle 2 in terms of residents and fellows receiving feedback on the outcome of a patient safety event report submitted into the CLE's central reporting system.

Queried separately, the median percentage of residents and fellows who reported a near miss/close call event increased in the second set of visits. In Cycles 1 and 2, the median (IQR) findings were 18.1% (11.5%–26.8%) and 23.3% (15.7%–33.3%), respectively (P < .001). In contrast to these results, the median percentage of residents and fellows who reported participating in an interprofessional patient safety event investigation decreased in the second set of visits, with a median (IQR) finding of 41.2% (31.4%–51.1%) in Cycle 1 and 37.3% (28.6%–50.0%) in Cycle 2 (P < .05).

Results related to qualitative information collected on walking rounds and interviews with patient safety and quality leaders are presented in TABLE 1. Across CLEs, the proportion of residents and fellows with a working knowledge of basic patient safety terminology differed between cycles (P < .01). A significantly higher percentage of CLEs in Cycle 2 (76.6%) tracked the number of patient safety event reports submitted by residents and fellows than in Cycle 1 (34.3%, P < .001).

FIGURE 2 presents changes on selected measures in health care quality and health care disparities.

Compared with Cycle 1, a larger median percentage of residents and fellows (postgraduate year 2 [PGY-2] and above) in Cycle 2 reported awareness of the priorities in quality improvement (QI) at their clinical site—a median percentage change of 3.9% (P < .01). Between-cycle differences were also noted in the proportion of residents and fellows with a working knowledge of QI concepts (TABLE 2); these differences were not statistically significant.

A modest increase in the percentage of residents and fellows (PGY-2 and above) who reported that they had participated in a QI project of their own design or 1 designed by their program or department was noted between Cycle 1 and Cycle 2, with median (IQR) findings of 77.4% (66.4%–87.4%) and 79.4% (70.7%–87.9%), respectively (P < .05).

Of the residents and fellows who reported that they had participated in a QI project, a significantly smaller percentage in Cycle 2 reported that the project was linked to the clinical site's QI goals (median [IQR], 45.8% [33.3%–64.9%]) compared with those in Cycle 1 (median [IQR], 54.5% [38.4%–69.0%]; P < .01). A slightly higher percentage of residents and fellows, however, reported being engaged in an interprofessional QI project linked to the clinical site's QI goals. In Cycles 1 and 2, the median (IQR) findings were 75.0% (61.5%–87.8%) and 77.4% (66.7%–86.9%), respectively. No statistically significant difference was observed.

In the area of health care disparities, a slightly smaller percentage of residents and fellows in the second set of visits reported knowing their clinical site's priorities in addressing health care disparities. In Cycles 1 and 2, the median (IQR) findings were 60.0% (42.4%–74.7%) and 58.7% (42.9%–75.0%), respectively. The difference was not statistically significant. There was also little change in the percentage of CLEs that appeared to have a systematic approach to addressing health care disparities among the at-risk patients receiving care at these clinical sites—less than 5.0% of CLEs in both sets of visits.

FIGURE 3 presents between-cycle changes in selected measures related to care transitions. Whereas in Cycle 1, a median (IQR) of 81.8% (71.4%–91.0%) of residents and fellows reported following standardized processes for handling transitions of care from inpatient to outpatient, in Cycle 2, 64.8% (51.0%–78.3%) reported doing so. This change was statistically significant (P < .001).

Across CLEs, the percentage of residents and fellows who reported following a standardized process for handling transitions of care during change-of-duty handoffs was higher in Cycle 1 than in Cycle 2, with a median (IQR) finding of 91.7% (83.6%–100%) and 85.7% (77.8%–93.3%), respectively (P < .001). Similarly, of those who reported following a standardized process, a higher percentage in the first set of visits indicated that the process included a standardized written template for communication for handling transitions of care during change-of-duty handoffs, with a median (IQR) finding of 78.9% (70.3%–88.9%) in Cycle 1 and 77.3% (68.4%–86.9%) in Cycle 2 (P < .05).

Based on observations during walking rounds, the change-of-duty handoff processes in most CLEs (69.5%) did not appear to be standardized in Cycle 1 (TABLE 3). At the time of the second visit, it appeared there was some standardization in most CLEs (94.8%).

Between-cycle changes on selected measures in supervision are presented in FIGURE 4. In Cycle 1, a median (IQR) of 19.6% (10.0%–28.3%) of residents and fellows reported having been placed or witnessing 1 of their peers placed in a situation where they believed there was inadequate supervision. In contrast, a median (IQR) of 25.0% (16.7%–33.3%) of residents and fellows reported the same in Cycle 2 (P < .001).

In both sets of visits, a high percentage of residents and fellows reported knowing what they were allowed to do without direct supervision, with a median (IQR) of 100% (95.0%–100%) in Cycle 1 and 96.2% (91.5%–100%) in Cycle 2.

Compared with those in Cycle 1, a higher percentage of residents and fellows in Cycle 2 reported having an objective way to know what procedures residents and fellows from other services were allowed to do without direct supervision when they consulted on patients (median [IQR] of 29.1% [16.7%–54.4%] versus 39.1% [22.5%–60.2%], respectively; P < .01).

In the majority of CLEs—89.9% in Cycle 1 and 93.7% in Cycle 2—nurses indicated on walking rounds that in the absence of an attending physician, they relied primarily on trust when residents and fellows performed clinical procedures (TABLE 4).

Overall, from Cycle 1 to Cycle 2, there was a statistically significant increase in the percentage of residents and fellows who reported that they would power through to handoff if placed in a situation in which they were impaired by fatigue (FIGURE 5), with median (IQR) findings of 30.4% (17.9%–41.3%) and 46.3% (31.3%–58.3%), respectively (P < .001).

In addition, the percentage of CLEs where the patient safety and quality leaders recalled 1 or more patient safety events related to resident or fellow fatigue in the past year increased slightly (6.0% in Cycle 1 versus 6.5% in Cycle 2). The change was not statistically significant.

FIGURE 6 presents selected measures in professionalism. In the first set of visits, a greater median percentage of residents and fellows reported that their clinical site provided a supportive, nonpunitive environment for coming forward with concerns regarding honesty in reporting compared with those in the second set of visits. Median (IQR) findings were 95.0% (90.4%–100%) for Cycle 1 and 89.7% (83.9%–95.1%) for Cycle 2 (P < .001).

In Cycle 1, a median (IQR) of 35.2% (21.7%–48.5%) of residents and fellows reported that they had documented a history or physical finding in a patient medical record that they did not personally elicit. In Cycle 2, a median (IQR) of 33.3% (18.0%–43.3%) of residents and fellows reported the same. The change was statistically significant (P < .01).

The percentage of residents and fellows who reported that they felt pressure to compromise their honesty or integrity to satisfy an authority figure during their training at their CLE was slightly lower in Cycle 2 than in Cycle 1. In Cycles 1 and 2, the median (IQRs) findings were 14.8% (8.6%–20.9%) and 12.5% (6.7%–20.1%), respectively. The difference was not statistically significant.

Collectively, the results indicate both progress and challenges across the CLER Focus Areas. The improvements in resident and fellow engagement in patient safety are encouraging, especially those in the recognition of reportable patient safety events and use of the patient safety event reporting system. In addition, the data show that, compared with Cycle 1, a significantly larger percentage of CLEs were tracking the number of patient safety event reports submitted by residents and fellows in Cycle 2. These improvements must be tempered by the relatively small change in the median percentage of residents and fellows receiving feedback on the outcome of patient safety event reports submitted. Additionally, the median percentage of residents and fellows reporting participation in an interprofessional patient safety event investigation has declined since the first cycle of visits.

Similar to patient safety, early results indicate progress in health care quality. Challenges continue to exist in addressing health care disparities. Although a larger median percentage of residents and fellows reported knowing the QI priorities at their clinical site and participating in QI projects of their own design or 1 designed by their program or department, the findings also indicate a significant decline in the median percentage of residents and fellows reporting participation in QI projects aligned with the clinical site's QI goals.

From Cycle 1 to Cycle 2, little measurable difference was found in the number of CLEs engaged in efforts to routinely monitor and systematically address disparities in health care among their patients. The median percentage of residents and fellows reporting awareness of their clinical site's priorities in addressing health care disparities has also declined slightly.

In general, the majority of the measures in care transitions moved in a direction opposite of desired change. Compared with the first set of visits, a significantly smaller median percentage of residents and fellows in the second set of visits reported following a standardized process for handling care transitions from inpatient to outpatient.

Across CLEs, fewer residents and fellows reported following a standardized process for handling transitions of care during handoffs between shifts in Cycle 2 than in Cycle 1. Fewer residents and fellows also reported using a standardized written template as part of a standardized process for change-of-duty handoffs. In contrast, it appeared, based on direct observation, that the change-of-duty handoff processes across programs were more standardized at the time of the second set of visits than at the first.

A high percentage of residents and fellows across CLEs continued to report knowing what they were allowed to do without direct supervision. At the time of the second set of visits, there was considerable positive change from the first set of visits in the median percentage of residents and fellows reporting that they had an objective way to know what procedures residents and fellows from other services were allowed to do without direct supervision when consulting on patients. However, challenges remain: it appeared that nurses in many CLEs continued to rely primarily on familiarity, trust, or year of training when residents and fellows performed clinical procedures in the absence of an attending physician. In addition, a larger median percentage of residents and fellows reported being placed or witnessing a situation in which they perceived there was inadequate supervision.

From Cycle 1 to Cycle 2, there was a marked increase in the median percentage of residents and fellows reporting that they would power through to handoff if impaired by fatigue. CLEs in both cycles had patient safety and quality leaders who recalled patient safety events related to resident and fellow fatigue in the past year.

In the area of professionalism, there was a modest improvement in the median percentage of residents and fellows reporting that they had documented a history or physical finding in a patient medical record that they did not personally elicit. From the first to the second set of visits, there was a small decrease in the median percentage of residents and fellows who reported that their clinical site provided a supportive, nonpunitve environment for coming forward with concerns regarding honest in reporting.

At the same time, the small decline in the median percentage of residents and fellows reporting that they felt pressured to compromise their honesty or integrity to satisfy an authority figure was promising.

The between-cycle findings across the CLER Focus Areas indicate that ongoing formative feedback may be having some effect in advancing CLEs. They also point to opportunities for improvement. In general, the results demonstrate modest progress in some areas (eg, percentage of residents and fellows personally reporting into the patient safety event reporting system), little or no movement in others (eg, percentage of CLEs with systematic and comprehensive efforts to identify and eliminate health care disparities), and undesired movement in other areas (eg, percentage of residents and fellows reporting they would power through when maximally fatigued).

Given their dynamic and intricate nature, CLEs can have a considerable time lag between the discovery of challenges, the implementation of systems changes to address these challenges, and the demonstration of results. The selected trends offer a perspective on how CLEs can continue their journey to assess and explore innovative ways to improve the learning environment and to ensure safe and high-quality patient care.