Improving Value-Based Care Education in a Fellowship by Incorporating ACGME Competencies

Although controlling costs is a stated priority for the medical profession as well as a societal responsibility, clinical educators are provided little evidence as to how to teach value-based care effectively. Moreover, best practices for measuring learner competency in this domain are unclear. The competency of cost awareness and risk-benefit analysis is embedded within the Accreditation Council for Graduate Medical Education (ACGME) competency of systems-based practice.¹ 

Gaining the skills necessary to provide value-based care requires mastering related competencies within medical knowledge and patient care and systematically analyzing one's practice, which falls under the practice-based learning and improvement competency.

Most educational interventions in value-based care focus on a single dimension of this competency—cost—which can be assessed through various tests of knowledge.^2–9  However, the integration of clinical judgment with knowledge of cost is crucial to ensure the preservation of high-quality care. This interplay is illustrated in a study by Rudy and colleagues,⁹  in which residents were presented with a hypothetical case. The authors found that residents receiving charge data before ordering tests spent less but also had lower appropriateness scores for ordered tests,⁹  thus demonstrating the importance of measuring the practical application of knowledge through clinical practice review when assessing competency in value-based care.

In a gynecologic oncology fellowship program, we developed a consistent format for an existing conference to integrate high-value care practices with medical knowledge and patient care competencies to routinely identify high- and low-value care practices and develop fellow-led quality practices, with measurement of preintervention and postintervention patient-care practices in the identified quality areas.

This 3-year fellowship program is based in a large urban health care institution that is part of a university and includes 3 fellows. Conference attendees included 5 gynecologic oncology attending physicians; 2 advanced practitioner nurses, who independently practice in the outpatient setting; and 3 gynecologic oncology fellows. For the purposes of the analysis, learners were considered to be all conference attendees.

At the beginning of the academic year, an introductory didactic session on value-based medical care was undertaken. As part of an effort to improve the quality of value-based teaching in our gynecologic oncology fellowship program, faculty and fellows contributing to education conference sessions were instructed to routinely and robustly incorporate cost- and value-based care considerations for each clinical topic addressed in all education conferences. Sessions included information such as cost-effectiveness studies that relate to the diagnostic or treatment strategies being discussed and “choosing wisely” recommendations.¹⁰  Faculty and fellows were also encouraged to identify areas in which robust value-based data were absent. Educational sessions spanned 90 minutes each. The clinically oriented conference format was interactive, case based, and team based, with the team collaborating on questions related to the case and in discussions regarding quality improvements in our practice.

The first 5 sessions were provided in successive weeks by the lead author (K.Z.), program director for the gynecologic oncology fellowship, to model the new format for the group. Cervical cancer was chosen as a focus for this initial teaching module, and it is these 5 sessions that represent the scope of this study. Curricular details are described in table 1 and in further detail in the online supplemental material. Subsequent modules for other disease sites were provided by core faculty members teaming up with fellows.

Based on discussions facilitated by one author (K.Z.) during each interactive didactic session, low- and high-value practices were identified by the group to target in our outpatient practice for reduction and enhancement, respectively.

A quality review of the gynecologic oncology practice was then undertaken by the fellows based on discussion generated during the conferences. Patient care decisions in the outpatient setting were analyzed by chart audit for 6 months prior to the educational intervention and compared with the 6 months after the intervention. The frequency of identified low- and high-value practices were compared before and after the intervention, with numerators as the low- or high-value practice being considered and the denominator as the number of opportunities for that particular care decision to be undertaken in the observation period. The percentage of change was calculated, and the effect size was analyzed with the phi coefficient (φ).

Care decisions were documented collectively for the gynecologic oncology group. Fellows participated actively in many of those care decisions, but their individual care decisions could not be distinguished from attending care decisions in our quality review.

This study was reviewed and approved through our Institutional Review Board at University Hospitals Cleveland Medical Center.

Table 2 depicts the low- and high-value practices that the group identified during the didactic sessions as targets for improvement. The evidence used to support those designations is described in further detail in the online supplemental material.

A total of 126 patients with a diagnosis of cervical cancer were seen by providers in the outpatient setting on the gynecologic oncology team during the entire observation period. The findings of the quality review are depicted in the figure.

Among low-value practices, there was a 33% reduction in routine postradiation positron emission tomography/computed tomography (PET/CT) imaging after the intervention (8 of 13 patients [62%] before intervention versus 2 of 7 patients [29%] after; effect size φ = 0.3), 33% reduction in the percentage of colposcopies performed for patients with low-grade squamous intraepithelial lesions (2 of 6 [33%] before intervention versus 0 of 2 [0%] after; effect size φ = 0.3), and a 12% reduction in the rate of radical hysterectomies performed in women with high-risk pathologic features (1 of 8 [12%] before versus 0 of 5 [0%] after; effect size φ = 0.2).

Among high-value practices, there was a 100% increase in the rate of smoking cessation referrals given to patients after the intervention period (0 referrals before versus 36 referrals after).

The utilization of pretreatment imaging for early stage cervical cancer did not change (86% [6 of 7 eligible patients] versus 89% [8 of 9 eligible patients]) nor did routine imaging for surveillance in asymptomatic patients (19% [22 of 115] versus 20% [19 of 97]).

In our practice, smoking cessation counseling was identified to be a high-value practice that was underutilized by our group. A direct result of the educational intervention was to expose the omission of this practice among those in our group. As a result of this educational intervention, our fellows were sufficiently impressed by the data presented to collectively undertake a quality initiative in this area. The difference in smoking cessation referrals was the result of several practice changes depicted in the online supplemental material.

Using an existing conference structure for our gynecologic oncology fellows, this high-value care curriculum produced marked changes in the provision of 3 low-value and 1 high-value services as well as stimulated fellows to undertake an additional, successful quality improvement project. The fellows' work also allowed assessment of their skills in carrying out a quality improvement project.

Of interest, there was no difference in the frequency of 2 low-value practices in our quality review. It is unclear why those 2 practices were not influenced by our educational intervention, although several possibilities exist. While the plan-do-study-act cycle provides a framework for iterative health care improvement,²⁴  one can also apply plan-do-study-act cycles to quality improvement projects in a teaching program.²⁵  As such, curricular refinements are planned for the next teaching cycle on this subject.

The study findings may partially be explained by the Hawthorne effect, which “is the inclination of people who are the subjects of an experimental study to change or improve the behavior being evaluated only because it is being studied, and not because of changes in the experiment parameters.”^26(p31),27  The improvement in the high-value care initiative, to increase smoking cessation referrals, may have been due in part to a Hawthorne effect. Thus, an important next step for our group is to assess the sustainability of those behaviors after the educational intervention as well as to determine whether other beneficial clinical actions declined when smoking cessation referrals increased.

Our binary quality endpoints of interest were a simplification of the complexity of measuring value-based care. Ideally, value is measured by the care of a patient's medical condition with direct clinical outcomes measured and total costs of care over the entire care cycle.²⁸  Fellows chose the endpoints of interest for their simplicity and applicability to those particular care decisions for the general population of patients with cervical cancer, irrespective of clinical context. However, ultimately, the fellows found that those captured only a small percentage of the care decisions undertaken in this population of 126 patients with cervical cancer, and the number of care decisions we were able to analyze was small. Indeed, measuring quality that accurately reflects the full spectrum of patient care is a complex and time-intensive task that requires access to high-quality data as well as contextual interpretation.^29,30  While health care is in the midst of a transition from fee-for-service to value-based reimbursement,³¹  the medical community still finds itself in the earliest stages defining metrics and effective methodology for measuring quality. This further underscores the vital need for our learners to be active participants in conversations regarding how to measure quality through these types of exercises.

The study is limited by the small numbers of trainees from a single site, which limits generalizability to other programs, particularly other types of fellowships. In addition, while fellows regularly participated in care decisions, it was impossible to distinguish decisions undertaken or influenced by fellows versus attending physicians. Thus, the impact of this educational intervention on the fellow participants could not be distinguished. The short follow-up period did not allow conclusions about sustained changes to fellow practices. Finally, without determination and measurement of balancing processes, it is not known whether other beneficial clinical actions declined or other low-value actions increased during the study period.

Next steps include analyzing the sustainability of this intervention in our group as well as the generalizability of these findings among more diverse faculty educators and training programs.

Overlaying value-based concepts consistently with the routine medical and patient care teaching points in our education conference is feasible. Having fellows engage in a subsequent quality practice review served both as a part of their educational experience and also allowed assessment of competency in its practical application.