Setting and Problem
Government agencies, insurance companies, health systems, and accreditors have developed numerous outcome measures that providers should consider and achieve. However, effective outcomes measurement faces barriers in teaching settings. In ambulatory practice, measurement tends to emphasize individual physician performance. This is problematic in ambulatory training practices, as residents are frequently absent for significant periods of time. In addition, measures selected are often chosen from outside the practice, and buy-in can suffer. This environment can lead to trainees and allied health providers accepting measures without critically analyzing what should be measured and why.
Intervention
University of Cincinnati internal medicine residents rotate through a yearlong ambulatory long block, occurring from months 17 to 29 of residency. The practice is divided into 6 mini-teams with 4 residents, and 1 nurse per team. Other team members include pharmacists, educators, administrators, a social worker, and a nurse practitioner. The team meets weekly to review care delivery, including performance measures. Before the start of the long block, residents are assigned a single outcome measure and asked to perform a literature-based review of the measure, termed the Defense of the Measures (DOM). Measures assessed may include items currently being followed in the practice, or new measures the group deems worthy of review (table). During long block orientation, residents present their review to the entire team, including nurses and allied health professionals. The review uses the PICOT question format (Population, Intervention, Comparison, Outcome, Time) to present evidence for or against a given measure, and concludes with a recommendation whether the measure should be kept, changed, removed, or added. Next, the entire team discusses the evidence presented, and votes on what should be done. The results of this vote become the list of outcome measures the practice team will follow for the year. Each month the team receives data on these measures for the practice as a whole, the 6 mini-teams, and each individual physician.
Outcomes to Date
Since the inception of DOM, outcomes measured in the practice have changed substantially (table). The team has rapidly adapted to newly available information, including a change in glycemic targets (HbA1c levels), new vaccinations (herpes zoster), or new screening recommendations (HIV, hepatitis C). The team has added measures of importance based on local need, such as the Ohio Automated Rx Reporting System (OARRS), which is mandated for patients receiving chronic opioids, and local research findings, such as measuring pulmonary function testing in patients with chronic obstructive pulmonary disease. In addition, the teams have reviewed and rejected several measures including prostate and lung cancer screening. All decisions about outcome measures will be revisited in the coming year in light of new evidence for or against a measure or changes in screening guidelines.
Ownership of the measures appears to have increased. After DOM, the team does not spend time arguing the value of measurement. The discussion during the voting process takes into account the viewpoint of multiple parties. For example, nurses often are tasked with the work a measurement creates (eg, injections), and they appreciate having a voice in the matter. Team members realize they cannot track and measure everything in every way, even if there is good evidence. For each measure, the team must assess its resources and the population served, and decide the benefit of the change. Virtually every new measure added has shown rapid increases in the years that follow. For example, herpes zoster vaccination increased from 10% to 49% 2011–2013, and OARRS reporting increased from 0% to 67% 2012–2013. In addition, several long block physician graduates have developed and managed outcome measures within their postresidency practice.
Conclusion
The DOM intervention empowers practice team members to critically examine the outcomes they are measured by, and use these choices to guide improvement work. It also teaches improvement practice related to outcomes measurement and prepares graduates to use these skills in practice.
