Using the American Board of Internal Medicine Practice Improvement Modules to Teach Internal Medicine Residents Practice Improvement

Current expectations require physicians to collect, analyze, plan, and implement changes to improve their own practice as well as the larger systems in which they work.1,2 This competency has previously not been an explicit part of the traditional residency curriculum and was formally added when the Accreditation Council for Graduate Medical Education adopted the 6 competencies in 2002.1 Specialty boards now incorporate practice improvement (PI) skills into maintenance of certification processes, but many faculty have not yet had to recertify. Correspondingly, faculty preceptors differ in their knowledge of PI and are similar to residents in their level of confidence with PI.3–5 Consequently, residency programs have struggled to prepare faculty preceptors who can teach and demonstrate practice-based learning and improvement (PBLI) and systems-based practice (SBP) and to develop and implement effective educational materials and data management systems.6,7 

Although individual programs can develop PBLI and SBP curricula for residents, the practice improvement modules (PIMs) offered by the American Board of Internal Medicine (ABIM) provide an efficient curricular supplement. The PIMs, developed for practicing physicians, include an educational and structured approach to data collection and analysis of performance and process indicators.8,9 The tutorial nature of the ABIM PIMs allows faculty to join residents in learning SBP and PBLI, to enhance their skills, and to guide residents toward improvement projects that have real value and potential impact in the clinics that faculty supervise.

Communication with patients is a target area for resident improvement. Studies have found patient satisfaction ratings to differ between residents and faculty physicians in outpatient settings.10–12 Although some disparity may be attributable to experience, some may reflect practice behaviors and systems issues that are amenable to change.12 At our own veterans' affairs hospital, patient focus groups indicated need for improvement in communication. Residents' yearly clinic evaluations revealed that they found it challenging to return calls, communicate test results, and complete patient paperwork promptly. Consequently, we chose to include the Communication-Primary Care PIM in our ambulatory PI curriculum.9 

Recent publications have described graduate medical education PI curricula and several have included a PIM. Reports have noted the importance of faculty development but provide little guidance for engaging faculty in the development and delivery of the curriculum.6,13–18 

The shifting emphasis toward faculty development that is workplace centered makes it imperative that faculty learn in the context of their real work.19 We describe the implementation of an ambulatory PI curriculum, which includes a PIM, for University of California, San Francisco internal medicine residents at the San Francisco Veterans' Affairs Medical Center (SFVAMC) outpatient practice.

We reviewed published PI curricula to select best practices. Reports confirmed our previous experience that didactics were suboptimal to practical application.20,21 The PIM enabled incorporating SBP and PBLI into our residents' clinical setting with support from clinic preceptors who learned the PIM content within the context of their actual work. The PIM provided step-by-step guidance for developing a PI project and further supported residents' and clinic preceptors' attainment of knowledge and skills needed to conduct and complete PI projects (Table 1). Critically, the preceptors modeled the necessary skills as they colearned with the residents.

We added curricular content to support the use of the PIM. We invited local experts to teach basic PI principles in our PI ambulatory didactics (Table 2).22–24 One clinic preceptor (the PI champion) developed teaching materials, based on the PI content in the PIMs, for the other PI clinic preceptors to use during PIM sessions (Table 3). The PIM sessions provided protected time for residents to work on their projects. The residents completed the PI curriculum during their 6-month ambulatory block in postgraduate years 2 and 3.

The first goal of our new curriculum was to provide preceptors with materials to facilitate the PI projects and residents with a foundation in PI concepts. The second goal was to identify specific areas for improvement in practice and/or systems using the ABIM PIM Communication-Primary Care. The third goal was to have residents use their PBLI and SBP skills in a PI project with support from PI clinic preceptors.

Table 2 summarizes the PI didactics showing the PI topics, PI concepts, teaching methods, and time devoted to each topic. General topics were discussed in the large group sessions with small group activities. Many concepts were reinforced in separate PIM sessions (Table 1).

The PIM included a patient survey, an examination of the clinic microsystem, a practice analysis, and a guide to develop a PI plan (Table 1). Once the patient survey and systems evaluation were complete, the ABIM web-based software produced a practice analysis. The program guided the resident team through the development of an improvement plan, providing step-by-step prompts and information about setting a performance goal and developing plans for remeasurement.

Residents rotated in 2-month blocks of alternating inpatient and outpatient months and attended the PI didactics during their outpatient months. To include all residents, each lecture was held twice. Residents worked on the PIM during prescribed continuity clinic times (PIM sessions). The 3 PI clinic preceptors, including PI champion, ambulatory chief resident, and the clinic site director, participated in PIM sessions. These preceptors used specially developed teaching materials to facilitate the meetings, review the results, and provide guidance as needed (Table 3).

Our evaluation included assessment of the curriculum, resident self-assessment of skills, and results of the PI projects. Residents completed a program survey about their satisfaction with the PIM and the ambulatory PI curriculum rating 8 questions on a Likert scale from strongly disagree (1) to strongly agree (5). Residents completed an ABIM retrospective pre–self-assessment and post–self-assessment of competence in SBP and PBLI knowledge and skills in these areas: describing an issue (5 items), building a team (4 items), defining the problem (5 items), choosing a target (4 items), testing the change (4 items), and extending improvement efforts (9 items).5 Each item was rated from not at all competent (0) to very competent (3). Residents also indicated if they participated in learning activities relevant to skills in each area. The PI projects developed by the residents were described.

Implementation began July 2008 with a special orientation to clinic. Residents and PI clinic preceptors were provided with information sheets on the general principles of PI and PIMs that were developed by the PI champion. Implementation was guided by change management strategies such as providing a rationale for the change, creating and empowering a small leadership team, discussing the vision and priorities to all people potentially affected by the change, and involving as many as possible in implementation.25 The PIM reinforced these principles, emphasizing that change must be realistic, achievable, and measurable. Our ambulatory chief resident actively facilitated all aspects of the curriculum and the clinic site director guided feasibility decisions and reviewed PI projects for alignment with clinic priorities.

The SFVAMC has an electronic medical record with clinical reminders and quarterly quality measure report cards. The residents benefited from these systems that facilitate improvement but are disadvantaged if they move to less sophisticated work environments. Our PIM-based ambulatory PI curriculum allowed residents to practice all of the steps involved and to learn how they can make changes in their own clinic.

Residents were divided into 3 groups of 6 to 13 residents based on their assigned continuity clinic day. Residents remained on the same clinic day and attended clinic conference together thereby developing a supportive group dynamic and learning environment. The groups focused on PI during 4 to 5 clinic conferences (PIM sessions [Table 1]) spaced throughout the year and facilitated by 1 to 3 of the PI clinic preceptors.

We engaged clerical staff, nurses, residents, and other clinic preceptors to support PIM activities, particularly the patient survey. The PI clinic preceptors worked through the microsystem evaluation using locally developed materials and guided the residents to complete the material as a group. The residents designed each segment of the PI project chosen by their group. Residents volunteered to take on portions of the project and were given up to 5 hours away from clinical duties to work on their tasks.

The average attendance at the PIM sessions was 70%. All residents participated in their team projects. Sixteen residents (64%) responded to the program survey. Seventy-four percent agreed or strongly agreed with the statement “projects will improve patient care,” 68% agreed or strongly agreed that their “PI knowledge improved,” and 68% agreed or strongly agreed with the statement that the “curriculum was effective for teaching PI to residents.” Residents' feedback to the PI champion and to the resident clinic site director indicated that they were able to complete a project that was valuable to their practice and did not require time outside of clinic.

Twenty-three residents (96%) completed the self-assessment. They reported significant improvement in all areas (Table 4). However, in most areas, the residents were just reaching a score near 2 or “somewhat confident.” Residents were more likely to have participated in activities associated with describing the issue and defining a problem and less likely to have participated in activities related to testing the change and extending improvement.

All 3 PI projects were completed. The first project was to improve compliance with ordered labs and imaging tests. Residents infrequently used an existing discharge planning handout; 4 of 6 residents reported no use. Residents designed and implemented a new handout that included an improved checklist of instructions for their patients. Residents added clinic information and tips for good communication with physicians. At the completion of the project, 6 of 6 residents used the discharge-planning handout. Subsequently, the resident PI team distributed the handout clinic-wide and it is now broadly used by faculty, residents, and nurse practitioner providers.

The second project involved creation of patient notification templates. Providers inform patients of their test results by letters generated in the electronic medical record or by phone call; however, our practice analysis showed that only 56% of patients reported they always received the results of their diagnostic tests. Residents developed templates within the electronic medical record to streamline the notification process. The templates included simple explanations of basic labs and patient education content and are easily imported into patient letters. A retrospective chart review of patients seen during a 1-month period showed that preintervention each resident notified 0% to 50% (mean, 29%) of their patients of their diagnostic test results (n  =  6 residents, 45 patients). After these templates were distributed, a subsequent 2-week chart review showed residents' notification rates ranged from 50% to 80% (mean, 62%) of their patients (n  =  6 residents, 53 patients). Residents agreed that the templates made informing patients of their test results more efficient. Many providers now use these templates daily.

The third project was the development of antihypertensive medication protocols for nurses. Residents chose this project to address a deficiency in the use of algorithms for the management of common problems and to improve resident-nurse communication. During preintervention the residents analyzed registered nursing blood pressure management visits for the prior 3-month period and found that 85% (23 of 27) of patients had a blood pressure goal stated and 52% (14 of 27) had a heart rate and/or lab goal stated. The postintervention analysis indicated that 96% (23 of 24) had blood pressure goal stated and 63% (15 of 24) had a heart rate and/or lab goal stated. The protocol templates were only used by 3 of 10 residents and in 4 of 30 patient encounters. Templates provided a guide, but residents found them too generic. Some residents were not present when the final versions of the templates were developed and, despite e-mail notification, were not aware of their availability. Residents and nurses use the templates and see them as an ongoing opportunity for improvement.

Our curricular approach of combining PI didactics with an ABIM PIM has facilitated resident and faculty engagement with PI. Although individual projects varied in their degree of success, residents' self-reported knowledge and patient care improved, and resources were produced that are now used by providers throughout the clinic. Additionally, we found the PIMs' structure and robust educational content useful for faculty who were not PI experts to enhance their own PI knowledge as they guided residents through the practical application of PI concepts while addressing real clinical problems.

Although the PIMs were designed for practicing physicians, we found them successful for resident teams. Using PIMs in teams required collaboration and good communication across months as residents transitioned between inpatient and outpatient duties. Also, faculty involvement and time specifically set aside for projects were necessary to make the teams function effectively. Importantly, we used faculty development that enhanced knowledge of PI and provided resources to help facilitate group projects in the ambulatory clinic.

The ambulatory PI curriculum includes didactics that allowed residents to envision how their PI projects fit into their current education, their future career, and the health care system overall. The SFVAMC medical practice clinic was conducive with a culture of ongoing improvement facilitated by the electronic medical record. The didactic and PIM projects enhanced this culture by engaging residents and faculty together in the design and conduct of PI projects. This model illustrates how learning takes place in the workplace.

One key to our success was involving the right people. The ambulatory chief resident facilitated the PIM implementation by encouraging resident participation. Representation from clinic leadership at PIM sessions was critical in directing residents toward useful projects. Additionally, cohesive resident teams allowed for the division of workload into manageable amounts, enhanced accountability, and allowed for completing meaningful projects.

Although we were gratified with our pilot results, there are several recommendations for improvement. First, identify ways to engage residents such as decreasing scheduling conflicts and identifying effective ways to communicate. Despite our best efforts, not all of the team members had the information needed regarding the PI projects. Second, embrace faculty development concurrent with implementation. We demonstrated that a successful PI curriculum using PIMs can be implemented by clinician educators who are not PI experts but who now have expertise that can be extended to others. Our newly trained faculty will expand the PI curriculum to our 3 other continuity clinic sites outside the SFVAMC using materials developed in this implementation year. Faculty members will still need to be committed to devoting considerable time to preparing and facilitating sessions. Third, identify assistance with particular aspects. For the residents, the PIMs provided an overwhelming volume of data. Residents' ability to manage and analyze their project data varied considerably, necessitating additional faculty oversight and instruction. The Communication-Primary Care PIM patient survey was very long and difficult for patients with low literacy or limited English proficiency and required many demands on the clinic system. Notably many of these challenges would be required of most PI projects. It is likely that the challenges are more specific to developing a meaningful PI curriculum than a specific challenge of the PIMs.

We acknowledge that this was a pilot study with limited though valuable information. Despite the limitations we found, our curriculum, which combined didactics, PIMs, and committed faculty, enhanced self-assessed knowledge and skills and resulted in systems improvements. For other programs considering the PIMs, we recommend clear time commitments and strong administrative support to augment success.