Charting the Road to Competence: Developmental Milestones for Internal Medicine Residency Training Open Access

In July 2002, the Accreditation Council for Graduate Medical Education (ACGME) Outcome Project changed the currency of accreditation from process and structure (capturing a program's potential to educate) to outcomes (capturing a program's actual accomplishments).1 Residency program directors were asked to provide more than a schedule of rotations, a written curriculum, and agreements with clinical training venues. They also must objectively document that their residents achieve competence in 6 general dimensions of practice. In phase 1 of the Outcome Project, programs defined objectives to demonstrate learning in the competencies. In phase 2 they integrated the competencies into their curricula and expanded their evaluation systems to assess performance in them. Programs are currently in phase 3, which requires them to use aggregate performance data for curriculum reform. Phase 4 intends to focus on identification of benchmark programs. This paradigm shift in training, hailed as the Flexnerian revolution of the 21st century,2 is aimed at enhancing our profession's ability to verify that graduates of residency programs are competent, at a minimum, to deliver safe and effective patient care.

Although the Outcome Project has advanced residency training in important ways, it has not resulted in widespread “operationalizing” of outcomes in the evaluation of residents or in the accreditation of programs. This may be partly because of the complex nature of the competencies, which reflect high-level syntheses of more operationally measurable learning objectives. To move the process forward, the ACGME has engaged the medical education community in “articulating milestones of competency development in each discipline.”3 The milestones would explicate the 6 ACGME general competencies by describing a developmental progression of observable behaviors. Programs would use the milestones to provide more specific feedback and evaluation to residents and ensure that they acquire the necessary knowledge, skills, and attitudes for advancing in the program and entering the next phase of their careers. The ACGME would use program performance on the milestones as currency for accreditation actions.

In November 2007, the American Board of Internal Medicine (ABIM) and the ACGME sponsored an initiative to develop milestones for internal medicine residency training. Herein we report the development process and the resulting list of proposed milestones for each ACGME competency.

The ABIM and ACGME convened a 33-member milestones task force ( appendix 1) composed of program directors, experts in evaluation and quality, and representatives of internal medicine stakeholder organizations, including the Alliance for Academic Internal Medicine, American College of Physicians, American Medical Association, Association of Program Directors in Internal Medicine, and the Society of General Internal Medicine. These individuals participated with the understanding that the resulting milestones document would not signify official policy of their respective organizations. Even though ABIM and ACGME provided funding, meeting space, and administrative support for this project, they agreed to maintain the editorial independence of the task force. An initial 2-day meeting included an overview of the Alliance for Academic Internal Medicine Education Redesign Task Force Consensus Report,4 a brainstorming session on the potential utility of developmental milestones, presentations of milestones initiatives at 3 residency programs (Michigan State,5 Lehigh Valley, and Baystate), a facilitated discussion of several conceptual frameworks of competence,6–9 and division of the task force into subcommittees representing the 6 ACGME general competencies.

The subcommittees worked independently, via conference calls and a PBwiki collaboration Internet site (PBworks, San Mateo, CA), to develop an initial set of milestones and suggested evaluation strategies for each competency. In their work, they reviewed the revised ACGME common program requirements and Residency Review Committee for Internal Medicine program requirements (effective July 2009),10 relevant medical education literature, and several internal medicine program curricula. The larger task force assembled for a second 2-day meeting in May 2008. The agenda included (1) achieving a consensus for a minimum standard for “competence” for internal medicine residents, (2) aligning evaluation strategies with particular milestones, and (3) considering practical issues, such as the resource and expertise requirements for programs. In addition, a writing committee was constituted from representatives from each subcommittee. Its charge was to refine and standardize the milestone language, reconcile redundancies and conflicts, and compose a document that articulated the need for developmental milestones, recorded the task force's process, and placed the initiative in the context of ongoing graduate medical education reform.

The members of the larger task force reviewed the document individually and provided additional commentary at a third meeting in December 2008. In particular, the task force recommended condensing the original detailed 64-page document to a briefer overview for the purpose of sharing the milestones with the broader medical education community. The writing committee revised the document accordingly and prepared it for external review. In total, members of the writing committee participated in 6 conference calls, 2 in-person meetings, and numerous e-mail exchanges.

The task force adopted the Dreyfus model8,9 of skill acquisition as a framework for developing milestones for internal medicine residency training. Specifically, we calibrated the milestones with the expectation that residents achieve, at a minimum, the “competency” level in the 5-step progression before completion of residency training. This threshold is consistent with other applications of the Dreyfus model to medical education.11–15 By the time a learner reaches “competence,” he or she has already progressed from simply applying rules to facts and features without context (novice) to considering the specific features of concrete “situations” (advanced beginner). The competent learner considers both context-free and situational elements but also hierarchically organizes and reduces them to a smaller set on which to base a decision. In addition, he or she becomes more intimately involved in the process and feels more responsibility for the outcome. In the next stage, proficiency, learners solve problems with an intuition that usually derives from some time in independent practice. Thus, although it is expected that some residents will achieve proficiency in some competencies, the task force decided not to set “proficiency” as a minimum threshold.

 appendix 2 lists the developmental milestones for internal medicine residency training, organized in terms of the ACGME general competencies and the extended specialty-specific requirements added by the Residency Review Committee for Internal Medicine. The ACGME-proposed bullets subdividing the competencies were used as the framework to organize the milestones. These subdivisions are either included verbatim or collapsed into a smaller number of categories. Recognizing that competence can be observed only in performance, we phrased the milestones in behavioral terms. We also suggested approximate time frames for residents to reach each milestone, recognizing a certain amount of arbitrariness in the process and anticipating that, for some milestones, achievement times may vary widely among programs with different curricula. For example, residents in a program that does not offer a quality improvement curriculum until the third year may not meet many of the practice-based learning and improvement milestones until then.

Finally, we confined our recommendations for evaluating the milestones ( appendix 2) to general strategies. The writing committee decided that a detailed discussion about the availability, formats, feasibility, and psychometric characteristics of specific assessment instruments was beyond the scope of this initiative, which focused on articulating the milestones. Among the general strategies, we did not include global ratings because faculty scoring fails to distinguish between performance in the 6 competencies.16 We did not link learning portfolios to particular milestones, as these collections may include evaluation items from all 6 competencies. The recommended evaluation strategies are intended not as prescriptions but rather as a range of options for program directors, who may choose among them or develop their own, based on their expertise, resources, programmatic objectives, and institutional values.

We propose this list of milestones to promote competency-based training in internal medicine. Residency program directors may use them to track the progress of trainees in the 6 general competencies and inform decisions regarding promotion and readiness for independent practice. In addition, the milestones may guide curriculum development, suggest specific assessment strategies, provide benchmarks for resident self-directed assessment-seeking,17 assist remediation by facilitating identification of specific deficits, and provide a degree of national standardization in evaluation. Finally, by explicitly enumerating the profession's expectations for graduates, they may improve public accountability for residency training.

It is worth noting that many of the milestones—particularly in practice-based learning and improvement, systems-based practice, communication and interpersonal skills, and professionalism competencies—are not unique to internal medicine. Physicians in any specialty should demonstrate competence in these “horizontal” dimensions of clinical practice. Thus, educators in other specialties may adopt some of our work as they develop milestones for their residency programs.

Some may find that the “generous” time frames set a low bar, believing that residents should reach some of the early milestones sooner. Indeed, in keeping with our decision to set a floor rather than a ceiling, we set the time frames with the expectation that a resident's failure to reach them would trigger further assessment and possibly remediation. At a programmatic level, a significant deviation from the expected progression along the milestones may trigger an accreditation action. Thus, we expect that many normally progressing residents will reach many of the milestones in advance of the “deadline.” A few exceptional graduating medical students may even begin their internship part of the way “down the road.” Finally, these time frames represent a starting point of an ongoing dialogue. We expect them to be refined based on the implementation pilot projects planned for the next phase.

We also anticipate that some program directors, weary from complying with the “musts” and “shoulds” handed down from the ACGME, may receive the milestones as yet another bureaucratic burden. On the contrary, we foresee the milestones making their jobs easier. The specific observable behaviors embodied in them, for instance, should assist program directors, who have hitherto struggled to translate the more general language of the 6 competencies into concrete assessments.18 Nor should this initiative stifle creativity and innovation. In the spirit of the Outcome Project, program directors remain free to develop innovative structures, curricula, and evaluation systems, provided they demonstrate learning “outcomes” in the 6 competencies, which are now elaborated in the milestones. Finally, we expect that residents, who often receive feedback lacking a specific action plan,19 will welcome the more actionable feedback afforded by the milestones framework.

Of course, “This is not the end,” as Churchill said in 1942. “It is not even the beginning of the end. But it is, perhaps, the end of the beginning.” More work is needed before these milestones can be successfully integrated into competency-based evaluation systems. Specifically, we will solicit commentary from the broader medical education community to help us refine the proposed milestones and correct any omissions or redundancies. We must also articulate concrete behavioral anchors for each developmental stage, identify psychometrically robust and feasible evaluation instruments to assess residents' progress, and train faculty to use these instruments effectively.20 Finally, we will learn practical lessons from the initial implementation experience, as diverse residency programs, beginning with pilot projects, integrate developmental milestones into their evaluation systems.

This will be a challenging task but, we believe, one that is well within our reach. We do not share the skepticism of others who lament the perceived inadequacy of currently available evaluation instruments.21 On the contrary, the “tool box” contains many robust instruments.22–29 The problem lies in the variable use of the instruments by faculty who do not share a common understanding of expected behaviors.30 The milestones provide a set of consistent expectations that should reduce this variability.

As representatives of the internal medicine education community, we articulated the milestones to embody our vision of the development of a competent internist. We ask the ACGME only to hold us to this standard.

Eva M. Aagaard, MD

Jane H. Barnsteiner, PhD, FAAN

Thomas A. Blackwell, MD

Karen Hsu Blatman, MD

Donald R. Bordley, MD

Kelly Caverzagie, MD

Davoren A. Chick, MD, FACP

Charles P. Clayton

Thomas G. Cooney, MD

Rosemarie L. Fisher, MD

Michael L. Green, MD, MSc

Luke Hansen, MD

Linda A. Headrick, MD

Kevin T. Hinchey, MD

Eric S. Holmboe, MD, FACP

Holly J. Humphrey, MD, FACP

William F. Iobst, MD

Gregory C. Kane, MD

David Karlson, PhD

Charles M. Kilo, MD, MPH

Lynne M. Kirk, MD

Catherine R. Lucey, MD, FACP

Thomas J. Nasca, MD, MACP

Eileen E. Reynolds, MD

Eugene C. Rich, MD, FACP

Paul H. Rockey, MD, MPH

William E. Rodak, PhD

Michele Sanders, MD

Henry J. Schultz, MD

Lawrence Smith, MD

Abraham Verghese, MD, MACP, DSc

Steven E. Weinberger, MD, FACP

Brent C. Williams, MD, MPH