Educational Milestone Development for Transitional Year Residency Training Open Access

The Accreditation Council for Graduate Medical Education (ACGME) Outcome Project has shifted the focus for accrediting residency training programs from structure and process to documented outcomes. A central part of the ACGME's Next Accreditation System (NAS) is “articulating milestones of competency development in each discipline.”1 Resident achievement of educational Milestones will be an indicator of the educational effectiveness of residency programs.

Most specialties have developed their Milestones through a collaborative effort involving representatives from the ACGME, the pertinent American Board of Medical Specialties (ABMS) organization, the pertinent professional specialty society, and the pertinent program directors' association. In contrast, the approach was quite different for Transitional Year (TY) residency programs, which comprise only 1 year of accredited postgraduate medical education, where there is no corresponding ABMS board certification entity or professional specialty society. In addition, graduates of TY training programs are not expected to demonstrate sufficient competency to enter the independent practice of medicine, as is the case with other specialty (categorical) residency programs.

In October 2010, the ACGME appointed Danny M. Takanishi Jr, MD, chair of the ACGME TY Review Committee, and Steven R. Craig, MD, chair of the Council of Transitional Year Program Directors (CTYPD) as cochairs of the TY Milestone Working Group (2a1box 1). In January 2011, TY program directors were invited to apply to serve on the Milestone Working Group. The cochairs and ACGME staff assigned to the working group (Steven P. Nestler, PhD, initially Linda Thorsen, MA, and subsequently Lorraine Lewis, EdD, RD) selected the other members of the working group. The working group membership included members of the ACGME TY Review Committee, the CTYPD, at-large members, and a resident representative. Community-based programs, academic health centers, and university programs from all regions of the country were represented. A number of these educators had served as TY program directors for more than 20 years, and 1 was a Parker J. Palmer Courage to Teach Award recipient. Specialties represented in the group included dermatology, emergency medicine, family medicine, internal medicine, obstetrics and gynecology, and surgery. Finally, the working group was editorially independent from the ACGME as it endeavored to craft the TY Milestones.

At its initial meeting in May 2011, the working group reviewed as guiding principles the aspirational goals and objectives of the ACGME Milestone Project, the developmental framework that would serve as the blueprint going forward, and empiric foundations derived from the literature.2–10 The group also reviewed the initial Milestone drafts formulated by the Internal Medicine, Pediatrics, and General Surgery Milestone Working Groups.

The conceptual underpinning of creating Milestones to document competence observed in performance required the working group to articulate these expected benchmarks of desired characteristics in behavioral language. The compelling need to provide a catalogue of evaluation instruments to document competence in each of the 6 competency areas required the working group to couple the genesis of Milestones with general evaluation strategies. The working group acknowledged at the outset that defining competence is a complex process and occurs in the context of the educational continuum. The group philosophically agreed that achievement of all desired Milestones will signify that a resident is ready for entry into certain career paths or categorical specialty training programs that require a clinical base year, and it will not infer that the graduating TY resident is safe to enter independent practice. Finally, the working group determined that it was vital that all stakeholders endorse the developmental process and a timeline was established for the design of the Milestones and the dissemination of the draft for vetting by the graduate medical education community.

The working group was charged to create a rubric consistent with a 5-level model, beginning with Milestones expected for matriculation into the training program, progressing on to a graduation target, and ending at an advanced level with Milestones that are aspirational and would only be achieved by a few exceptional residents.11 The 5-tiered construct was derived from the Dreyfus model of skills acquisition.12 

The Internal Medicine, Pediatrics, and General Surgery Milestone Working Groups had been meeting and working on creation of their specialty-specific Milestones for more than 1 year prior to the first meeting of the TY Milestone Working Group. The group reviewed the process used by these other specialties in drafting their Milestones and the initial work products that emerged from these efforts. This was especially important as TY Milestone Working Group members unanimously concurred that the Milestones created by the group needed to be generally comparable to the Milestone requirements for the postgraduate year (PGY)-1 of these core specialties. Graduating medical students and specialty programs requiring a clinical base year of training prior to entering certain career paths or categorical specialty training programs need assurances that all of these training pathways will prepare them well with equivalent training expectations in the acquisition of fundamental clinical skills.

The ACGME TY program requirements provided the rudimentary scaffolding on which the TY Milestones were built, incorporating and adding elements identified as essential for the PGY-1 from other Milestone working groups. The initial focus of attention was Milestone development in the patient care and medical knowledge competency domains.

The working group developed a total of 23 Milestones across the 6 ACGME competencies. Milestones in the patient care and medical knowledge competency areas were created independently by the working group. For the other competencies of interpersonal and communication skills, professionalism, practice-based learning and improvement, and systems-based practice, the group reviewed the work product of an ACGME Expert Panel on Milestones released in September 2011. The cross-specialty Milestones had been developed for the express purpose of adoption and/or adaptation by the specialty Milestone groups. The TY Milestone Working Group adopted these proposed Milestones with some significant revisions to ensure that the product was aligned with the goals of a TY year.

Each Milestone was uniquely stratified across 5 levels of competency. The 5 levels are shown in 3a1box 2.

The working group tacitly recognized that the establishment of Milestone validity was central in safeguarding the success of its efforts. Participation of a multidisciplinary cohort of content experts and review of the theoretical and empirical literature on the subject matter3–10 ensured the initial validity of the TY Milestones. In addition, the group solicited feedback from a large number of constituents and made revisions to the draft Milestones based on the feedback from these diverse groups. Constituents whose input was sought included an advisory group of TY program directors and representatives from several categorical programs TY graduates pursue. Two organizations in particular provided valuable input into the final Milestone draft. The Military TY program directors during their meeting in November 2011 and TY program directors who attended the CTYPD annual meeting in Fort Lauderdale, Florida, in May 2012 provided vigorous commentary that was invaluable in drafting the final set of TY Milestones.

During academic year 2013–2014, TY programs are being encouraged to pilot the TY Milestones and to provide feedback about the results of these efforts. Moreover, the CTYPD is planning a session for the 2014 annual meeting in May on best practices for TY Milestone implementation, based on pilot projects conducted by member programs.

The TY Milestones will help programs develop curricula designed to better teach and assess resident performance in each of the 6 competency areas. Residents will find the Milestones especially helpful in self-assessment and in developing their individualized learning plans. Residency programs will find the Milestones helpful in identifying residents in need of early remediation.

Collective assessment of performance by a program's residents in each of the Milestone areas compared to national norms will provide programs with a gap analysis to identify areas of training that need to be strengthened. Finally, assessment of resident performance at the end of TY training (summative end-of-training evaluation) in each of the Milestones will need to be clearly communicated to categorical programs that subsequently accept these graduating residents so remediation can be provided to help bring these residents up to the expected level of performance, where needed, early in the categorical training process.

The new residency training accreditation requirements that go into effect on July 1, 2014, require TY programs to establish a Clinical Competency Committee (CCC). The requirements call for the program director to appoint the CCC, which must comprise at least 3 members of the program faculty. Faculty from other programs and appropriate nonphysician members may also be appointed. The CCC should review all resident evaluations semiannually; prepare and assure the reporting of Milestones evaluations of each resident semiannually to the ACGME; and advise the program director regarding resident progress, including promotion, remediation, and dismissal.13 

The TY Milestones will allow programs to enhance their assessment and feedback to residents. At the program level, Milestone data will be used as 1 indicator of the educational effectiveness of TY programs in the outcomes-based approach of the NAS. Use of the Milestones will generate further data on their usability, practicality, and validity. Collection of this data over the first years after TY Milestone implementation will lead to informed modifications of this initial iteration of the Milestones to preserve and sustain criterion that are meaningful and relevant to TY residents and programs.