Use and Potential Misuse of Milestones

By 2001, the Accreditation Council for Graduate Medical Education (ACGME) 6 core competencies were established, which introduced graduate medical education's (GME's) transition to competency-based medical education (CBME).¹  Competencies are teachable attributes that residents and fellows must develop in order to carry out professional roles. Milestones describe the developmental pathway for achieving each competency in more than 150 GME specialtes and subspecialties. Yet confusion remains about their optimal use as well as misuse, which may produce adverse consequences for all stakeholders (eg, residents, faculty, programs, sponsoring institutions [SIs]).

Milestones are narrative statements that describe skills, knowledge, and behaviors for performance in each of the 6 core competency domains and are arranged sequentially from attributes of novices to attributes that are aspirational.²  Milestone-based assessments enable faculty to provide GME learners specific formative feedback and improvement goals through the defined progression of levels toward expertise.³  Aggregated Milestone data from within and across programs can facilitate program, SI, and review committee (RC) learning and improvement.

Milestones themselves are not assessments. They are constructs used by the GME community to build assessments. As constructs, Milestone assessments are open to interpretation and may be affected by context and other factors such as the skill and biases of assessors. These features can be mitigated during formative feedback sessions, but they make Milestone-based assessments inappropriate for high-stakes summative assessments of learning or competence. Thus, Milestone assessments should never be shared with certifying, credentialing or licensing boards or agencies, or potential employers, or provided as evidence of competence for legal purposes.⁴  When used for these unintended purposes, faculty may be reluctant to give honest assessments and clinical competency committees (CCCs) may be tempted to inflate Milestone judgements. Milestone security can facilitate the integrity of the assessment processes.

1. Orient the program director, teaching faculty, residents/fellows, and program coordinator/administrator(s) to the current specialty-specific Milestones, how they are operationalized through the program's curricula and assessment tools, and specific (mis)uses.

2. Implement an annual CCC orientation program that defines the accepted and potential misuses of Milestones and Milestone-based assessments, to include emphasizing what they should not be used for:

   • “Cut and paste” end-of-rotation assessments

   • Sole summative remediation or advancement decisions

   • Assessment for a credentialing entity (eg, to seek hospital privileges)

   • Assessment for a potential employer, certifying specialty, or subspecialty board

   • Malpractice litigation (inferring competence of a resident/fellow involved in a malpractice suit)

3. Review program and SI policies that ensure the security of individual Milestone assessments.

1. Encourage direct feedback: Milestone-based assessments are formative, not a summative “grade.” Use Milestones to initiate interactive feedback/coaching conversations on goals. Cultivate ongoing faculty and learner development, in a culture of supportive assessment and feedback.^2,5 

2. Utilize the learning trajectory: Use other data sources with serial Milestone assessments to codevelop individualized learning plans, and promote learner self-confidence and self-efficacy. Use longitudinal specialty Milestone data to estimate the likelihood a Milestone rating will fall below an acceptable level at program completion. This learning analytic approach can predict residents with concerning performance earlier.² 

3. Incorporate Milestones into program evaluation and share lessons learned: Incorporate aggregate and serial program Milestone data with national benchmarks in your program evaluation plan, to monitor overall curriculum and the effect of curricular changes. Share lessons learned with other programs.

4. Ensure ongoing security: Milestone assessments are aggregated when reviewed by the ACGME and their RCs. The ACGME uses aggregate specialty or program Milestones, as the “unit of analysis” in its reports. This prevents the misuse of Milestone assessment data as indicators of overall competence of an individual resident or fellow, and further ensures the security of individual Milestone data.