Observations: Another Possible Use of Milestones

In the past 2 years, the Milestone Project¹  has arrived with a big bang. It also stirred up mixed and ambivalent feelings in the medical education community. Most of the criticism has focused on added work for programs, on what is perceived as somewhat unclear mission and goals, and the issue of how this initiative will benefit training programs versus the Accreditation Council for Graduate Medical Education, which will use milestones for evaluating programs. Correspondingly, much of milestone literature has focused on how to make their use easier, on their role in performance assessment,²  and on the need to train faculty to use the milestones as an assessment tool.³ 

In their more in-depth criticism of the Milestone Project, Dewan and colleagues⁴  acknowledged that, for the first time, the milestones and the new accreditation system incorporate “outcomes.” However, as they pointed out, like other system changes in medical education, the Milestone Project was devised without supporting evidence, and it may not result in meaningful outcomes, such as improved patient outcomes or producing better physicians. According to Dewan et al,⁴  the “outcomes” of the milestones are problematic, as (1) verification of their completion is not an outcome, but a measure of adherence to a requirement on the input side; and (2) the “objective” assessments are done by faculty who have been identified as being notoriously “subjective.” The authors also emphasized that the quality of training ultimately must be judged on clinical outcomes and, unfortunately, those outcomes are not immediately available.⁴ 

As the milestones were primarily developed for evaluating the training programs, not much has been discussed about their possible usefulness to programs. Dewan and colleagues⁴  correctly noted the need for outcomes. Measuring outcomes is 1 part of the educational process. Of more interest and relevancy is how we use specific outcomes measures to improve our training programs—either teaching methods or curriculum. The questions then become: How long do we have to wait for those outcomes or their surrogate measures to be able to make meaningful changes in our programs? Is there possibly a way to use the milestones to improve training outcomes more immediately in a way that is useful to the program?

While waiting for trends in outcomes measures, we decided to begin to use milestones in 1 such possible way. Prior to the semiannual resident evaluation, we asked residents to evaluate themselves using the Psychiatry Milestones. Then, during the semiannual evaluation, we compared each resident self-evaluation (milestone by milestone) with that done by the clinical competence committee (CCC). Some resident self-evaluations closely matched the assessment by the CCC. Others varied tremendously. In both cases, they served as the basis of a practical, concrete conversation about performance. In-depth discussion of the differences in individual milestone ratings has been quite useful for residents to understand what is being evaluated and what are the expectations. While we do not know whether this feedback approach will improve the outcomes discussed by Dewan and colleagues,⁴  we found it useful in pointing out specific areas for resident improvement. Finally, we can affirm that the milestones have offered our psychiatry residency training program an immediate benefit!