Nuclear Medicine Milestones Open Access

As part of the Next Accreditation System (NAS), all accredited specialties have developed specialty-specific educational Milestones as competency-based learning and assessment tools. In the NAS, Milestone data will be used in the assessment and provision of feedback to residents and will be a key indicator of programs' educational effectiveness. Nuclear medicine participated in this process as a Phase II specialty. The guiding principles behind Milestone development in nuclear medicine were to create an appropriate number and variety of Milestones that would be widely applicable across all programs and provide meaningful and valid data, while simultaneously making the Milestones manageable and not overly burdensome.

The project was started with an initial face-to-face planning meeting in February 2012. To foster contributions from diverse perspectives, the Nuclear Medicine Milestone Working Group encompassed broad representation from the nuclear medicine community with its members involved and experienced in graduate medical education. The members were drawn from Accreditation Council for Graduate Medical Education (ACGME) staff, the Nuclear Medicine Residency Review Committee (RRC), the American Board of Nuclear Medicine (ABNM), the Academic Council/Program Directors Association of the Society of Nuclear Medicine and Molecular Imaging (SNMMI), as well as resident representatives. Lorraine M. Fig, MD, MPH, served as chair. A smaller advisory group of 4 nuclear medicine representatives and 2 ACGME leaders provided feedback to the working group throughout the development process. Members of both bodies are shown in the 2a1box.

A preliminary draft of the Nuclear Medicine Milestones was created to include specific definitions of relevant subcomponents (subcompetencies) in the 6 domains of general competence. Contributory reference documents included the ACGME Nuclear Medicine program requirements, the ABNM standards, and published literature on nuclear medicine training.1 Working group members began to define the levels of performance expected of trainees across the continuum of their education. A particular challenge in developing the specific Milestone levels was the current framework of nuclear medicine training: the length of the education program varies from 1 to 3 years depending upon the prior training of the resident. Thus, trainees (by virtue of their different educational backgrounds) enter the program at different levels and are expected to advance at different rates. Another difficulty was finding an appropriate balance between Milestones that were very broad or generalized and those that were too specific (and thus only narrowly applicable).

At a second meeting, the working group made extensive revisions to the Milestones to expand their scope. For the Patient Care Milestone group, 3 Therapy Milestones were created to emphasize the importance and complexity of this aspect of nuclear medicine practice. The collaborative process continued via several conference calls through the remainder of the year. Concurrently with Milestone development, the working group also created scenarios for each Milestone subcomponent to provide specific, concrete guidance on how each level would be achieved. The working group developed the scenarios as a useful educational adjunct, and users are cautioned that the examples are not Milestone requirements per se, but provide representative examples.

Members of the working group and 6 program director volunteers participated in a pilot field-based testing process at the end of 2012. The pilot provided valuable information about the utility and feasibility of the draft Nuclear Medicine Milestones, under similar conditions to actual usage in the NAS. Specific questions of interest related to (1) time required to complete the assessments; (2) feasibility of the Milestone reporting process; (3) clarity of the Milestones; and (4) extent to which the Milestones discriminate between residents of different skills and abilities. Several program directors and education committees expended significant efforts to respond to the pilot. Comments were overwhelmingly positive and supportive of the concept, with high scores on such questions as “I understand the knowledge, skills, behaviors, etc that are being referred to in most Milestones” and “I have enough information to enable accurate selection of a level of performance for most Milestones.” Concerns were raised by some, which included the length of document, the time involved (averages of 16 to 22 minutes per resident), overlapping and repetitive Milestone items, and the difficulty for residents to achieve the highest assessment levels.

The pilot resulted in final refinements and a reduction in the number of Milestones from 31 to 19. This was achieved by eliminating duplicate Milestones and consolidation of Milestones in every competency category.

The pilot program also clearly showed that additional education is needed in Milestone usage. For example, some pilot participants were not aware that the Level 5 Milestone is an aspirational goal, one that is advanced beyond performance targets set for residency. To date, education efforts have included publications in specialty journals2 and newsletters (eg, published by the SNMMI Academic Council and the American College of Nuclear Medicine) as well as presentations at national conferences. Additional education is planned during and following the roll-out period.

There are 19 Nuclear Medicine Milestones. Their distribution across the 6 general competencies is shown in the table.

In keeping with the vision of the use of educational outcomes in NAS, the Nuclear Medicine Milestones will be used for semiannual review of resident performance and reporting to the ACGME. One of the goals of the Milestones is to provide benchmarks during residency that will facilitate the early identification of residents who are not progressing along the continuum or are considerably delayed when compared to their peers. In the process of developing the Milestones, the working group took care to review the structure of a variety of nuclear medicine training programs to ensure that the various levels of progression accurately reflected topic mastery, were logical, and were widely applicable.

A stated aim of the NAS is to ensure that physicians are competent to provide high-quality and safe medical practice by the time of residency program graduation. Thus the working group critically examined the appropriateness of the Level 4 Milestones, which is the level designated as the graduation target. We believe that residents who substantially demonstrate the Level 4 Nuclear Medicine Milestones are prepared for unsupervised practice. (Note that it is possible that some residents may achieve the Level 4 Milestones at an earlier time point during residency, and that few residents attain Level 5 by the end of training.)

The Nuclear Medicine Clinical Competency Committee (CCC) should ideally be composed of a widely representative membership. The majority of CCC members will consist of core medical faculty, yet the inclusion of technical and other nonphysician staff is encouraged as this will provide a more rounded assessment and allow the contribution of specialized assessments. For example, radiation safety staff may provide input into the trainee's level of knowledge of radiation physics. CCC members are expected to make a consensus decision on the progress of each resident, as the Milestone reporting process consists of a single report per resident at each reporting time point. Training to facilitate consistent ratings and education of the residents in the expectations at each Milestone level will strengthen the evaluations, foster transparency, and improve the feedback process. It is emphasized that the Nuclear Medicine Milestone levels do not correspond with the postgraduate year of education.

The Nuclear Medicine Milestones were developed by a broadly representative group to facilitate 2 key aims of outcomes-based accreditation in the NAS—assessment of and provision of feedback to residents using specialty-specific dimensions of competence, and review of programs' effectiveness through the aggregated data on residents' attainments of competence for independent practice in the specialty. In that context, assessing the validity of the Nuclear Medicine Milestones by comparison to established data points and methodologies is an important future goal. Accumulation of robust data for statistical correlation of Milestone results with external data such as the ABNM board scores is critical in this. The process of collecting this information may be slower in nuclear medicine than other specialties, owing to the relatively limited numbers of graduates in the specialty.