The Development of the Radiation Oncology Milestones Open Access

The Radiation Oncology Milestones were developed to assist programs with resident evaluation and meaningful feedback, and to stimulate faculty development. In the Next Accreditation System (NAS), resident performance of educational Milestones will be an indicator of the educational effectiveness of residency programs.

The process of Milestone development for radiation oncology began in the fall of 2011 and was led by 1 of the authors (W.R.L.) with assistance from the Accreditation Council for Graduate Medical Education (ACGME). Two groups were recruited for the effort: a Radiation Oncology Milestone Working Group and an Advisory Group (2a1box). The working group comprised 11 individuals. The working group included members of the Radiation Oncology Residency Review Committee, leadership of the Association of Directors of Radiation Oncology Programs (ADROP), 2 resident members, and 2 ACGME representatives. Seven members of the group were board-certified radiation oncologists with expertise in resident education. The advisory group included 6 board-certified radiation oncologists, including representatives from the American Board of Radiology (ABR) and the American Society of Radiation Oncologists (ASTRO), as well as ACGME representatives.

The working group met in Chicago on 3 separate occasions (October 2011, February 2012, and September 2012). The first meeting focused on initial development tasks with an emphasis on developing Milestones for the Patient Care competency. The second meeting continued the development of Milestones for all 6 ACGME competencies; during this meeting feedback from the Milestone Advisory Group was reviewed and incorporated. The third meeting completed the development of the Milestones, and the working group also identified potential assessment tools.

After the third meeting, a draft of the Radiation Oncology Milestones was posted on the ACGME website for public comment. Comments from a range of constituents were considered and incorporated, and a final version of the Milestones was completed in May 2013.

The Radiation Oncology Milestones encompass a total of 22 Milestone sets (with each set describing the Milestones for a given subcompetency ranging from the novice to entry into the unsupervised practice of radiation oncology). The distribution according to the 6 competencies is as follows: Patient Care (11), Medical Knowledge (2), Professionalism (2), Interpersonal and Communication Skills (2), Practice-Based Learning and Improvement (2), and Systems-Based Practice (3). For the Patient Care Milestones, 9 are categorized according to disease site and 2 are procedural in nature (brachytherapy and stereotactic radiosurgery/stereotactic body radiotherapy).

There are 2 important aspects of the Radiation Oncology Milestones. First, the Milestones are not intended to be comprehensive (eg, there are no Milestones for Sarcoma or treatment of pediatrics patients), and second, the Milestones are designed to be used solely for the evaluation of resident physicians and are not designed to be relevant in any other context.

The Milestones were developed by a group of content experts that comprised the members of the working group and the advisory group, who sought input from the radiation oncology community. The working group explicitly identified critical aspects of the practice of radiation oncology, and used a framework with graduated measures to reflect resident acquisition of competence for unsupervised practice. As part of the Milestone development process a small number of faculty and residents at various radiation oncology programs completed the Milestones and the vast majority of participants judged the Radiation Oncology Milestones to be understandable and useful.

The Radiation Oncology Milestones are designed to assist faculty to assess the professional development of residents in training. The Milestones use levels (1–5) that are synonymous with moving from novice to expert. The levels do not correspond with postgraduate year of education. The Milestones are intended to assist faculty in the identification of residents who may be appropriate for remediation and also to assist in the documentation of competency to enter unsupervised practice. Level 4 is designed as a graduation target, and does not represent a graduation requirement.

All radiation oncology programs are required to have a Clinical Competency Committee (CCC). The members of the CCC must be appointed by the program director and, at a minimum, the CCC must include 3 faculty members. The CCC is responsible for reviewing all resident evaluations semiannually and reporting the Radiation Oncology Milestones for each resident to the ACGME after these evaluations. It is expected that the size of CCCs, and the methods employed in resident evaluation, will differ across programs.

The Radiation Oncology Milestones will be used by programs to assess resident progress in the acquisition of competence for practice in the specialty, and will be used in the NAS as 1 measure of the effectiveness of residency programs. Future work is needed to test the feasibility, practicality, and validity of the Radiation Oncology Milestones, including studies to assess their correlation with established measures used to assess competence in the specialty, such as the ABR Radiation Oncology Board Examination.