Formulating the Allergy and Immunology Milestones Open Access

The Accreditation Council for Graduate Medical Education (ACGME) rolled out the initial phase of the Next Accreditation System (NAS) in July 2013. A key component of the NAS includes the establishment of educational Milestones for interns, residents, and ultimately fellows. The initial phase of Milestone development focuses on residency programs leading to initial board certification. As a conjoint specialty populated by graduates of both pediatrics and internal medicine residencies, allergy and immunology was 1 of 2 specialties at the fellowship level to complete Milestone development, forging the way for other subspecialties yet to come. The Allergy and Immunology Milestones provide a streamlined overview of fellow competence in core aspects of care in the specialty, as well as scholarship and lifelong learning.

In December 2011, the Allergy and Immunology Residency Review Committee (RRC) and the American Board of Allergy and Immunology (ABAI) established a joint Milestone Working Group. Experts from key stakeholders were assembled to produce a high-quality, meaningful product. The dedicated experts on the Milestone Working Group and the Advisory Group included representatives from the Allergy and Immunology RRC, the ABAI, the Program Directors Assembly (PDA) for Allergy and Immunology, key leaders from specialty societies, and an ACGME facilitator (2a1box). The group included program directors and a trainee member of the RRC to provide invaluable guidance on relevance of the Milestones and practical advice on implementation.

The working group used a series of assumptions to develop the Milestones, including emphasis on a final product that would be competency-based, concise, easy to use, and versatile. Versatility and a system in which Milestones would not be time-based were considered essential to accommodate the variety of curricular structures, multiple tracks, and widely variable scheduling of clinical rotations across allergy and immunology training programs. The working group also included the assumptions that entering fellows would have prior training and variable competency in interpersonal and communication skills, professionalism, practice-based learning and improvement, and systems-based practice.

The formative process adopted was an iterative one predesigned to incorporate lessons learned and feedback from stakeholders at multiple stages of development. An initial draft was developed at the December 2011 initial meeting in time for a January 2012 PDA meeting, where an orientation to the Milestone concept, working group, and initial draft were presented. Related presentations were provided to the entire Allergy and Immunology RRC and ABAI memberships during the groups' respective spring 2012 meetings. The working group continued to incorporate feedback from the boards, the PDA, and other ACGME Milestone working groups in the spring and summer of 2012. The Milestones also matured with successive input from specialty societies and ACGME annual meeting updates in 2012 and 2013, comments from program directors and faculty members, and feedback from the ACGME expert panel charged with developing common Milestones for the 4 cross-cutting competencies. The key constituencies in the specialty were engaged and fully knowledgeable of the Milestone project, and committed to assisting in the creation of a manageable and meaningful product. In particular, feedback fostered modifications that increased clarity and ensured the Milestones captured essential elements of the scope of practice of the specialty. Furthermore, feedback confirmed the prevailing development assumption adopted by the working group, particularly the need to keep the Milestone set simple and with minimal ambiguity.

The effort resulted in the development of 10 subcompetencies and Milestone sets for Allergy and Immunology, distributed among the 6 ACGME competencies. Each subcompetency has a Milestone set for the levels of competence from entry into the specialty to preparedness for unsupervised practice. Two subcompetencies pertain to medical knowledge (medical knowledge and research/scholarship), whereas 4 address patient care and represent the multiple phases of specialty patient care: medical interviewing, diagnostic measures, management plans, and coordination of care. This approach to patient care competency assessment was favored over discrete conditions or entrustable professional activities that would yield an unworkable number of competencies to capture the full breadth of the specialty.

The final 4 subcompetencies/Milestone sets cover the competencies of systems-based practice, practice-based learning and improvement, professionalism, and interpersonal and communication skills. A series of multiple discrete Milestone frameworks provided by the ACGME expert panel were combined into a single Milestone set for each competency. The Allergy and Immunology Milestone Working Group and constituencies were in full agreement that a single combined Milestone for each of these competencies would suffice, with the knowledge that entering fellows will have been evaluated on these competencies in their prior training and will have a level of competence in these areas at entry into allergy and immunology training above what is expected for entry into an initial residency program. The focus of training and assessments in these competencies will be allergy and immunology subspecialty aspects and application of these competencies.

The working group and the RRC designed the Allergy and Immunology Milestones for the required semiannual program reporting to provide aggregate data for use in conjunction with other data to enable an annual assessment of the effectiveness of training programs. However, the Milestones were also designed for potential use by fellows, programs, program Clinical Competency Committees (CCCs), ABAI, and credentialing committees (figure 1). Accordingly, the Milestones can be used as clinical rotation evaluation instruments, self-assessments, and peer feedback. When coupled with fellow-specific performance assessments and goal-oriented feedback and recommendations, the Milestones will serve as an effective means of providing timely meaningful feedback throughout training. Finally, the closing Milestone assessment of graduating fellows can serve as a source of primary data and appropriate terminology for completion of end of training documentation and recommendations for independent clinical practice.

Formation of a CCC is a common program requirement effective July 1, 2014, for all allergy and immunology training programs. CCCs for allergy and immunology programs must have at least 3 members, which may include the program director or other physicians and nonphysicians who teach and evaluate the trainees.

One of the main responsibilities of the CCC will be to assist with Milestone competency determinations for individual residents. The method used to determine final Milestone assessments that are reported to the RRC will likely vary to some degree from program to program. Accordingly, the Allergy and Immunology Milestone Working Group and the RRC decided to defer to individual program leaders the determination of the best method for Milestone completion and CCC composition. Submission of Milestone assessments of allergy and immunology trainees to the RRC will be the ultimate responsibility of the program director. Options for the program director include independently conducting Milestone assessments with input from multiple data sources, submitting Milestones on behalf of the CCC, or jointly completing assessments with the CCC. Regardless, all program components contributing to Milestone assessments should make use of all available data to include rotation evaluations, self-assessments, peer and 360-degree assessments, objective structured clinical examinations, in-training examination scores, academic presentations, and research products. Each Milestone assessment should be a holistic measurement of competency based on multiple inputs that best match the achievement descriptions of a given Milestone level.

Literature searches revealed a paucity of data characterizing allergy and immunology–specific educational interventions and outcomes.1–6 This absence of a validated approach or instrument added to the challenges encountered by the Allergy and Immunology Milestone Committee. The validity of the approved Milestone framework for allergy and immunology currently rests solely on the results of pilot testing and the substantial subject matter expertise of the developers. Feedback received from 10 programs piloting a draft of the Milestones in the winter of 2012–2013 was instrumental in finalizing the Milestones (figure 2). One example of incorporated feedback received from multiple sources was the separation of medical knowledge and research/scholarly activity into separate Milestones.

Although allergy and immunology fellowship educational outcome research is not a primary objective of the Milestone project, there is a need to conduct research in this area to foster the incorporation of proven educational interventions into training program curricula, program requirements, and future Milestone enhancements. Our nearly 300 trainees and the patients who will be entrusted to their care deserve only the best that their program, the accreditation body, and the certifying body have to offer.