When the Accreditation Council for Graduate Medical Education (ACGME) launched the Milestone Project in 2013, it anticipated “an iterative process, and that a periodic review and revision of the Milestones would be beneficial, as experience and research evidence accrued.” It promised that a taskforce would reexamine the project within 3 to 5 years.1 In 2016, the ACGME began this review process and included public members working alongside physician members.
Public members provide a unique lay perspective on strategic and accreditation issues, bring the voice of both the community-at-large and patients, engage with these populations, and enhance the effectiveness of review committees, boards of directors, and other committees and taskforces.2 Public members of the ACGME committees are selected based on qualifications including professional and educational experience; prior committee, board, and group experience; and personal or patient experience with the health care system.3 They are not physicians and typically cannot have a direct affiliation with a graduate medical education (GME) program or work for the ACGME's member organizations or a review or recognition committee organizations (latter 2 instances apply to all review/recognition committee members). Typically, health care professionals are permitted provided they are not employed by an organization with ACGME-accredited residency or fellowship programs, to avoid conflicts of interest.
For the Milestone Project, the ACGME defined public members more broadly as any non-physician, including GME staff. The requirements regarding public members were relaxed to exclude participation only with a specialty within which the public member currently works. This was an important difference, as typical public members on review and recognition committees have a 6-year term, allowing time to learn terminology and process, whereas the Milestones 2.0 is an approximately 1-year term, requiring that public members be able to start working upon introduction to the group. For Milestones 2.0, the ACGME solicited public members to join work groups from review committees, a call for volunteers, and recommendations from certification boards. As of February 2021, the ACGME has convened 103 Milestones 2.0 work groups, covering 106 specialties and including 57 public members.
This article describes the experience of 6 public members who participated in the formation and implementation of Milestones 2.0. We are non-physician colleagues in the GME arena with diverse backgrounds from academic medical centers and hospitals. We have experience in GME, health care and program administration, curriculum design and implementation, and medical education assessment and evaluation. Consistent with the definition of public members for Milestones 2.0, as defined above, we use the term “public members” in this article to refer to our role in the process, with the understanding that it was a specific experience. We assert, however, that these learnings are broadly applicable. To produce this article, all authors completed an independent, critical reflection to examine our experiences. Based on these reflective narratives, we summarized our experiences, highlighting successes and improvements for future initiatives to include public members.
What the Milestones 2.0 Work Groups Did
Each specialty's Milestones 2.0 work group was charged with revising the existing Milestones required of residents and fellows to achieve competency, and ultimately independent practice. In addition, work groups developed supplemental guides with examples to assist programs in the interpretation and assessment of Milestones. Each work group met at ACGME headquarters in Chicago for collaborative sessions lasting 1 to 2 days. Between sessions, members used shared documents to review, comment, and edit the work product, and to finalize the Milestones and the supplemental guide.
In all cases, our work groups had thoughtful, nuanced conversations. Together, we discussed specialty-specific board requirements and credentialing, evolving expectations in GME, scholarly activity, and the need for public accountability. We carefully crafted Milestone narratives, standardized language, considered interpretation implications, and addressed what constitutes varying levels of progressive competency. All the while, we accounted for the needs of and implications to programs of varying sizes and resources.
What Value Do Public Members Bring?
We all found ourselves fully involved in the work groups and valued as contributors by our physician colleagues. Developing Milestones to evaluate physician competency progression is by nature complex. Sometimes the forest gets lost for the trees. By bringing an overarching non-clinical perspective, we often expanded deliberations from a solely clinical discussion to one more balanced with the overlying objectives. To this end, public members' perspectives complemented those of our physician colleagues.
Across specialties, all participants benefited from the ability to ask questions. We provided balance by asking questions not steeped in specialty-specific details. For example, because public members are from outside the specialty, we would occasionally ask clarifying questions to better understand a quandary facing the group. Providing an answer for the public member would often result in clarity that would ease or eliminate the group's stumbling block.
Similarly, dedicated physicians in the specialty would often find themselves in spirited debate over a nuanced element of a Milestone. While not unique to the non-physician, a question from a public member could go far in breaking through the subtleties of the issue and bring the work group to a decision.
Without a doubt, public members' participation in Milestones 2.0 expanded the work groups' perspectives from what could have been a physician-centric lens. This expansion did not dilute the technical nature of the Milestones, but rather enriched discussions. Since public members did not enter the Milestones 2.0 work groups with specific knowledge of the specialty under consideration, we were able to offer suggestions without being affected by prior experiences or assumptions.
Finally, public member involvement in Milestones 2.0 should provide the public-at-large with confidence that physicians trained in ACGME-accredited programs are well evaluated throughout their progression toward competent, independent practice. As public members, we added to the growing body of knowledge that supports the ACGME's decision to include the public in its many efforts.
What Could Be Improved?
Reflecting on our experiences, we agree that it could be challenging for an individual public member with no background in Milestones or learner assessment to participate effectively. Given our varied roles within GME, most of us had a general understanding of the Milestones before joining the work groups or had served as public members on other ACGME initiatives, allowing us to contribute more effectively from the outset. We encourage the ACGME to continue including public members, but to make selection criteria and process more clearly defined and transparent to ensure future public members are appropriate for and properly utilized in assignments.
Another area for improvement is the preparation process for public members. Although the organizing team shared materials prior to the first meeting, it was unclear what to expect from each member of the work group or to what extent physician members were aware of our involvement. The ACGME team did extraordinary work orienting members during the first meeting; however, some of us initially felt apprehensive and unsettled about whether our contributions would be universally accepted. To better prepare public members for meeting dynamics, the ACGME may want to consider creating resources to set work group expectations.
Conclusions
The ACGME is well served by participation from its public members. The positives described far outweigh the opportunities for improvement. Our unique professional training contributed to diversity of thought and expertise specific to GME; this value-add had not been realized with past Milestones efforts. As public members, we substantially contributed to the refinement of the Milestones and benefitted from the opportunity for professional growth and development. Including a public member on the Milestones work groups benefitted the members themselves, the ACGME, and ultimately the public. Including public members with knowledge of and/or experience in GME may further strengthen the outcomes.
In addition to membership on the board of directors and review committees, this is ACGME's third effort at including public members in its deliberations. We believe involvement of public members throughout the organization is an excellent opportunity to give the public-at-large an active role in the accreditation process. Next, we recommend the ACGME analyze the impact of public members and determine where else public members may provide positive contributions.
