Promising Practices in US Sponsoring Institutions to Advance Diversity, Equity, and Inclusion in Graduate Medical Education

The primary outcome of health care delivery is optimal patient health outcomes.^1,2  Physician diversity is a critical factor in improving care for patients of diverse demographic and social backgrounds.³  With the goal of improving diversity, equity, and inclusion (DEI) in graduate medical education (GME), in 2019, the Accreditation Council for Graduate Medical Education (ACGME) introduced a diversity accreditation standard, mandating all US training programs engage in “practices that focus on mission-driven, ongoing, systematic recruitment and retention of a diverse and inclusive workforce of residents, fellows (if present), faculty members, senior administrative staff members, and other relevant members of its academic community.”⁴ 

Despite initiatives to promote excellence in DEI within clinical learning environments, concerns have been raised about the degree to which medical education prepares learners to care for patients from diverse backgrounds. Additionally, academic health systems have variably incentivized and supported efforts to enhance DEI within the health care delivery context.⁵  The nested relationship between clinical learning environments and trainee practice, teaching, and learning directly influence patient health outcomes. Informed by both accreditation and raising transparency for evidence that suggests improved outcomes from DEI efforts, residency programs are working to improve DEI within their institutions.

The ACGME established the Barbara Ross-Lee, DO, Diversity, Equity, and Inclusion Award in 2020 to recognize efforts to achieve DEI in GME.⁶  Each year, ACGME-accredited residency programs, sponsoring institutions, and specialty organizations working to diversify the physician workforce and create inclusive workplaces are invited to apply. The application consists of 6 questions that ask sponsoring institutions to describe recruitment and retention activities, how they ensure an inclusive learning environment, activities to reduce health disparities in their local community, novel approaches to increasing DEI, how their programs can be adopted widely, and how they measure the impact of all these efforts. Using the narrative responses to these questions and accompanying letters of support from sponsoring institutions as the primary data source, we carried out an exploratory content analysis to examine system-level strategies for improving DEI.

The Barbara Ross-Lee, DO, Diversity, Equity, and Inclusion Award process required programs and institutions to submit a narrative-based application that included a description of efforts to increase diversity in the learning environment and an assessment of the impact, both quantitative and qualitative (testimonials from learners and faculty). Each application included letters of support from a current resident/fellow, a program director, a designated institutional official or chair of a GME Committee, and from another person involved with GME who could comment on the importance of the initiatives.

The ACGME requested the authors (led by D.B.) to analyze the submitted applications and explore the research question. To ensure the integrity of the research, prior to analysis we members of the research team voiced our own preconceived notions and perceptions related to DEI.⁷  Several investigators in this project were leaders in the field of DEI, which raises the possibility of bias during the analysis; we sought to address this concern prior to our work by creating a diverse team with some individuals having limited experience in DEI. We also acknowledged that when conducting this type of analysis within our constructivist framework, we must continually be aware of how our own experiences, perceptions, and biases may influence our findings. We were careful to avoid overinterpretation or make judgments of the quality of interventions demonstrated in the data during research team adjudication sessions. We additionally implemented cross checks during data analysis to verify our results.⁶ 

Since limited literature existed related to a cohesive framework of DEI strategies developed by sponsoring institutions, we used an exploratory inductive approach and the constant comparative method in the analysis.⁸  Four members of our research team had prior experience in qualitative research and led the analytical approach. In a prior study, we analyzed all US GME program applications, submitted between August 17, 2020 and January 11, 2022 (n=29), to identify specific strategies used by GME programs that led to enhanced DEI practices.⁶  These results provided the basis of a preliminary codebook that our research team used in this study,⁹  which involved the evaluation of the applications from sponsoring institutions submitted in 2020 (n=6) and 2021 (n=6). The applications varied in length from 8 to 39 pages and comprised a total of 234 pages. Submissions demonstrated geographical diversity, including the West (n=1), East (n=5), Southwest (n=2), and Midwest (n=4) regions of the United States. Two of the 12 sponsoring institution submissions were miscategorized (ie, were not about sponsoring institutions) and were removed from the dataset. Three investigators (S.S., M.W., J.D.G.) first collectively analyzed and coded 2 applications (total 18 pages) to modify and expand the codebook. Each investigator then independently analyzed 3 or 4 applications transcripts. Seven adjudication sessions were held with the 3 investigators to discuss the findings, compared for consistency and agreement, and disagreements were discussed until agreements were reached. Through several iterations, this process resulted in final themes. The data was managed using the data management support program Dedoose. Building upon the research team’s previous work, sample strategies were identified for each subtheme and categorized as either foundational (high impact and achieved with minimal effort) or aspirational (high impact but requiring extensive effort and investment).⁶  The study was exempt from further review by the Institutional Review Board at Yale University School of Medicine.

Our analysis identified 5 broad themes and 10 subthemes that characterized institutions and their approach to DEI. Each of these are described below.

Every sponsoring institution commented on their institution’s commitment to DEI. Sponsoring institutions identified an approach that addressed the vision, mission, and strategic planning within the education and health care delivery fabric. These efforts were described with intentionality and not described as “one-offs” or transient efforts, but rather a proactive and strategic approach that was part of the organizational culture. Three subthemes (Table 1) were identified in the transcripts that exemplified this demonstrated commitment to DEI: A culture that fosters DEI, allocation of capacity within the institution to support DEI, and initiatives and policies that reflect the DEI mission.

Nearly every institution referred to the importance of creating an inclusive “culture” that fosters DEI. While culture can be amorphous and difficult to pinpoint, it is evident throughout the transcripts that creating an inclusive culture is critical to DEI work. An institution’s culture is defined as its values and beliefs that inform the way individuals behave within the organization.¹⁰  These shared beliefs are communicated and reinforced through various methods (policies/procedures/hiring practices) and can ultimately shape the perceptions and behaviors of those within the institution and can be a driving force for change. Most transcripts explicitly named diversity and inclusion as part of a larger institutional mission statement or a key strategic goal, describing diversity and inclusion as an “integral part of [our institutional] identity” and “part of our institutional DNA.”

Institutions demonstrated a commitment to DEI by their investment in people. Every institution in the analysis has invested capacity and resources in programs that support the people according to their mission around diversity and inclusion. This investment, whether in designated programs, or in creating and elevating dedicated faculty and staff roles, specifically for the purpose of DEI, provides the support and structure to advance DEI work. While the job titles varied across institutions, from Diversity & Inclusion Officer, Director of Diversity, Vice Chair for Diversity and Inclusion, or Health System Diversity Committee with representatives from all clinical departments, all served analogous goals of improving diversity and fostering inclusion at their respective institutions.

Some system-wide initiatives and policies demonstrated explicit institutional commitment to advancing DEI efforts and showed how they were “walking the walk.” For example, many institutions cited pathway programs as critical to their success in recruitment of underrepresented in medicine (UIM)⁴  trainees and stated the importance of supporting and funding these programs to help meet the mission of the institution. In addition to pipeline programs, in a system-wide effort to improve UIM recruitment, one institution described a GME-wide policy recommendation to eliminate United States Medical Licensing Examination scores as a tool in the screening and ranking processes. In another example, several institutions required employees to complete unconscious bias training.

Most institutions were able to examine the ongoing work and outcomes of their DEI efforts through a data infrastructure that allowed the capture, collection, and transparent communication about the structure and process measures. Data infrastructure played a pivotal role in creating institutional memory. Two subthemes (Table 2) that elucidated this continuous quality improvement approach at the institution level include tracking systematic processes for recruitment and retention and monitoring inclusion efforts.

Most institutions developed databases, utilizing data available in the Electronic Residency Application Service and National Resident Matching Program, to track various diversity metrics of their trainees, such as number of UIM trainees that applied, were offered an interview, accepted an interview, and ultimately matched in the program. A few institutions tracked retention of UIM graduates with attention to those selected as chief residents, those offered and accepted into fellowship or faculty positions at their home training institution, and graduates remaining to practice in medically underserved communities. Most have tracked this data for at least 5 years and utilized this data as an outcomes measure to determine the impact of interventions and subsequently to develop actionable goals.

Some applicants utilized dashboards to track and share DEI efforts across the institution. For example, at one institution, a dashboard incorporated an annual assessment of individual residency and fellowship programs’ DEI curricula and strategies for recruitment and retention of UIM trainees. Another institution developed a Clinical Learning Environment Review (CLER) dashboard that measured institutional elements that reflect the 6 pathways of CLER incorporating health care disparities. This dashboard is shared annually with chairs, program directors, and institutional leaders. A subset of institutions utilized exit interviews to inform DEI needs. Many institutions tasked committees to analyze the data and recommend actionable goals using a continuous quality improvement lens. This ongoing use of data facilitated a regular check on DEI efforts and sharing of best practices and strategies.

Nearly every institution explicitly linked their DEI efforts with the importance of integration in the community and the goal of mitigating health disparities. Multiple institutions recognized the historical context in which they were operating and how it may impact their relationship with the community, particularly given the recent attention to systemic racism and health inequities highlighted by the COVID-19 pandemic. Three subthemes (Table 3) demonstrated these intricate linkages: diversifying the workforce, building partnerships within the community, and community engagement and service-learning opportunities.

Many institutions highlighted the importance of training and diversifying their workforce to meet the needs of the community. As an example, one institution partnered with 3 of the nation’s largest federally recognized tribes within their community to improve recruitment and training efforts.

Most institutions identified key intermediaries in the relationship between the medical community and the larger community. In some institutions, this relationship was managed by a particular department, and, in others, by a diversity committee that serves as a clearinghouse for requests for community initiatives and disseminating community engagement opportunities within the health system.

At one institution, in collaboration with the county department of health and human services, an extensive community health needs assessment was performed to identify medically underserved areas and identify at-risk populations due to health disparities. The key findings from this assessment were used to develop a community health implementation plan, which included building a new medical facility in the community identified to be at the greatest risk for early disease and death. The findings and the community health implementation plan are now shared widely with GME program leadership to enable GME program leaders to have a better understanding of the community that their residents serve.

Many institutions described concerted efforts to increase resident and fellow engagement in the community by involvement in service-learning opportunities. Most of the community engagement opportunities were created in collaboration with community-based organizations. Many institutions referred to partnerships with community-based organizations that informed specific institutional initiatives to mitigate identified health care disparities. Examples of initiatives born out of identified community needs at one institution included: deployment of mobile COVID-19 testing, enhanced high risk diabetic retinopathy screening, and hosting “community conversations” on health care equity for communities of color.

Multiple perspectives and a diversity of voices involved in promoting and advancing DEI work is essential. All institutions described the importance of diversity of those involved in DEI work across the institution, and many illustrated the concept of coproduction, with residents playing a pivotal role. Two representative subthemes (Table 4) identified in the transcript include multiple people across all levels involved in DEI, and more specifically, coproduction with residents being critical to advancing DEI.

Many institutions cited the breadth of those involved in DEI efforts as not only beneficial, but necessary to advance diversity and inclusion efforts. The transcripts identified that all individuals in the academic health center and community are potential collaborators. Most importantly, individuals involved in DEI existed across all levels of the sponsoring institution including students, residents, staff, faculty, administrators, and community members who galvanized the work. The diversity of faculty, staff, and community engaged in DEI work serves as a catalyst for change.

A traditional hierarchical approach to medical education was supplanted by coproduction, trainees and faculty working together on an equal playing field to inform and implement DEI efforts. In numerous institutions, diversity committees were often comprised of multiple stakeholders and in almost every instance were reliant on resident participation. Trainees (students, residents, and fellows) often were in positions of leadership in diversity committees to advise and provide critical direction for DEI. In several institutions, resident ambassadors or “champions” were often identified to spearhead the DEI work. Trainees often interfaced with the GME office and institutional leadership to voice needs and provide input on recommended changes and helped with planning and organizing DEI events such a diversity inclusion symposiums, second look weekends for diverse applicants, and informing DEI curricular needs. This resident engagement was the fuel that progressed the strategies over the long term.

No institution described a single approach to advance DEI work but rather multiple, systematic strategies across the education continuum. Our analysis identified that a multifaceted approach to advancing DEI was vital. The themes identified did not exist in silos, and efforts to advance DEI work were not all concentrated in recruitment but with attention to inclusion, promotion, and community. This theme represents a key point of emphasis that is foundational to a comprehensive and meaningful approach to DEI. One institution commented:

Every institution made a substantial effort to grow the pool of diverse applicants, improve the residency application and interview process, and strengthen residency and fellowship training inclusion efforts, faculty development and retention, and community engagement. Even when programmatic strategies existed, sponsoring institutions needed to be on the same page to create the foundation and environment to elevate these strategies and foster their success.

Our analysis revealed 5 themes and 10 subthemes that work to promote and enhance DEI within US sponsoring institutions. Approaches to advance DEI work did not operate in silos but rather in multiple, methodical schemes at all phases of a learner’s pathway through GME. Our work builds on the understanding that redressing structural inequities that have led to the marginalization of UIM students and vulnerable communities requires a concerted effort and multipronged approach. No one strategy is effective alone and requires medical institutions to engage in radical introspection and recognize the ways that they fall short in achieving justice for their learners and ultimately, the population they are intending to serve. We created a summary infographic of the themes we discovered as a potential guide for US sponsoring institutions aspiring to improve DEI in GME^6,11  (Figure).

A theme identified is an organizational culture centered on DEI. A transformational culture is one that creates DEI-mission-driven relationships among members of the institution, which then leads to enhanced results in the form of improved infrastructure to improve DEI within the organizations.¹²  It is crucial to acknowledge that, while mission statements provide a foundational commitment to DEI, fostering an inclusive culture necessitates sustained efforts that go beyond mere statements and encompass comprehensive actions, policies, and practices throughout the institution. Using data infrastructure and continuous quality improvement, organizations should not only track DEI-improvement progress, but also be held accountable for their lack of advancement in DEI work. Critical data may include the amount of funds allocated toward DEI work, funds available to DEI leadership to advance the field, and data reporting how the current students and administration reflect the racial/ethnic and socioeconomic communities they serve. Another key promising practice is that of coproduction, a concept that has been less described in the literature but seems to be important across sponsoring institutions working to advance DEI. The coproduction requires institutions to acknowledge the importance of working with community and the cultural wealth that community members possess, which can augment DEI initiatives when community members are involved in the process.¹³  Trainee involvement is critical to advancing diversity efforts; however, it is equally important to acknowledge the concept of minority tax and implement effective mitigation strategies.

The 2019 ACGME diversity accreditation standard is progress toward building a diverse and inclusive workforce, and successful strategies shared across programs and institutions to meet this standard can accelerate progress toward meeting, maintaining, and exceeding this standard. While there are existing requirements at the institution level that aim to enhance and support DEI, they do not provide explicit guidance on how to effectively achieve these goals.^14,15  Exploring how institutions can have some accountability in this work may be critical for success at the program level. A business analysis by McKinsey & Company that includes 15 countries and over 1000 large companies highlights the profitability of the most diverse companies, often outperforming those that are less diverse.¹⁶  Similarly, further research in medicine can help to elucidate further outcome measures of DEI work and continue to build the case for why improved DEI in US sponsoring institutions is imperative.

This analysis illuminates what several institutions with progressive DEI efforts are doing to advance this work. Describing these themes provides a better understanding and a guide that other sponsoring institutions across the country can implement when crafting their own approaches to advancing DEI. In addition, the themes identified in this review will likely enhance the chances of success for strategies implemented at the individual GME program levels as well.

This study has several limitations. First, the applications were from sponsoring institutions that self-identified as exemplars in DEI, raising the possibility of selection bias. Additionally, the applications inconsistently described the level of involvement across all programs within the sponsoring institutions, making it challenging to determine whether the DEI efforts highlighted were driven by a few programs or encompassed all programs within the institution. However, given there is limited information in the literature regarding strategies used across multiple GME programs and sponsoring institutions, we believe these results have significant credibility and can apply to other academic institutions. The absence of detailed information on funding sources in the narratives is a limitation as it restricts the potential adoptability and replicability of these efforts. Another limitation is the lack of clear descriptions of the obstacles and challenges overcome in developing and sustaining the DEI efforts showcased in the award submissions. This highlights the critical need for institutions to disseminate their work, providing a valuable blueprint for other institutions to adopt and operationalize similar DEI initiatives.

This qualitative study identified 5 themes that can inform and guide sponsoring institutions in promoting DEI: organizational commitment, data infrastructure, community and diverse team engagement, and a systematic approach to DEI throughout the medical education continuum.

The authors would like to acknowledge all US graduate medical education and sponsoring institutions who submitted applications for the Barbara Ross-Lee, DO Diversity, Equity, and Inclusion Award.

Funding: This work is supported by the Health Resources and Services Administration (HRSA) of the U.S. Department of Health and Human Services (HHS) as part of an award totaling $3,791,026 with 0% financed with nongovernmental sources. The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by HRSA, HHS or the U.S. government. For more information, please visit HRSA.gov.

Conflict of interest: The authors declare they have no competing interests.

Disclaimer: The content of and views expressed in this paper reflect the views of the authors and do not necessarily represent the views of the Accreditation Council for Graduate Medical Education, American Medical Association, Josiah Macy Jr. Foundation, HRSA, or other participants in this initiative.