Establishing Competencies for Leadership Development for Postgraduate Internal Medicine Residents

Physicians lead complex, diverse teams in the domains of research, education, and patient care.^1,2  Yet physician-leaders have been described as “accidental administrators,” often lacking training in skills necessary to be an effective leader.^3–5  Given current challenges facing health care, the need for physicians to develop effective leadership skills is paramount.^1,2,6,7 

The call to action to increase formal and comprehensive leadership training for physicians and trainees is not new,^1,2,8–10  and it has been the focus of several recent systematic reviews.^11–13  Graduate medical education (GME) accrediting bodies have recognized the importance of leadership training for residents and have encouraged curricular development through published educational policy and guidance. The US Institute of Medicine report, Crossing the Quality Chasm: A New Health System for the 21st Century, emphasized the need to develop physician leaders across all levels.¹⁴  Subsequently, in 2013, the Association of American Medical Colleges added interprofessional collaboration and personal and professional development to its list of key competencies.¹⁵  Similarly, the Accreditation Council for Graduate Medical Education (ACGME) has listed leadership domains, including interpersonal communication, quality improvement, and systems-based practice, as milestones of residency education.¹⁶  Though trainees desire more leadership development during residency,^17,18  few GME programs provide opportunities for leadership development, and published curricula are heterogenous in terms of content and delivery.^12,13 

Existing GME leadership development programs most commonly employ classroom-based didactics coupled with independent reading or discussion and reflection in the small group setting—neither of which provide opportunity for skills practice.^11–13  Described curricula place a disproportionate level of focus on technical/conceptual knowledge and individual skill development, as opposed to other domains of effective leadership such as self-awareness, emotional intelligence, and organizational understanding.^11–13  As stated by Sultan et al, the “competency-centric medical education culture” represents a likely driver of this skewed focus toward cognitive, skills-based leadership competencies.¹³  However, there have been increasing calls for a more comprehensive approach to leadership development—one that incorporates a deliberate focus on developing the (often neglected) leadership domains of emotional intelligence and character.^19–21  The “character-competence entanglement” paradigm from Sturm et al describes the existence of a mutually reinforcing effect between character and competence that leads to extraordinary leadership performance.²² 

It remains unclear which domains and competencies are most important for early career physicians.^11,23,24  Furthermore, published curricular interventions for trainees employ a variety of pedagogical methodologies.^11–13  It is unknown if specific leadership skills or domains are best suited to a particular approach. We conducted a Delphi study with the primary aim of creating consensus around a prioritized list of leadership skills and competencies to guide curricular development for internal medicine (IM) residents.

The Delphi approach generates consensus from a panel of experts, is used to close gaps when empiric data is lacking,^25–27  and works best with high-level concepts.²⁸ 

We utilized a purposive sampling strategy to assemble a geographically diverse group of panelists with expertise in leadership and GME who represented clinical practice, research, education, administration, and quality improvement domains of leadership in keeping with the established Delphi practice.²⁹  We were particularly interested in physician leaders within an academic environment who could speak to pedagogical considerations for content delivery. As such, in 2019, we recruited IM physicians in leadership roles with training and expertise in medical education and/or leadership programming within national professional societies (eg, Alliance for Academic Internal Medicine, Society of General Internal Medicine). We sought individuals who met at least one of the following inclusion criteria: (1) has presented on leadership topic at the national level; (2) has served in a GME leadership capacity with a designated leadership title within their institution; and/or (3) has served in a leadership capacity with a designated leadership title on a national GME committee within the aforementioned professional societies.

We emailed 44 potential panelists an invitation to participate in the study, which was conducted via a custom online Delphi survey system. There is no consensus on what constitutes an appropriate sample size of a Delphi study; studies with 6 panelists have been published, and a sample size of 12 or more is deemed reasonable.^26,30  Given that a 70% response rate between rounds is needed to maintain study rigor,²⁵  we sought to enroll 15 panelists in round 1 with at least 12 completing round 2. No incentive was offered.

Two authors (S.M., J.C.) with advanced training in medical education, national experience in teaching leadership development, and GME program leadership (J.C.) developed this survey (provided as online supplementary data). We identified a preliminary list of leadership skills to be included in this study through review of published literature regarding competencies for health care professions.^{11,24,31–35}  We employed a framework that uses 3 established leadership development domains (character, emotional intelligence, and cognitive skills) to classify leadership skills, as described in a recent systematic review of leadership curricula in postgraduate medical education.¹³  This holistic perspective defines leaders (eg, those who inspire, engage, influence, empower) as demonstrating expertise across a variety of intellectual and affective competencies. This definition is also echoed in the broader business literature.²¹  Skills included were vetted by local experts with experience in leadership development and education to establish content validity. Five local, general IM, core residency faculty with expertise in leadership piloted the survey to establish response process validity. We employed 2 rounds of survey administration for all panelists.

Our initial contact included a standardized definition of physician leadership adapted from the American Medical Association: “consistent use by a physician of the leadership knowledge, skills, and expertise necessary to identify, engage, and elicit from each team member the unique set of contributions needed to help patients achieve their care goals.”³⁶  During round 1, panelists ranked 28 leadership skills on importance (1 = not important, 2 = of little importance, 3 = somewhat important, 4 = very important, 5 = essential) for categorical IM residents to achieve competence by the second half of residency and apply during the first 3 to 5 years post-residency. For each skill, we also asked panelists to identify the number of content hours required for training and the ideal pedagogical method. We categorized pedagogical approaches as outlined in a recent systematic review¹³  of leadership development programs in GME. These included experiential, mentorship/coaching, work-based, theoretical learning, interactive discussion, reflection, and project-based. Round 1 also included a demographic questionnaire and an open-ended question to identify additional important topics.

We calculated median importance scores with IQRs from pooled, de-identified data. Consensus was defined a priori. Any skill ranked by > 70% of panelists as a Likert 4 or 5 (very important or essential) met automatic inclusion criteria. Any skill ranked by > 70% of panelists as a Likert 1 or 2 (not important or of little importance) met automatic exclusion, in accordance with typical definitions of consensus.²⁶  Any skill not meeting these criteria was sent to round 2 for re-ranking.

During round 2, panelists reviewed their answers from round 1 along with pooled median importance scores with IQRs as well as descriptive pooled data regarding number of content hours and pedagogical methods. We then allowed panelists to adjust their responses after reviewing group distribution; however, we made it clear that they were not required to do so if their thoughts had not changed. Round 2 also included any new skills identified from free text responses in round 1. During round 2, respondents were also asked to identify up to 5 topics they felt to be critical to teach IM residents. The final list of consensus topics included all of those ranked as a Likert 4 or 5 by > 70% of panelists. We considered skills with a pooled median importance rating of > 4 as “essential” and those with a pooled median importance rating = 4 as “very important.”

As this was a Delphi survey, results are largely descriptive in nature. Medians and IQRs were computed for Likert-type responses, and frequencies and percentages were calculated for nominal variables. To compare methods for delivering leadership curricula, we computed and tested all pairwise differences in proportions between each of the 8 methodology options using Wald tests and assuming a multinomial distribution. Bonferroni-adjusted P values were computed, as we performed 28 pairwise comparisons. A familywise type I error rate of 0.05 was assumed.

This study was deemed to be exempt by our local institutional review board.

Sixteen panelists completed round 1 (of 44, 36% response rate); 14 (of 16, 88% response rate) completed round 2. Table 1 shows demographics of the 14 panelists who completed demographic items. Panelists held a variety of high-level university, hospital, and national professional organization leadership roles (Table 1).

Of 28 leadership skills presented in round 1, 15 met automatic inclusion criteria, 0 met automatic exclusion criteria, and 13 did not generate consensus. Panelists generated 5 additional skills via free text response. Thus, round 2 was comprised of 18 skills, 13 without consensus from round 1 plus 5 additional skills (Figure). Of 18 skills included in round 2, 6 met automatic inclusion criteria, 0 met automatic exclusion criteria, and 12 did not generate consensus. Thus, our final consensus list included 12 “essential” and 9 “very important,” across all 3 leadership domains (6 character, 7 emotional intelligence, 8 cognitive skills; Table 2). Five of these skills were identified by > 30% of panelists as critical for residents to be taught (Table 3). Skills not meeting consensus for inclusion fell across the following domains: 1 character, 1 emotional intelligence, and 10 cognitive skills (Table 4).

All panelists agreed that GME programs should include dedicated curricular time for leadership development for IM residents. Highest-ranked methods for delivering leadership curricula were “mentorship/coaching,” “work-based reflection,” and “interactive discussion,” which were chosen 27.4%, 27.7%, and 22.4% of the time, respectively. Pairwise comparisons in percentages between these top 3 formats and all other formats were significant with adjusted P values < .001. “Lecture” was the lowest-ranked format for content delivery. While experts generally recommended between 2 and 5 content hours per skill (range 0 to > 5 hours; median 2–3 hours; IQR 205 hours), they had varying recommendations regarding total number of content hours dedicated to leadership development (4–8 hours: 33%; 9–12 hours: 20%; > 12 hours: 40%).

This study describes results of a rigorous Delphi survey of national experts to generate consensus on essential leadership competencies for IM residents. In addition to the prioritized list it generated, our study lends insight into optimal curricular methods to provide this education. These findings, including those that address feasibility and implementation, can help to inform best practices for future GME leadership curriculum development.

Interestingly, nearly two-thirds of skills identified in the final consensus list (13 of 21) and 80% (4 of 5) of skills identified as “critical” fell within character and emotional intelligence domains. Nearly all skills (10 of 12) excluded from the final list were categorized within the cognitive skills domain. Though the majority of published leadership development curricula focus on cognitive skills-based leadership competencies,¹³  our findings add to a growing body of evidence suggesting that leadership development programs should take a more comprehensive approach and include a strong focus on building character and developing emotional intelligence.^19,20,22,37  Further, this work indicates that these more affective domains are perceived by experienced leaders to be equally as important for early career leaders and, in fact, might be more foundational than cognitive skills focused on by more executive-level leadership development programs.³⁸  It is notable that, although identified as the most critical skill, collaboration was not identified as “essential” for IM residents by our participants. The inclusion of collaboration as one of the Association of American Medical Colleges' core competencies and its direct relation to the ACGME systems-based practice competency substantiates its critical designation.^15,16 

To date, few specialty-specific evaluations of specific leadership competencies for teaching leadership to residents have been undertaken. While it is possible that competencies may vary across specialty, a review of existing publications support the generalizability of our findings. More specifically, critical skills identified in this study, including collaboration, communication, and feedback, are identified as key competencies within surgical, emergency, and critical care settings.^35,39,40  Similarly, character, emotional, and relational domains of leadership are emphasized both in the aforementioned settings^39–41  and within pharmacy³²  and nursing⁴²  domains. Based on reviews of existing literature,^39,40,43  one area in which leadership competencies for surgical specialties may diverge from our population of IM residents may be the need to incorporate additional cognitive-based skills such as group process, followership, emergent decision-making, and stress management, although these skills are also very important in procedure-based IM subspecialties.

Most recommended pedagogical methods involved small group teaching or direct observation and debrief through coaching and reflection. Didactic formats were rarely recommended for content delivery, suggesting that formal curricular time and space, a commonly cited barrier in GME, may not impede the development of programming in this content area. Our results, which address feasibility, allow GME programs to tailor work-based leadership programming to meet individual needs and curricular space. These findings are congruent with prior work examining best practices for leadership training and indicate curricula that employ multiple work-based pedagogical approaches and experiential learning are most effective and favored by learners.^11,12,32,44 

Unlike many of the leadership skills within the cognitive skills domain, which lend themselves well to competency-based medical education, no clear standards exist to evaluate the competency of medical trainees regarding emotional intelligence and character. GME needs validated character and emotional intelligence assessment tools applicable to our population of learners and may look to the larger business literature to adapt tools to our learners.^45,46 

Limitations of our Delphi study include that our results are reflective of the opinions of our expert panel, as is inherent in Delphi methodology. Our panel was made up of senior leaders in academic IM—their opinions may not be generalized to other populations (eg, levels of trainees, non-academic settings, specialties outside of IM) or speak to the perspectives of other members of the health care team. Further, we did not specifically collect data on the types of teams that our panelists had direct experience in leading, or if panelists had pursued formal leadership training or degrees. Panelists who participated have all accrued significant academic credentials and/or formal administrative titles. However, it cannot be assumed that panelists' experience directly translates to either expertise in the management of multidisciplinary teams, specifically, or to expertise in the most appropriate pedagogical method to teach each leadership skill.

Next steps in the continuation of this work might be to vet this consensus list of essential leadership skills with other key stakeholders (eg, recent graduates across a variety of academic and non-academic settings, patients, other members of the health care team) to corroborate or refute content.^12,13  Finally, the total number of curricular hours required to teach all leadership skills that were deemed essential may exceed the total number of curricular hours most programs have available to dedicate to leadership development, unless leadership development curricula are embedded within the program's existing structure and framework. A key component for how GME programs should structure their leadership curricula should be based on a local needs assessment and available curricular space.

In summary, prior work has demonstrated the importance of creating targeted leadership skills development programs for IM residents, and our results describe a list of skills deemed most important by national experts. These results suggest that GME leadership curricular interventions should emphasize emotional intelligence and character domains of leadership, in addition to cognitive domains, and should prioritize coaching, discussion, and reflection as methods of delivery.

The authors would like to thank Ethan Lennox, MA, Division of General Internal Medicine, University of Pittsburgh School of Medicine, and Linda Quinn, Systems Analyst, Center for Research on Health Care Data Center, Department of Medicine, University of Pittsburgh School of Medicine.