ABSTRACT
Nonclinical skills (eg, self-regulation, team leadership, conflict resolution) are essential for success as a chief resident (CR). The literature on programs teaching these skills reports few if any effectiveness outcomes.
We reported the outcomes of a leadership course for CRs using participants' self-reported outcomes and assessments from their program directors (PDs).
A 2-day curriculum focused on emotional intelligence competencies, including self-awareness, self-management, social awareness, and relationship management. We used a logic model to align 2017–2018 curriculum with targeted outcomes. Questionnaires before and after the course assessed short-term and intermediate outcomes for the participants and PD interviews evaluated observed changes in CRs' performance attributable to the course.
A total of 74 residents participated in the course, and 65% and 59% responded to the post-course and follow-up questionnaires, respectively. Over 95% of respondents indicated developing leadership knowledge and skills and connecting with new CRs in the post-course questionnaire. During follow-up, CRs reported applying concepts learned during chief residency, using tools to address conflict, engaging in quality and patient safety projects, and continuing to interact with other participants. The relationships between reported outcomes and participants' gender/prior leadership training were not significant (P > .05), with small to medium effect sizes (0.01–0.32). All 14 PDs offered positive appraisal of the CRs, but we could not specifically attribute this growth to the course.
Participation in this CR leadership development course was associated with enhancement and application of leadership competencies in immediate and intermediate time frames.
Being a successful chief resident (CR) requires several nonclinical skills, but the literature does not report the effectiveness of programs that teach these skills to residents.
A leadership course for CRs using participants' self-reported outcomes and assessments from their program directors.
The study examined self-reported outcomes, and it was not possible to attribute skill growth among CRs directly to the course.
Participation in the course was associated with enhancement and application of leadership competencies, in immediate and intermediate time frames.
Introduction
Being a chief resident is a role that requires management and leadership skills (eg, communicating, aligning groups, advocating, and implementing change). Recommended curricula of health systems science1 and of CanMEDS2 explicitly include leadership as a key competency for trainees. Yet, chief residents (CRs) frequently report low familiarity with these skills.3–5 There is consensus regarding a need for leadership development for physicians and that such training should begin early in training.6–8 For example, Blumenthal et al and others have highlighted residency as an opportunity for developing leadership3,4,9–11 ; they recommend nurturing non-clinical skills such as self-reflection, self-awareness, and self-regulation; team leadership; change management; negotiation; and maintaining professional networks. Because residents provide a significant portion of the care in large teaching hospitals, those who undertake leadership responsibilities can have a significant impact on patient care.
In response to this need to develop leadership skills among CRs, various programs have been established in single specialties, multiple specialties, single institutions, and national programs.12–21 Still, published evidence demonstrating the effectiveness of such programs remains sparse, with studies focusing mostly on immediate program outcomes (ie, improvement in participants' knowledge, self-awareness and self-confidence, and anticipated behavior changes16,21–23) and self-reported outcomes. Several systematic reviews of physician leadership development programs have shown substantial evidence gaps, especially regarding long-term measures and high-impact outcomes.24–26
The current study was undertaken to address these gaps. Specifically, we measured immediate and intermediate outcomes associated with a longstanding leadership workshop3 offered to all CRs at the Cleveland Clinic. Outcomes included participants' self-reports as well as assessments from the CRs' program directors (PDs) regarding workshop participant performance and application of these concepts during the subsequent CR year.
Methods
Site and Setting
Cleveland Clinic is a fully integrated, multispecialty academic medical center where 985 residents and fellows trained in Cleveland in 75 Accreditation Council for Graduate Medical Education (ACGME) accredited programs and 80 additional programs in the 2017–2018 academic year. At Cleveland Clinic Florida (CCF), 100 residents and fellows trained in 11 ACGME-accredited programs.
Since 2008, a 2-day Chief Residents Leadership Workshop has been offered annually to all CRs in the initial weeks of their chief residency year.3
Leadership Curriculum
The curriculum was based on longstanding leadership development programs at CCF (eg, Cleveland Clinic Learning Academy, Leading in Health Care, and Samson Global Leadership Academy).3,27,28 These and the current curriculum are grounded in the emotional intelligence (EI) framework of Goleman et al.29 Goals and topics addressed in the course (table 1) correspond to 4 areas of EI competencies: self-awareness, self-management, social awareness, and relationship management. The 12 course sessions were taught by CCF faculty and selected extramural invited experts.
Course Participants
Current CRs starting their chief residency tenure were nominated to attend by their PDs. They represented all specialties at CCF campuses in Cleveland, Akron, and Florida, and other health centers in Northern Ohio. The course, offered in early August, was announced in various venues, including the CCF GME website,30 program leadership and council meetings, and to ACGME designated institutional officials at other hospitals in Northern Ohio who were invited to nominate CRs.
Course Evaluation
We applied a logic model31 to track and align course activities with resulting outcomes (table 2). Components of this logic model include resources devoted to the course, activities targeted to attain desired outcomes, outputs of course participation, and outcomes from CR course participation. The following 2 questions were investigated in the 2017–2018 academic year course evaluation: What are the immediate (last day of course) and intermediate outcomes (9 months after course completion) for the CRs who participate in the course? What suggestions for improvement, if any, do stakeholders have about the course?
Data Collection
Data collection for this mixed methods evaluation included questionnaires and interviews. All data collection instruments were piloted and modified based on feedback from medical students in a similar leadership course (pre- and post-course questionnaires), prior CR course attendees (CR follow-up questionnaire), and former PDs (PD interview guide). These documents are provided as online supplemental material.
We used a pre-course questionnaire to assess participants' baseline familiarity with key concepts covered in the course. To investigate short-term or immediate outcomes (table 2), we administered questionnaires with closed- and open-ended questions to participants before and immediately after the course. Intermediate outcomes were assessed using an online questionnaire administered on REDCap, a web-based platform for developing and managing surveys (Vanderbilt University, Nashville, TN). Two authors (S.M. and C.F.F.) also interviewed 14 PDs purposefully selected32 to represent core programs (anesthesiology, family medicine, general surgery, internal medicine, neurological surgery, pediatrics, obstetrics and gynecology, ophthalmology, orthopedic surgery, pathology, pediatrics, psychiatry, radiology, and urology) that had the largest number of residents and who had sent multiple CR course participants. Three of these PDs were from other hospitals in Northern Ohio. Interviews were one-on-one, semistructured, and held either in PDs' offices or by telephone. Interview duration was approximately 10 to 30 minutes. Interview questions were developed from the course content and outcomes and assessed whether PDs observed any changes in CRs' behavior, attitude, or performance attributable to the course, as well as their overall evaluation of the course (interview guide provided as online supplemental material).
This continuous quality improvement project was deemed exempt by the Cleveland Clinic Institutional Review Board.
Data Analysis
Responses to the Likert scale questions on the pre- and post-course and 9-month follow-up questionnaires were entered into SPSS 24 (IBM Corp, Armonk, NY) to compute descriptive statistics of responses and summarize participants' demographic characteristics, including gender, specialty, institutional affiliations, and prior leadership training. CR responses to open-ended questions on the abovementioned questionnaires were analyzed using Dedoose (University of California, Los Angeles, CA), a web-based software for analyzing qualitative and mixed methods research, to uncover emerging themes related to the perceived immediate and intermediate course outcomes. Fisher's exact 2-tailed tests were used to assess differences in CRs' reported immediate and intermediate outcomes across levels of gender and prior leadership training. The a priori level of statistical significance was P < .05. Cramér's V statistic served as a measure of the effect size (ES). This statistic has a lower limit of 0 and an upper limit of 1. We used Cohen's criteria33 to classify ES magnitude (0.10 = small, 0.30 = medium, and 0.50 = large). Data from the PD interviews were transcribed verbatim and coded using Dedoose. Two authors (S.M. and C.F.F.) performed a qualitative content analysis of PD interview transcripts independently and then discussed to reach consensus.34
Results
Seventy-four new CRs (64% [47 of 74] male) representing 28 specialties and 5 health systems in Ohio and Florida attended the course. Participants included 64 CRs from Cleveland Clinic (Ohio and Florida) and 10 CRs from other medical centers across Ohio.
Achievement of Immediate Outcomes
Pre-Course Questionnaire:
All participants were queried regarding their perceived familiarity with different leadership competencies, including EI, cultural competence, organizational change, conflict resolution, teamwork, and team building. Most indicated levels of familiarity that were below moderate, which corresponded to a mean score of 4 points on a 5-point scale.
In open-ended responses, participants reported a number of motivations for taking this course. Among the top reasons were to (1) develop or improve leadership skills, (2) better serve as CRs, and (3) comply with the recommendation of their program leadership. Some CRs indicated being motivated by their future leadership career aspirations, not having received leadership training before, and personal interest in course content. Participants indicated various reasons for seeking future leadership opportunities in health care organizations, ranging from a desire to pursue careers in academic medicine, improve and innovate health care, or assume specific leadership roles, such as department chairs, residency PDs, and chief of staff.
Post-Course Questionnaire:
Forty-seven of 74 CRs (57% [27 of 47] male) responded to the post-course evaluation (64% response rate), most of whom (57%, 27 of 47) had not had prior leadership training. Over 95% (45 of 47) of respondents indicated they developed leadership knowledge and skills represented by the first 10 curricular goals, related learning objectives of the course, and connected with CR peers they did not know before (table 3). All 47 respondents recommended the course to future CRs. Participants' responses to open-ended questions supported their quantitative ratings (table 4). For example, respondents reported increased awareness of the characteristics of effective teams, change management strategies, and conflict styles and approaches to conflict resolution. No significant relationships were found between reported immediate outcomes and participants' gender or prior leadership training (P > .05). Effect sizes ranged from 0.01 to 0.32, suggesting weak to moderate associations between immediate outcomes and CRs' gender and prior leadership training.
Achievement of Intermediate Outcomes
Follow-Up Questionnaire:
Forty-four of 74 CRs (61% [27 of 44] male) responded to the 9-month follow-up questionnaire (59% response rate). Twenty-three of 44 respondents (52%) reported prior leadership training; 4 of these CRs were from institutions other than CCF. Ninety-five percent (42 of 44) of the respondents indicated they had applied concepts learned in the course during their chief residency, and 82% (36 of 44) used tools learned in the course to address conflicts in their programs (table 5). The most widely applied concepts included EI, conflict resolution, and communication, followed by leadership, teamwork, and team building. Forty-one percent (18 of 44) engaged in various quality and patient safety and other improvement projects (table 5). Fifty-two percent (23 of 44) reported continued interactions with other participants through various media following the course (eg, discussions by e-mail, connecting at local conferences, and collaborating on didactics and joint patient care). Those who did not report staying in contact with other CRs cited lack of time, opportunity, or distance. CRs' gender and prior leadership training were not significantly associated with the above intermediate outcomes (P > .05) and the magnitude of associations were small (ES ≤ 0.20).
Program Director Interviews:
All 14 PDs offered positive appraisal of their CRs who had attended the course and expressed intent to continue nominating CRs for future offerings. Reasons offered for continued nominations of CRs ranged from exposing their chiefs to foundational leadership knowledge and skills to enabling and promoting a community of leaders in order to support one another and share experiences. While the PDs reported growth in the CRs over the course of their chief residency, attribution of this growth to the course was not possible, partly because some PDs were insufficiently familiar with the course curriculum. Others suggested that it would have been helpful if they had been alerted beforehand to assess CR performance in specific domains addressed by the course. Several PDs had discussed the impact of the course with their CRs before the study interviews.
Follow-up feedback from the PDs and CRs was favorable. Additional topics suggested by PDs who knew the course curriculum well included the importance of role modeling to juniors, giving feedback, leveraging one's own strengths, and practice management. Suggestions from CRs centered mainly on promoting more interactivity and including experiential learning activities like case-based scenarios, small group sessions, specific scenarios to help apply concepts taught in the workshop, and discussion of concrete examples whenever possible to illustrate a topic. Some suggested more facilitation of participant introductions and assigning seating to encourage interactions among newly acquainted attendees. To refresh the curriculum throughout the year, some PDs and CRs suggested periodic check-ins with the participants or webinars.
Course Expenses and Feasibility:
Course expenses mainly reflected the cost of travel and lodging for those CRs attending from long distances away, estimated at approximately $1,200 per person (supported by a grant), and an honorarium for one outside faculty member. Costs related to catering for 2 days were modest (∼$30 per attendee). In a full salary model like the Cleveland Clinic, costs related to course director (J.K.S. and C.F.F.) contribution of 2 full days and CRs' time were covered. Administration of the course was supported by a program manager for whom course organization was estimated to require 30 hours (∼0.02 full-time equivalent).
Discussion
In this mixed methods study investigating the outcomes of a leadership course for CRs from different specialties and health centers across Northern Ohio and Florida, baseline familiarity with leadership competencies was generally low and the self-reported impact of the course on participants' knowledge of leadership competencies was high. During 9-month follow-up interviews, PDs reported growth in their CRs. In line with feedback from the study participants, additional course time will be reserved in the future for CRs to discuss application of course content to their work and solve case-based scenarios in small groups. Course goals and objectives will be shared with PDs to help with assessment of CRs' performance and selection of CRs for the course.
This study extends available findings regarding leadership development for CRs in several ways. In their meta-analysis of leadership training in graduate medical education (GME), Sadowski et al35 pointed out that “leadership curricula are heterogeneous and limited in effectiveness.” Furthermore, most available studies of leadership training programs in GME have assessed only short-term effects, most frequently in post-curriculum or pre-post surveys, and rarely included longitudinal follow-up with participants or measured higher-level outcomes at Kirkpatrick's level 3 (transfer of learning to workplace attributed to educational program) or level 4 (organizational/performance changes directly attributed to educational program).36 In this study, CRs provided specific examples (9 months post-course completion) where they applied leadership tools to alleviate interpersonal conflicts encountered during chief residency (level 3) and implemented quality improvement projects based on course concepts (level 4). Thus, by obtaining both immediate and intermediate term outcomes, the current study addresses a gap in the available literature regarding physician leadership development.24–26 Although attribution to the course was largely uncertain, the current study distinctively addressed impacts of the course on participants' leadership behaviors by surveying their PDs, and is unique in including CRs from different specialties and health systems in a single course.37
Several limitations of the study warrant comment. Bias toward course impact was introduced because CRs are almost uniformly selected for their leadership potential.38,39 Such selection bias and the lack of a control group precludes attributing outcomes to course participation. As with most other studies, this study included self-reported outcomes,40 thereby falling short of more rigorous measures of leadership competency acquisition that might be provided by serial multi-rater feedback on EI or leadership competence. Finally, while CRs from multiple institutions participated, small numbers preclude assessing whether the impact of the course is generalizable beyond the majority of participants from the sponsoring institution.
Future studies of CR leadership development should continue to focus on the concrete, intermediate to long-term impact of course participation. Additional evaluations of the impact of such leadership programs (eg, by addressing questions like: Were participating CRs' programs improved? Was learning by trainees in these programs demonstrably enhanced? Did quality and patient safety improve in the CRs' programs?) will enhance current understanding of the value of such activities and provide opportunities for future research.
Conclusions
This leadership development course for CRs was associated with self-reported enhancement of leadership competencies, both in immediate and intermediate time frames. It was acceptable to the participants and the PDs who recommended them.
References
Author notes
Editor's Note: The online version of this article contains the pre- and post-course questionnaires, the chief resident follow-up questionnaire, and the program director interview guide.
Funding: This work was funded in part by a grant from The Doctors Company Foundation.
Competing Interests
Conflict of interest: The authors declare they have no competing interests.
A subset of this study was presented as an innovation in medical education poster at the Association of American Medical Colleges Central Group on Educational Affairs (CGEA) Meeting, Rochester, MN, March 21–23, 2018.
The authors would like to thank Dr. Elias Traboulsi at the Cleveland Clinic for his insights and knowledge regarding graduate medical education and the need for special training of chief residents and The Doctors Company Foundation for their generous support.