Background

Previous faculty-driven residents-as-teachers (RAT) models have had limited efficacy and sustainability.

Objective

To evaluate the acceptability and effects of a resident-led RAT program on resident teaching.

Methods

In October 2016, obstetrics and gynecology (OB/GYN) residents at a large academic institution implemented a resident-led RAT program, consisting of a steering committee of peer-selected residents with 2 faculty mentors who planned education-focused resident didactics and journal clubs, organized resident involvement in clerkship activities, and recognized residents who excelled in teaching as Distinguished Educators (DEs). From July 2016 through June 2019, using the Kirkpatrick Model, we evaluated the program with annual resident surveys assessing self-perception of 13 teaching skills (5-point Likert scale) and value of RAT program, institutional end-of-clerkship student evaluations of resident teaching, and resident participation in DE award.

Results

Annual resident survey response rates ranged from 63% to 88%. Residents' self-reported teaching skills improved significantly in 11 of 13 domains from 2016 to 2018 (improvements ranging from 0.87–1.42; 5-point Likert scale; P < .05). Of the 2018 respondents, 80% agreed that the resident-led RAT program added value to the residency. For 2017–2018 and 2018–2019 academic years, 47% and 48% of medical students (100% response rate) strongly agreed that residents provided effective teaching compared to 30% in 2016–2017 (P < .05). Ten residents have graduated as DEs during this time period.

Conclusions

A resident-led RAT program increased residents' self-reported teaching skills, improved medical student perceptions of teaching quality, and was sustainable and acceptable over a 3-year period.

Objectives

To evaluate the acceptability and effects of a resident-led residents-as-teachers (RAT) program on resident teaching over a 3-year time period.

Findings

Both residents' self-reported teaching skills and medical student ratings of resident teaching showed a significant improvement over the study time period.

Limitations

This was a single institution study, and self-assessment was used to measure resident teaching skills.

Bottom Line

A resident-led RAT program can result in sustainable improvements in medical student perception of resident teaching.

Resident teachers play a key role in medical student education, with up to one-third of student knowledge gained during clinical rotations coming from residents.1  The Accreditation Council for Graduate Medical Education and the Liaison Committee on Medical Education include resident teaching ability as a core competency and emphasize resident preparation for this role.2,3  Students frequently learn from residents in informal interactions that supplement the formal curriculum and can acquire skills such as effective communication, navigation of the medical system, and direct patient care logistics.4,5 

Overall, residents-as-teachers (RAT) programs have been shown to improve teaching skills and enthusiasm for teaching.6  However, nationally, obstetrics and gynecology (OB/GYN) residents have consistently been rated among the lowest specialties in core clerkship teaching effectiveness.7  RAT programs specific to OB/GYN, including direct observations, simulations, and multiday workshops, have demonstrated temporary improvement in resident comfort and students' ratings of teaching.814  The majority of these reported programs, within OB/GYN and other specialties, have been short term and lack evidence regarding efficacy and sustainability over time.15  Additionally, current curricula in OB/GYN to train residents as teachers are largely limited to faculty-driven workshops or simulations.

Our goal was to create a resident-led RAT program that would foster an environment to promote effective resident teaching. The program would be longitudinal, include teaching through direct medical student instruction and educational leadership experiences, be acceptable to residents, and be potentially attractive to incoming interns with an interest in teaching.

In October 2016, a resident-led RAT program was initiated in the Baylor College of Medicine OB/GYN residency, a large academic 4-year residency with 12 residents per year. This initiative was named the Academy of Resident Educators (AoRE), an adaptation from a similar program in the pediatrics department at our institution.16  Prior to this, the RAT curriculum was faculty-led and consisted of lectures during protected didactic time.

There were 2 main components to the resident-led program (Figure). The first was the creation of a steering committee with one peer-selected resident from each postgraduate year (PGY) and 2 faculty advisors. This committee organized 4 educational workshops during protected resident didactics and 2 educational journal clubs per year that were open to all 48 residents in the program. They also facilitated resident involvement in clerkship didactics, clerkship simulation, and medical student interest group activities. Second, residents with additional interest in education were given the opportunity to be recognized as Distinguished Educators (DEs) at graduation. Requirements of a DE were creation of an educational portfolio documenting 25 hours of educational work over the 4 years of residency and completion of a personal statement on the resident's role as a clinician educator. Residents who were in their fourth year and third year at the time of the initiation of the program had to complete 10 and 20 hours to be recognized, respectively.

Figure

Academy of Resident Educators—A Resident-Led Residents-as-Teachers Program

Figure

Academy of Resident Educators—A Resident-Led Residents-as-Teachers Program

Close modal

The program was evaluated over a 3-year period using the Kirkpatrick Model for program evaluation.17  The main components of this evaluation were an anonymous resident survey with questions addressing teaching skills and RAT program perception, anonymous institutional end-of-clerkship student evaluations of resident teaching, and DE graduation metrics. The resident survey was distributed to all 48 residents in October 2016, 2017, and 2018. This 16-item survey was based on the Clinical Educator Self-Assessment, an unpublished instrument developed by Dennis Baker, PhD, at Florida State University. The first 3 questions collected basic demographic information on age, identified gender, and training level. The remainder of the survey (provided as online supplementary data) asked residents to rate their teaching skills on a 5-point Likert scale in 13 teaching domains (1, low, to 5, high). This assessment was chosen as there are no published surveys to measure residents' teaching skills with robust validity evidence, and it has previously been used as an outcome measure for RAT programs.11,18  It was distributed annually via a SurveyMonkey link sent to the residents' institutional emails, with 2 reminder emails sent weekly after initial distribution. In addition, 3 survey questions were added in 2018 regarding the value of AoRE to the residency program, intention to graduate as DE, and influence of AoRE in decision to choose program.

Clerkship evaluations from July 2016 through June 2019 were also reviewed. All students were required to complete these evaluations during the last week of the OB/GYN clerkship per institutional policy. Responses to the prompt, “In general, residents provided effective teaching during the clerkship,” were tracked and stratified by academic year (July 2016–June 2017, etc). Academic years are referred to as Year 1, Year 2, and Year 3 in the text for clarity. This question assessed residents as a whole and was not used to assess individual residents. Students responded to this question on a 5-point Likert scale (1, strongly disagree, to 5, strongly agree).

Level 1 (response) of the Kirkpatrick model was measured with the Year 3 resident survey question regarding the value of AoRE. Level 2 (learning) was evaluated by monitoring for development of teaching skills as residents moved through the program by comparing data from the PGY-1 and PGY-2 cohorts in Year 1 to the PGY-3 and PGY-4 cohorts from Year 3. Level 3 (behavior) was tracked by assessing changes in student ratings of teaching throughout the duration of the program. Finally, Level 4 (results) was monitored by graduation of residents as DEs, and Level 5 (return on investment) was evaluated with residents' views of AoRE in choosing the residency program.

Two-way ANOVA with the Šidák correction for multiple comparisons was used for individual comparison of the 13 resident teaching skills stratifying by PGY level while the Mann–Whitney U test was used to compare mean scores of respondent groups by PGY level. Descriptive statistics were used to analyze additional resident survey questions added in Year 3 and graduation as a DE. Chi-square testing was used to compare answers to clerkship evaluations grouped by individual Likert scale answer, with Kruskal–Wallis test with Dunn's correction used to compare average medical student ratings of residents, by academic year. All statistical analysis was performed using Prism version 7.01 (GraphPad Software, San Diego, California).

This study was approved by Baylor College of Medicine's Institutional Review Board.

Table 1 displays results of outcome measures framed by the Kirkpatrick Model.17  Demographic data and baseline characteristics of the resident respondents are reported in Table 2. A total of 42 (88%), 34 (71%), and 30 (63%) residents completed the resident teaching skills survey in Year 1, Year 2, and Year 3, respectively. Residents were predominantly female and aged 25 to 29. In Year 3 of the program, 80% (24 of 30) of residents completing the survey agreed that the AoRE added value to the residency (Kirkpatrick Level 1). Table 3 displays the changes in self-reported teaching skills noted for residents who were PGY-1s and PGY-2s in Year 1 to their skills as PGY-3s and PGY-4s in Year 3 (Kirkpatrick Level 2). Over this time period, residents showed significant improvement in 11 of the 13 teaching domains, ranging from 0.87 to 1.42 (5-point Likert scale, P < .05).

Table 1

Evaluation of the Academy of Resident Educators Utilizing the Kirkpatrick Model of Evaluation

Evaluation of the Academy of Resident Educators Utilizing the Kirkpatrick Model of Evaluation
Evaluation of the Academy of Resident Educators Utilizing the Kirkpatrick Model of Evaluation
Table 2

Characteristics of Residents Completing Resident Teaching Skills Survey

Characteristics of Residents Completing Resident Teaching Skills Survey
Characteristics of Residents Completing Resident Teaching Skills Survey
Table 3

Responses to Resident Teaching Skills Self-Assessment Survey—Comparison of PGY-1 & PGY-2 Residents in Year 1 and PGY-3 & PGY-4 Residents in Year 3

Responses to Resident Teaching Skills Self-Assessment Survey—Comparison of PGY-1 & PGY-2 Residents in Year 1 and PGY-3 & PGY-4 Residents in Year 3
Responses to Resident Teaching Skills Self-Assessment Survey—Comparison of PGY-1 & PGY-2 Residents in Year 1 and PGY-3 & PGY-4 Residents in Year 3

Table 4 presents end of clerkship survey responses to the prompt, “In general, residents provided effective teaching during the clerkship,” stratified by academic year (Kirkpatrick Level 3). Response rate was 100%, as all clerkship students were required to complete the end-of-clerkship evaluation. At baseline, 30% (52 of 171) of students during Year 1 of the program strongly agreed that the residents provided effective teaching (Likert scale 5). This increased to 47% (86 of 185) for Year 2 (P = .038) and 48% (89 of 187) for Year 3 (P = .027), both statistically significant when compared to Year 1 of the program. The mean resident rating also significantly improved throughout the duration of the program, with a baseline of 3.92 in Year 1, increasing to 4.17 for Year 2 (P = .015) and 4.26 for Year 3 (P < .001).

Table 4

Medical Student Responses to Prompt: “In General, Residents Provided Effective Teaching During the Clerkship”

Medical Student Responses to Prompt: “In General, Residents Provided Effective Teaching During the Clerkship”
Medical Student Responses to Prompt: “In General, Residents Provided Effective Teaching During the Clerkship”

The first year of initiation of the AoRE, 1 of 12 (8%) residents was recognized as a DE, while the number increased to 3 (25%) in Year 2 and 6 (50%) in Year 3 (Kirkpatrick Level 4). In Year 3, all 9 PGY-1 respondents agreed that the option to participate in the AoRE positively influenced their decisions to train in our residency program (Kirkpatrick Level 5).

Although there was initially a large time investment required of the founding members of the AoRE (10 hours per month for first 3 months) for both residents and faculty mentors, this experience is now a minimal time commitment of approximately 4 hours a month. There has also been no need for additional IT resources. Teaching workshops were performed by faculty recruited from our own institution who were known to excel at teaching. Estimated cost of $1,000 per year was primarily for food at journal clubs and the yearly DE certificate.

Utilizing a resident-led RAT program, we have been able to strengthen the effectiveness of our resident teachers from our medical students' perspective. At baseline, less than a third of our clerkship students strongly agreed that our residents provided effective teaching, and now this perception exists among almost half our students. We have also seen this improvement sustained over a significant time period.

We think that having residents lead efforts to improve their student teaching has contributed most to the improved teaching effectiveness. The residents are actively engaged in continuous programmatic evaluation of their program, by leading journal clubs to uncover what students look for in resident teachers and organizing didactic sessions to provide easily implementable teaching methods. Their participation and leadership of the program has improved their confidence in teaching and has likely led to the improvement seen in resident self-perception of student teaching since program inception.

Compared to previous reports affording shorter follow-up of RAT programs,6,19  we demonstrated sustainability over 3 years. We believe there are 2 major factors in creating sustainability: (1) promoting resident autonomy over program maintenance and (2) low costs. Our resident-led RAT has a clear mission to foster growth of clinician educators through educational leadership and professional development, with clear resident-established criteria for recognition of DEs. It is now embedded within the residency program through blocked time for resident-led educational workshops and journal clubs. While other effective RAT programs utilize simulation and nonrecurring seminars that can increase the need for IT resources and provide short-term gains,14,20,21  the main driver for this program is the steering committee that is dedicated to the education mission and utilizes existing infrastructure, decreasing needs for additional costs and staff support.

Several limitations of this study should be noted. First, the changes over the 3-year time period may not solely be associated with the AoRE alone. Second, resident views were obtained through self-assessment and survey methodology, potentially limiting exploration of relevant RAT program metrics. Also, as a Likert scale was used to judge perception of teaching skills, respondents may interpret answer options in different ways. Although we believe that the AoRE was genuinely well-liked by residents, we cannot dismiss the potential for agreeability bias in the survey responses regarding the perception and recruitment value of the AoRE among our residents. Finally, the current study's findings are based on a small sample size within only one institution, potentially limiting generalizability to other programs and specialties.

In the future, we will continue to track the program's outcomes and the development of our DEs. With overwhelming resident interest in AoRE, 5 new subcommittees have formed: medical student mentorship, clerkship education, educational resources development, liaisons with other specialties, and education research coordination. Next steps for our initiative include developing and further studying these subcommittees. We would also like to specifically study how the AoRE affects recruitment of residency candidates.

A resident-led RAT program is acceptable to residents, time-effective, sustainable, and results in improved medical student perception of resident teaching.

1. 
Snell
L.
The resident-as-teacher: it's more than just about student learning
.
J Grad Med Educ
.
2011
;
3
(
3
):
440
441
.
2. 
Accreditation Council for Graduate Medical Education.
Common Program Requirements
.
2021
.
3. 
Liaison Committee on Medical Education.
Functions and Structure of a Medical School: Standards for Accreditation of Medical Education Programs Leading to the MD Degree. March 2017.
2021
.
4. 
Karani
R,
Fromme
HB,
Cayea
D,
Muller
D,
Schwartz
A,
Harris
IB.
How medical students learn from residents in the workplace: a qualitative study
.
Acad Med
.
2014
;
89
(
3
):
490
496
.
5. 
Ramani
S,
Mann
K,
Taylor
D,
Thampy
H.
Residents as teachers: near peer learning in clinical work settings: AMEE Guide No. 106
.
Med Teach
.
2016
;
38
(
7
):
642
655
.
6. 
Hill
AG,
Yu
T-C,
Barrow
M,
Hattie
J.
A systematic review of resident-as-teacher programmes
.
Med Educ
.
2009
;
43
(
12
):
1129
1140
.
7. 
Association of American Medical Colleges.
Medical School Graduation Questionnaire: All Schools Summary Report
.
2021
.
8. 
Berkowitz
LR,
McDermott
CH,
White
K,
Saavedra
A,
Hafler
JP.
Residents as teachers of the pelvic floor and perineal anatomy
.
Female Pelvic Med Reconstr Surg
.
2010
;
16
(
5
):
268
271
.
9. 
Cullimore
AJ,
Dalrymple
JL,
Dugoff
L,
et al
The obstetrics and gynaecology resident as teacher
.
J Obstet Gynaecol Can
.
2010
;
32
(
12
):
1176
1185
.
10. 
Frattarelli
LC,
Kasuya
R.
Implementation and evaluation of a training program to improve resident teaching skills
.
Am J Obstet Gynecol
.
2003
;
189
(
3
):
670
673
.
11. 
Gaba
ND,
Blatt
B,
Macri
CJ,
Greenberg
L.
Improving teaching skills in obstetrics and gynecology residents: evaluation of a residents-as-teachers program
.
Am J Obstet Gynecol
.
2007
;
196
(
1
):
87.e1
7
.
12. 
Katz
NT,
McCarty-Gillespie
L,
Magrane
DM.
Direct observation as a tool for needs assessment of resident teaching skills in the ambulatory setting
.
Am J Obstet Gynecol
.
2003
;
189
(
3
):
684
687
.
13. 
Ricciotti
HA,
Dodge
LE,
Head
J,
Atkins
KM,
Hacker
MR.
A novel resident-as-teacher training program to improve and evaluate obstetrics and gynecology resident teaching skills
.
Med Teach
.
2012
;
34
(
1
):
e52
e57
.
14. 
Ricciotti
HA,
Freret
TS,
Aluko
A,
McKeon
BA,
Haviland
MJ,
Newman
LR.
Effects of a short video-based resident-as-teacher training toolkit on resident teaching
.
Obstet Gynecol
.
2017
;
130
(
suppl 1
):
36
41
.
15. 
Bree
KK,
Whicker
SA,
Fromme
HB,
Paik
S,
Greenberg
L.
Residents-as-teachers publications: what can programs learn from the literature when starting a new or refining an established curriculum?
J Grad Med Educ
.
2014
;
6
(
2
):
237
248
.
16. 
Moza
R,
Villafranco
N,
Upadhya
D,
et al
Academy of Resident Educators: a framework for development of future clinician-educators
.
J Grad Med Educ
.
2015
;
7
(
2
):
294
295
.
17. 
Kirkpatrick
DL,
Kirkpatrick
JD.
Evaluating Training Programs: The Four Levels
.
San Francisco, CA
:
Berrett-Koehler Publishers Inc;
1994
.
18. 
Berger
JS,
Daneshpayeh
N,
Sherman
M,
et al
Anesthesiology residents-as-teachers program: a pilot study
.
J Grad Med Educ
.
2012
;
4
(
4
):
525
528
.
19. 
Dannaway
J,
Ng
H,
Schoo
A.
Literature review of teaching skills programs for junior medical officers
.
Int J Med Educ
.
2016
;
7
:
25
31
.
20. 
Hill
AG,
Srinivasa
S,
Hawken
SJ,
et al
Impact of a resident-as-teacher workshop on teaching behavior of interns and learning outcomes of medical students
.
J Grad Med Educ
.
2012
;
4
(
1
):
34
41
.
21. 
Miloslavsky
EM,
Sargsyan
Z,
Heath
JK,
et al
A simulation-based resident-as-teacher program: the impact on teachers and learners
.
J Hosp Med
.
2015
;
10
(
12
):
767
772
.

Author notes

Editor's Note: The online version of this article contains the survey used in the study.

Funding: The authors report no external funding source for this study.

Competing Interests

Conflict of interest: Dr. Kilpatrick has financial disclosures to the American Board of Obstetrics and Gynecology, UpToDate, and Merck.

This work was previously presented as an oral presentation at The Association of Professors in Gynecology and Obstetrics Annual Meeting, Orlando, Florida, February 26–29, 2020.

Supplementary data