Residents-as-Teachers Publications: What Can Programs Learn From the Literature When Starting a New or Refining an Established Curriculum?

Most teaching that medical students receive during their clinical clerkships is performed by residents.1 Acknowledging the importance of residents as teachers in medical education, the Liaison Committee on Medical Education requires that residents be provided with training and be evaluated to ensure their competency as teachers.2 Likewise, the Accreditation Council for Graduate Medical Education mandates that residency programs provide residents with resources to develop teaching skills to effectively educate patients, families, students, fellow residents, and other health care professionals.3 External requirements notwithstanding, training programs have a vested interest in developing residents into excellent teachers.

The importance of formal residents-as-teachers (RAT) curricula has been noted throughout the literature, yet not all residency programs have such a curriculum in place. In a recent study by Fromme et al,4 87% of pediatrics training program directors reported they have a formal RAT curriculum, but half of programs allocate 10 hours or fewer for it during residency. In addition, only one-quarter of the responding program directors felt their RAT program was very effective.

Using parts of, or full curricula that have already been successfully implemented and published is an efficient way for residency programs to initiate and fine-tune required RAT training. However, few publications on RAT programs are sufficiently detailed to provide enough outcome data to be reproducible. A 2009 literature review by Post et al,5 looking at interventions and outcome measures, found that one-half of the articles on this topic were descriptive without measurable outcomes or pertained to fellows and faculty, not residents, and thus could not be included in their study. The other 50% of articles involved varying methodologies with most being nonrandomized or uncontrolled.5 The lack of measurable outcomes has made it difficult for residency programs to use the literature to initiate or to improve their own RAT programs.

The purposes of our study were to (1) review the extant RAT literature to assess the reproducibility of published RAT curricula, (2) evaluate the type of outcomes achieved in those studies using the Kirkpatrick model of evaluation, and (3) identify curricula that training programs interested in starting or modifying a RAT program could feasibly adopt.

In 2011, the authors performed a systematic review of the literature using PubMed, Medline, Scopus, PsycINFO, ERIC, and Embase. Key search words included residents, residents as teachers, teaching, internship and residency, and curriculum. In addition, a search of MedEdPORTAL was performed using the same key terms.

Results from these searches were then reviewed for pertinence to the subject of residents as teachers. The authors identified articles that were deemed pertinent (ie, those that described a RAT curriculum or intervention that was implemented to improve residents' teaching skills).

Selected articles were evaluated based on the reproducibility of the curriculum and the assessment tools and educational outcomes achieved.

Reproducibility was determined by the presence of clearly delineated goals and objectives with descriptions of study design, intervention assessment, lesson plans and materials (videos, PowerPoint slides, handouts, etc), and duration of intervention. To assess our reproducibility scale for interrater reliability, 3 articles that met inclusion criteria were selected at random. All authors then evaluated each article using a 3-point scale (1, not reproducible; 2, partially reproducible; 3, fully reproducible). The authors discussed the articles and an agreement in scoring was achieved through a consensus methodology to ensure that 1 author (K.K.B.) selected and scored articles appropriately.

Educational outcomes were evaluated using the Kirkpatrick model6 described in table 1. Interrater reliability was then established for the Kirkpatrick outcomes using the same method described above. One author (K.K.B.) then read all subsequent articles in their entirety and scored each for reproducibility and educational outcomes.

table 2 summarizes RAT programs in a multitude of residency programs comprising various specialties. The number of residents in RAT programs ranged from 1 to 443. Duration of the programs ranged from 1 hour to 1 year and included various teaching modalities, including workshops, lectures, feedback sessions, standardized learner training, role playing, and teaching manuals. Some of the programs reported using elective time ranging from 1 month to a year.

The RAT programs used a variety of assessment methodologies, with many combining assessment modalities (eg, self-assessment and learner evaluation of the resident). A frequently used evaluative tool in 36% (14 of 39) of the studies was self-assessment, typically in the form of a preintervention and postintervention questionnaire. Learner evaluation by medical students, peers, or attending physicians was the second most commonly used assessment (28%, 11 of 39). Other assessments included objective structured teaching exercises (OSTEs), objective structured clinical examinations (OSCEs), direct observation, and videotaped evaluation.

figures 1 and 2 summarize the Kirkpatrick scores and reproducibility of the curricula.

The primary purpose of our study was to review the RAT literature for published curricula that could relatively easily be modified and/or reproduced by other residency programs. Although there are articles in the literature that delineate the interventions provided, the objectives, and detailed examples of the workshop activities, few of those interventions were randomized, controlled studies. There were articles that provided more rigorous assessment tools (OSTE, direct observation), but they lacked the necessary detail for other programs to use.

A reproducible curriculum includes a clear set of goals and objectives with descriptions of lesson plans and materials used (videos, PowerPoint slides, handouts, etc), as described above. For example, Morrison et al7 clearly delineated the curriculum and provided a website where readers could find additional resources. However, other articles deemed less reproducible simply listed discussion topics without providing clear objectives or describing how readers could incorporate the curricula in their own programs. Although helpful in aiding other programs to develop important topics to include in future workshops, a list of skills was not considered sufficient to replicate the program at different institutions.

Likewise, we considered a reproducible assessment to be one that described in detail the tools used to assess residents and provided examples. If a preassessment and postassessment evaluation were administered to residents, an example of the assessments needed to be included in the article. As an example of a reproducible assessment, D'Eon8 evaluated videotapes of residents' teaching following a 2-day workshop using an 8-point Likert scale. The scoring scale was provided in the article, allowing other programs to easily reproduce that assessment.

Most curricula and assessment/evaluations reviewed were “partially reproducible.” For example, an overview of the intervention was provided, indicating when mini-lectures or small group discussions were performed, but no objectives or curricular resources were provided. Similarly, some articles stated that a questionnaire was used for assessment with a list of topics being evaluated, but the questionnaire was not published. Most articles used some form of survey, questionnaire, or written learner assessment. Including those assessment tools would help other programs hoping to establish or improve RAT programs to reproduce those curricula or curricular elements in their own institutions.

Although there may be limitations by some journals regarding publishing curricula and assessments in their entirety, other journals have accepted those items as appendices. Many authors have indicated their willingness to be contacted for readers to obtain materials; however, having them easily and readily accessible for any program to use would be more beneficial.

Of the articles reviewed, just over 25% (10 of 39) had educational outcomes that met Kirkpatrick's fourth level of evaluation, indicating that most of the studies focused on resident reactions, attitudes and knowledge, and/or behaviors, but not more than that. As Miller9 pointed out in his classic commentary, performance at the simulation level (“shows how”) does not always accurately predict what trainees do in an actual clinical setting. Chen et al10 commented that educational studies need to consider all the stakeholders in the outcomes (eg, those who benefit from residents' improved teaching, including other trainees, faculty, and patients). We have highlighted those programs that sought these higher-level outcomes as models to emulate in a start-up or advanced RAT curriculum.

table 2 provides an overview of the articles reviewed and important information for programs hoping to create or improve their own RAT program. Although we recognize that a single article may not satisfy every program's potential needs for resources, duration of intervention, number of residents, and other attributes, table 2 is intended as a resource to quickly locate articles that may provide helpful resources for enhancing current RAT programs or starting a program de novo. Specific articles may not contain all curriculum and assessment tools in their entirety, but some authors do provide contact information so that specific materials may be requested. Encouraging all authors to upload their resources to MedEdPORTAL or establishing a clearinghouse website as a resource for residency programs would allow program directors easy access to those materials without having to contact several different authors.

table 2 provides a concise review of published articles in the literature; however, searching the published literature is not the only modality for locating materials and ideas that would be helpful in curriculum development. MedEdPORTAL also serves as a valuable resource for RAT resources. Simpson et al11 published their interactive teaching method “From Madness to Methods,” an engaging exercise for teaching educators new instructional methods. The short exercise can easily be adapted to fit into any current or new RAT program. Online modules,12,13 PowerPoints,14 and videos15 have all been published on the MedEdPORTAL site and allow programs to incorporate multimedia into their RAT curriculum. Finally, MedEdPORTAL provides some descriptive manuscripts and facilitator's guides, complete with sample schedules and assessment tools, such as that provided by Fromme.16 

With the current duty hour limitations, educators should be seeking to develop RAT programs with abbreviated class schedules (ie, a more blended learning approach). Having residents prepare for a workshop allows faculty to focus on application versus theory and, most important, allows residents to practice skills learned.

Our review of the medical education literature on RAT curricula, including peer-reviewed resources in MedEdPORTAL, identified a number of RAT curricula, yet only 1 published resource provided a reproducible approach to curriculum and assessment. This suggests a need for further research/scholarship that allows for improved reproducibility of the curriculum, as well as efforts to develop approaches for curricular evaluation at the educational outcomes level, which may require a multicenter, collaborative approach.