AuduBon-Bons: Bite-Sized Learning for Residents in the Ambulatory Obstetrics and Gynecology Clinic

Although the majority of health care occurs in the outpatient setting, most graduate medical education (GME) occurs in inpatient wards.^1,2  In an effort to improve outpatient education, many residency programs have developed ambulatory curricula. One internal medicine program developed a curriculum spanning the length of the residency which improved residents' satisfaction with the ambulatory rotation.³  Another obstetrics and gynecology (OB/GYN) curriculum utilized clinical cases, which effectively addressed gaps in residents' ambulatory women's health learning.⁴ 

Training residents in OB/GYN must encompass the intensity of surgery, the acuity of obstetrics, and the broad spectrum of office practice. Currently, guidelines for ambulatory OB/GYN training are limited to the Council on Resident Education in Obstetrics and Gynecology (CREOG) Educational Objectives. Graduate surveys completed by our residents at NewYork-Presbyterian Hospital/Columbia University Irving Medical Center consistently reflected a lower level of comfort and confidence in readiness for practice in the outpatient setting compared to inpatient care. The literature demonstrates efforts by OB/GYN programs to compile lists of educational goals in the outpatient setting; however, no examples of a studied curriculum exist.^4-6  This is not a challenge unique to OB/GYN, as demands on ambulatory providers in all fields have increased exponentially without concurrent innovation in pedagogical methods.

The objective of this project was to develop and study a flipped classroom ambulatory curriculum delivered in 15 minutes to fit within the time demands of a clinic setting. We hypothesized that it would be feasible to implement, and that the curriculum would be effective in increasing residents' comfort and confidence with ambulatory OB/GYN topics. These methods and the lessons learned could be applied to update other residency programs' ambulatory curricula.

Phase I of the curriculum spanned the 2019-2020 academic year. Our resident clinic is located on Audubon Avenue. In the spirit of the “bite-sized” teaching employed, we named this program “AuduBon-Bons for the Ambulatory Clinic.” Columbia trains 6 OB/GYN residents per postgraduate year (PGY). All 24 residents agreed to participate in this program and its study. Residents were allowed to discontinue their participation at any point.

The curriculum design team consisted of 4 members: 2 clinician educators (residency director/vice chair of education in OB/GYN and associate director of the OB/GYN student clerkship) and 2 medical students. A chief resident (PGY-4) also served as a consultant to the curriculum design team to ensure resident input was incorporated. Six academic specialists in OB/GYN, one of whom was represented in the curriculum design team, were selected to create and deliver modules. These faculty were chosen due to their academic and teaching expertise and also their availability as preceptors to deliver the modules at every clinic session during the week.

The curriculum design team compiled a list of 100 topics utilizing the CREOG Educational Objectives, 11th edition. The flipped classroom curriculum was structured to teach a different topic each week in the resident continuity clinic over the course of 24 months. Over an 18-month period, 100 modules were created, 95 by faculty and 5 by the curriculum design team. Due to COVID-19 disruption, the implementation and response to only the initial 36 modules was studied.

A template was created by the curriculum team with a standardized slide design. Submitted modules were reviewed by the curriculum design team to ensure accuracy and consistency of format before dissemination. The 6 OB/GYN faculty were responsible for teaching the module that was scheduled during their precepting session, regardless of whether they personally authored the module. The first appointments for the morning and afternoon clinics were delayed by 15 minutes, and the number of patients seen by the residents was decreased by one per panel to accommodate this curriculum. This number was chosen as it would not be predicted to dramatically decrease patient access. The reduction was approved by the director of the ambulatory care network.

Pre-work that required approximately 15 minutes to review was electronically distributed to residents the week before the module was delivered to allow for a flipped classroom session. Assignments drew on reading the American College of Obstetricians and Gynecologists practice guidelines, listening to podcasts, and watching instructional videos. There was no way to monitor residents' completion of pre-work.

At 7:30 am daily, a text message reminder was sent to residents and preceptors scheduled for clinic. The module was reviewed during the first 15 minutes of morning and afternoon sessions from Monday through Thursday. Four residents are present in clinic daily, divided between morning and afternoon panels. Residents participate in clinic throughout all rotations except for 8 to 10 weeks of nights per PGY and two 4- to 5-week blocks of elective time. After delivery, all modules were housed on an internal web-based platform made available to faculty and residents.

In the 2020-2021 academic year, we added 2 components to each module: a Social Determinants of Health slide and an Epic SmartPhrase slide. Epic is an electronic health record (EHR) utilized by 26% of acute care US hospitals.⁷  These slides were added retroactively to modules that had already been disseminated and integrated into future modules. The Social Determinants of Health slide aimed to identify disparities, inequities, or barriers related to the topic and present actionable steps to address these issues, and the Epic slide showed suggested SmartPhrases. Epic SmartPhrases are selections of text inserted into the body of an EHR note that contain both prewritten default text and areas requiring completion to individualize content, provide a mental checklist for providers, standardize documentation of procedures/counseling across clinicians, and maximize billing.

A member of the curriculum design team was present to conduct in-person attendance and time tracking for the first 54 learning sessions. Residents received a weekly pre-module and post-module survey (provided as online supplementary data), created by the design team and distributed via email through REDCap, a secure web application for building and managing online surveys and databases. The pre-module survey was a 4-question form used to establish residents' baseline comfort and confidence with a given subject. The post-module survey was a 9-question form that assessed comfort and confidence after the module, established whether pre-work was completed and helpful, and assessed the preceptor's delivery of the content. Questions were Yes/No or utilized a 5-point Likert scale, and residents had the option to provide additional feedback via free text at the end of each survey. Preceptors received post-module surveys (provided as online supplementary data) created by the design team weekly to be completed after their clinic session and asked about resident engagement and preceptor workload. Residents and preceptors were instructed to complete the surveys whether they participated in clinic during the week or not. The first question of all surveys asked whether the resident/preceptor was in clinic for the given week, and if the response was “No,” the survey ended with no further data collected. This encouraged consistent survey completion. Residents and preceptors also completed a retrospective survey created by the curriculum design team in May 2020 to assess the overall program.

Data were analyzed manually with Microsoft Excel by the curriculum design team. Resident data were further broken down by PGY. More comprehensive statistical analysis with mean change and corresponding 95% confidence interval (CI) calculations was performed by utilizing only responses from residents who had completed paired pre-module and post-module surveys. Subgroup analysis of paired survey responses is only exploratory as numbers do not allow firm statistical conclusions from this many statistical tests.

Columbia University Institutional Review Board approval was attained for this project.

Weekly surveys were completed from July 1, 2019, through March 20, 2020, corresponding with the first 36 modules. The program was halted for the remainder of the academic year due to the COVID-19 pandemic. Sixty-four additional modules were developed but not delivered.

On average over 36 weeks, 11 residents were present for the weekly teaching sessions, with a range of 5 to 14. Average preceptor survey completion rate was 74.6% (191 of 256). Average resident pre-survey completion rate was 71.6% (619 of 864). Average resident post-survey completion rate was 68.1% (588 of 864).

Table 1 describes preceptors' experiences creating and delivering modules. Modules were submitted to the investigative team on average 3 months before their dissemination. Each preceptor dedicated an estimated 230 hours to create 19 modules.

Table 2 explores start and run time of modules. Morning modules began late more frequently than afternoon modules. Morning modules were often completed within the designated 15 minutes, while approximately half of the afternoon modules ran longer than 15 minutes. Both preceptors and residents were on time for 66.7% (36 of 54) of sessions that had in-person time tracking.

Residents were asked on the pre-module and post-module surveys to rate their comfort level regarding their knowledge base on the module's topic utilizing a 5-point Likert scale, where 1 indicated “I have no baseline knowledge on this topic” and 5 indicated “I'm very comfortable with my knowledge on this topic.” For every weekly session, the average resident comfort level with the module's topic increased from the pre-module survey to the post-module survey (Figure 1). The gain in comfort persisted when modules were analyzed by PGY. The average mean increase in comfort from pre-module to post-module was 0.79 (95% CI 0.66-0.92) on the 5-point Likert scale.

Similarly, residents were asked on the pre-module and post-module surveys to rate their confidence level in feeling prepared to counsel a patient regarding the module's topic utilizing a 5-point Likert scale, where 1 indicated “I'm not at all ready to counsel on this topic by myself” and 5 indicated “I feel completely ready to counsel on this topic independently.” The mean change in resident confidence counseling a patient regarding the ambulatory topic increased from pre-module survey to post-module survey in every module (Figure 2). The gain in confidence persisted when modules were analyzed by PGY. The average mean increase in confidence from pre-module to post-module was 0.94 (95% CI 0.81-1.07) on the 5-point Likert scale. Residents completed pre-module assignments 64.8% (236 of 364) of the time. Of residents who completed the pre-work, 89.4% (211 of 236) reported that the assignments were useful.

Almost all (97.9% 190 of 194) preceptor surveys reported that residents were extremely engaged during the session; all surveys reported that information provided to residents was well-received and short learning modules were an effective means of teaching ambulatory care. Faculty almost exclusively received 4 or 5 (where 1 was poor and 5 was excellent) in their assessment by residents.

Residents were asked to assess the overall program in the retrospective survey. Ninety-two percent (22 of 24) of residents ranked the curriculum as satisfactory or very satisfactory. Responses were broken down by PGY: 100% (6 of 6) of PGY-1/PGY-3 and 83.3% (5 of 6) of PGY-2/PGY-4 residents found the curriculum satisfactory or very satisfactory. Fifty percent (12 of 24) of residents and 60% (3 of 5) of preceptors reported that text message reminders were helpful. When asked for strengths of the curriculum, themes emerged from free-text responses: “Review of basic concepts that we may not actually ever formally learn,” “Especially helpful when we completed the associated homework,” and “Formal didactics each week with basic concept, reinforced immediately by patient interactions.”

This flipped classroom design with a 15-minute educational intervention was feasible in a busy ambulatory training environment with high trainee and faculty satisfaction. Institutional graduate surveys completed in the past by outgoing residents reflected a lower level of confidence and comfort in readiness for practice in the outpatient setting as compared to inpatient care. This curriculum demonstrated an increase in both measures after dissemination of modules through the program's surveying of residents. Therefore, this curriculum has the potential to fill a gap in GME training.

Other ambulatory curricula have demonstrated the utility in accommodating different levels of learners.^3,4  Similarly, one of the strengths of this program were modules that provided basic knowledge to junior residents and served as valuable reinforcement and review for senior residents preparing for board examinations. In contrast to other ambulatory programs, this curriculum was deemed effective because of its brief structure and flipped classroom format. With increasing institutional pressure to see more patients, it is essential that didactic programs in the clinic setting fit into a busy schedule. The small group learning method has proven to be a more fruitful academic environment that maximizes learning, and it is the preferred approach to pedagogy in health professional education as compared to lecture-based learning.^8-10  We predict this model is sustainable because module creation, which required the largest amount of time, is a one-time requirement.

Long term, we plan to compare graduate surveys of residents who completed the curriculum to the responses of residents before the curriculum was implemented to evaluate whether participants feel better prepared for ambulatory practice. We also hope to assess whether the introduction of these modules translates to changes in clinical practice or alters patient outcomes. This curriculum could be directly utilized by other OB/GYN residency programs, or the format could be adapted for use in other GME residency programs.

It is feasible to create and implement an in situ ambulatory OB/GYN curriculum in an outpatient clinic using short learning modules and a flipped classroom format.