Curriculum Transition From Lecture-Based to Team-Based Learning is Associated With Improved Performance on Internal Medicine In-Training Examination

Residency training requires quality educational activities to ensure residents have the requisite medical knowledge to care competently for patients and to pass their board certification examinations. Medical knowledge is traditionally taught in a series of 1-hour noontime lectures,¹  though other approaches have emerged such as the academic half-day^2,3  and active learning approaches.^1,4  Team-based learning (TBL) is an active learning method beginning to generate traction in graduate medical education (GME).⁵  With TBL, learners acquire knowledge through advance reading assignments and then learn to apply this knowledge through real-world problem-solving exercises lead by a faculty facilitator. These activities commonly include an Individual Readiness Assurance Test (IRAT), Group Readiness Assurance Test (GRAT), and application exercises.⁶  In GME, TBL has been shown to promote learning and teamwork,⁷  learner engagement,⁸  clinical skills development,⁹  and learner and faculty satisfaction.¹⁰  There is limited medical literature, however, describing its effect on medical knowledge acquisition and standardized testing performance in GME.

In 2013, the internal medicine residency program at Albany Medical College transitioned from a lecture-based medical knowledge curriculum to TBL. In 2018, we described the planning, design, faculty development, lesson structure, satisfaction with, and feasibility of this curriculum.¹¹  We hypothesized that an active learning approach using TBL would result in more effective knowledge acquisition than lectures and lead to improvements on residents' Internal Medicine In-Training Examination (IM-ITE) scores. We now report on IM-ITE performance of 2 groups of residents, one exposed to a lecture curriculum and one to a TBL curriculum.

Our program is a medium-sized, university-based internal medicine (IM) residency with 48 categorical and 25 preliminary residents. In 2013, we transitioned from a lecture-based medical knowledge curriculum to TBL. In the primary analysis, residents enrolled in academic years 2011–2012 and 2012–2013 who took the ITE comprise the lecture cohort, and those enrolled in 2015–2016 and 2016–2017 comprise the TBL cohort. Five residents in the lecture cohort and 3 in the TBL cohort were excluded as they had not taken the United States Medical Licensing Examination Step 2 Clinical Knowledge (USMLE Step 2 CK). This left 60 residents in each of the 2 cohorts. In a secondary analysis, 28 residents enrolled in 2012–2013 and 2013–2014 and exposed to both lecture and TBL in successive years were analyzed for within-resident score changes. One additional programmatic change of note involved the clinical rotation scheduling structure. Prior to 2013, residents attended afternoon ambulatory clinic twice weekly during each rotation. Beginning in 2013, we switched to a 4+1 block schedule in which 5 staggered cohorts of approximately 10 residents rotate through 4 weeks of inpatient/elective service followed by an ambulatory week with daily clinic.

Prior to 2013, the lecture-based curriculum consisted of four 60-minute case-based lecture conferences totaling 240 minutes weekly. Cases were selected from all specialty areas to illustrate specific topics and presented to a senior faculty member for discussion. Faculty discussants, who were unfamiliar with the case to be presented, focused on diagnostic reasoning, evidence-based therapies, and additional curricular elements illustrated by the case. Presenting residents independently prepared and reviewed summaries of key teaching points of the topic of interest relating to the case. The conference learning was passive in nature with limited opportunity for interactive discussion. There were no preparatory reading assignments.

In 2013, we implemented 2 separate TBL curricula. The table of contents of the Medical Knowledge Self-Assessment Program served as the basis of both curricula. A 75-topic hospital curriculum covered 1 topic weekly in a 90-minute conference for all residents, repeating every 18 months. The ambulatory curriculum was taught during a 4-hour academic half-day of the 1-week ambulatory rotation. It was attended by 10 categorical residents weekly and offered 2 separate 120-minute TBL topic lessons. Residents had 10 ambulatory weeks yearly, 20 topics per year, and 60 over 3 years. Total TBL conference time averaged over the 5-week block schedule amounted to 138 minutes weekly. Each TBL lesson had an advance reading assignment requiring an estimated 45 minutes on average reading time. With reading factored in, approximately 200 minutes per week were devoted to TBL.

For all residents, we collected data on medical school attended (US allopathic or osteopathic, Caribbean, or other international), gender, USMLE Step 2 CK scores, and IM-ITE percent correct scores. Over the 6-year span of the study, the mean national ITE percent correct score increased from 63 to 66, and the mean national USMLE Step 2 CK score increased from 230 to 240.¹²  There were also changes in variance of scores. To make accurate comparisons, we converted both scores to Z-scores based on national means and SDs for each year. The primary study outcome variable was the difference in ITE Z-scores, reflecting knowledge acquisition under lecture and TBL. A Z-score of 0 represents the national mean and a Z-score of 1.0 represents 1 SD from the national mean. Additionally, we examined how the residents' ITE performance affected the program's overall ITE performance compared to all IM programs nationally.

Data are presented as means and SDs for variables measured on a continuous scale with unadjusted comparisons between groups performed by Student's t test. Categorical variables are presented as counts and proportions or percentages with associations between variables assessed by chi-square analysis. Data were analyzed for 3 postgraduate years (PGYs) combined. A multiple regression analysis was conducted of ITE Z-scores as a function of independent variable of curriculum (lecture or TBL) with categorical covariables of school, gender, and PGY (1, 2, or 3) and continuous covariable of USMLE Step 2 CK Z-score. To assure statistical independence, data were analyzed for PGY subgroups (in the combined PGY analysis there may be bias as some residents would have 2 likely related ITE outcome scores and others only one). Adjusted comparisons of ITE Z-scores were by multiple regression with effects of curriculum (lecture or TBL), USMLE Step 2 CK Z-score, school, and gender. In an additional analysis, the within-resident changes in ITE Z-scores for residents exposed to both lecture and TBL (2012–2013 and 2013–2014) were assessed by a paired t test. Statistical analysis was completed using R or Minitab statistical software with significance accepted at P < .05.

The study was declared exempt by the Albany Medical College Institutional Review Board.

The primary analysis included 120 residents with 60 each in the lecture and TBL cohorts. Both cohorts had similar distributions of gender and graduation from a US medical school. The TBL cohort had a larger proportion of international and a smaller proportion of Caribbean graduates. Though exact figures are not available, attendance appeared to remain steady at about 60% of on-duty residents for both cohorts, with the exception of the 10 resident ambulatory week academic half-day TBL, which was close to 100%. At all 3 PGY levels, the unadjusted resident ITE percent correct scores were significantly higher with the TBL curriculum compared to lecture. Similarly, in a univariate analysis, the transition to TBL was associated with significantly higher ITE Z-scores at all PGY levels (Table 1). In a multivariable analysis of all 3 PGYs combined, incorporating effects of curriculum (lecture or TBL) and adjusting for gender, school, PGY, and USMLE Step 2 CK Z-score, the transition from lecture to TBL resulted in an increase in ITE Z-score of 0.415 (P < .001; Table 2). In a multivariable analysis of PGY subgroups of the lecture and TBL cohorts, PGY-1 and PGY-2 residents demonstrated statistically significant increases in ITE performance after adjustment for USMLE Step 2 CK, school, and gender. PGY-1 ITE Z-score increased by 0.393 (P = .049); PGY-2 ITE Z-score increased by 0.435 (P = .007). For PGY-3 residents the ITE Z-score increase of 0.336 was not significant (P = .06; Table 3). To further investigate the effects of the TBL transition, an additional analysis was performed of the 28 PGY-1 and PGY-2 residents exposed to lecture in 2012–2013 and then to TBL in 2013–2014 as PGY-2s and PGY-3s. Adjustments for covariates were not necessary as this paired analysis was within-resident. Among these residents, ITE Z-scores improved from -0.413 to 0.067 for a net change of +0.48 (95% CI 0.226–0.734, P = .001), further supporting the study hypothesis. Finally, in relation to all IM residency programs, these improvements resulted in our program's percentile rank for ITE performance increasing from the 15th percentile nationally in 2011–2012 to the 65th percentile in 2016–2017 (Figure).

Consistent with our hypothesis, our program's transition from a lecture-based curriculum to TBL was associated with an improvement in residents' overall IM-ITE performance even after adjustments for the increase in national mean ITE scores (Z-score) and for prior clinical standardized testing performance (USMLE Step 2 CK). To our knowledge this is the first report in the literature of such a finding. The IM-ITE has been demonstrated to correlate with rate and degree of knowledge acquisition during residency¹³  and predicts performance on the American Board of Internal Medicine Certification Examination (IM-CE).^14–16  Furthermore, higher performance on the IM-CE has been associated with better patient care outcomes.^17–19 

Studies have demonstrated that TBL in undergraduate medical education can lead to improvements in students' subject area standardized testing performance.^20–23  In contrast, the GME literature on TBL is limited, and most studies are descriptive or report on pilot applictions.^5,7,10,24  Studies reporting on standardized testing learning outcomes in GME are scarce. One report describes an improvement in the clinical pathology sections of the ITE in a pathology residency, but the increase was not significant.²⁵  Another study shows that higher attendance at 18 rheumatology TBL lessons within an IM residency curriculum was associated with significantly higher rheumatology section IM-ITE scores compared to lower attendance.⁴ 

The reasons our TBL curriculum improved our residents' ITE performance are likely multifactorial. One may be the advance reading assignments. In a survey of IM program directors addressing the decline of programs achieving the minimum 80% IM-CE pass rate, the top reason listed was residents spending “less time independently reading.”²⁶  Additionally, self-directed reading is associated with medical knowledge acquisition in IM training and IM-ITE performance.²⁷  We previously demonstrated that the majority of our residents prepared for TBL by completing reading assignments ahead of time.¹¹  Our residents hold each other accountable for their team's performance; therefore, the incentive to prepare by reading is high.

Educational method and learning theory may also help explain our results. Lecture-based teaching results in low learner engagement, with retention rates estimated as low as 5%.²⁸  TBL is grounded in constructivist learning theory and its 4 elements: (1) it is learner centered; (2) it focuses on problem-solving; (3) it emphasizes dialogue and interaction; and (4) learner reflections help guide integration of new knowledge.²⁹  Our TBL curriculum embodies all of these elements with its learner-centered instruction by a faculty “facilitator” guiding small group work and leading to lively discussion and immediate feedback on real-world, case-based exercises. Though our lecture-based curriculum provided cases as a starting point, the focus of both faculty and presenting residents was directed at gain of factual knowledge as opposed to problem-solving and knowledge application. Given that the ITE emphasizes synthesis and judgment rather than factual recall,¹³  it naturally draws on residents' critical thinking abilities. TBL, with its grounding in constructivist learning theory, has been demonstrated to improve critical thinking.²⁹  These higher-level analytical skills which our residents gain from TBL are likely beneficial to their ITE performance.

Our study is limited by its implementation at a single residency program and retrospective and observational design. It is also possible that the transition to the 4+1 block schedule may have afforded residents more reading time. And though we controlled for some variables associated with ITE performance, we did not collect data for other variables linked to ITE performance such as age, precise resident conference attendance, self-directed resident use of other electronic educational resources, and USMLE Step 1 scores.²⁷ 

As active learning and TBL make further gains in GME, future studies should examine the generalizability of transitioning to a TBL curriculum, its impact on other standardized testing such as board certification examinations, and ultimately on objective measures of the quality of actual patient care.

Implementation of a comprehensive medical knowledge curriculum in IM using TBL is feasible, and compared to a lecture-based curriculum, TBL is associated with improved resident medical knowledge acquisition as evidenced by higher IM-ITE scores.

The authors would like to thank the residents and faculty at Albany Medical College for their participation and enthusiastic embrace of the TBL curriculum, and Maximilian Schynoll for his invaluable technical support in the manuscript preparation.