JGME-ALiEM Hot Topics in Medical Education: Analysis of a Multimodal Online Discussion About Team-Based Learning

Team-based learning (TBL) is a small group instructional method that promotes active learning and provides students with opportunities to learn from one another while applying conceptual knowledge to real-world problems.¹  A TBL session is informed by a sequence of activities, including individual presession study and review, individual and team-based testing prior to group work, and team-based solutions of clinical scenarios.

Since being developed in the 1970s for business education at the University of Oklahoma, TBL has increasingly been incorporated into health professions education, mainly in undergraduate medical education.^1,2  Although there is relatively less experience with the use of TBL in graduate medical education (GME), the focus on active learning, collaboration, and application to real-world problems makes it an attractive instructional strategy in an era of increasing intraprofessional and interprofessional collaboration, shared decision making, and increasing calls from accrediting and certification bodies to emphasize cocreation of learning in residency education.^3,4 

A recent Journal of Graduate Medical Education (JGME) article reported on the “Use of Team-Based Learning Pedagogy for Internal Medicine Ambulatory Resident Teaching.”⁵  In January 2016, this article was the focus of a virtual, open-access, health professions education journal club hosted by JGME and the Academic Life in Emergency Medicine (ALiEM) blog.

Virtual journal clubs in health professions education have become increasingly popular and are being utilized across clinical specialties.⁶  Social media platforms provide a way for health professions practitioners to connect with colleagues around the world in online communities of practice for the purpose of disseminating and evaluating the latest medical literature.⁷  ALiEM (http://www.aliem.com) is a public, not-for-profit, education innovation organization that has recently implemented unique multimodal online journal clubs, allowing for curated summaries of online discussions to be widely and rapidly disseminated—something not possible prior to the advent of social media.^8–10 

In this article, we summarize the themes that emerged from the multiplatform online discussion during the JGME-ALiEM Hot Topics in Medical Education journal club on the topic of TBL.

The JGME and ALiEM editorial boards collaboratively selected the featured article in this virtual journal club. Four facilitators (J.R., C.P., M.L., J.S.) were selected by the editorial boards for their expertise in health professions education and facility with social media. The journal club was hosted by ALiEM. Promotion for the journal club began 3 days before the discussion period, and was primarily conducted on Twitter using the facilitators' accounts (@Sherbino, @PatockaEM, @Jeff_—Riddell, @M_Lin), the ALiEM account (@ALiEMteam), and the JGME account (@JournalofGME), using the #JGMEscholar hashtag. The term “hashtag” refers to a search term used on Twitter to identify messages linked to a specific topic. The facilitators also contacted health professions educators and organizations with expertise or interest in TBL via a snowball technique to promote the virtual journal club.

The weeklong JGME-ALiEM Hot Topics in Medical Education was launched January 11, 2016, via an ALiEM blog post (http://www.aliem.com/team-based-learning-2016-jgme-aliem-hot-topics-in-medical-education). We followed the timeline outlined in our previous JGME publication about the virtual journal club innovation,¹¹  and adopted a similar educational design to last year's inaugural virtual journal club.¹⁰  The post included a brief review of TBL, the featured article, and 4 discussion questions (box).

On January 14, 2016 (day 4), a Google Hangout on Air was live-streamed to the public, featuring a video panel discussion with the first and second authors of the study, an invited subject matter expert (Ali Jalali, MD, University of Ottawa), and a facilitator (J.S.). The video, which was automatically uploaded to the “ALiEM Interactive Videos” YouTube channel, was embedded and archived within the ALiEM post for asynchronous viewing (https://www.youtube.com/watch?v=4h4C6MH5zIQ). The audio version of the YouTube discussion was also immediately published on the ALiEM SoundCloud account (https://soundcloud.com/academic-life-in-em/team-based-learning-jgme-aliem-hot-topics-in-medical-education). Quotes and key ideas from the panel discussion were live-tweeted by other facilitators (C.P. and M.L.) during the broadcast. We estimate that the 4 facilitators spent approximately 30 hours collectively preparing and managing the multimodal online journal club.

Blog comments, a transcript of the video panel discussion, and Twitter comments (tagged with #JGMEscholar or linked to the hashtag) were qualitatively analyzed using NVivo 10 (QSR International, Burlington, MA). One author (J.R.) independently conducted a thematic analysis. Several strategies were used to maintain study rigor, including creating an electronic audit trail (record of key analysis decisions) via NVivo and a team communication app (Slack Technologies Inc, San Francisco, CA) and engaging in reflexivity (investigator reflecting on the influence of his or her experience to lead to assumptions during analysis). Finally, 2 investigators (C.P. and M.L.) reviewed the results of the analysis. All suggestions from this audit were merged into the final code via a consensus process.

As a measure of the impact of this virtual journal club event, web analytics were captured for the week of the launch, as well as the following week to incorporate any late discussion. Participation was captured using analytics available from Google (blog website), Symplur Healthcare hashtag (#JGMEscholar hashtag), YouTube (video), and SoundCloud (podcast).

Web analytics are reported in table 1. The figure shows the geographic distribution of participants. The thematic analysis included 4 domains: benefits and barriers to TBL, design of teams, role of assessment and peer evaluation, and crowdsourced TBL resources.

A general theme across all platforms of communication was an enthusiasm for TBL. During the video panel discussion, the first author (Dr Balwan) of the discussed article commented on the “buzz” among learners when TBL was first implemented. It led her to document the interactions with her camera. The second author (Dr Fornari) described the excitement of faculty and residents when a redesigned curriculum was informed by good pedagogy. Residents generally reported being more engaged and perceived that they learned more. Commenting on his program's implementation of a modified TBL curriculum, 1 resident said it was a “huge improvement to just getting death by PowerPoint.”

Several participants commented on the efficiency of faculty-resident ratios in TBL, where 1 faculty member facilitates a large group of learners. The panel expert indicated that TBL allowed 2 faculty members to facilitate the same number of learners that had previously required 20 faculty using a small group design. All panelists agreed that although TBL may require more initial preparation time in designing the examinations and materials, there is a scalable benefit with TBL regarding ongoing faculty participation.

The discussion also explored ideas of why TBL is so effective. Some posited the importance of the social construction of knowledge.¹²  One blog commenter suggested that in TBL “you are asking for more than rote memorization on content,” suggesting that TBL moves beyond memorization of facts into higher levels of Bloom's taxonomy.¹³  Residency graduates should not only master the basic foundational facts, but also must be able to apply the content to solve real-world problems. TBL allows them to model these behaviors by working effectively with and accessing the knowledge of a diverse team.¹ 

Barriers to TBL also surfaced in the discussion. Convincing faculty to adopt a novel instructional method, increased preparation necessary to launch a TBL curriculum, and nonacceptance of peer evaluation were all identified as barriers. The consensus among discussants was that barriers related to design and development could be overcome via “faculty champions”: experienced TBL educators available to help others convert traditional courses into a TBL format. One participant suggested that junior faculty, who still need to develop a lecture bank, could redirect their energies toward designing TBL modules.

A plurality of comments focused on the nature of teams in TBL. It was suggested that since health professionals work in teams they should also learn in teams. One participant went further to suggest that as interprofessionalism grows in health care, we must consider competency not as a measure of an individual, but as a measure of a team.¹⁴  The participant added that TBL may be the best way to foster this.

There was near unanimity in the need for teams to be carefully and thoughtfully chosen ahead of time. Commenters argued that teams should be “specifically cultivated” and “chosen well in advance.” Most participants recommended keeping teams together, advocating that longitudinal consistency within teams helps to form relationships by providing a familiar, comfortable forum for residents to share knowledge and skills with each other. One speaker shared how a program used “families”: groups of 8 learners and 2 faculty members who progress through a longitudinal curriculum as a cohort.

A challenge to team dynamics was raised by an emergency medicine faculty member, who argued that varying work schedules and duty hour restrictions led to erratic attendance by learners. This led them to drop TBL in favor of a flipped classroom model,¹⁵  with new teams forming and disbanding each week.

Participants suggested that the ideal team size is 4 to 8 members, with the literature recommending 6 to 8 members.¹  Several blog commenters and tweeters suggested that diversity is essential, citing the inherent benefits of having groups with a mix of training years represented. One suggested that “there is inherent benefit to having mixed-year groups.” Another added, “Having a diverse set of foundational knowledge, perspectives, and backgrounds lends itself to a broader range of opinions and potentially more creative solutions/ideas.” One blog commenter noted that senior residents can develop teaching and mentoring skills via TBL. Another participant shared a graduated responsibility model where postgraduate year 1 (PGY-1) residents are responsible for basic core content, PGY-2 residents for advanced content, and PGY-3 residents for synthesizing cutting-edge literature.

The discussion highlighted differing opinions related to the use of assessments and peer evaluation. One blog commenter said that individual and team assessments prior to group work created an undesired culture of “checking up” on the residents. Another added that “it is often uncomfortable for learners.” Several blog commentators felt that assessments wasted precious group time that could be better spent targeting higher levels of Bloom's taxonomy during the application exercise.¹³  During the video panel discussion, the content expert indicated that there is flexibility in the implementation of the assessment strategies. He cautioned that unbundling and using only portions of the TBL approach could undermine the process and the defining features of TBL. For instance, in response to suggestions to remove Readiness Assessment Tests (RATs) and focus on just the application exercise during classroom time, he tweeted: “No RAT? Just call it a flipped classroom then.” Two discussants cited literature on the importance of retrieval (ie, assessments) for improving learning.^16,17 

Uncertainty also surfaced in the discussion around what to do with the assessment results. While some individuals argued that learner accountability (adequate preparation for a TBL session) is 1 of the most important principles of TBL, several discussants struggled to understand how graded assessments translate into GME, where residents generally do not get formal grades. A blog commenter expressed his uncertainty over what to do regarding accountability: “I'm not sure what to do with a resident that didn't prepare. Remove them from the session? If not, does it only serve to make the resident feel a bit humiliated?” To address this challenge, 1 individual suggested including individual assessments within the resident's portfolio under the clinical knowledge section, while including team assessments and peer evaluations in the portfolio under the teamwork and collaboration section.

Table 2 includes a variety of TBL resources identified by discussants during the virtual journal club.

Our curated summary reports on the themes that emerged from a multiplatform virtual journal club on the topic of TBL. A thematic analysis identified 4 domains relevant to GME: benefits and barriers to TBL, the design of teams, the role of assessment and peer evaluation, and crowdsourced TBL resources.

The discussion suggested 2 important areas for future scholarship related to TBL. One of the unanticipated themes in the discussion was the debate over the necessity of inclusion of assessments (of the individual or team) and peer evaluations. While individual and team assessments have been reported in the literature in pathology, primary care, neurology, and psychiatry residencies,^3,18–20  a number of faculty commented that they do not use any or all of the prescribed assessments typical of TBL. This raises the possibility that, in lieu of the whole “TBL bundle,” select elements of TBL are being applied based on local preferences and resources. While these deconstructed approaches to TBL may be subjectively reported as effective, they have not been evaluated against the complete TBL bundle. The second area identified for future scholarship involves how individual assessments, team assessments, and peer evaluations should be incorporated into the high-stakes residency evaluation.

This virtual journal club is an illustration of a growing trend in recent years toward building virtual communities of practice and changing the way health practitioners engage in professional development and lifelong learning.^6,11  Social media–based journal clubs are providing postpublication peer review and discussion while building a network of engaged academicians.¹¹  This annual collaboration between JGME and ALiEM allows broad dissemination of educational innovations while providing a platform for an international group of educators “who share a concern or a passion for something they do” to discuss, reflect, and potentially “learn how to do it better.”²¹ 

Although some have used web traffic and free data analytics as a proxy for “success,” there is no “gold standard” for how to measure the quality or effectiveness of online journal clubs.¹¹  Some virtual journal clubs have presented no analytic data, focusing their discussion on the content of the online discourse.²²  In studies presenting analytics, page views of published online journal clubs have ranged from 370 in a month²³  to 1324 in 2 weeks.¹⁰  Analytics at their best are an inadequate and variable proxy for impact. It may be more important going forward to spend less time comparing analytic outcomes and more time understanding what makes virtual journal clubs meaningful for those who participate. Further research might explore these issues of meaning and impact for online journal clubs beyond web analytics. Following up with participants to inquire about behavior or practice change would enrich our understanding of the influence of the virtual journal club.

This online journal club likely was limited by sampling bias, with individuals comfortable on social media more likely to participate. Also, participant demographics are unknown, raising the possibility that important perspectives (eg, clinical specialty, stage of training, content expertise, etc) have not been included. Finally, the online discussion was truncated based on an artificial time period. Late comments were included in our analysis up to 1 week after the close of the journal club to encourage maximal participation. We were unable, however, to complete a constant comparative or iterative sampling, and as such we are unable to ensure sufficiency of sampling. This limits the generalizability of these findings.

The JGME-ALiEM Hot Topics in Medical Education virtual journal club provided a novel discussion forum across multiple social media platforms, engaging authors, content experts, and the health professions education community in a discussion about TBL. A thematic analysis identified 4 domains relevant to GME: benefits and barriers to TBL, the design of teams, the role of assessment and peer evaluation, and crowdsourced TBL resources.