JGME-ALiEM Hot Topics in Medical Education Online Journal Club: An Analysis of a Virtual Discussion About Resident Teachers

A physician is a teacher. A physician facilitates not only the education of patients, colleagues, and other health professionals, but physicians-in-training also serve an important role in the education of junior colleagues, including medical students. The CanMEDS framework¹  and the Accreditation Council for Graduate Medical Education Outcome Project²  explicitly include the teacher role as a core competency for physicians. While North American medical education has historically emphasized clinically based (ie, bedside) teaching, the shift to competency-based medical education only further emphasizes the value of teaching in authentic (ie, clinical) environments.³  At times, the organization of clinical, administrative, and teaching responsibilities in academic centers frequently requires the partial delegation of teaching responsibilities from faculty to residents. As a result, senior residents have a significant influence on the learning of other trainees, such as junior residents and medical students.⁴ 

The influence of medical student and junior resident education provided by residents is reflected in the growing literature that addresses the design and impact of resident-as-teacher curricula.^5–7  In fact, a recent Journal of Graduate Medical Education (JGME) article⁸  reported on “What Makes a Great Resident Teacher? A Multicenter Survey of Medical Students Attending an Internal Medicine Conference.” In January 2015, this article was the focus of an open-access, online, health professions education journal club hosted by JGME and the Academic Life in Emergency Medicine (ALiEM) blog. Here we describe the themes and summarize the virtual, multiplatform online discussion that arose during the JGME-ALiEM Hot Topics in Medical Education journal club on the featured topic of resident teachers.

The JGME and ALiEM editorial boards collaboratively selected the featured article in the online journal club. Three facilitators (J.S., N.J., M.L.) were selected by the editorial boards for their expertise in medical education and facility with social media. The discussion was hosted by ALiEM (www.aliem.com), a public, not-for-profit education blog with more than 1.2 million page views in 2014 and 20 000 Facebook followers.

Promotion for the journal club began 3 days before the discussion period, and was primarily conducted on Twitter using the facilitators' accounts (@sherbino, @NJoshi8, @M_Lin), the ALiEM account (@ALiEMteam), and the JGME account (@JournalofGME), and included the #JGMEscholar hashtag. The facilitators also e-mailed health professions educators and social media thought leaders to promote the journal club.

The weeklong JGME-ALiEM Hot Topics in Medical Education was launched January 12, 2015, via an ALiEM blog post (www.aliem.com/what-makes-great-resident-teacher-jgme-aliem-hot-topic). The post included a written summary of the paper with links to the original manuscript, a video interview with the primary author (Lindsay Melvin, MD, University of Toronto; www.youtube.com/watch?v=qe31YjsXhhQ), and 4 discussion questions (box). On January 15, 2015 (day 4), a Google Hangout on Air was live-streamed to the public, featuring a video panel discussion with the primary author, an invited expert panelist (H. Barrett Fromme, MD, University of Chicago), and a facilitator (J.S.). The video was embedded and archived within the ALiEM blog post for asynchronous viewing (www.youtube.com/watch?v=9B-h4vIIrTo). Quotes and key ideas from the panel discussion were live-tweeted by the other facilitators (N.J., M.L.). Further details about the educational design of an online journal club can be found in the companion Rip Out in this issue of JGME.⁹ 

We recorded and analyzed participation in the journal club discussion via the ALiEM blog and Twitter.

A thematic analysis of the blog comments, Twitter comments with the #JGMEscholar hashtag, and a video panel discussion were conducted by 1 author (N.J.). The results were independently reviewed by the other authors (J.S., M.L.). Points of disagreement were resolved by consensus.

As a measure of the impact of this innovation, web analytics were captured for the week of the journal club, as well as the following week to incorporate any late discussion. Viewership and engagement were measured using such tools as Google Analytics, the ALiEM social media post widget, YouTube Analytics, and Symplur.

The #JGMEscholar hashtag garnered similar participation when compared with established Twitter-based journal clubs, such as #UroJC in urology and #NephJC in nephrology, using public Symplur analytic data for a 14-day interval during a January 2015 journal club. The #UroJC hashtag garnered 179 tweets from 45 participants, resulting in 251 140 Twitter impressions, and #NephJC garnered 783 tweets from 112 participants, resulting in 617 883 Twitter impressions. In comparison, #JGMEscholar garnered 360 tweets from 86 participants, resulting in 459 857 Twitter impressions. We acknowledge that Twitter metrics are not definitive measures of impact, yet #JGMEscholar achieved an equivalent audience when compared to established online clinical medicine journal clubs. In contrast with other online journal clubs, the JGME-ALiEM online medical education journal club featured a more diverse audience across different specialties, including emergency medicine, internal medicine, and pediatrics, and across multiple nations. Presumably, this diversity resulted in part from a discussion topic broadly applicable across specialties.

The amount of time needed to prepare, host, and curate the online journal club was small. Production times were approximately 3 hours (preparation), 5 hours (promotion), 10 hours (journal club event), 5 hours (video discussion), and 40 hours (curation/dissemination). These time costs do not include the development of the blog, which was preexisting. The curated summary enables those who missed the journal club event to track participants' diverse viewpoints (and perhaps continue the discussion), a feature absent in other online journal clubs.

Web analytics are reported in table 1. The blog post garnered 1324 page views from 372 cities in 42 countries (figure 1), with 209 tweets directly sent from the blog post during the 14-day analysis period. Although multiple social media platforms were available to facilitate discussion, there was a general dichotomous adoption of platforms. Twitter was used primarily as a promotional tool to highlight or endorse a theme developing in the journal club discussion. Blog comments were used primarily to provide opinions on a question or respond to an issue.

The analysis from the blog comments (n = 40), tweets (n = 569), and video panel discussions for the 4 stimulus questions is summarized below.

One of the most surprising elements of the featured paper was the devaluing of feedback as useful for learning by the medical students who were surveyed. This controversial finding was the focus of much of the online discussion across social media platforms. The majority of the audience emphasized that feedback was an important element for teaching medical students.^10,11  One discussant suggested that feedback is essential for resident-as-teacher programs (figure 2).

A common speculation regarding the devaluing of feedback was that medical students may not recognize feedback as such when it is delivered. Many commentators concluded that if feedback is not recognized, it is not valued or appreciated for its influence on learning. This issue is not unique to medical students. Research has shown that learners at various levels of training have difficulty recognizing feedback, partly because of nuanced distinctions between feedback, coaching, and assessment.¹² 

A proposed solution offered by discussants was to explicitly label feedback, and to use clear language that signals to learners that feedback is being delivered, such as “I am going to give you some feedback.” Providing feedback without labeling it may leave learners unaware that they received feedback during a clinical shift.

Several Twitter discussants suggested the actual issue was the word “feedback.” They proposed to rebrand feedback as “coaching” because of the positive connotations of this term, and the implication of the longitudinal relationship typical in coaching.¹³ 

Participants suggested that medical students truly value feedback, but that ineffective delivery reduces its value.¹⁴  Medical student discussants criticized generic and untimely feedback, which prevented the development of learning plans. This is reflected in research that shows medical students value teachers who tailor education to the students' individual needs.¹⁵ One discussant proposed enhanced assessment training for learners and teachers, and to create a feedback culture to address this issue. Some discussants suggested that medical students place a lower value on feedback from residents. Students may value feedback in general, but not from other trainees with limited clinical experience. While students may appreciate general teaching from a resident, whom they perceive as a near-peer, they may not value a constructive critique with performance recommendations.

Another explanation for the devaluing of feedback was that medical students do not fully understand their learning needs. Several studies suggest that self-assessment is inaccurate and unreliable.^16–19  The consensus from the discussants was that professional inexperience may leave medical students to rank feedback as less effective to their learning as other modalities.

Several blog commenters and tweeters, some of whom are medical students, suggested that perhaps threats to the self-images of medical students led to the devaluing of resident feedback (figure 2). Other participants suggested that medical students looked to feedback as a form of internal validation of their clinical abilities, as opposed to a means of identifying areas for learning.²⁰  Thus, constructive feedback that does not validate an internal perception may be devalued.

The discussion identified differences between the needs of a student-centered versus a teacher-centered curriculum. Several medical students clarified the perceived needs of learners, suggesting they evolve during the progression of training. A number of faculty members acknowledged the challenge of balancing the needs of various stakeholders, but the “end-user” (ie, the student) should remain a primary focus.²¹ 

During the video panel discussion, an invited expert panelist felt that the devaluing of feedback further underlined the importance of effectively training residents in the delivery of feedback.²²  She cautioned that many resident teacher curricula are modeled on faculty teaching programs; however, this assumes that residents and faculty share the same learning needs. Therefore, more scholarship on effective resident teacher curricula is required.

The featured article's single specialty nature (internal medicine), and the opportunistic sampling of resident teacher performance immediately after an intensive, weekend-long workshop, limited the generalizability of the findings to other learning environments and specialties. Several discussants commented on the recall bias of retrospective surveys. On satisfaction bias, an individual commented that “the literature has shown that students often do not appreciate feedback as a phenomenon on the whole.²³ As such, they may not report these experiences as valuable.” Recognizing these limitations, 1 discussant asked whether there is anything to be learned from a study limited by methodological design (figure 2). The invited expert panelist countered that despite nonperfect methods, this study generates hypotheses to inform future scholarship.

Participants in the online journal club identified a rich and varied set of resources for residents as teachers, shown in table 2.

Our curated summary reports the multiplatform social media discussions from the inaugural JGME-ALiEM Hot Topics in Medical Education online journal club. The virtual health professions education community discussed a study of medical students' perceptions of feedback from resident teachers. The primary focus was why feedback was devalued by medical students, and many explanations were proposed, including feedback not being labeled as such, the process of giving feedback, the source of feedback, limitations of students' self-assessments, and threats to student self-image. A crowdsourced repository of resident teacher resources was developed.

Traditional journal clubs are challenged by the scheduling of participants and facilitators, finding local experts to guide the discussion, and exclusive sharing of conclusions only with participants. In response to these challenges, online journal clubs have become more popular.^24,25  Using online platforms, participation can be synchronous or asynchronous, global experts can be recruited, and the discussion is digitally archived for broader dissemination. As a testament to their popularity, PubMed Commons recently announced a partnership with several online journal clubs.²⁶  In a similar manner, the Harvard Business Review partnered with the New England Journal of Medicine in 2013 to create an annual open-forum series to discuss major health care issues.²⁷  The common theme is to bring different communities of practice together in a virtual online space to discuss, cocreate, and disseminate knowledge in an open and flat hierarchical environment.²⁸ 

A limitation of this online journal club is sampling bias. Familiarity with social media was required for participation. Also, participant demographics are unknown, thus raising the possibility that important perspectives (eg, clinical specialty, stage of training, content expertise, etc) have not been considered in the analysis. Both of these elements limit the generalizability of the findings. Presumably, with increasing attention on the role of social media in health professions education, and the increasing incorporation of digital technologies into curricula, more educators will feel comfortable posting a blog comment or tweeting a response during virtual discussions. An online health professions education journal club has the potential to grow over time with minimal incremental costs because the available social media technologies are free and scalable.

Next steps for the JGME-ALiEM Hot Topics in Medical Education online journal club will be a thoughtful interpretation of web analytics to guide development, ensuring that a broad and representative audience contributes to future discussions on key topics for the health professions education community. Finally, the process of archiving and disseminating the curated summary of journal club findings should be examined to optimize reach and help influence related health professions education scholarship.

The JGME-ALiEM Hot Topics in Medical Education online journal club provided a novel discussion forum across multiple social media platforms, engaging authors, content experts, and the health professions education community. This approach can be adopted by other disciplines to promote an international asynchronous discussion on important topics. Crowd-sourced analysis of the resident teacher role suggests that feedback from residents to medical students is important, and that barriers to feedback acceptance can be overcome. Resident teacher curriculum resources are recommended.