Teaching Laboratory Stewardship in the Medical Student Core Clerkships Pathology-Teaches

The case scenarios and project goals of teaching appropriate ordering and interpretation of laboratory tests were identified by a multifactorial needs analysis.¹⁰  Briefly, the needs analysis consisted of semistructured interviews with core clerkship directors and residency program directors at our institution, a national survey to the Undergraduate Medical Educator Society of the Association of Pathology Chairs, and a review of online resources that teach HVC/stewardship principles. Based on the needs analysis, learning objectives and the individual components of learning that the students would need to understand to grasp key concepts within a specific case scenario were developed. This framework was then used as the case design, and input was sought from subject matter experts from anatomic pathology, family medicine, laboratory medicine, internal medicine, emergency medicine, infectious disease, microbiology, obstetrics-gynecology (OB-GYN), neurology, and transfusion medicine in order to ensure realistic, high-fidelity case content. Each Pathology-Teaches case was developed with the clerkship director and included clinical scenarios relevant for that particular clerkship.

In the online modules, the students work through a clinical case in which they must make decisions regarding the ordering or interpretation of laboratory tests (see “Family Medicine Pathology-Teaches case example”: https://cortex.acr.org/Presenters/CaseScript/CaseView?CaseId=stetczNz+i0%v3d&Preview=true&FromPortal=false). Immediate feedback is provided, with links to the appropriateness criteria (such as Choosing Wisely) and/or cost information. Additional links within the Pathology-Teaches case are also provided for supplemental information/references for review after mastering a concept. The Pathology-Teaches modules are hosted by the educational content management system of the American College of Radiology (ACR) and were implemented online with the Information Technology assistance of the ACR analyst.

The effectiveness of the educational module itself was evaluated by pretest and posttest flanking the intervention. The pretest and posttest questions were not identical; however, each pretest and posttest question was mapped to a learning objective of the case module. In the first 2 months, the pretest and posttest were initially launched from the ACR Web site. After review of initial 2 month data showing that students were spending an average of greater than 15 minutes within the module, the pretest and posttest were shortened from 10 questions to 5 questions each. From September 2018 to April 2019, the pretest and posttest were exclusively launched from our institution's Blackboard Web site. The pretest and posttest questions were swapped in January 2019 for the 4 Pathology-Teaches clerkship cases implemented before December 2018 (Family Medicine, Emergency Medicine, Neurology, and Internal Medicine), as a validity-promoting measure to ensure that any observed increase in performance could not be attributed to one assessment being easier than the other.

Descriptive statistics, paired-sample t tests, and Hedge g effect sizes were calculated to compare the student performance, pre and post test. Student feedback was collected from 2 questions on the existing end-of-rotation, voluntary-response curriculum evaluation surveys. These consisted of one 5-point scale question: “How valuable did you find Pathology-Teaches in enhancing your learning on the appropriateness of ordering laboratory tests?” (1 = No Value to 5 = Extremely Valuable), and a second, open-ended question “How could this Pathology-Teaches case study be improved?” All student information was de-identified before analysis.

This project was submitted to our institutional review board (IRB) and was deemed a routine program evaluation not requiring IRB approval. Five Pathology-Teaches online modules were introduced into the following clinical clerkships at the following times: Family Medicine (July 2018); Emergency Medicine (November 2018); Internal Medicine (November 2018); Neurology (November 2018); and OB-GYN (January 2019). In brief, an introductory email was sent to the medical students at the beginning of their rotation, introducing the pilot and the educational opportunity. Students who chose to participate completed the online pretest, case module, and posttest, and were also provided with an opportunity to assess the intervention on the end-of-rotation feedback evaluation. The outcomes were collected through April 2019.

Overall, 203 students completed the Pathology-Teaches pilot from July 2018 to April 2019, including 72 in Family Medicine, 72 in Emergency Medicine, 24 in Internal Medicine, 24 in Neurology, and 11 in OB-GYN. Key assessment outcomes are summarized in the Table. Overall, across all clerkships, posttest scores (mean, 83.5%; SD = 19.0%) were significantly higher than pretest scores (mean, 63.1%; SD = 24.2%). This mean increase of +20.3% in knowledge test outcomes was statistically significant (paired-sample t test, P < .001), and the corresponding Hedge g effect size of 0.93 indicates a large effect. Within clerkships, all mean increases were also statistically significant (all P < .05), with tabulated effect sizes all indicating large effects (Hedge g ranging from 0.77 to 1.14).

Of the 494 students who completed the Pathology-Teaches questions on the end-of-rotation evaluation surveys, 251 provided specific feedback (ie, did not answer “N/A”), with most perceiving value; 38.6% (97 of 251) rated the activity as “extremely valuable” or “very valuable,” and 41.4% (104 of 251) rated as “some/moderate value” (Figure 1). This trend of overall favorable feedback was observed throughout all 5 clerkship rotations (Figure 2). The highest rate of participation was observed in Family Medicine and Emergency Medicine, which both incorporated additional reminders into their clerkships. In addition, the Family Medicine clerkship director emphasized how the cases would help students with medical decision-making, and the Emergency Medicine clerkship director stressed the professionalism aspect of completing assigned tasks.

Qualitative student feedback from the end-of-rotation evaluations included 38 specific comments, of which 45% (17 of 38) were positive; 42% (16 of 38) were constructive in nature, and largely centered on the technical aspects of the module or timing; and 13% (5 of 38) were negative, and questioned the value of such a project. Positive comments included “I liked the brevity but the valuable teaching points”; “The case selected was very helpful and inclusion of the invasive adenocarcinoma was well done”; “It was pretty nice. I liked it. But I liked it being optional”; “I think just scale it up. It was helpful!”; and “…I did find the one case module helpful, but a more longitudinal approach would be ideal.” There were 6 specific requests for more cases. Examples of constructive comments regarding the technical aspects included “make the visuals larger by default, increase the size of the viewing window,” and “streamline the UI [user interface]—some aspects didn't work” and “the module was a bit clunky to navigate. It would be helpful to have a ‘next slide' button.” Other constructive suggestions for improvement also suggested a need for reminders or dedicated, carved-out time (“It should be an in-class activity; would encourage more people to complete it.”) or requests for different information taught (“less statistics, more clinical-based facts.”). The 5 negative comments included “I'm not sure it is needed”; “There is not really a single element that is helpful. It is disjointed and does not make sense, there is very little pathology so am not sure why you are calling it Pathology-Teaches. It is an unnecessary element of the course”; and “I'm not advocating for additional work for students on this extremely busy rotation, but the module was minimal in length and offered very little in terms of applicable or new knowledge.”

We had previously identified a gap in systematically teaching laboratory stewardship as part of HVC within the clinical curriculum at our medical school.¹⁰  Pathology-Teaches was designed to bridge this gap and focused on 2 major themes: appropriate test ordering (knowledge of laboratory test indications, pretest/posttest probability, appropriateness criteria, recognition of unnecessary testing) and interpretation (test specifications, factors that impact the test result, recognizing inaccurate results). These themes align well with previously identified core goals of a medical student laboratory medicine curriculum, as well as an overall initiative to better prepare students to deliver cost-effective, safe health care.^11,12  The Pathology-Teaches cases were designed for medical students in their core clinical clerkships, and represent a synthesis of the expertise from a diverse team of clinician educators.

The Pathology-Teaches cases assisted medical students in understanding HVC laboratory stewardship concepts, and overall were perceived as valuable by the students. At our institution, medical students do not have a required laboratory medicine clinical rotation, and therefore our goal was to increase awareness of the appropriate use of laboratory testing as well as the negative downstream effects of low-value testing. The brevity of the modules was cited as a strength in student comments; however, it also limited the scope of what could be taught. A long-term goal is a longitudinal approach to teaching HVC, and future efforts will explore opportunities to extend this pilot into other aspects of the Undergraduate Medical Education curriculum, perhaps with other health systems science–related topics, such as quality improvement. A longitudinal approach would enable a more in-depth examination of these important topics, and an ideal combination would blend the online modules with small group sessions or other types of face-to-face sessions to reinforce key concepts. This more comprehensive approach to teaching these subjects could then be evaluated with more robust, simulation-type assessments to evaluate behaviors, such as the standardized patient scenarios the Mayo Clinic School of Medicine has developed to assess the student's communication skills to appropriately respond to a patient's request for unnecessary and potentially harmful testing.¹³  In addition, some medical schools with a formal science of health care delivery curriculum have mandated the new National Board of Medical Examiners Health Systems Science Examination, which covers quality improvement, patient safety, and teamwork and collaboration.¹⁴ 

Both radiology and pathology have shared interests in promoting appropriate utilization, and a unique aspect of this project is the partnership with the ACR. A parallel project (Radiology-TEACHES) is designed to teach appropriate use of radiology studies and has been successfully piloted at our institution, with more than 96% of students reporting that the virtual classroom experience was valuable in learning about appropriate utilization (41.18%, some value; 41.18%, very valuable; 14.71%, extremely valuable), and with a statistically significant improvement in overall performance in posttest scores, as compared to the pretest scores.¹⁵  Future extension of these educational efforts could include case studies that combine ordering of both radiology and laboratory studies as part of the overall diagnostic evaluation, or a required resource-stewardship–focused clerkship within the undergraduate medical curriculum with input from both diagnostic specialties, similar to that described by Dr Huang at Oakland University William Beaumont School of Medicine (Rochester, Michigan).¹⁶ 

It is critical to target the early learner in order to change the mindset that leads to overutilization. An encouraging trend is the emergence of student-led initiatives such as the Choosing Wisely Canada Students and Trainees Advocating for Resource Stewardship (STARS) campaign.^17,18  As part of Choosing Wisely Canada, Leon-Carlyle et al¹⁹  have described the incorporation of resource stewardship into their curriculum, resulting in a shift in the mindset from “what test do we order” to “what do we need to know?” In addition to raising awareness of the wide-ranging effects of health care spending on the systems level, it will also be crucial for future physicians to build an understanding of the importance of determining what outcomes are most important to patients, in order to understand patient preferences around testing and screening decisions.²⁰  Preparing students to have these productive conversations with patients that promote value will strengthen the physician-patient bond, promote patient engagement, and highlight the patient-centered aspect of HVC.

The authors would like to acknowledge the American College of Radiology (ACR) for use of their educational platform that supports the educational activity, as well as Michele Huneke, BS, MT(ASCP) (ACR) and William McKinney, BS (Baylor College of Medicine) for excellent technical assistance. The authors would like to acknowledge Thomas Wheeler, MD, Stacey Rose, MD, Sridevi Deveraj, PhD, Daniel Rosen, MD, Meredith Reyes, MD, and Elizabeth Hartwell, MD, for contributing their expertise in reviewing and refining the Pathology-Teaches cases.