Virtual Pathology Elective, Real Education: The PathElective.com Experience as a Model for Novel Pathology Pedagogy and a Primer for Curricular Evolution

PathElective.com was developed using Squarespace (Squarespace, Inc). All traffic data, including page views, visits, unique visitors, and geographic location, were obtained through the internal functionality provided by Squarespace.⁷  Course content in areas of anatomic pathology and clinical pathology was designed, composed, and curated by a course coordinator, typically a pathologist or experienced medical laboratory professional. The course content was input into the website by the website developer and reviewed by the course director before publishing. Content was protected using GoPayWall (GoPaywall), a subscription service that allows subscribers to access the curated educational content. Subscription to PathElective is free, but the free sign-in protection on content was developed to ensure users had a valid email to which their certificates would be emailed and to make sure students agreed to our website's terms and conditions and consented to the use of their quiz data being used in research. Students and trainees from around the globe can sign up to be a member of PathElective and view the educational content on the website for free.

Students were assessed before and after interacting with course materials using a dual-form crossover quiz design on Google Forms (Google LLC). The dual-form crossover quiz design was chosen as the means of assessing the assimilation of new knowledge and has been favored as a means of assessing students in standardized test settings as well.⁸  Randomization between the forms was accomplished using the user's birth year (Figure 1, A). The different forms, though they contained different questions, assessed the same content areas and attempted to assess similar levels of questioning (Supplemental Table 1; see supplemental digital content containing 2 tables and 1 figure at https://meridian.allenpress.com/aplm in the May 2024 table of contents). Questions were assessed for difficulty by 2 reviewers before the quiz was launched. Additionally, quiz data were analyzed via paired t test after the study period to ensure the average score for each of the forms did not differ significantly from one another. Summary statistics were also collected between forms to ensure no statistical differences were present between forms. Prelesson and postlesson quiz data were paired and anonymized, and percentage of improvement was determined using a paired t test. A paired t test was selected to correct for variations in starting score due to the diversity in student backgrounds. These data were organized into a forest plot with the null hypothesis set to a 0% improvement. The 95th percentile was calculated and plotted on the forest plot to visualize statistical significance using Prism GraphPad by Dotmatics.

Upon completion of the postlesson quiz, a satisfaction survey was completed by students (Figure 1, B). The satisfaction survey included a Likert-scale assessment of (1) overall lesson quality, (2) lecturer quality, (3) social media engagement, (4) lecture quality, and (5) satisfaction with the online format. In addition to Likert-scale data collection, voluntary written feedback was elicited at the end of the survey. Likert-scale satisfaction data were analyzed using the calculation of descriptive statistics for each category for each lesson. Though the satisfaction data are ordinal, means were used in the analysis since all courses had similar, if not identical, median analyses that were not able to distinguish the subtle differences in satisfaction across courses and categories. These means were visualized using Microsoft Excel (Microsoft Corporation). Means were compared visually based on a scale of total possible responses (1–5) to assess overall student satisfaction, against the mean across all categories (cross-category) to assess categories that need improvement, and against the mean for each category to assess courses that need improvement (Supplemental Table 2).

Based on the variety in course layouts provided by course directors, some courses contained a single lesson, while others contained multiple lessons. For courses composed of multiple lessons that were all released within a week of one another, a novel analysis was performed aimed at examining the attrition rate of students as more modules are added to an online course (Figure 1, C). This analysis was developed to assess the level of participation, measured by number of students who took the postlesson quiz, in each lesson within a course and to determine whether or not students tended to complete a course with multiple lessons, and if so, to what extent. The number of students who engaged with each lesson was recorded during the first 18 months. The percentage of change in student participation between lessons was determined and aggregated for each of the courses. The average and 95% CI were determined for each of the lesson additions across all courses. The data were plotted and statistically analyzed with polynomial regression using Prism GraphPad. The degree of polynomial regression and line of best fit was determined using the extra sum of squares f-test with an alpha set to .05, which was also calculated using Prism GraphPad. In addition to polynomial regression analysis, attrition was also visualized and plotted using violin-plot analysis to depict the entire spread of the data.

This protocol, as well as the consent procedures, were reviewed by the Loyola University Chicago Institutional Review Board (Maywood, Illinois) and was determined to be exempt (LU#215076).

During the study period (May 1, 2020 to December 31, 2021), PathElective.com received 622 681 page views (number of times a page within the website was loaded including internal links), 139 081 visits (number of times the page was loaded by a visitor coming from an external site), 67 670 unique visitors, and 10 278 registered users who earned 15 305 certificates. Sixty-eight and one half perfect (68.5%) of visitors (95 300/139 081) arrived at the site via a direct URL input, 17.2% (24 862/139 081) through a search engine, and 10.9% through social media (16 361/139 081). Of the social media channels, most visitors came from X (formerly known as Twitter [X Corp.]) (6.7%, 9286/139 081) followed by Facebook (Meta Platforms, Inc.) (3.5%, 4927/139 081), Instagram (Meta Platforms, Inc.) (0.8%, 1152/139 081), YouTube (Google LLC) (0.3%, 487/139 081), LinkedIn (Microsoft Corporation) (0.2%, 280/139 081), and Reddit (Advance Publications) (0.1%, 205/139 081). Most visits (44.4%, 61 740/139 081) were from the United States of America. The top 4 high-traffic states in the United States were Illinois (5701 visits), with Chicago (3050) hosting a majority of visits; Texas (5069), with Houston (1646) hosting a majority of visits; California (4321), with Los Angeles (670) hosting a majority of visits; and New York (4303), with New York City (955) hosting a majority of visits. After the United States, India was the second most popular country (10.8%, 14 901/139 081) from which PathElective was accessed, mostly in Tamil Nadu (1939), Karnataka (1829), Delhi (1694), and Maharashtra (1473), all exhibiting a similar level of website traffic.

Student performance was assessed as above and organized in Figure 2. In total, 7609 paired assessments were analyzed, with 6150 assessments originating from anatomic pathology (AP) courses and 1756 originating from clinical pathology (CP) courses. The course with the single highest number of student participants was lesson 1 of Gross Pathology, with 563 participants. The CP course with the highest level of participation was Hematology, with 323 participants. On average, student performance on course assessments improved by 13.8% (95% CI, 10.2%–17.5%). Separately, AP courses experienced a 14% (95% CI, 10.5%–17.5%) average improvement in score, while CP courses experienced a 13.2% (95% CI, 9.2%–17.5%) average improvement in score. All courses experienced a statistically significant increase in student performance except Dermatopathology: lesson 40 (n = 80), Cytopathology: lesson 3 (n = 75), and Cardiac Pathology: lessons 2 and 3 (n = 82 and 75, respectively). The course with the highest improvement in student performance was Gross Pathology: lesson 4, with an improvement of 30.2% (95% CI, 27.2%–33.1%) (n = 323). Courses with significantly higher than average performance were Autopsy: lesson 1; Gross Pathology: lessons 3, 4, and 6; and Hematopathology: lesson 1. Courses with significantly lower than average performance were Cardiac Pathology: lessons 1, 2, and 3; Cytopathology: lesson 3; Dermatopathology: lessons 1, 2, 4, and 5; and Gross Pathology: lessons 1 and 5.

Satisfaction data obtained after completion of each lesson quiz assessed overall quality of the lesson, quality of the lecturer, level of social media engagement, qualities of the constituent lectures, ability to get help when needed, and satisfaction with the online format. When taken together, the aggregate average of all quality metrics across all lessons was 4.46 (out of 5). The averages of each category for every lesson were organized into Supplemental Table 2. All courses received, on average, above a 3.88, with a range of 3.88 to 4.72 (Figure 3, A). When comparing all categories and courses to the aggregate average, it is evident that the category with the lowest average scores across all courses was social media engagement (Figure 3, B). When comparing each of the courses for each satisfaction category, courses with higher than average satisfaction metrics included Dermatopathology, Gross Pathology, Neuropathology, Illustrations of Histology, and Hematology (Figure 3, C). Courses with lower than average satisfaction metrics included Breast Pathology and Cardiac Pathology. Other courses that had lessons with some lessons falling higher and others falling lower than average according to satisfaction data included Cytopathology, Hematopathology, Clinical Microbiology, and Molecular Microbiology. It is important to mention that the differences between courses that were above average and below average were within 0.3 to 0.59 points. However, these small differences were noted to help improve course layouts, lectures, etc. to improve the learner experience. Additionally, written student feedback was relayed to the course director and was typically used to make certain topics clearer and provide additional resources.

For courses with more than 1 lesson (n = 12), student retention/attrition data were collected and analyzed. For the nonlinear regression of the mean taking into account standard deviation and sample size, the second-order polynomial function was significantly better than the first (P = .002), but the third-order polynomial was not significantly better than the second order (P = .21). The unique polynomial relationship with each added lesson highlights the precipitous drop during the transition between lessons 1, 2, and 3. After lesson 3, the drop in student participation was less precipitous. The latter half of the data set was limited by sample size, so the projection of the regression is less reliable. The equation yielded from the polynomial regression analysis was Y = −1.643 − 26.19(x) + 2.838 (x)² , where x is equal to lesson number of lessons after the first lesson (ie, lesson no. − 1). The R² for the polynomial regression was found to be 0.656 (Figure 4, A and B).

Data harnessed from the PathElective.com module usage and satisfaction surveys provide a powerful and unique perspective that can help guide pedagogical approaches in medical education. Keeping a finger on the pulse of what students voluntarily choose to learn from, which modules are most effective, and which teaching frameworks lead to the highest increases in postmodule student scores can provide valuable insight that can be harnessed to guide postpandemic hybrid learning and medical school curricular reform. Data from the geographic origin of the students, satisfaction scores, and attrition rates in light of performance characteristics provide directed data for the PathElective.com site to adjust and refocus modules that are rated lower or wherein students are not performing as well as in others.

When analyzing the geographic distribution of students accessing PathElective.com, it is no surprise that the 2 major foci of interest were the United States and India, given the initial interest reported in Lilley et al 2021.⁶  However, when analyzing the geographic distribution of students within the top 4 high-traffic states (Illinois, Texas, New York, and California), the hypothesis that interest would be centered around medical schools and residency programs remained only partially true. Interestingly, the geographic distribution of students accessing PathElective.com in Illinois, Texas, and New York exhibited a scattering of access points across the states, outside of the major cities that house large medical centers and medical schools. This finding could be due, in part, to students learning remotely during the height of the COVID-19 pandemic but could also support the claim that PathElective.com is being accessed by a broader audience than those in medical school or residency—the audiences that predominated the platform's user database in earlier studies.⁶  Additionally, website traffic data support the hypothesis that a registration requirement decreases the number of people who access the website's educational content since, out of the 59 928 unique visitors, only 10 278 registered to access the educational content behind the registration wall on PathElective.

The ability to assess 7609 paired assessments allows us a powerful lens to evaluate modules and focus energy toward those modules that are not as effective as others. Overall, the paired assessment data show most improvement linked to those modules wherein information would be new for students. Autopsy, gross pathology, and basics of hematology are all subspecialized topics typically not covered in detail during medical school education. Autopsy education is always of great interest to students given its presence in popular media and the intrigue of being a detective of disease after death. This likely drives the interest, engagement, and improvement in scores for that module. Similarly, the importance of understanding of gross pathology for trainees in pathology, surgery, and pathologists' assistant education drives heightened interest in that module. These modules also utilized variable pedagogical methods (multimedia and interactive), which suggests that more engaging content could play a role in online medical education. While we did not systematically examine course content, pedagogy, and the difficulty level for specific questions, doing so in the future will help dissect these data even further.

Overall, usage of the website resulted in high satisfaction scores across all categories except social media engagement. The categories of overall PathElective.com experience, quality of the website, and functionality were generally scored as very satisfied or satisfied. This positive skew in satisfaction data was found to exist across all questions assessed. In fact, the average for each category was above 4 out of 5. The weakest category, however, was social media engagement. The reasons for this are probably multifold. On the site, we have encouraged students to interact with their course instructors and other PathElective users on social media. The idea behind this is to establish a feeling of a shared experience, even if virtual. This is aided by the fact that all our faculty members have an active social media presence. While we emphasize this point to the trainee, the onus is on them to make that interaction a reality. It is difficult for our site faculty to identify, keep track of, and initiate social media interactions with all PathElective.com users. Many students also feel inertia or hesitation in initiation of these interactions as well.⁹  Additionally, it is important to consider the range of scores being analyzed. For the social media category, the range of averages across all courses was between 3.88 and 4.46 and the range of all course averages was between 3.88 and 4.75 (Supplemental Table 2). Though social media performed below average when compared to other categories, the average scores were still above the “neutral” choice of 3 out of 5. Taken together, our emphasis on how useful social media can be for their overall experience, but lack of a concrete follow-through on how to make that happen, is likely the underlying reason for the slightly lower satisfaction rating. These data allow us to consider modifications in strategies of engagement moving forward, which could include faculty having “virtual office hours” via X or Instagram Live, or even using voice-based platforms such as X Spaces or ClubHouse (Alpha Exploration Co), which have been shown to have added utility in such settings, for example, in their use during journal clubs.¹⁰  There were no correlations found between any of the assessed satisfaction domains and average score improvement (Supplemental Figure 1).

Our novel attrition analysis offers a first-of-its-kind look into student participation and behavior in online courses with multiple lessons. As hypothesized, our data support the claim that students tending to terminate participation in online classes correlates with the increasing number of lessons; however, with our limited data set, the prediction of student retention past 4 lessons is less reliable. Additionally, most students tend to start at lesson 1 and work their way through the course even though the courses offer a modular, adaptable experience. The exception to the latter claim was seen in gastrointestinal pathology and cardiac pathology. Interestingly, there were no similarities between these 2 courses that would explain this difference, but in the gastrointestinal pathology course, each lesson takes an organ-based approach to teach its system's pathologies, and in the cardiac pathology course, the first lesson covered basic histology, which seems to have been skipped by a number of students. The trend of skipping the introductory module was not replicated in other courses with introductory lessons (ie, Dermatopathology, Head And Neck Pathology, Pulmonary Pathology, and Breast Pathology). Taken together, the regression plotting the mean and 95% CI, and the violin plots visualizing data distribution, median, and interquartile ranges give a holistic view of student activity in the voluntary, self-paced, online courses with multiple lessons in one course.

PathElective.com has proven to be an effective educational resource for medical students, residents, fellows, and physicians across their training journey that has been well received. These data support the use of PathElective in a wide range of in-person and virtual educational settings as an adjunct to their curriculum. Future research will focus on the utility of PathElective in select educational settings and in certain student populations as well as on the effect of different teaching styles on student performance outcomes. Further research also needs to be done to investigate the equitability and inclusivity of PathElective across the United States and globally to assess the ability to access the platform and any barriers there are that typically prohibit students from pursuing further education in pathology and medicine more broadly.

Members of the PathElective team who have contributed significantly to the content creation or curation, development, and/or design of the website but not to the construction of this paper include Aadil Ahmed, MD; Vijayalakshmi “Viju” Ananthanarayanan, MD, MBBS; Natalie Banet, MD; Andrew M. Bellizzi, MD; Narendra Bhattarai, MD; Matthew Cecchini, MD; Joe Chaffin, MD; Bonnie Choy, MD; Nicole A. Cipriani, MD; Amy Deeken, MD; Christopher Demas, MD; Kyle Devins, MD; David J. Escobar, MD, PhD; Lucy Fu, MD; Sarah Garner, MS; Lorenzo Gitto, MD; Isabel Gómez, MD; Brittany Grantham, MS; Emily Hagen, BS; Daniela Hermelin, MD; My-Linh Ho, MD; Daniel Johnson, MD; Constantine E. Kanakis, MD, MSc; Walter Kemp, MD, PhD; Azmer Khan; Nigar Khurram, MD; Giselle López, MD; Nolan Maloney, MD; Maximo J. Marin, MD; Patrick J. McIntire, MD; Sanjay Mukhopadhyay, MD; Cory Nash, MS; Megan Parilla, MD; Ami Patel, MD; Maryam Pezouh, MD; Raghavindra Pillappa, MD; Meredith Pittman, MD; Sameera Rashid, MD; Dana Razzano, MD; Kara Roncin, MD; Jaclyn Rudzinski, MD; Katherine Saunders, DO; Teresa Scordino, MD; Anurag Sharma, MD; Alexis Snyder, MHS; Michael Swete, DO; Cody Thomas, MD; Emily Towery, MD; Anna Trzcinska, DMD; Imran Uraizee, MD; Kartik Viswanathan, MD, PhD; Kristy A. Waite, DO; Michael Williams, MD, MS; Sara E. Wobker, MD, MPH; Eva M. Wojcik, MD; Joseph R. Wiencek, PhD; Kristy Wolniak, MD, PhD; and Brandon Zelman, DO.