Educational Value of Digital Whole Slides Accompanying Published Online Pathology Journal Articles: A Multi-Institutional Study

Ten journal articles, primarily case reports on a variety of anatomic pathology topics, were selected from a journal that publishes pathology articles and offers accompanying digital slides. The articles represented a variety of organ systems and lesion types, with an emphasis on cancers. Digital links to the articles (online journal) and separate links to the WSI were provided to all the participants. For details please see the supplemental digital content at www.archivesofpathology.org in the July 2016 table of contents.

Participants were drawn from voluntary responses from 8 US pathology residency training programs. Thirty-one pathologists in training, including junior (postgraduate year 1–2) and senior (postgraduate year 3–4) level residents, and 6 trained pathologists with general surgical pathology background, were recruited to participate in the study from a variety of training programs in the United States. Subjects were randomized into 2 groups based on month of birth (odd or even). One group was instructed to read the journal articles and view only the fixed images published within, whereas the other group was instructed do the same and additionally view the associated WSI for each article. No remuneration was offered to the participants aside from acknowledgment and thanks. Some of the training programs used the exercise as a group project, whereas in others it was an optional activity.

An online assessment tool (Qualtrix, Provo, Utah; licensed to University of Oklahoma Health Sciences Center, Oklahoma City) was used to query participants about the topics in the journal articles and related diagnostic skills. The tool allows development of a robust variety of queries that can be linked, randomized in terms of presentation, and subjected to rule-based presentation. Validation methods are also built in. Access to this online evaluation tool was available from March 15 to May 31, 2015.

A total of 65 multiple-choice questions were designed based on the 10 selected articles. Among them, 50 questions were fixed image–based questions intended to assess the ability to integrate that viewed content with other material. The other 15 questions were strict content-recall questions drawn from written material in the articles; these also functioned as the internal control questions. The participants were asked to record the time taken to complete the exercise. Subjective confidence in rendered answers was also recorded for each query. Demographic data were solicited from each participant, and in addition, the tool offered the ability to enter free-text comments about the experience, the articles, or the tool. The preface material in the assessment and in the study instructions provided to participants indicated only that the study was an attempt to better define the educational value of WSI in journal publications.

Two sets of questions were prepared based on each article: strict content-recall questions drawn from written or tabular material in the articles, and fixed image–based questions intended to assess recall of WSI viewed material and an ability to integrate that viewed content with other material. One group of questions covering an article not included in the study set was also included as a control. Respondents were also asked to self-attest as to which protocol they followed (WSI or no WSI) and to report an estimate of the amount of time spent on each phase of the study. Free-text feedback was also made available. Participants were informed that the goal of the study was to assess the value of WSI in journal publications. For details please see the supplemental digital content.

Participants were stratified by whether they viewed the WSI or not, and subgroup analysis was performed for different levels of training. Text-based questions were selected as the internal control questions. Descriptive statistics including frequency and percentages were reported. A Fisher exact test was implemented to test the association between the WSI reading status and answers to questions for the whole group. Statistical significance was defined as a P value <.05.

Thirty-seven participants completed the article review and survey. Sixteen (43.2%) were junior pathology residents (postgraduate year 1–2), 15 (40.5%) were senior pathology residents (postgraduate year 3–4), and 6 (16.2%) were board-eligible or certified pathologists. Nineteen (51%) were male and 18 (49%) were female. On average, the participants reported devoting 2.33 hours to reading the articles, 1.15 hours to study of the WSI, and 45 minutes to completing the survey. The participants born in an odd-numbered month were assigned to the non-WSI group, and the participants born in an even-numbered month were assigned to the WSI group. Both groups were instructed to read the articles and take the survey, but only the WSI group was asked to review the WSI accompanying each article. The characteristics of survey participants in the non-WSI and WSI groups are summarized in Table 1. Distributions of certain sociodemographic characteristics, such as race, age, and ethnicity, were not collected from our survey.

The rates of correct answers were calculated in both groups for each question, and the group with the higher rate of correct answers for the question was deemed as having better performance. The total better performance counts are summarized in Table 2. The WSI group had higher performance scores in 72% of image-based questions (36 out of 50 questions) as compared with the non-WSI group. As an internal study control, the WSI group had higher performance scores in only 40% of the text-based questions (6 out of 15 questions). The WSI group had significantly better performance than the non-WSI group for image-based questions compared with text-based questions (P = .03, 2-tailed Fisher exact test).

Each participant's training level appeared to be positively correlated with the test performance. Specifically, for the image-based questions, the rates of correct answers increased according to the level of pathology training. Consistent with the better performance counts presented in Table 2, the rates of correct answers were higher in both training levels when they reviewed WSI (Figure).

The free-text responses from participants supported that WSI added significant value to accompanying journal articles. Many expressed an increased sense of confidence as they navigated the slide, adjusted magnification, and mentally compared findings with other cases in their experience. Several felt that the whole slide images were crisper or of better quality than the published fixed images in some of the cases. Some participants felt that WSI was especially valuable for unusual entities, as are often presented in case reports. A few participants also noted relative disadvantages in WSI, such as difficulty loading images with certain browsers, delays in loading images, and difficulty recognizing pathologic features in the absence of annotations (Table 3).

The past decade has witnessed a rapid shift from conventional light microscopy toward digitally based pathology workflow.^10–12  Whole slide imaging has already played an important role in research and education, and undoubtedly will be essential in future pathology practice, such as off-site frozen section interpretation, telepathology consultations, and laboratory quality assurance.^13–17  Even with its stunning success, however, the general adoption of WSI in the pathology community has been slow. It has been proposed that in the future, pathology textbooks and journal articles will incorporate links to WSI,¹⁸  but mainstream journals appear to still be puzzled about or unable to cost justify any potential benefits of adding this capability to the publication.

To the best of our knowledge, our study provides the very first quantitative evidence supporting the added value of having WSI in journal articles. The present multi-institutional survey demonstrates that within the same context, the group of residents and pathologists viewing WSI materials had significantly better performance than the group that didn't view WSI for image-based questions compared with internal control questions. It should be noted that our non-WSI group had a slightly higher percentage of senior pathology residents and board-certified pathologists than the WSI group (Table 1). Given the positive correlation between training level and test performance (Figure), the added value of WSI might still be underestimated in the current study. On average, participants in both groups spent approximately the same amount of time reading the articles, and the WSI group spent extra efforts studying the online imaging materials. Although this could be raised as a confounding factor, it might also be argued that the higher total time spent studying a topic was facilitated by the added interest and interaction offered by the WSI, and this led to better performance on the visual, integrative component of the survey. Certainly this trend could be a useful avenue for further investigation as to which kinds of articles should include a WSI component.

Among 8 participating pathology residency training programs, 2 programs used the exercise as a group project, whereas other programs used it as an optional activity. For the group project, each participant performed the study and test independently, while the group was in the same classroom within a similar block of time. The influence of being in a group or individual setting on learning was outside the scope of this study.

A limitation of the current study is the relatively small number of participants. We found better performance in the WSI group for image-based questions (P = .03; Table 2). We also consistently observed added value of WSI in different training groups, and the quantitative improvement ranges from 3.6% in the junior pathology residents subgroup (63.0% versus 60.8%) to 8.8% in the senior pathology residents and pathologists subgroup (68.3% versus 62.8%) (Figure). However, because of small sample size, WSI-related added values approached but did not reach statistical significance in both subgroups. A large-scale study will therefore be helpful to achieve a generalized consensus on this matter.

Our survey also provided the opportunity for each participant to give feedback on the value of WSI in journal articles. Most WSI group participants gave positive comments on their experience. Favored features supporting the use of WSI included the ability to fully evaluate a specimen by choosing different fields and by switching among different magnifications, which allowed the reader to decide if the presented figures were truly representative; WSI also helped readers have a better understanding of the journal articles because it gave a broader visual representation of the words in the articles. The negative comments were mainly focused on the slow process experienced while using the software.

Our study showed encouraging evidence for the added value of WSI in journal articles. Our participants agreed that viewing WSI when reading the articles enhanced the overall educational experience. Whole slide imaging also provides mobility and convenience to readers. With the adoption of other techniques such as quick-response bar codes, readers will be able to use mobile devices to access the URL for the WSI anywhere and anytime. All these advantages will greatly boost the value of WSI-enabled journal-based education in the future.

The authors sincerely thank the participating programs and individuals, including Adrienne Lawton, MD; Christopher Jones, MD; Jack Wilson, MD; Shanedelle Norford, MD; Aravindhan Sriharan, MD; Omid Rouhi, MD, PhD; Pamela Ferreira, MD; Daniel Sanchez, MD; Ha Nguyen, MD; Tania Mendoza, MD; Robert Seifert, MD; Sherma Zibadi, MD, PhD; Rania Shamekh, MD, MS; Diana Braswell, MD; Carolina Dominguez Marquez, MD; Cory Porteus, DO; Michael Black, MD; Janet McNaughton, MD; Victor Zota, MD; and Stacey Simons, MD. We thank Anne Champeaux, MD, the program director of University of South Florida, Tampa, for her support. We also acknowledge, thank, and apologize to those whose names are not included here because of the anonymous nature of the study.