Context.—Despite great interest in using whole slide imaging (WSI) in pathology practice and education, few pathology journals have published WSI pertinent to articles within their pages or as supplemental materials.

Objective.—To evaluate whether there is measurable added educational value of including WSI in publications.

Design.—Thirty-seven participants, 16 (43.3%), 15 (40.5%), and 6 (16.2%) junior pathology residents (postgraduate year 1–2), senior pathology residents (postgraduate year 3–4), and board-certified pathologists, respectively, read a sequence of 10 journal articles on a wide range of pathology topics. A randomized subgroup also reviewed the WSI published with the articles. Both groups completed a survey tool assessing recall of text-based content and of image-based material pertinent to the diseases but not present in the fixed published images.

Results.—The group examining WSI had higher performance scores in 72% of image-based questions (36 of 50 questions) as compared with the non-WSI group. As an internal study control, the WSI group had higher performance scores in only 40% of text-based questions (6 of 15 questions). The WSI group had significantly better performance than the non-WSI group for image-based questions compared with text-based questions (P < .05, Fisher exact test).

Conclusion.—Our study provides supporting evidence that WSI offers enhanced value to the learner beyond the text and fixed images selected by the author. We strongly encourage more journals to incorporate WSI into their publications.

The advent of the digital pathology age has been hailed for its many advantages over the “yoked to glass and light microscope” era of the past several hundred years. But in some respects, the pace of adoption or penetration into certain niches has lagged expectations.1  Although digital slides are now routinely used in most educational settings for pathology and histology courses,26  and most current texts include access to an enriched array of fixed magnification digital images, user-controlled, manipulatable whole slide images are still vanishingly rare as components of published pathology scientific studies or textbooks. Almost every pathology meeting featuring case studies includes user-accessible digital slides, acknowledging the added convenience of digital slides and the added value of whole slide examination over fixed, selected images, yet these very same sponsoring organizations have not included digital slides routinely or ever in their publications.7  To our knowledge, only one pathology journal routinely offers authors the option of posting pertinent digital slides relating to their work.8 

“Show me the data!” We can almost hear the editors' rebuffs to any insistence or encouragement to publish digital slides as a routine part of the work they do in obtaining, preparing, and publishing the work of scientists working in the fields that use morphologic, histologic, or immunohistochemical data sets. There are no compelling data documenting added value in publishing whole digital slides. Use statistics from journals that do or have attempted to use whole slide imaging (WSI) have not been made public. The initial trial use in Archives of Pathology & Laboratory Medicine9  has not been followed with any additional article with linked digital slides. Furthermore, although many online libraries exist featuring WSI of many pathologic entities (see, for example, https://digitalpathologyassociation.org/whole-slide-imaging-repository), and even more exist within the confines of various training programs, not a single pathology text has yet been published with a link to any sort of comprehensive slide library featuring digital slides. Presently, those seeking to learn about an entity must rely predominantly on author-selected, fixed-magnification images published within the journal or text, or independently search the available WSI libraries for a suitable matching case example.

This study attempts to begin to answer the question of what additional value WSI offers to the learner beyond the text and fixed images selected by the author. Believing that this value may be reflected in the level of confidence individuals may feel about their knowledge of a subject, and suspecting that the experience of reading about an entity as well as examining a pertinent whole slide may enhance their integrative ability to deal with future similar issues, we undertook this study involving learners at varying professional stages after they had interacted with selected journal articles, either with or without also seeing the WSI.

Study Materials

Ten journal articles, primarily case reports on a variety of anatomic pathology topics, were selected from a journal that publishes pathology articles and offers accompanying digital slides. The articles represented a variety of organ systems and lesion types, with an emphasis on cancers. Digital links to the articles (online journal) and separate links to the WSI were provided to all the participants. For details please see the supplemental digital content at www.archivesofpathology.org in the July 2016 table of contents.

Test Population

Participants were drawn from voluntary responses from 8 US pathology residency training programs. Thirty-one pathologists in training, including junior (postgraduate year 1–2) and senior (postgraduate year 3–4) level residents, and 6 trained pathologists with general surgical pathology background, were recruited to participate in the study from a variety of training programs in the United States. Subjects were randomized into 2 groups based on month of birth (odd or even). One group was instructed to read the journal articles and view only the fixed images published within, whereas the other group was instructed do the same and additionally view the associated WSI for each article. No remuneration was offered to the participants aside from acknowledgment and thanks. Some of the training programs used the exercise as a group project, whereas in others it was an optional activity.

Online Assessment Tool

An online assessment tool (Qualtrix, Provo, Utah; licensed to University of Oklahoma Health Sciences Center, Oklahoma City) was used to query participants about the topics in the journal articles and related diagnostic skills. The tool allows development of a robust variety of queries that can be linked, randomized in terms of presentation, and subjected to rule-based presentation. Validation methods are also built in. Access to this online evaluation tool was available from March 15 to May 31, 2015.

Test Questions and Answers Recording

A total of 65 multiple-choice questions were designed based on the 10 selected articles. Among them, 50 questions were fixed image–based questions intended to assess the ability to integrate that viewed content with other material. The other 15 questions were strict content-recall questions drawn from written material in the articles; these also functioned as the internal control questions. The participants were asked to record the time taken to complete the exercise. Subjective confidence in rendered answers was also recorded for each query. Demographic data were solicited from each participant, and in addition, the tool offered the ability to enter free-text comments about the experience, the articles, or the tool. The preface material in the assessment and in the study instructions provided to participants indicated only that the study was an attempt to better define the educational value of WSI in journal publications.

Assessment/Data Collection

Two sets of questions were prepared based on each article: strict content-recall questions drawn from written or tabular material in the articles, and fixed image–based questions intended to assess recall of WSI viewed material and an ability to integrate that viewed content with other material. One group of questions covering an article not included in the study set was also included as a control. Respondents were also asked to self-attest as to which protocol they followed (WSI or no WSI) and to report an estimate of the amount of time spent on each phase of the study. Free-text feedback was also made available. Participants were informed that the goal of the study was to assess the value of WSI in journal publications. For details please see the supplemental digital content.

Data Analysis and Subgroups

Participants were stratified by whether they viewed the WSI or not, and subgroup analysis was performed for different levels of training. Text-based questions were selected as the internal control questions. Descriptive statistics including frequency and percentages were reported. A Fisher exact test was implemented to test the association between the WSI reading status and answers to questions for the whole group. Statistical significance was defined as a P value <.05.

Thirty-seven participants completed the article review and survey. Sixteen (43.2%) were junior pathology residents (postgraduate year 1–2), 15 (40.5%) were senior pathology residents (postgraduate year 3–4), and 6 (16.2%) were board-eligible or certified pathologists. Nineteen (51%) were male and 18 (49%) were female. On average, the participants reported devoting 2.33 hours to reading the articles, 1.15 hours to study of the WSI, and 45 minutes to completing the survey. The participants born in an odd-numbered month were assigned to the non-WSI group, and the participants born in an even-numbered month were assigned to the WSI group. Both groups were instructed to read the articles and take the survey, but only the WSI group was asked to review the WSI accompanying each article. The characteristics of survey participants in the non-WSI and WSI groups are summarized in Table 1. Distributions of certain sociodemographic characteristics, such as race, age, and ethnicity, were not collected from our survey.

Table 1. 

Characteristics of Survey Participants in Non–Whole Slide Imaging (WSI) Group and WSI Group

Characteristics of Survey Participants in Non–Whole Slide Imaging (WSI) Group and WSI Group
Characteristics of Survey Participants in Non–Whole Slide Imaging (WSI) Group and WSI Group

The rates of correct answers were calculated in both groups for each question, and the group with the higher rate of correct answers for the question was deemed as having better performance. The total better performance counts are summarized in Table 2. The WSI group had higher performance scores in 72% of image-based questions (36 out of 50 questions) as compared with the non-WSI group. As an internal study control, the WSI group had higher performance scores in only 40% of the text-based questions (6 out of 15 questions). The WSI group had significantly better performance than the non-WSI group for image-based questions compared with text-based questions (P = .03, 2-tailed Fisher exact test).

Table 2. 

Better Performance Counts for Image-Based and Context-Based Questions

Better Performance Counts for Image-Based and Context-Based Questions
Better Performance Counts for Image-Based and Context-Based Questions

Each participant's training level appeared to be positively correlated with the test performance. Specifically, for the image-based questions, the rates of correct answers increased according to the level of pathology training. Consistent with the better performance counts presented in Table 2, the rates of correct answers were higher in both training levels when they reviewed WSI (Figure).

Percentage of correct responses for image-based questions. Abbreviation: WSI, whole slide imaging.

Percentage of correct responses for image-based questions. Abbreviation: WSI, whole slide imaging.

Close modal

The free-text responses from participants supported that WSI added significant value to accompanying journal articles. Many expressed an increased sense of confidence as they navigated the slide, adjusted magnification, and mentally compared findings with other cases in their experience. Several felt that the whole slide images were crisper or of better quality than the published fixed images in some of the cases. Some participants felt that WSI was especially valuable for unusual entities, as are often presented in case reports. A few participants also noted relative disadvantages in WSI, such as difficulty loading images with certain browsers, delays in loading images, and difficulty recognizing pathologic features in the absence of annotations (Table 3).

Table 3. 

Free-Text Comments From Participants Answering the Question “What Do You Believe Is the Value of Whole Slide Imaging Accompanying a Publication in Pathology?”

Free-Text Comments From Participants Answering the Question “What Do You Believe Is the Value of Whole Slide Imaging Accompanying a Publication in Pathology?”
Free-Text Comments From Participants Answering the Question “What Do You Believe Is the Value of Whole Slide Imaging Accompanying a Publication in Pathology?”

The past decade has witnessed a rapid shift from conventional light microscopy toward digitally based pathology workflow.1012  Whole slide imaging has already played an important role in research and education, and undoubtedly will be essential in future pathology practice, such as off-site frozen section interpretation, telepathology consultations, and laboratory quality assurance.1317  Even with its stunning success, however, the general adoption of WSI in the pathology community has been slow. It has been proposed that in the future, pathology textbooks and journal articles will incorporate links to WSI,18  but mainstream journals appear to still be puzzled about or unable to cost justify any potential benefits of adding this capability to the publication.

To the best of our knowledge, our study provides the very first quantitative evidence supporting the added value of having WSI in journal articles. The present multi-institutional survey demonstrates that within the same context, the group of residents and pathologists viewing WSI materials had significantly better performance than the group that didn't view WSI for image-based questions compared with internal control questions. It should be noted that our non-WSI group had a slightly higher percentage of senior pathology residents and board-certified pathologists than the WSI group (Table 1). Given the positive correlation between training level and test performance (Figure), the added value of WSI might still be underestimated in the current study. On average, participants in both groups spent approximately the same amount of time reading the articles, and the WSI group spent extra efforts studying the online imaging materials. Although this could be raised as a confounding factor, it might also be argued that the higher total time spent studying a topic was facilitated by the added interest and interaction offered by the WSI, and this led to better performance on the visual, integrative component of the survey. Certainly this trend could be a useful avenue for further investigation as to which kinds of articles should include a WSI component.

Among 8 participating pathology residency training programs, 2 programs used the exercise as a group project, whereas other programs used it as an optional activity. For the group project, each participant performed the study and test independently, while the group was in the same classroom within a similar block of time. The influence of being in a group or individual setting on learning was outside the scope of this study.

A limitation of the current study is the relatively small number of participants. We found better performance in the WSI group for image-based questions (P = .03; Table 2). We also consistently observed added value of WSI in different training groups, and the quantitative improvement ranges from 3.6% in the junior pathology residents subgroup (63.0% versus 60.8%) to 8.8% in the senior pathology residents and pathologists subgroup (68.3% versus 62.8%) (Figure). However, because of small sample size, WSI-related added values approached but did not reach statistical significance in both subgroups. A large-scale study will therefore be helpful to achieve a generalized consensus on this matter.

Our survey also provided the opportunity for each participant to give feedback on the value of WSI in journal articles. Most WSI group participants gave positive comments on their experience. Favored features supporting the use of WSI included the ability to fully evaluate a specimen by choosing different fields and by switching among different magnifications, which allowed the reader to decide if the presented figures were truly representative; WSI also helped readers have a better understanding of the journal articles because it gave a broader visual representation of the words in the articles. The negative comments were mainly focused on the slow process experienced while using the software.

Our study showed encouraging evidence for the added value of WSI in journal articles. Our participants agreed that viewing WSI when reading the articles enhanced the overall educational experience. Whole slide imaging also provides mobility and convenience to readers. With the adoption of other techniques such as quick-response bar codes, readers will be able to use mobile devices to access the URL for the WSI anywhere and anytime. All these advantages will greatly boost the value of WSI-enabled journal-based education in the future.

The authors sincerely thank the participating programs and individuals, including Adrienne Lawton, MD; Christopher Jones, MD; Jack Wilson, MD; Shanedelle Norford, MD; Aravindhan Sriharan, MD; Omid Rouhi, MD, PhD; Pamela Ferreira, MD; Daniel Sanchez, MD; Ha Nguyen, MD; Tania Mendoza, MD; Robert Seifert, MD; Sherma Zibadi, MD, PhD; Rania Shamekh, MD, MS; Diana Braswell, MD; Carolina Dominguez Marquez, MD; Cory Porteus, DO; Michael Black, MD; Janet McNaughton, MD; Victor Zota, MD; and Stacey Simons, MD. We thank Anne Champeaux, MD, the program director of University of South Florida, Tampa, for her support. We also acknowledge, thank, and apologize to those whose names are not included here because of the anonymous nature of the study.

1
Hassell
LA,
Glassy
E.
The (not yet) willingly adopted tool
.
J Pathol Inform
.
2013
;
4
:
13
.
2
Hamilton
PW,
Wang
Y,
McCullough
SJ.
Virtual microscopy and digital pathology in training and education
.
APMIS
.
2012
;
120
(
4
):
305
315
.
3
Triola
MM,
Holloway
WJ.
Enhanced virtual microscopy for collaborative education
.
BMC Med Educ
.
2011
;
11
:
4
.
4
Dee
FR.
Virtual microscopy in pathology education
.
Hum Pathol
.
2009
;
40
(
8
):
1112
1121
.
5
Pantanowitz
L,
Szymas
J,
Yagi
Y,
Wilbur
D.
Whole slide imaging for educational purposes
.
J Pathol Inform
.
2012
;
3
:
46
.
6
Hassell
LA,
Fung
KM,
Chaser
B.
Digital slides and ACGME resident competencies in anatomic pathology: an altered paradigm for acquisition and assessment
.
J Pathol Inform
.
2011
;
2
:
27
.
7
Cagle
PT,
Glassy
EF.
Whole slide images add value to journal article figures
.
Arch Pathol Lab Med
.
2014
;
138
(
5
):
592
.
8
Kayser
K.
Diagnostic pathology in 2012: development of digital pathology in an open access journal
.
Diagn Pathol
.
2013
;
8
:
3
.
9
Hwang
DH,
Szeto
DP,
Perry
AS,
Bruce
JL,
Sholl
LM.
Pulmonary large cell carcinoma lacking squamous differentiation is clinicopathologically indistinguishable from solid subtype adenocarcinoma
.
Arch Pathol Lab Med
.
2014
;
138
(
5
):
626
635
.
10
Ghaznavi
F,
Evans
A,
Madabhushi
A,
Feldman
M.
Digital imaging in pathology: whole-slide imaging and beyond
.
Annu Rev Pathol
.
2013
;
8
:
331
359
.
11
Arnold
MA,
Chenever
E,
Baker
PB,
et al.
The College of American Pathologists guidelines for whole slide imaging validation are feasible for pediatric pathology: a pediatric pathology practice experience
.
Pediatr Dev Pathol
.
2015
;
18
(
2
):
109
116
.
12
Allen
TC.
Digital pathology and federalism
.
Arch Pathol Lab Med
.
2014
;
138
(
2
):
162
165
.
13
Weinstein
RS,
Graham
AR,
Richter
LC,
et al.
Overview of telepathology, virtual microscopy, and whole slide imaging: prospects for the future
.
Hum Pathol
.
2009
;
40
(
8
):
1057
1069
.
14
Isaacs
M,
Lennerz
JK,
Yates
S,
Clermont
W,
Rossi
J,
Pfeifer
JD.
Implementation of whole slide imaging in surgical pathology: a value added approach
.
J Pathol Inform
.
2011
;
2
:
39
.
15
Wilbur
DC,
Madi
K,
Colvin
RB,
et al.
Whole-slide imaging digital pathology as a platform for teleconsultation: a pilot study using paired subspecialist correlations
.
Arch Pathol Lab Med
.
2009
;
133
(
12
):
1949
1953
.
16
Ho
J,
Parwani
AV,
Jukic
DM,
Yagi
Y,
Anthony
L,
Gilbertson
JR.
Use of whole slide imaging in surgical pathology quality assurance: design and pilot validation studies
.
Hum Pathol
.
2006
;
37
(
3
):
322
331
.
17
Rocha
R,
Vassallo
J,
Soares
F,
Miller
K,
Gobbi
H.
Digital slides: present status of a tool for consultation, teaching, and quality control in pathology
.
Pathol Res Pract
.
2009
;
205
(
11
):
735
741
.
18
Glassy
EF.
Rebooting the pathology journal: learning in the age of digital pathology
.
Arch Pathol Lab Med
.
2014
;
138
(
6
):
728
729
.

Author notes

The authors have no relevant financial interest in the products or companies described in this article.

Supplemental digital content is available for this article at www.archivesofpathology.org in the July 2016 table of contents.

Competing Interests

Presented at the Pathology Visions 2015 Conference; October 13, 2015; Boston, Massachusetts.

Supplementary data