Whole slide imaging (WSI) offers a convenient, tractable platform for measuring features of routine and special-stain histology or in immunohistochemistry staining by using digital image analysis (IA). We now routinely use IA for quantitative and qualitative analysis of theranostic markers such as human epidermal growth factor 2 (HER2/neu), estrogen and progesterone receptors, and Ki-67. Quantitative IA requires extensive validation, however, and may not always be the best approach, with pancreatic neuroendocrine tumors being one example in which a semiautomated approach may be preferable for patient care. We find that IA has great utility for objective assessment of gastrointestinal tract dysplasia, microvessel density in hepatocellular carcinoma, hepatic fibrosis and steatosis, renal fibrosis, and general quality analysis/quality control, although the applications of these to daily practice are still in development. Collaborations with bioinformatics specialists have explored novel applications to gliomas, including in silico approaches for mining histologic data and correlating with molecular and radiologic findings. We and many others are using WSI for rapid, remote-access slide reviews (telepathology), though technical factors currently limit its utility for routine, high-volume diagnostics. In our experience, the greatest current practical impact of WSI lies in facilitating long-term storage and retrieval of images while obviating the need to keep slides on site. Once the existing barriers of capital cost, validation, operator training, software design, and storage/back-up concerns are overcome, these technologies appear destined to be a cornerstone of precision medicine and personalized patient care, and to become a routine part of pathology practice.
Pathologists make crucial diagnoses and issue reports that directly affect patient care, primarily using microscopes and hematoxylin-eosin (H&E)–stained tissue with techniques that have not changed since the 19th century1,2 and that are likely to remain for decades to come with supplements from the advancing technology. Developing techniques such as immunohistochemistry (IHC) and molecular diagnostics/theranostics have brought up the necessity and ability to measure discrete molecules or “biomarkers” within tissue; however, there are a number of analytical variables that can affect the results of these tests, including the accurate measurement of signals.3,4 Given this inherent variability, there have been efforts to standardize these tests, and the US Food and Drug Administration, American Society of Clinical Oncology, and College of American Pathologists (CAP) have each provided special direction on some testing (eg, human epidermal growth factor 2 [HER2/neu]).5,6 Whole slide imaging (WSI) with algorithmic analysis is one potential technology to improve the assessment of such markers (as shown in the Table). This has ranged from studies on basic H&E slides to IHC to multiparametric quantum dot staining.
Abbreviations: ER, estrogen receptor; IA, image analysis; HCC, hepatocellular carcinoma; HER2/neu, human epidermal growth factor 2; HLA-DR, human leukocyte antigen–antigen D related; IHC, immunohistochemistry; MVD, microvessel density; PR, progesterone receptor; WSI, whole slide imaging.
Genomic Health, Inc, Redwood City, California.
Currently, the most widely used application of WSI is arguably “telepathology,” that is, sharing of images with remote computers. This has enabled immediate access of pathologic evaluation from a remote observer. In addition to bringing expertise regarding tissue that may be located thousands of miles away for diagnostic purposes, it is gradually revolutionizing pathology education and pathology research as well. In this article, we will discuss the use of WSI in image analysis (IA), telepathology, and some of the challenges that can be encountered in the utilization of this technology.
DAILY APPLICATIONS OF WHOLE SLIDE IMAGING ALREADY IN USE
The simple viewing of WSIs provides an application that can be used in a wide variety of settings. Whole slide images are used regularly for educational purposes. For example, slides are scanned for our departmental “Unknown Conferences” that are geared toward residents and other trainees. In this manner, trainees can view slides from any location, and multiple trainees can view the same slide simultaneously. Slides are shared among numerous individuals without the risk of slide breakage or loss. After these conferences are completed, these slides and other slides can be assembled into a bank of slides that can serve as study sets for the trainees. In the same manner, these slides can be used to present tumor boards and other clinical conferences. We have found this useful, since many clinical conference rooms do not have working microscopes; however, almost all conference rooms have a computer with projection capability and an Internet connection, allowing access to slides stored on network-connected servers.7–10
Whole slide images can also be used to retain consult/second-opinion cases from other institutions. We have found this to be particularly useful when we are required to send the slides back owing to CAP or legal obligations. If scans are performed, the images can still be available for comparison with subsequent resections. Such images can be shared for a second opinion with other sites (especially multihospital centers such as ours), and in a similar manner, such images can be shared with our pathologists for another opinion by using techniques adopted at other centers.11
Digital image capture has been used in our department for immediate cytology evaluations, since it is not cost-effective to send cytopathologists to all immediate evaluations, particularly for procedures lasting an hour or more. In addition, there are often simultaneous procedures ongoing at different departments and sites. For this reason, our cytologists have instead devised methods for trainees or cytotechnologists to send images from the rapid on-site evaluation site to the pathologist, similar to methods presented by others.12–14
We have used WSIs for many applications in research, as will be discussed further below. For example, we have conducted international multiobserver agreement studies that have been facilitated through the use of WSIs.15–17 In a study of gallbladder epithelial atypia and early gallbladder cancer staging, WSI was used to circulate cases among 18 experts. If WSIs had not been used, boxes of slides would have had to be transferred from one expert to another; this would have been costly and would have taken approximately 2 years. Using WSIs, this was completed in 1 month.16,17
APPLICATION OF WHOLE SLIDE IMAGING AND IMAGE ANALYSIS FOR ORGAN SYSTEMS
Breast and Gynecologic Pathology
Image cytometry has been used in our department for some time.18 Perhaps the most common application has been in the breast. Early investigation in image analysis involved ploidy analysis, which at the time was believed to have a major role in patient stratification to treatment protocols.19–22 In cases with no fresh tissue available, ploidy (nuclear DNA content) could be estimated by image analysis of Feulgen-stained slides.18 Currently, the most common use is in measurement of theranostic markers by IHC.20,23–28 This includes studies on HER2/neu,29–31 estrogen and progesterone receptors (ER and PR),23–28 Ki-67 (MIB-1),32,33 proliferating cell nuclear antigen,20 epidermal growth factor receptor,20 and cathepsin.20 Most noteworthy, we currently analyze the membranous staining for HER2/neu and the nuclear staining for ER, PR, and Ki-67 by using digital algorithms (Figure 1, A through H). We have established that ER and PR by IA and also by visual inspection correlate with biochemical dextran-coated charcoal assays for ER and PR.24–27 In many cases, we have found advantages of automated IA over visual inspection. For example, in a study on PR hormone status, image cytometric quantitation of PR immunohistochemical staining correlated with disease-free survival; however, visual quantitation of PR immunostaining did not relate to either overall or disease-free survival.23 We have found a similar result during an analysis of male breast carcinoma.20
Stains routinely used for breast carcinomas at our center include HER2/neu, ER, PR, and Ki-67, all of which are analyzed by digital methods (Figure 2). After using the Dako Automated Cellular Imaging System (ACIS; Dako, Carpinteria, California), we recently moved to a different platform, the Aperio whole slide scanner (currently marketed by Leica Biosystems Inc, Buffalo Grove, Illinois). There were challenges in reaching the optimum Aperio algorithm parameters to provide results that were equivalent to those of the ACIS.34 This transition was similar to other transitions that we have experienced in the past, since our department has also used other quantitation systems including the CAS 200 image analyzer (Becton Dickinson, San Jose, California).35 Our breast prognostic marker workflow represents a “computer-assisted” quantitation and is one input to the analytic process. For each breast cancer case, we digitally quantitate 6 scanned fields at ×20 magnification. The cases are also all visually reviewed by the pathologist to verify the accuracy of the image cytometric quantitation. In cases with unexpected results, the scanned images are revisited. The scoring in the final pathology report is ultimately at the discretion of the sign-out pathologist, although in most cases it is the one obtained by WSI and digital quantitation.
In establishing the protocols and applying them to daily practice, close correlation with other standard parameters is warranted. For example, in our concordance studies correlating the cytometric scores and fluorescence in situ hybridization (FISH) amplification results, we have found that when the IA 2+ equivocal range is changed from a range of 1.8 to 2.2 to a range of 1.8 to 2.6, then low positive (3+) specimens (>2.2–2.6) are included in an equivocal (2+) group; this change in the upper equivocal range cutoff improved concordance (from 84.4% to 94.4%) and eliminated the need to triage the IHC 3+ cases to further FISH testing.36
Image analysis and WSI utilization are also very valuable techniques to analyze the potential role of other ancillary markers. For example, we have investigated various markers including apoptotic markers in breast cancer. One study involved the quantitation of Bcl-2 and Bcl-x, which inhibit cell death, and Bax, which promotes cell death. This study revealed that a Bcl-2:Bcl-x ratio of 1 or greater is associated with an improved disease-free survival. However, the Bcl-2:Bax ratio was not predictive of overall or disease-free survival.35 Similar studies have been conducted in ovarian carcinoma, in which increased expression of Bax and Bcl-x was associated with increased overall and disease-free survival; Bcl-2:Bax and Bcl-2:Bcl-x ratios less than 1 had a non–statistically significant survival advantage.37 Quantitation of tumor-infiltrating lymphocytes may be important in breast carcinoma prognostics and theranostics38 ; and as we have shown with on-slide “flow cytometry” methods for counting lymphocytes on WSIs,39–41 IA may be useful in quantitating tumor-infiltrating lymphocytes in breast cancer. These and similar studies are crucial to identify next-generation theranostic markers to supplement ER/PR/HER2/neu in patient-specific precision therapy of breast cancer and other cancers, and IA/WSI can have very important roles in their establishment.
Gastrointestinal, Pancreatobiliary, and Liver Pathology
Ki-67 IHC is often used as a surrogate marker of proliferation in a variety of tumors, including neuroendocrine tumors42,43 and hepatocellular carcinoma,44 and has become one of the most important parameters in diagnosis, classification, and grading. We have used computerized approaches extensively for this purpose; however, we have discovered several shortcomings that need to be addressed before this method can be used reliably.42,43,45–47 For example, unless software modifications are made specifically or operators trained in pathology are engaged in the process, the scanner cannot distinguish other brown elements in the tissue (such as hemosiderin) from true positive units. For example, the scanner calculated the index as 68% in a case that was in reality only 3%, and this was due to the extensive hemorrhage in the tumor. Similarly, cells that are fairly abundant in the tissue, such as endothelial cells and lymphocytes, often show significant Ki-67 labeling, which leads to overcounting of the index unless some correction is applied. For these reasons, until the proper modifications are in place, we have had to revert for now to manual counting of printed images by a pathologist or other trained personnel, instead of using the automated count. Nonetheless, there is no question that with proper improvements the WSI approach will soon become the norm for this purpose. Two recent studies with sections and cytologic cell blocks show excellent correlation between results by manual counting and Aperio quantitation.48,49
Image analysis has been used by our group for many other applications in gastrointestinal, liver, and pancreatobiliary pathology. Image analysis has also been used for the analysis of HER2/neu staining in gastric carcinoma.50 We have used algorithms in the assessment of microvessel density in hepatocellular carcinomas with the Aperio microvessel density algorithm applied to CD31 and CD34 IHC, showing decreased survival in tumors with high microvessel density.51 We have implemented algorithms in the quantitation of steatosis (Figure 1) and have worked with our radiology colleagues to correlate these findings with radiology measurements.52 Using special stains such as trichrome, we have explored the use of IA in the quantitation of hepatic fibrosis, primarily using positive pixel count algorithms applied to trichrome slides.53–55 Using routine H&E slides, we have used basic measurements and also the positive pixel count algorithm to measure features of dysplastic nuclei in colonic adenomas and Barrett esophagus–associated dysplasia,56,57 and recently, we have used similar methods to recognize differences in sessile serrated adenomas.58 Investigators in our department have also used IA to measure apoptosis in colonic mucosa by using the apoptosis inhibitor Bcl-2 and the apoptosis promoter Bax.59 We have also used IA in an on-slide “flow cytometry”–type approach to quantitate inflammatory cells in autoimmune pancreatitis,39 and it is possible that such methods may become important in quantitating inflammatory cells in a variety of applications (eg, tumor-infiltrating lymphocytes in gastrointestinal tract cancers). These diverse applications demonstrate the many possibilities of WSI IA.
Renal and Genitourinary Pathology
Renal allograft rejection is one area in which IA has several potential applications that can be incorporated into daily practice. To establish more objective methods for the quantitation of inflammation in rejection, we have been engaged in efforts to establish on-slide “flow cytometry”–type methods, using cell-counting algorithms in the measurement of inflammatory cell density (eg, CD3+ cell density).40,41 Finding useful molecular correlates of rejection that can be easily probed on the tissue level has also been a goal of our group. For example, we have looked at the quantitation of human leukocyte antigen–antigen D related (HLA-DR) staining in renal allografts and correlated this with the severity of rejection.60 In an attempt to further characterize antibody-mediated rejection in the kidney, we also applied the microvessel density algorithm to C4d-stained slides and showed correlations of the C4d staining with mean fluorescence intensity of donor-specific antibody to HLA.61 Providing surrogate markers of allograft deterioration has been a goal of our department in studies using IA to measure renal fibrosis (Figure 1, A through H) in sections stained with trichrome, periodic acid–Schiff, Sirius red, and collagen III for IHC.15,62 Some of these studies have indicated that IA may provide benefits, given the challenge of standardizing fibrosis assessment among a number of reviewers in multicenter settings.15
Individuals in our department have collaborated with informatics specialists to devise ways for computers to determine the Fuhrman grade of renal cell carcinoma63 and to normalize images of renal tumors so that they can be interpreted by computer algorithms.64 In another study,65 the mean epidermal growth factor staining density measured by IA correlated with skin adjacent to hypospadias.
Neuropathologic diagnosis of tumors has also proven very amenable for the utilization of WSI,66 and Emory University pathologists and informaticists have explored a number of aspects of this technology.66,67 Earlier studies from our department established the role of Ki-67 quantitation in glial neoplasms.68,69 Recently, using multimodal, multiscale approaches and machine-based classification, researchers have devised ways to mine scanned histologic data on glioblastoma in The Cancer Genome Atlas Project and other sources; the resulting quantitative morphometric analysis findings have been further integrated with molecular data to provide in silico cancer research70–80 and with radiologic data to provide clinicopathoradiologic correlation.81 Insights from this work also include findings on the importance of tumor-infiltrating lymphocytes in glioblastoma,72 and in silico approaches from these studies have uncovered novel findings regarding the regulation of asymmetric cell division in glioblastoma by such mediators as the human Brat ortholog TRIM3.80
IMAGE ANALYSIS IN OTHER ORGAN SYSTEMS AND APPLICATIONS, INCLUDING QUALITY ANALYSIS/CONTROL
Image analysis has also garnered use in other organ systems. For example, angiogenesis measured by IA of CD31 IHC shows strong correlation with regional recurrence of laryngeal cancer.82 In an attempt to improve the quality analysis of our special stains, we used WSI of control slides to objectively measure the hue of trichrome stain so that “Westgard”-type rules could potentially be applied to histology staining quality control measures.83 On a daily basis, positive controls for ER, PR, HER2/neu, and Ki-67 are scanned and quantitated by IA. Results are graphed, and both IHC and automated quantitation are assessed.
There are different approaches to telepathology. The most simplistic approach involves the acquisition of static images that are then sent to the consultant. This can involve acquisition devices as ubiquitous as smart phones, as demonstrated by individuals at our institution,84,85 and forms of WSIs can be obtained using these methods.85 Some provide pathologists with the ability to move a microscope remotely by using a robotically controlled microscope stage (eg, the “Trestle” system).86,87 Others involve the acquisition of WSIs that are transferred over the network and reviewed remotely (eg, the Aperio system). All methods have their own advantages, including different costs and ease of implementation. Static image methods in which the image “donor” obtains the images may not benefit from the insight that an expert “recipient” may have in choosing the most diagnostic areas; however, these methods may be quite inexpensive to implement. Methods such as remote robotic control or WSIs provide recipients with the ability to view the slide at their leisure and speed and with the opportunity to use their own insight in choosing the proper diagnostic areas; however, these methods may be quite expensive.87
Whole slide scanners have been used for telepathology at our institution; however, challenges can sometimes be encountered in their implementation.88,89 As one example, an Aperio scanner was installed at Emory University Midtown Hospital, which is separated from the Emory University Hospital (EUH) on the Emory University campus (all in Atlanta, Georgia). Telepathology was needed between the 2 campuses, primarily because neuropathologists were only routinely stationed at the EUH location. The solution implemented was a hybrid whole slide scan/robotic approach (Figure 3), in which WSI is rapidly obtained at relatively low magnification (approximately ×5). For higher optical magnification, the remote pathologist may optionally enable a “Telepath Live” feature in the Aperio ImageScope program to take control of the scanner and use it as a robotic microscope with the caveat that the scanned slide still has to be on the scanner stage. Anecdotally, reception from the pathologists using this system has been mixed. Some pathologists adapted quite well to the new technology; however, others had qualms about the solution. Pathologists who had difficulty with this solution complained about the delay needed for scanning. For the solution to be feasible, scanning had to be performed at ×20 magnification, and some pathologists did not feel that this provided enough detail, particularly when they needed to examine fine cytologic features (eg, on smear preparations examined in conjunction with the frozen sections). Some surgeons also noted an increase in frozen section turnaround time. However, all things considered, this telepathology implementation provided a solution where subspecialty expertise was needed and would have otherwise been unavailable.
Our experience as well as that of other institutions has shown that in the implementation of telepathology, proper assessment of network connections and file server requirements maintaining Health Insurance Portability and Accountability Act (HIPAA) regulations should be considered carefully. The CAP has established guidelines for the validation of WSIs, and these should also be thoughtfully considered when implementing a WSI solution.90 At our institution, addressing the issues related to WSI implementation required a great investment of time from our pathologist and informatics leaders. Implementation of such a solution needs consideration of whether the projected utilization justifies the investment of time from the pathology and informatics teams.
Methods used in telepathology can eventually be used to essentially replace routine light microscopy.87,91 In multihospital centers, such implementations could eliminate the need to send thousands of slides and paperwork from one site to another and could also eliminate problems such as slide loss or breakage. Such methods may also improve the ergonomic issues that some pathologists may confront (eg, neck problems).92 Scanning speed and quality are now quite high; however, infrastructure issues and issues with economic storage of large numbers of images need to be solved before there will be widespread implementation.91
Overall, WSI, novel stains, IA techniques, and telepathology offer a great deal of promise but also a number of challenges. Individuals at our institution have implemented clinical decision support tools93 and have been involved in efforts to define “computational pathology.” 94 It is anticipated that such methods will complement standard techniques even more in the future and will hopefully help complement WSI and IA.95 Overall, WSI and IA provide a wide variety of quantitative approaches for the examination of pathologic specimens. Once the perilous tasks of investment in technology and validation of these techniques are overcome, these techniques can potentially provide great promise for precision medicine and optimal personalized patient care.
The authors have no relevant financial interest in the products or companies described in this article.