Context.—

Digital pathology (DP) implementations vary in scale, based on aims of intended operation. Few laboratories have completed a full-scale DP implementation, which may be due to high overhead costs that disrupt the traditional pathology workflow. Neither standardized criteria nor benchmark data have yet been published showing practical return on investment after implementing a DP platform.

Objective.—

To provide benchmark data and practical metrics to support operational efficiency and cost savings in a large academic center.

Design.—

Metrics reviewed include archived pathology asset retrieval; ancillary test request for recurrent/metastatic disease; cost analysis and turnaround time (TAT); and DP experience survey.

Results.—

Glass slide requests from the department slide archive and an off-site surgery center showed a 93% and 97% decrease, respectively. Ancillary immunohistochemical orders, compared in 2014 (52%)—before whole slide images (WSIs) were available in the laboratory information system—and 2017 (21%) showed $114 000/y in anticipated savings. Comprehensive comparative cost analysis showed a 5-year$1.3 million savings. Surgical resection cases with prior WSIs showed a 1-day decrease in TAT. A DP experience survey showed 80% of respondents agreed WSIs improved their clinical sign-out experience.

Conclusions.—

Implementing a DP operation showed a noteworthy increase in efficiency and operational utility. Digital pathology deployments and operations may be gauged by the following metrics: number of glass slide requests as WSIs become available, decrease in confirmatory testing for patients with metastatic/recurrent disease, long-term decrease in off-site pathology asset costs, and faster TAT. Other departments may use our benchmark data and metrics to enhance patient care and demonstrate return on investment to justify adoption of DP.

Applications for digital pathology (DP) have seen increased growth; however, few departments have successfully implemented a full-scale operation of a DP platform. Digital pathology has tremendous potential to disrupt the practice of pathology. During the last 2 decades, whole slide imaging has shown improvement in image resolution, scan times, scanner capacities, image management software, and most importantly, integration into laboratory information systems (LISs). In addition, the utility of whole slide images (WSIs) has been thoroughly investigated and been approved for primary diagnosis in the United States.1  However, the utility of WSIs needs to be demonstrated in routine laboratory workflow in order to become truly effective in clinical practice. There is reluctance among many pathologists to exchange their principal diagnostic tool, the optical microscope, for a digital solution. As such, many of the forces that will bring about the implementation of whole slide imaging for routine diagnostic use will be external. Aside from being a new tool in the pathology community, DP promises a higher caliber of patient care. This is accomplished in conjunction with barcoding and tracking solutions for case management, image management software, expedited sharing of WSIs, and load balancing. Decreasing interpretive errors of challenging cases can be managed by sharing of WSIs for consultations to expert or subspecialty pathologists, or across multiple sites in a distributed health network. The relative few institutions with fully deployed DP operations have shown benefits of using a DP platform.211  Although there are several DP platforms available, there is a daunting financial commitment to support a return on investment (ROI). Full deployment of DP at minimum requires an investment toward personnel, hardware, software, integration, and other supplementary acquisition costs. Owing to the additional required need of scanning the glass slide, DP is additive to current histopathology workflows.12  For these reasons, hospital administrators and departments debating DP may be concerned with encountering high initial overhead and continued investments, since only few studies have shown meaningful metrics pertaining to efficiency or cost savings.

This study aims to establish practical metrics and benchmark data that demonstrate clinical and operational utility of DP in a large anatomic pathology laboratory.

## MATERIALS AND METHODS

This evaluation is based on the experience of a high-volume, academic, tertiary care cancer center. The institution is a 473-bed comprehensive cancer center, with more than 10 000 employees, including 1148 attending physicians and 133 institute member scientists. The institution admits more than 23 506 patients per year and has more than 722 238 outpatient visits at local and regional facilities.13,14  The Department of Pathology includes more than 80 pathologists and 39 fellows. In 2017, more than 160 000 total accessions and more than 1.5 million glass slides were produced. The department has a long-standing history with DP and was an early adopter, having used WSIs for more than a decade. The clinical workflow in reviewing the digital slides is currently for retrospective review.

### Glass Slide Scanning Protocol

During the past decade, the institutional glass slide scanning protocol has evolved. The scanning timeline was developed as a phased approach for retrospective and prospective glass slide digitization. Selected glass slides are marked for scanning in each case and then follow a department scanning protocol. The aim of the DP implementation was concerted around operational workflow and efficiency as opposed to an enterprise-wide full DP deployment.

### Scanning Operation (Hardware and Software)

The pathology department at Memorial Sloan Kettering Cancer Center (MSKCC) has dedicated whole slide scanning rooms (Figure 1), as well as other scanning locations that house a variety of whole slide scanners including Leica Aperio AT2 and XT (Leica Biosystems, Buffalo Grove, Illinois); 3DHistech Pannoramic 250, Pannoramic 1000, and confocal scanner (3DHistech, Budapest, Hungary); Hamamatsu Nanozoomer 2.0HT and S60 (Hamamatsu Photonics, Hamamatsu City, Shizuoka, Japan); Philips Ultra-Fast Scanner (Philips, Amsterdam, the Netherlands); and VisionTek M6 digital microscopes (Sakura Finetek, Torrance, California). The predominant scanners used for the clinical operation during the time of this study were the Leica/Aperio AT2 whole slide scanners. The number of AT2 scanners has varied through the DP implementation. At the time of this study, 6 AT2 scanners and 3 digital scanning technicians (ie, 1:2 scan tech to scanner ratio) were responsible for digital slide scanning. Selected glass slides for digitization follow a department scanning protocol (see Results). Slides are scanned predominantly at ×20 (0.5 μm/pixel); however, ×40 (0.25 μm/pixel or 0.16 μm/pixel) scans are performed on the basis of clinical need (ie, hematopathology, cytology). The WSIs are transferred and stored in the institution's data center. The department has 1 gigabit/s network connections for each computer workstation and whole slide scanner. The whole slide scanners share a 10 gigabit/s connection to the institutional data center. The importance of interfacing the WSIs with the department's LIS has been described.15,16  The advanced barcoding and tracking module implemented at our institution uses 2D barcodes that are decoded by the Leica Aperio whole slide scanners, and in turn interface the eSlide Manager database software with the LIS (Cerner CoPathPlus, Cerner Corporation, Kansas City, Missouri). Whole slide images are launched from within the LIS and are viewed in a separate WSI viewer application (ie, Web-based MSK Slide Viewer or client-based Leica Aperio Imagescope v12.3).

Figure 1

Digital pathology suite. Whole slide scanners and computers are arranged ergonomically for digital pathology supervisors and technicians to manage scanning workflow.

Figure 1

Digital pathology suite. Whole slide scanners and computers are arranged ergonomically for digital pathology supervisors and technicians to manage scanning workflow.

### Glass Slide Retrieval

The departmental glass slide archive is maintained by dedicated staff and is accessible only to them. All other staff members are restricted to requesting slides via a glass slide requisition for all request types (ie, clinical, conference, research). Glass slide requisitions are scanned and stored for auditing. Requisitions were collected and tabulated between the years 2014–2017. In addition, an off-site surgical center opened in 2016, where pathologists are stationed away from the main campus for intraoperative consultations. Pathologists review the surgery schedule the day prior, and could similarly request glass slides for review. These glass slide requests were also tabulated and reviewed.

### Clinical Review of WSIs

When reviewing a case prospectively, the patient's complete clinical and pathology history is routinely examined. When prior materials (ie, glass slide or WSIs from other specimens) are reviewed, pathologists may document in their report whether the patient's prior material was comparatively reviewed. A natural language search of our LIS was queried to retrieve these cases, and subsequent comparative analysis of immunohistochemical (IHC) orders. Evaluated cases excluded prognostic or predictive IHC stain ordering.

### Operational Cost Analysis

Comparative costs before and after implementing the DP system were analyzed, as based on costs for departmental scanning of select slides from prospective cases. Factors considered include personnel (ie, slide file clerks, slide scanning), hardware, software, service agreements, information technology (IT) infrastructure, digital storage, glass slide physical asset storage, and off-site storage vendor services (ie, filing, retrieving, delivery costs). Cases from recent years are stored on-site in the department glass slide archive for ready access retrieval; however, older pathology assets (glass slides, blocks, documents) are stored in an off-site storage facility. Costs for off-site storage, retrieval, and other vendor services were calculated and compared to the costs of implementing and maintaining a DP operation based on prescanning and postscanning department requirements.

### Assumptions in Data Generation

The operational cost analysis was conducted by using certain assumptions. Every effort was made to include costs that were obtained from actual departmental data of incurred costs. Comparative operational analysis was acquired from predigital pathology personnel, annual increase in surgical pathology accession volume, and vendor pathology glass slide requests. Postdigital pathology factors were based on scanning a minimum of 40 000 glass slides per month (480 000 per year), which reflects departmental intended scanning throughput and personnel for operation, and IT infrastructure including vendor-laboratory information system interface. Internal data were used for calculations based on current hardware, personnel, vendor costs, and institutional digital file storage.

### Turnaround Time

The LIS was queried to identify cases with and without prior WSI. Turnaround times (TATs) for each specimen class (ie, biopsy, surgical resection, consultation) were compared. Turnaround time was defined as the point of case accession to the time of reported diagnosis.

### Experience Survey

A DP experience survey was created and distributed to all pathologists and trainees (n = 124). The survey included questions based on their clinical utilization and perspective using DP. Questions used a 5-point Likert rating scale for each response.

## RESULTS

### Glass Slide Scanning

As of July 2018, with the first scanned slide in 2007, a total of 1 039 340 slides were scanned, comprising 270 terabytes in digital storage. These WSIs include clinical, research, and educational slides, and may underestimate average file sizes if measured as a whole. Before August 2015, glass slide scanning was performed on a research or ad hoc basis. Whole slide images were scanned and stored locally, available only in network-attached storage. Clinical scanning was initiated after implementation of the LIS interface. The LIS supported WSI integration through a Health Level 7 (HL7) messaging interface between the barcoding and tracking module and a customizable add-on solution from the vendor, PICSPlus (Cerner Corporation). Scanning was initiated for glass slides sent to the institution for consultation in August 2015. Subsequent scanning of specimens from biopsies performed at our institution commenced in January 2016. Glass slides from surgical resection specimens began to be digitized in February 2017. This phased approach allowed clinical operations the flexibility to scale while accommodating for revisions to workflow, IT, networking, and security. Clinical scanning between 2015 and 2017 included 424 901 WSIs that were scanned and linked within the LIS (Figure 2; and Figure 3, a and b). For clinical whole slide scanning, the average file size was 482 MB per slide. There was an average of 3 glass slides scanned per case during this period (range, 1–97 slides scanned per case). Whole slide images from within 2 years were available on hot/tier 1 storage for image retrieval, with WSIs older than 2 years on warm storage/tier 2. No additional compression of digital slides was performed for digital storage.

Figure 2

Increase in monthly scanning volumes. Clinical whole slide scanning initiated in August 2015; each month shows (nonadditive) individual monthly scan volumes as each phase of the digital pathology implementation commenced. As scaling increased, in April 2017, networking issues were encountered, which were subsequently resolved.

Figure 2

Increase in monthly scanning volumes. Clinical whole slide scanning initiated in August 2015; each month shows (nonadditive) individual monthly scan volumes as each phase of the digital pathology implementation commenced. As scaling increased, in April 2017, networking issues were encountered, which were subsequently resolved.

Figure 3

Subspecialty distribution and increase in scale by subspecialty between 2015 and 2017 (n = 424 901 whole slide images [WSIs]): Peds (pediatrics; n = 39); Neuro (neuropathology; n = 8212); Cytology (n = 15 272); BST (Bone and soft tissue; n = 20 794); Head and Neck (n = 21 825); Heme (hematopathology; n = 25 728); GYN (gynecology, n = 28 954); Thoracic (n = 38 878); Derm (dermatopathology; n = 50 155); Breast (n = 72 103); GI (gastrointestinal; n = 69 920); GU (genitourinary, n = 73 021). a, At our institution, the 3 largest subspecialties with WSIs include the breast, gastrointestinal, and genitourinary pathology services. b, Increasing scale of clinical scanning across the department shows additive scanning volumes between 2015 and 2017.

Figure 3

Subspecialty distribution and increase in scale by subspecialty between 2015 and 2017 (n = 424 901 whole slide images [WSIs]): Peds (pediatrics; n = 39); Neuro (neuropathology; n = 8212); Cytology (n = 15 272); BST (Bone and soft tissue; n = 20 794); Head and Neck (n = 21 825); Heme (hematopathology; n = 25 728); GYN (gynecology, n = 28 954); Thoracic (n = 38 878); Derm (dermatopathology; n = 50 155); Breast (n = 72 103); GI (gastrointestinal; n = 69 920); GU (genitourinary, n = 73 021). a, At our institution, the 3 largest subspecialties with WSIs include the breast, gastrointestinal, and genitourinary pathology services. b, Increasing scale of clinical scanning across the department shows additive scanning volumes between 2015 and 2017.

### Departmental Scanning Protocol

Specimens are accessioned in the LIS by using an advanced barcoding and tracking module and follow traditional histology workflow. After case assembly and glass slide distribution, cases are disseminated to subspecialty mailboxes. Fellows or attending pathologists then review the clinical case and designate select slides in each case to be digitized. Guidelines for glass slide scanning are as follows: (1) for biopsies, 1 block from each specimen part is selected; (2) for resection cases, all frozen section slides and frozen section controls, glass slides with diagnostically relevant grading/staging information, predictive or prognostic immunohistochemistry; (3) for consultation cases, all diagnostic and prognostically relevant glass slides. Rush and priority scanning services have also been made available to the department, where requisition forms are filled out and provided with the glass slides and are prioritized to be scanned for clinical, educational, or research initiatives.

Workflow for scanning consultation material was accomplished in conjunction with the vendor. For tracking purposes, the addition of our institution label (with 2D barcode) is placed on all consultation material. Placement of the label atop the outside institution label would often cover key information (ie, stain, patient identifier). Therefore, placing our institution label below the outside consulting laboratory label was preferred; however, this region was not captured in the macrocamera's field of view, which would not be visualized and decode our institution's barcode, and the WSIs would not be linked in the LIS. After discussions with the vendor, this was rectified and the camera's field of view was expanded to cover the area below the label.

This change introduced a secondary issue of potentially having 2 barcodes in the same field of view of the whole slide scanner's macrocamera. Our department initially used color-coding label stickers (eg, red dot stickers). These red dot label stickers were placed over outside institutional barcodes to hide them from the whole slide scanner camera, such that the camera would only recognize and decode our department's barcode. However, as our implementation scaled, these were too cumbersome to be used for placement atop the outside barcodes, as well as inefficient for removal after scanning (ie, before returning to the consulting institution). The department then instituted liquid chalk ink markers, which facilitated easier placement and removal of “erasable chalk ink” for scanning workflow purposes (Figure 4, a and b).

Figure 4

Glass slide scanning indicator: composite red dot stickers with chalk ink. a, Color-coding label stickers; inset: sticker shown covering outside hospital barcode. b, Liquid chalk ink markers; inset: chalk ink dot shown in upper right hand corner of slide label as slide scanning indicator.

Figure 4

Glass slide scanning indicator: composite red dot stickers with chalk ink. a, Color-coding label stickers; inset: sticker shown covering outside hospital barcode. b, Liquid chalk ink markers; inset: chalk ink dot shown in upper right hand corner of slide label as slide scanning indicator.

After clinical review of glass slides, the glass slides marked for scanning are collected and cleaned. Cleaning the glass slides includes removal of all marker ink annotations with alcohol, which also acts to remove dust particles or fingerprints from the glass slides. All glass slides received as marked for scanning (eg, red dot on slide label) are scanned; in the event of a slide being received broken, these slides are manually scanned separately, if possible. Glass slides with extensive preanalytic artifacts (ie, air bubbles, knife cutting) are ameliorated in the laboratory before scanning, dependent on the extent of the artifact, and then included in the subsequent scanning workflow. All slides marked for scanning are aggregated on shelving racks outside the whole slide scanning room. The scanning room is in the Department of Pathology, located on the same floor as the glass slide archive, pathologists' offices, sign-out areas, and histology laboratory. This allows for efficient transportation and management of the glass slides in the scanning workflow. On the shelving racks, a queue of glass slides includes a thorough quality assurance (QA) process by DP technicians. This process includes a barcoding and tracking station to scan each glass slide; this ensures each barcode is readable by handheld barcode readers and marks the glass slide scanned (received for scanning in the LIS). Glass slides without barcodes, or barcodes that failed reading by the handheld reader, have new patient labels with barcodes printed and placed on the glass slide. The QA process also involves macroevaluation of preanalytic slide artifacts to be resolved before scanning whenever possible. After the glass slides are scanned, a second QA by a digital imaging coordinator or supervisor is performed to ensure the WSI is of adequate quality and is present within the LIS. This second QA process involves investigation in the eSlide Manager database where all slides are provided a quality factor (range, 0–100) by the vendor's algorithm and thumbnail overview; all slides with a quality factor below 90 are reviewed—whereby the policy transitioned from iterative inspection for every 10, 4, and 1 WSI(s) for surgical pathology, molecular studies, and cytology, respectively—and evaluated for typical WSI artifacts (ie, out of focus, tissue detection). Any slide requiring a rescan is included in the subsequent batch of slides or is manually scanned. After scanning, when viewing a digital slide in the MSK Slide Viewer, an additional QA tool has been implemented whereby the reviewer can provide direct feedback to the digital scanning personnel on image quality, artifacts, or network performance. All glass slides then proceed to be filed in the department slide library archive. A fully loaded whole slide scanner (eg, 400 slides) scanned at 0.25 μm/pixel (×40 equivalent resolution) takes approximately 43 hours to be completed. Turnaround time from initial glass slide retrieval to glass slide archive storage for clinical and priority scanning is 24 to 48 hours; nonclinical scanning otherwise has a 4- to 5-day TAT. Digital slide scanning is performed in 2 shifts, 6 days per week.

### Glass Slide Requisitions

In 2014, before initiation of the digital scanning operation, there were 19 369 archival glass slide requisitions. In 2015, a total of 20 745 archival glass slide requests were retrieved by the slide library staff, while 2016 and 2017 had 12 336 and 1426 archival glass slide requests, respectively. Archival glass slide requests showed an overall 93% decrease (Figure 5, a). At the off-site surgical center, pathologists' requests for prior archived material (ie, glass slides) from patients with anticipated intraoperative consultations showed a 97% decrease (range, 0–32; average 8 requests per month), dramatically reducing requests, owing to remote access of the patient's WSIs (Figure 5, b). With this decrease in slide requests from the department slide library, 3 full-time employees from the slide file room were redistributed and incorporated into the DP operations workflow.

Figure 5

Decrease in glass slide requests with increasing availability of whole slide images (WSIs). a, With the increasing availability of WSIs, glass slide requests from the departmental slide library decreased by 93%. b, Off-site intraoperative consultation of glass slide requests decreased by 97% over time as the whole slide scanning operation scaled.

Figure 5

Decrease in glass slide requests with increasing availability of whole slide images (WSIs). a, With the increasing availability of WSIs, glass slide requests from the departmental slide library decreased by 93%. b, Off-site intraoperative consultation of glass slide requests decreased by 97% over time as the whole slide scanning operation scaled.

### Immunohistochemical Stain Orders

From the natural language query, 2801 cases were retrieved from the LIS where pathologists documented review of patient material from a prior specimen. In 2014, before glass slides were scanned, 52% of cases had IHC staining ordered when prior material was reviewed, as compared to 19% and 21% of cases with IHC staining ordered on prospective specimens in 2016 and 2017, respectively.

From the years after clinical scanning of glass slides commenced (2015–2017), pathologist case review showed 1884 documented comparisons to WSIs from prior scanned material (median, 756 cases per year). When prior WSIs were available in reviewing a prospective case, in 2016, a total of 745 cases did not have IHC staining ordered, as compared to 172 cases that had IHC staining ordered. In 2017, pathologists did not order confirmatory IHC tests in 767 cases, as compared to 200 cases that had IHC staining ordered (Figure 6). Pathologists ordered less ancillary studies by up to 75.4% when WSIs were available. A median of 3 supportive or confirmatory IHC tests were ordered per case. Given an average cost of $50 per IHC test,17 an anticipated savings of$113 400 can be expected at our institution, with a median number of 756 cases per year with documented WSI review.

Figure 6

Clinical cases with immunohistochemistry (IHC) test ordered. In 2014, the clinical whole slide image (WSI) database was not interfaced with the laboratory information system (LIS), which had 52% of cases with IHC orders. After WSIs were available in the LIS, IHC orders decreased by 30% in cases with documented review of prior patient WSIs.

Figure 6

Clinical cases with immunohistochemistry (IHC) test ordered. In 2014, the clinical whole slide image (WSI) database was not interfaced with the laboratory information system (LIS), which had 52% of cases with IHC orders. After WSIs were available in the LIS, IHC orders decreased by 30% in cases with documented review of prior patient WSIs.

### Operational Cost Analysis

A cost savings analysis was conducted by calculating and comparing prescanning and postscanning costs with the following budgetary considerations reviewed: required personnel (ie, slide file clerks, slide scanning), hardware (including capital equipment purchases), software, service agreements, IT infrastructure, digital storage, glass slide physical asset storage, and off-site storage vendor services (ie, filing, retrieving, delivery costs) during a 5-year period (2014–2018), based on incurred costs while implementing a DP system at our institution. A projected savings of more than $267 000/y was calculated secondary to personnel restructuring, decreased vendor services (ie, pathology asset retrieval, labor and other vendor services) due to the decreased need for glass slide transport and ready availability of WSIs, and physical storage of glass slides (Figure 7, a). These calculations show a$1.3 million projected savings during a 5-year period (2019–2023; Figure 7, b).

Figure 7

Operational cost analysis. a, A projected savings of more than $267 000/y secondary to personnel restructuring; decreased vendor services (ie, pathology asset retrieval, labor, and other vendor services), due to the decreased need for transport and ready availability of whole slide images; and physical storage of glass slides. b, These calculations show a$1.3 million savings for a projected 5-year period (2019–2023). “Without digital pathology” includes the off-site storage vendor services (based on our volume tracking from pre and post digital pathology implementation) and also includes Personnel:FTE for slide file clerks. “With digital pathology” includes a decrease in off-site storage vendor services (based on our volume tracking from pre and post digital pathology implementation) and Personnel:FTE for slide scanning, digital storage, and hardware/software costs specific to the scanning operation. The increase in scaling of digital storage costs is contrasted to the decrease in hardware/software costs over time and overall decrease in personnel costs. Abbreviations: FTE, full-time employee; IT, information technology.

Figure 7

Operational cost analysis. a, A projected savings of more than $267 000/y secondary to personnel restructuring; decreased vendor services (ie, pathology asset retrieval, labor, and other vendor services), due to the decreased need for transport and ready availability of whole slide images; and physical storage of glass slides. b, These calculations show a$1.3 million savings for a projected 5-year period (2019–2023). “Without digital pathology” includes the off-site storage vendor services (based on our volume tracking from pre and post digital pathology implementation) and also includes Personnel:FTE for slide file clerks. “With digital pathology” includes a decrease in off-site storage vendor services (based on our volume tracking from pre and post digital pathology implementation) and Personnel:FTE for slide scanning, digital storage, and hardware/software costs specific to the scanning operation. The increase in scaling of digital storage costs is contrasted to the decrease in hardware/software costs over time and overall decrease in personnel costs. Abbreviations: FTE, full-time employee; IT, information technology.

### Turnaround Time

Audit of biopsy, surgical resection, and consultation cases between 2014 and 2016 was reviewed. The average TAT for 59 571 surgical cases was tabulated. In 2014, 2015, and 2016, the average TAT was 4 days, 4 days, and 3 days, respectively. For surgical resection specimens, cases with prior WSIs were reported 25% sooner (1 day). This may be attributed to the decrease in glass slide retrieval (ie, patient prior material) and ready access to WSIs in the LIS. However, this effect was not noticed in other specimen class subtypes. Consults and biopsy specimens are typically the first accessioned case of a patient at a large cancer center, and may not have prior material. The effect of having prior digital images may not be applicable for these patients, or may be nullified in these other specimen types.

### Experience Survey

The DP experience survey was distributed to 124 users and the results from 71 respondents (57.3%) were recorded. Of the 71 respondents, 28 (39%) were fellows and 43 (61%) were from faculty of varying ranks. Figure 8 shows the respondents' chart and respective questions with Likert-scale responses. Seventy respondents (99%) indicated they agree or strongly agree that they check the LIS to identify if WSIs are available before requesting glass slides from the department slide library. Fifty-nine (83%) responded that they view a patient's prior WSIs during clinical sign-out. Sixty-eight respondents (96%) indicated that the availability of WSIs from a prior patient specimen aided in deciding if ancillary studies were needed. Sixty-five respondents (91%) agreed with the perception that review of WSIs improved clinical TAT. Fifty-seven respondents (80%) said using WSIs improved their clinical sign-out experience. Thirty-eight respondents (54%) would feel comfortable using WSIs for primary diagnosis with the availability of glass slides upon request, compared to 16 (23%) who agreed or strongly agreed that they would feel comfortable using WSIs for primary diagnosis if glass slides would not be available. These figures may represent the unfamiliarity of using WSIs for primary diagnosis in our large academic center; it would be of interest to follow up with a similar survey in several years to better reflect the increased use of WSIs in the clinical setting.

Figure 8

Digital pathology experience survey questions and responses. A digital pathology experience survey was distributed with Likert-scale responses.

Figure 8

Digital pathology experience survey questions and responses. A digital pathology experience survey was distributed with Likert-scale responses.

## DISCUSSION

When considering implementation of a DP operation, a comprehensive and conservative analysis must be performed to determine a cost-benefit ratio. Aside from addressing departmental cultural barriers involved in transitioning to a digital microscopy platform, WSIs may have considerable additional benefits.18,19  Building a business case is also necessary to solicit buy-in from hospital administration. The upfront one-time purchases are a long-term investment in a potential new standard of care. However, there is potential for outdated hardware to be updated or replaced. There may be additional costs during times of scanner upgrades; however, with a given increase in productivity, the costs would be quickly recuperated.20  The incurred additional cost would be significantly lower than the initial upfront costs, since a portion of the budget should already be established (ie, network upgrades, file storage, software). Storage considerations may also depend on institutional policy regarding the longevity of patient data, where the decision may be to never purge any patient-related data (ie, WSIs). This commitment may vary depending on the type of institution. A different digital blueprint may be required for distributed health networks compared to single institutions or community hospitals, or based on the applicable scale of each operation (ie, clinical, educational, research, and commercial needs).

In the past few years, a few, yet increasing number of laboratories around the world have deployed a full-scale DP operation.29  Although there are many published applications and advantages of undergoing this digital transformation, practical metrics for evaluating its cost are largely based on direct costs (ie, hardware, software) and comparative equivalency to traditional microscopy (ie, diagnostic accuracy). In 2011, Washington University Department of Pathology evaluated a DP solution, based on cost savings, time savings, and improvements to patient care.9  Based on their value-added assessment, WSIs had advantages to glass slides; though enhancements to patient care, education, and research were observed, these did not result in an overall cost or time savings. This was primarily due to the length of time to scan all glass slides, and capital investment in estimated hardware and software, a significant time for QA review, as well as other costs for storage and personnel. Their results may reflect higher costs and now dated hardware and software from that time. For example, their reported ×20 and ×40 scan times were 6 minutes and 33 minutes, respectively. By comparison, current whole slide scanners have much higher efficiency and throughput.9  Although their results did not show cost or time savings, they concluded that there were opportunities in which WSIs enhanced patient care. These included generating a permanent record of the glass slides for medicolegal purposes21  in the event the original glass slide would be unavailable, lost, or broken, or for comparison from a digitized frozen section glass slide to subsequent excisional or posttreatment specimen, or if the original glass hematoxylin-eosin–stained slide would need to be sacrificed as part of additional ancillary testing when no other tissue is available for molecular or immunohistochemical testing. Other opportunities also included QA activities; tumor boards or multidisciplinary management discussions; and digital image analysis. A cost analysis from the University of Pittsburgh Medical Center in 2014 projected an overall $17.73 million 5-year cost savings across their large distributed health network.22 These calculations included but were not limited to 100% DP utilization, gains in productivity and laboratory consolidation, reduction in cancer interpretative errors, and payer/provider sharing and cost savings. Implementing this digital transformation has also allowed for an increase in domestic and international consultation services.23,24 The existing literature describing recent successful implementations of DP systems does not discretely comment on criteria that were used to evaluate their economic- or metric-based assessment. Our study illustrates internal data from a large academic center implementing a DP operation, with emphasis on practical return on investment from our evidence-based data. Using the metrics provided in our study, we believe there are tangible criteria that may be used by departments considering implementing a DP system. The advantage of having WSIs interfaced with an LIS is the increased ease of access and ready availability of patient material that can be reviewed on demand. Hitherto, pathology departments have had a local glass slide archive for recent cases (usually a few years), and owing to facility requirements, have had older cases stored in an off-site archive for long-term storage. These storage requirements stem from retention regulations citing slides must be retained for a minimum of 10 years from their date of examination.25 Additional state regulations may require more conservative retention times. With a decrease in archival glass slide transport between off-site storage facilities, due to the decreased demand for the physical glass slides in lieu of high-fidelity WSIs, departments have potential to further decrease glass slide storage costs by moving their physical storage facilities to remote, less costly locations. Digital pathology allows for these glass slide requests, usually managed by several departmental personnel, to be minimized or eliminated as based on the scale of whole slide scanning. Our findings may be applied to multiple institutional models. In our case, where a separate facility routinely requires pathology glass slide review, access to WSI material remotely and on-demand access becomes possible. At our off-site surgery center, pathologists screen the following day's surgical patient list, and in the past, have routinely requested glass slides from the slide file room for pertinent prior patient pathology. As WSIs became increasingly available, glass slide requests for this intended use were dramatically decreased. Along with the decrease in glass slide requests, due to the ready availability of WSIs from scanned prior patient material, pathologists have on-demand access to a patient's entire WSI history. This enables pathologists to compare single or multiple prior patient pathology encounters to prospective cases and allows for more efficient patient care. Our data show a decrease in IHC stain ordering, which may have represented confirmatory IHC stain orders. At a tertiary care center, this figure may be magnified; however, the data being referenced also do not capture all applicable cases owing to lack of standardized documentation when reviewing a prior digital slide for each patient. Without digital slide availability in the LIS, pathologists could request glass slides for comparison; the time delay between ordering IHC tests and receiving confirmatory test results, including with requesting and delivery of the archived glass slides, would be comparable, hence more confirmatory-type IHC tests may have been ordered. For hospital-based laboratories, this may result in a reduction in revenue from immunohistochemistry billing; however, with packaged/bundled billing in a value- or case-based reimbursement health care system, a reduction in immunohistochemistry utilization could further lead to significant laboratory savings. However, the decrease of IHC ordering as a result of implementing a DP system may have other contributing factors (ie, pathologist practice, reimbursement regulations) and may not be the sole contributor to these findings. Additionally, the increase in efficiency and decrease in laboratory testing as a result of WSI availability may decrease the burden on the laboratory staff, provide increased antibody reagent longevity, decrease TAT, and ultimately provide higher downstream satisfaction owing to the increased productivity. Comparing the department's off-site slide storage costs in the era of our DP operation, including assessment of pathology asset retrieval, physical storage, vendor labor, scanner hardware and software, WSI storage, and personnel (eg, slide file clerks, DP staff), we demonstrate, through operational cost analysis, a 5-year$1.3 million savings when compared with a non-DP operation (excluding potential decreased IHC stain ordering when reviewing prior patient pathology material). These factors included slide file clerk personnel redistribution, where owing to the decrease in glass slide requests from the intradepartmental slide library, full-time employees were redistributed to the DP operation. Of the vendor services calculations, in the DP operation, there was a decrease of 16% in asset retrieval, filing, and delivery costs. Digital file storage, which has increased exponentially, and IT infrastructure costs were also incorporated into the calculations. We determined an anticipated operational break-even point in the first quarter of 2021, about 7 years after the phased clinical implementation of a DP operation.

Other departments may have different operational initiatives and prerequisites for implementing a DP operation, and similarly, not all respective costs may have been included in this assessment. For instance, an advanced barcoding and tracking solution is a requirement for storing and interfacing WSIs with the LIS and, as this was already in place at our institution, was not incorporated in the operational cost analysis. Conversely, there are numerous operational efficiency metrics that have been reported in another cost analysis12  that were not incorporated into this cost analysis, such as interpretive accuracy, laboratory consolidation, or administrative-secretarial time, which may also contribute additional cost savings.

Importantly, aggregating a business case for DP is not purely related to vendor direct costs. There are numerous quality improvements to patient care, safety, workflow, and other intangible benefits (ie, building a pathology data warehouse, vendor interactions to improve current products, commercialization opportunities). A time and motion study by Stratman et al26  showed an overall 13.4% (43 minutes and 9 seconds) time savings in the pathologist's workday, where DP could save time by automating case assembly, queries, requests, retrieval, and delivery. A review of the DP business case can further provide information to departments seeking advice on structuring an assessment.27,28  This review discusses the increasing incidence of cancers, as well as the histopathology volume growth. There are estimated shortages of pathologists, and DP is one solution to ameliorate this decrease. The investment in a DP solution includes personnel, space/facilities, hardware, software, IT resources, and their integration. The alternative solution is not to implement a DP solution; however, under the continued pressures of forecasted trends, departments that do not adapt may find it difficult to provide novel services to their patients (ie, telepathology, image analysis, virtual tumor boards). Ultimately, the financial costs and quality benefits must be weighed and extrapolated for the respective use cases of the institution or organization.

Efficiencies seen with WSIs or DP may additionally provide qualitative value, such as an increase in pathologist satisfaction. Typically, with any cultural change, there are criticisms due to the unfamiliarity with a new system. Anecdotally, in our institution, before implementation of a DP platform, many pathologists were skeptical of whether WSIs would have high fidelity in representing glass slides. In contrast, after implementing a DP system, pathologists have become frustrated if the patient's prior pathology specimen is not scanned and available within the LIS for review. As departments continue to deploy these technologies with increase in scale, their value may become increasingly omnipresent to pathologists and administrators. In our DP experience survey, 57 pathologists (80%) reported that using WSIs improved their clinical sign-out experience. However, this was largely based on their experience viewing WSIs for retrospective cases. The survey responses also showed 38 pathologists (54%) who would feel comfortable using WSIs for primary diagnosis with glass slides available if needed, compared to 16 pathologists (23%) who would feel comfortable using WSIs for primary diagnosis if glass slides would not be readily available. This may reflect the unfamiliarity of using WSIs for primary diagnosis to date in our institution and may change over time with increasing use of these technologies.

## CONCLUSIONS

Our DP implementation has shown a noteworthy increase in efficiency and operational utility. An ancillary endpoint of increased pathologist satisfaction was also noted. A DP deployment and full-scale operation may be gauged by the following metrics, including but not limited to the number of glass slide requests as WSIs become readily available, decrease in confirmatory testing for patients with metastatic or recurrent disease, long-term decrease in off-site pathology asset costs, decrease in operational cost, and faster TAT for reporting cases. Other departments may use our benchmark data and the metrics to improve patient care and demonstrate return on investment to justify adoption of DP.

## References

1
.
FDA allows marketing of first whole slide imaging system for digital pathology: news release
. ,
2018
.
2
Cheng
CL
,
Azhar
R
,
Sng
SH
, et al.
Enabling digital pathology in the diagnostic setting: navigating through the implementation journey in an academic medical centre
.
J Clin Pathol
.
2016
;
69
(
9
):
784
792
.
3
Thorstenson
S
,
Molin
J
,
Lundström
C.
Implementation of large-scale routine diagnostics using whole slide imaging in Sweden: digital pathology experiences 2006-2013
.
J Pathol Inform
.
2014
;
5
(
1
):
14
.
4
Fraggetta
F
,
Garozzo
S
,
Zannoni
GF
, et al.
Routine digital pathology workflow: the Catania experience
.
J Pathol Inform
.
2017
;
8
:
51
.
5
Philips
.
News center. Philips and Spanish hospital Campus de la Salud start implementation of multi-year strategic partnership agreement
.
July 12, 2016
. ,
2018
.
6
Hartman
DJ
,
Pantanowitz
L
,
McHugh
JS
, et al.
Enterprise implementation of digital pathology: feasibility, challenges, and opportunities
.
J Digit Imaging
.
2017
;
30
(
5
):
555
560
.
7
Philips
.
News center. Philips supports global transformation of pathology with two new fully digitized labs in Austria
.
August 29, 2017
. ,
2018
.
8
The James
.
Ohio State set to launch first of its kind digital pathology platform. July 5, 2016
. ,
2018
.
9
Isaacs
M
,
Lennerz
JK
,
Yates
S
, et al.
Implementation of whole slide imaging in surgical pathology: a value added approach
.
J Pathol Inform
.
2011
;
2
:
39
.
10
Stathonikos
N
,
Veta
M
,
Huisman
A
, et al.
Going fully digital: perspective of a Dutch academic pathology lab
.
J Pathol Inform
.
2013
;
4
:
15
.
11
Volynskaya
Z
,
Chow
H
,
Evans
A
,
Wolff
A
,
Lagmay-Traya
C
,
Asa
SL
.
Integrated pathology informatics enables high-quality personalized and precision medicine: digital pathology and beyond
.
Arch Pathol Lab Med
.
2018
;
142
(
3
):
369
382
.
12
Cornish
TC
,
Swapp
RE
,
Kaplan
KJ
.
Whole-slide imaging: routine pathologic diagnosis
.
.
2012
;
19
(
3
):
152
159
.
13
Memorial Sloan Kettering Cancer Center
.
About Us. History & Milestones. Patient care in 2017
. ,
2018
.
14
Memorial Sloan Kettering Cancer Center
.
About Us. History & Milestones. Facts – 2009
. ,
2018
.
15
Guo
H
,
Birsa
J
,
Farahani
N
, et al.
Digital pathology and anatomic pathology laboratory information system integration to support digital pathology sign-out
.
J Pathol Inform
.
2016
;
7
:
23
.
16
Volynskaya
Z
,
Evans
AJ
,
Asa
SL
.
Clinical applications of whole-slide imaging in anatomic pathology
.
.
2017
;
24
(
4
):
215
221
.
17
Raab
SS
.
The cost-effectiveness of immunohistochemistry
.
Arch Pathol Lab Med
.
2000
;
124
(
8
):
1185
1191
.
18
Pantanowitz
L
,
Valenstein
PN
,
Evans
AJ
, et al.
Review of the current state of whole slide imaging in pathology
.
J Pathol Inform
.
2011
;
2
:
36
.
19
Evans
AJ
,
Salama
ME
,
Henricks
WH
, et al.
Implementation of whole slide imaging for clinical purposes: issues to consider from the perspective of early adopters
.
Arch Pathol Lab Med
.
2017
;
141
(
7
):
944
959
.
20
Griffin
J
,
Treanor
D.
Digital pathology in clinical use: where are we now and what is holding us back?
Histopathology
.
2017
;
70
(
1
):
134
145
.
21
Studdert
DM
,
Mello
MM
,
Gawande
AA
, et al.
Claims, errors, and compensation payments in medical malpractice litigation
.
N Engl J Med
.
2006
;
354
(
19
):
2024
2033
.
22
Ho
J
,
Ahlers
SM
,
Stratman
C
, et al.
Can digital pathology result in cost savings: a financial projection for digital pathology implementation at a large integrated health care organization
.
J Pathol Inform
.
2014
;
5
(
1
):
33
.
23
Zhao
C
,
Wu
T
,
Ding
X
, et al.
International telepathology consultation: three years of experience between the University of Pittsburgh Medical Center and KingMed Diagnostics in China
.
J Pathol Inform
.
2015
;
6
:
63
.
24
Vodovnik
A.
Distance reporting in digital pathology: a study on 950 cases
.
J Pathol Inform
.
2015
;
6
:
18
.
25
US National Archives and Records Administration
.
2006
.
Code of federal regulations. Standard: retention requirements. Title 42. Chapter 4. Section 493.1105
.
26
Stratman
C
,
Drogowski
L
,
Ho
J.
Digital pathology in the clinical workflow: a time and motion study
.
Abstract presented at
:
Pathology Visions
;
October 2010
;
San Diego, CA
.
27
Williams
BJ
,
Bottoms
D
,
Clark
D
,
Treanor
D.
Future-proofing pathology part 2: building a business case for digital pathology
[
published online ahead of print March 16
,
2018
].
J Clin Pathol
. doi:.
28
Williams
BJ
,
Bottoms
D
,
Treanor
D.
Future-proofing pathology: the case for clinical adoption of digital pathology
.
J Clin Pathol
.
2017
;
70
(
12
):
1010
1018
.

## Author notes

Dr Klimstra is a consultant with Paige.AI; Dr Klimstra and Ms Samboy are equity shareholders in Paige.AI. The other authors have no relevant financial interest in the products or companies described in this article.