Implementation of Digital Pathology Offers Clinical and Operational Increase in Efficiency and Cost Savings

This evaluation is based on the experience of a high-volume, academic, tertiary care cancer center. The institution is a 473-bed comprehensive cancer center, with more than 10 000 employees, including 1148 attending physicians and 133 institute member scientists. The institution admits more than 23 506 patients per year and has more than 722 238 outpatient visits at local and regional facilities.^13,14  The Department of Pathology includes more than 80 pathologists and 39 fellows. In 2017, more than 160 000 total accessions and more than 1.5 million glass slides were produced. The department has a long-standing history with DP and was an early adopter, having used WSIs for more than a decade. The clinical workflow in reviewing the digital slides is currently for retrospective review.

During the past decade, the institutional glass slide scanning protocol has evolved. The scanning timeline was developed as a phased approach for retrospective and prospective glass slide digitization. Selected glass slides are marked for scanning in each case and then follow a department scanning protocol. The aim of the DP implementation was concerted around operational workflow and efficiency as opposed to an enterprise-wide full DP deployment.

The pathology department at Memorial Sloan Kettering Cancer Center (MSKCC) has dedicated whole slide scanning rooms (Figure 1), as well as other scanning locations that house a variety of whole slide scanners including Leica Aperio AT2 and XT (Leica Biosystems, Buffalo Grove, Illinois); 3DHistech Pannoramic 250, Pannoramic 1000, and confocal scanner (3DHistech, Budapest, Hungary); Hamamatsu Nanozoomer 2.0HT and S60 (Hamamatsu Photonics, Hamamatsu City, Shizuoka, Japan); Philips Ultra-Fast Scanner (Philips, Amsterdam, the Netherlands); and VisionTek M6 digital microscopes (Sakura Finetek, Torrance, California). The predominant scanners used for the clinical operation during the time of this study were the Leica/Aperio AT2 whole slide scanners. The number of AT2 scanners has varied through the DP implementation. At the time of this study, 6 AT2 scanners and 3 digital scanning technicians (ie, 1:2 scan tech to scanner ratio) were responsible for digital slide scanning. Selected glass slides for digitization follow a department scanning protocol (see Results). Slides are scanned predominantly at ×20 (0.5 μm/pixel); however, ×40 (0.25 μm/pixel or 0.16 μm/pixel) scans are performed on the basis of clinical need (ie, hematopathology, cytology). The WSIs are transferred and stored in the institution's data center. The department has 1 gigabit/s network connections for each computer workstation and whole slide scanner. The whole slide scanners share a 10 gigabit/s connection to the institutional data center. The importance of interfacing the WSIs with the department's LIS has been described.^15,16  The advanced barcoding and tracking module implemented at our institution uses 2D barcodes that are decoded by the Leica Aperio whole slide scanners, and in turn interface the eSlide Manager database software with the LIS (Cerner CoPathPlus, Cerner Corporation, Kansas City, Missouri). Whole slide images are launched from within the LIS and are viewed in a separate WSI viewer application (ie, Web-based MSK Slide Viewer or client-based Leica Aperio Imagescope v12.3).

The departmental glass slide archive is maintained by dedicated staff and is accessible only to them. All other staff members are restricted to requesting slides via a glass slide requisition for all request types (ie, clinical, conference, research). Glass slide requisitions are scanned and stored for auditing. Requisitions were collected and tabulated between the years 2014–2017. In addition, an off-site surgical center opened in 2016, where pathologists are stationed away from the main campus for intraoperative consultations. Pathologists review the surgery schedule the day prior, and could similarly request glass slides for review. These glass slide requests were also tabulated and reviewed.

When reviewing a case prospectively, the patient's complete clinical and pathology history is routinely examined. When prior materials (ie, glass slide or WSIs from other specimens) are reviewed, pathologists may document in their report whether the patient's prior material was comparatively reviewed. A natural language search of our LIS was queried to retrieve these cases, and subsequent comparative analysis of immunohistochemical (IHC) orders. Evaluated cases excluded prognostic or predictive IHC stain ordering.

Comparative costs before and after implementing the DP system were analyzed, as based on costs for departmental scanning of select slides from prospective cases. Factors considered include personnel (ie, slide file clerks, slide scanning), hardware, software, service agreements, information technology (IT) infrastructure, digital storage, glass slide physical asset storage, and off-site storage vendor services (ie, filing, retrieving, delivery costs). Cases from recent years are stored on-site in the department glass slide archive for ready access retrieval; however, older pathology assets (glass slides, blocks, documents) are stored in an off-site storage facility. Costs for off-site storage, retrieval, and other vendor services were calculated and compared to the costs of implementing and maintaining a DP operation based on prescanning and postscanning department requirements.

The operational cost analysis was conducted by using certain assumptions. Every effort was made to include costs that were obtained from actual departmental data of incurred costs. Comparative operational analysis was acquired from predigital pathology personnel, annual increase in surgical pathology accession volume, and vendor pathology glass slide requests. Postdigital pathology factors were based on scanning a minimum of 40 000 glass slides per month (480 000 per year), which reflects departmental intended scanning throughput and personnel for operation, and IT infrastructure including vendor-laboratory information system interface. Internal data were used for calculations based on current hardware, personnel, vendor costs, and institutional digital file storage.

The LIS was queried to identify cases with and without prior WSI. Turnaround times (TATs) for each specimen class (ie, biopsy, surgical resection, consultation) were compared. Turnaround time was defined as the point of case accession to the time of reported diagnosis.

A DP experience survey was created and distributed to all pathologists and trainees (n = 124). The survey included questions based on their clinical utilization and perspective using DP. Questions used a 5-point Likert rating scale for each response.

As of July 2018, with the first scanned slide in 2007, a total of 1 039 340 slides were scanned, comprising 270 terabytes in digital storage. These WSIs include clinical, research, and educational slides, and may underestimate average file sizes if measured as a whole. Before August 2015, glass slide scanning was performed on a research or ad hoc basis. Whole slide images were scanned and stored locally, available only in network-attached storage. Clinical scanning was initiated after implementation of the LIS interface. The LIS supported WSI integration through a Health Level 7 (HL7) messaging interface between the barcoding and tracking module and a customizable add-on solution from the vendor, PICSPlus (Cerner Corporation). Scanning was initiated for glass slides sent to the institution for consultation in August 2015. Subsequent scanning of specimens from biopsies performed at our institution commenced in January 2016. Glass slides from surgical resection specimens began to be digitized in February 2017. This phased approach allowed clinical operations the flexibility to scale while accommodating for revisions to workflow, IT, networking, and security. Clinical scanning between 2015 and 2017 included 424 901 WSIs that were scanned and linked within the LIS (Figure 2; and Figure 3, a and b). For clinical whole slide scanning, the average file size was 482 MB per slide. There was an average of 3 glass slides scanned per case during this period (range, 1–97 slides scanned per case). Whole slide images from within 2 years were available on hot/tier 1 storage for image retrieval, with WSIs older than 2 years on warm storage/tier 2. No additional compression of digital slides was performed for digital storage.

Specimens are accessioned in the LIS by using an advanced barcoding and tracking module and follow traditional histology workflow. After case assembly and glass slide distribution, cases are disseminated to subspecialty mailboxes. Fellows or attending pathologists then review the clinical case and designate select slides in each case to be digitized. Guidelines for glass slide scanning are as follows: (1) for biopsies, 1 block from each specimen part is selected; (2) for resection cases, all frozen section slides and frozen section controls, glass slides with diagnostically relevant grading/staging information, predictive or prognostic immunohistochemistry; (3) for consultation cases, all diagnostic and prognostically relevant glass slides. Rush and priority scanning services have also been made available to the department, where requisition forms are filled out and provided with the glass slides and are prioritized to be scanned for clinical, educational, or research initiatives.

Workflow for scanning consultation material was accomplished in conjunction with the vendor. For tracking purposes, the addition of our institution label (with 2D barcode) is placed on all consultation material. Placement of the label atop the outside institution label would often cover key information (ie, stain, patient identifier). Therefore, placing our institution label below the outside consulting laboratory label was preferred; however, this region was not captured in the macrocamera's field of view, which would not be visualized and decode our institution's barcode, and the WSIs would not be linked in the LIS. After discussions with the vendor, this was rectified and the camera's field of view was expanded to cover the area below the label.

This change introduced a secondary issue of potentially having 2 barcodes in the same field of view of the whole slide scanner's macrocamera. Our department initially used color-coding label stickers (eg, red dot stickers). These red dot label stickers were placed over outside institutional barcodes to hide them from the whole slide scanner camera, such that the camera would only recognize and decode our department's barcode. However, as our implementation scaled, these were too cumbersome to be used for placement atop the outside barcodes, as well as inefficient for removal after scanning (ie, before returning to the consulting institution). The department then instituted liquid chalk ink markers, which facilitated easier placement and removal of “erasable chalk ink” for scanning workflow purposes (Figure 4, a and b).

After clinical review of glass slides, the glass slides marked for scanning are collected and cleaned. Cleaning the glass slides includes removal of all marker ink annotations with alcohol, which also acts to remove dust particles or fingerprints from the glass slides. All glass slides received as marked for scanning (eg, red dot on slide label) are scanned; in the event of a slide being received broken, these slides are manually scanned separately, if possible. Glass slides with extensive preanalytic artifacts (ie, air bubbles, knife cutting) are ameliorated in the laboratory before scanning, dependent on the extent of the artifact, and then included in the subsequent scanning workflow. All slides marked for scanning are aggregated on shelving racks outside the whole slide scanning room. The scanning room is in the Department of Pathology, located on the same floor as the glass slide archive, pathologists' offices, sign-out areas, and histology laboratory. This allows for efficient transportation and management of the glass slides in the scanning workflow. On the shelving racks, a queue of glass slides includes a thorough quality assurance (QA) process by DP technicians. This process includes a barcoding and tracking station to scan each glass slide; this ensures each barcode is readable by handheld barcode readers and marks the glass slide scanned (received for scanning in the LIS). Glass slides without barcodes, or barcodes that failed reading by the handheld reader, have new patient labels with barcodes printed and placed on the glass slide. The QA process also involves macroevaluation of preanalytic slide artifacts to be resolved before scanning whenever possible. After the glass slides are scanned, a second QA by a digital imaging coordinator or supervisor is performed to ensure the WSI is of adequate quality and is present within the LIS. This second QA process involves investigation in the eSlide Manager database where all slides are provided a quality factor (range, 0–100) by the vendor's algorithm and thumbnail overview; all slides with a quality factor below 90 are reviewed—whereby the policy transitioned from iterative inspection for every 10, 4, and 1 WSI(s) for surgical pathology, molecular studies, and cytology, respectively—and evaluated for typical WSI artifacts (ie, out of focus, tissue detection). Any slide requiring a rescan is included in the subsequent batch of slides or is manually scanned. After scanning, when viewing a digital slide in the MSK Slide Viewer, an additional QA tool has been implemented whereby the reviewer can provide direct feedback to the digital scanning personnel on image quality, artifacts, or network performance. All glass slides then proceed to be filed in the department slide library archive. A fully loaded whole slide scanner (eg, 400 slides) scanned at 0.25 μm/pixel (×40 equivalent resolution) takes approximately 43 hours to be completed. Turnaround time from initial glass slide retrieval to glass slide archive storage for clinical and priority scanning is 24 to 48 hours; nonclinical scanning otherwise has a 4- to 5-day TAT. Digital slide scanning is performed in 2 shifts, 6 days per week.

In 2014, before initiation of the digital scanning operation, there were 19 369 archival glass slide requisitions. In 2015, a total of 20 745 archival glass slide requests were retrieved by the slide library staff, while 2016 and 2017 had 12 336 and 1426 archival glass slide requests, respectively. Archival glass slide requests showed an overall 93% decrease (Figure 5, a). At the off-site surgical center, pathologists' requests for prior archived material (ie, glass slides) from patients with anticipated intraoperative consultations showed a 97% decrease (range, 0–32; average 8 requests per month), dramatically reducing requests, owing to remote access of the patient's WSIs (Figure 5, b). With this decrease in slide requests from the department slide library, 3 full-time employees from the slide file room were redistributed and incorporated into the DP operations workflow.

From the natural language query, 2801 cases were retrieved from the LIS where pathologists documented review of patient material from a prior specimen. In 2014, before glass slides were scanned, 52% of cases had IHC staining ordered when prior material was reviewed, as compared to 19% and 21% of cases with IHC staining ordered on prospective specimens in 2016 and 2017, respectively.

From the years after clinical scanning of glass slides commenced (2015–2017), pathologist case review showed 1884 documented comparisons to WSIs from prior scanned material (median, 756 cases per year). When prior WSIs were available in reviewing a prospective case, in 2016, a total of 745 cases did not have IHC staining ordered, as compared to 172 cases that had IHC staining ordered. In 2017, pathologists did not order confirmatory IHC tests in 767 cases, as compared to 200 cases that had IHC staining ordered (Figure 6). Pathologists ordered less ancillary studies by up to 75.4% when WSIs were available. A median of 3 supportive or confirmatory IHC tests were ordered per case. Given an average cost of $50 per IHC test,¹⁷  an anticipated savings of $113 400 can be expected at our institution, with a median number of 756 cases per year with documented WSI review.

A cost savings analysis was conducted by calculating and comparing prescanning and postscanning costs with the following budgetary considerations reviewed: required personnel (ie, slide file clerks, slide scanning), hardware (including capital equipment purchases), software, service agreements, IT infrastructure, digital storage, glass slide physical asset storage, and off-site storage vendor services (ie, filing, retrieving, delivery costs) during a 5-year period (2014–2018), based on incurred costs while implementing a DP system at our institution. A projected savings of more than $267 000/y was calculated secondary to personnel restructuring, decreased vendor services (ie, pathology asset retrieval, labor and other vendor services) due to the decreased need for glass slide transport and ready availability of WSIs, and physical storage of glass slides (Figure 7, a). These calculations show a $1.3 million projected savings during a 5-year period (2019–2023; Figure 7, b).

Audit of biopsy, surgical resection, and consultation cases between 2014 and 2016 was reviewed. The average TAT for 59 571 surgical cases was tabulated. In 2014, 2015, and 2016, the average TAT was 4 days, 4 days, and 3 days, respectively. For surgical resection specimens, cases with prior WSIs were reported 25% sooner (1 day). This may be attributed to the decrease in glass slide retrieval (ie, patient prior material) and ready access to WSIs in the LIS. However, this effect was not noticed in other specimen class subtypes. Consults and biopsy specimens are typically the first accessioned case of a patient at a large cancer center, and may not have prior material. The effect of having prior digital images may not be applicable for these patients, or may be nullified in these other specimen types.

The DP experience survey was distributed to 124 users and the results from 71 respondents (57.3%) were recorded. Of the 71 respondents, 28 (39%) were fellows and 43 (61%) were from faculty of varying ranks. Figure 8 shows the respondents' chart and respective questions with Likert-scale responses. Seventy respondents (99%) indicated they agree or strongly agree that they check the LIS to identify if WSIs are available before requesting glass slides from the department slide library. Fifty-nine (83%) responded that they view a patient's prior WSIs during clinical sign-out. Sixty-eight respondents (96%) indicated that the availability of WSIs from a prior patient specimen aided in deciding if ancillary studies were needed. Sixty-five respondents (91%) agreed with the perception that review of WSIs improved clinical TAT. Fifty-seven respondents (80%) said using WSIs improved their clinical sign-out experience. Thirty-eight respondents (54%) would feel comfortable using WSIs for primary diagnosis with the availability of glass slides upon request, compared to 16 (23%) who agreed or strongly agreed that they would feel comfortable using WSIs for primary diagnosis if glass slides would not be available. These figures may represent the unfamiliarity of using WSIs for primary diagnosis in our large academic center; it would be of interest to follow up with a similar survey in several years to better reflect the increased use of WSIs in the clinical setting.

When considering implementation of a DP operation, a comprehensive and conservative analysis must be performed to determine a cost-benefit ratio. Aside from addressing departmental cultural barriers involved in transitioning to a digital microscopy platform, WSIs may have considerable additional benefits.^18,19  Building a business case is also necessary to solicit buy-in from hospital administration. The upfront one-time purchases are a long-term investment in a potential new standard of care. However, there is potential for outdated hardware to be updated or replaced. There may be additional costs during times of scanner upgrades; however, with a given increase in productivity, the costs would be quickly recuperated.²⁰  The incurred additional cost would be significantly lower than the initial upfront costs, since a portion of the budget should already be established (ie, network upgrades, file storage, software). Storage considerations may also depend on institutional policy regarding the longevity of patient data, where the decision may be to never purge any patient-related data (ie, WSIs). This commitment may vary depending on the type of institution. A different digital blueprint may be required for distributed health networks compared to single institutions or community hospitals, or based on the applicable scale of each operation (ie, clinical, educational, research, and commercial needs).

In the past few years, a few, yet increasing number of laboratories around the world have deployed a full-scale DP operation.^2–9  Although there are many published applications and advantages of undergoing this digital transformation, practical metrics for evaluating its cost are largely based on direct costs (ie, hardware, software) and comparative equivalency to traditional microscopy (ie, diagnostic accuracy). In 2011, Washington University Department of Pathology evaluated a DP solution, based on cost savings, time savings, and improvements to patient care.⁹  Based on their value-added assessment, WSIs had advantages to glass slides; though enhancements to patient care, education, and research were observed, these did not result in an overall cost or time savings. This was primarily due to the length of time to scan all glass slides, and capital investment in estimated hardware and software, a significant time for QA review, as well as other costs for storage and personnel. Their results may reflect higher costs and now dated hardware and software from that time. For example, their reported ×20 and ×40 scan times were 6 minutes and 33 minutes, respectively. By comparison, current whole slide scanners have much higher efficiency and throughput.⁹  Although their results did not show cost or time savings, they concluded that there were opportunities in which WSIs enhanced patient care. These included generating a permanent record of the glass slides for medicolegal purposes²¹  in the event the original glass slide would be unavailable, lost, or broken, or for comparison from a digitized frozen section glass slide to subsequent excisional or posttreatment specimen, or if the original glass hematoxylin-eosin–stained slide would need to be sacrificed as part of additional ancillary testing when no other tissue is available for molecular or immunohistochemical testing. Other opportunities also included QA activities; tumor boards or multidisciplinary management discussions; and digital image analysis. A cost analysis from the University of Pittsburgh Medical Center in 2014 projected an overall $17.73 million 5-year cost savings across their large distributed health network.²²  These calculations included but were not limited to 100% DP utilization, gains in productivity and laboratory consolidation, reduction in cancer interpretative errors, and payer/provider sharing and cost savings. Implementing this digital transformation has also allowed for an increase in domestic and international consultation services.^23,24  The existing literature describing recent successful implementations of DP systems does not discretely comment on criteria that were used to evaluate their economic- or metric-based assessment. Our study illustrates internal data from a large academic center implementing a DP operation, with emphasis on practical return on investment from our evidence-based data.

Using the metrics provided in our study, we believe there are tangible criteria that may be used by departments considering implementing a DP system. The advantage of having WSIs interfaced with an LIS is the increased ease of access and ready availability of patient material that can be reviewed on demand. Hitherto, pathology departments have had a local glass slide archive for recent cases (usually a few years), and owing to facility requirements, have had older cases stored in an off-site archive for long-term storage. These storage requirements stem from retention regulations citing slides must be retained for a minimum of 10 years from their date of examination.²⁵  Additional state regulations may require more conservative retention times. With a decrease in archival glass slide transport between off-site storage facilities, due to the decreased demand for the physical glass slides in lieu of high-fidelity WSIs, departments have potential to further decrease glass slide storage costs by moving their physical storage facilities to remote, less costly locations.

Digital pathology allows for these glass slide requests, usually managed by several departmental personnel, to be minimized or eliminated as based on the scale of whole slide scanning. Our findings may be applied to multiple institutional models. In our case, where a separate facility routinely requires pathology glass slide review, access to WSI material remotely and on-demand access becomes possible. At our off-site surgery center, pathologists screen the following day's surgical patient list, and in the past, have routinely requested glass slides from the slide file room for pertinent prior patient pathology. As WSIs became increasingly available, glass slide requests for this intended use were dramatically decreased.

Along with the decrease in glass slide requests, due to the ready availability of WSIs from scanned prior patient material, pathologists have on-demand access to a patient's entire WSI history. This enables pathologists to compare single or multiple prior patient pathology encounters to prospective cases and allows for more efficient patient care. Our data show a decrease in IHC stain ordering, which may have represented confirmatory IHC stain orders. At a tertiary care center, this figure may be magnified; however, the data being referenced also do not capture all applicable cases owing to lack of standardized documentation when reviewing a prior digital slide for each patient. Without digital slide availability in the LIS, pathologists could request glass slides for comparison; the time delay between ordering IHC tests and receiving confirmatory test results, including with requesting and delivery of the archived glass slides, would be comparable, hence more confirmatory-type IHC tests may have been ordered. For hospital-based laboratories, this may result in a reduction in revenue from immunohistochemistry billing; however, with packaged/bundled billing in a value- or case-based reimbursement health care system, a reduction in immunohistochemistry utilization could further lead to significant laboratory savings. However, the decrease of IHC ordering as a result of implementing a DP system may have other contributing factors (ie, pathologist practice, reimbursement regulations) and may not be the sole contributor to these findings. Additionally, the increase in efficiency and decrease in laboratory testing as a result of WSI availability may decrease the burden on the laboratory staff, provide increased antibody reagent longevity, decrease TAT, and ultimately provide higher downstream satisfaction owing to the increased productivity.

Comparing the department's off-site slide storage costs in the era of our DP operation, including assessment of pathology asset retrieval, physical storage, vendor labor, scanner hardware and software, WSI storage, and personnel (eg, slide file clerks, DP staff), we demonstrate, through operational cost analysis, a 5-year $1.3 million savings when compared with a non-DP operation (excluding potential decreased IHC stain ordering when reviewing prior patient pathology material). These factors included slide file clerk personnel redistribution, where owing to the decrease in glass slide requests from the intradepartmental slide library, full-time employees were redistributed to the DP operation. Of the vendor services calculations, in the DP operation, there was a decrease of 16% in asset retrieval, filing, and delivery costs. Digital file storage, which has increased exponentially, and IT infrastructure costs were also incorporated into the calculations. We determined an anticipated operational break-even point in the first quarter of 2021, about 7 years after the phased clinical implementation of a DP operation.

Other departments may have different operational initiatives and prerequisites for implementing a DP operation, and similarly, not all respective costs may have been included in this assessment. For instance, an advanced barcoding and tracking solution is a requirement for storing and interfacing WSIs with the LIS and, as this was already in place at our institution, was not incorporated in the operational cost analysis. Conversely, there are numerous operational efficiency metrics that have been reported in another cost analysis¹²  that were not incorporated into this cost analysis, such as interpretive accuracy, laboratory consolidation, or administrative-secretarial time, which may also contribute additional cost savings.

Importantly, aggregating a business case for DP is not purely related to vendor direct costs. There are numerous quality improvements to patient care, safety, workflow, and other intangible benefits (ie, building a pathology data warehouse, vendor interactions to improve current products, commercialization opportunities). A time and motion study by Stratman et al²⁶  showed an overall 13.4% (43 minutes and 9 seconds) time savings in the pathologist's workday, where DP could save time by automating case assembly, queries, requests, retrieval, and delivery. A review of the DP business case can further provide information to departments seeking advice on structuring an assessment.^27,28  This review discusses the increasing incidence of cancers, as well as the histopathology volume growth. There are estimated shortages of pathologists, and DP is one solution to ameliorate this decrease. The investment in a DP solution includes personnel, space/facilities, hardware, software, IT resources, and their integration. The alternative solution is not to implement a DP solution; however, under the continued pressures of forecasted trends, departments that do not adapt may find it difficult to provide novel services to their patients (ie, telepathology, image analysis, virtual tumor boards). Ultimately, the financial costs and quality benefits must be weighed and extrapolated for the respective use cases of the institution or organization.

Efficiencies seen with WSIs or DP may additionally provide qualitative value, such as an increase in pathologist satisfaction. Typically, with any cultural change, there are criticisms due to the unfamiliarity with a new system. Anecdotally, in our institution, before implementation of a DP platform, many pathologists were skeptical of whether WSIs would have high fidelity in representing glass slides. In contrast, after implementing a DP system, pathologists have become frustrated if the patient's prior pathology specimen is not scanned and available within the LIS for review. As departments continue to deploy these technologies with increase in scale, their value may become increasingly omnipresent to pathologists and administrators. In our DP experience survey, 57 pathologists (80%) reported that using WSIs improved their clinical sign-out experience. However, this was largely based on their experience viewing WSIs for retrospective cases. The survey responses also showed 38 pathologists (54%) who would feel comfortable using WSIs for primary diagnosis with glass slides available if needed, compared to 16 pathologists (23%) who would feel comfortable using WSIs for primary diagnosis if glass slides would not be readily available. This may reflect the unfamiliarity of using WSIs for primary diagnosis to date in our institution and may change over time with increasing use of these technologies.

Our DP implementation has shown a noteworthy increase in efficiency and operational utility. An ancillary endpoint of increased pathologist satisfaction was also noted. A DP deployment and full-scale operation may be gauged by the following metrics, including but not limited to the number of glass slide requests as WSIs become readily available, decrease in confirmatory testing for patients with metastatic or recurrent disease, long-term decrease in off-site pathology asset costs, decrease in operational cost, and faster TAT for reporting cases. Other departments may use our benchmark data and the metrics to improve patient care and demonstrate return on investment to justify adoption of DP.