AI-Powered Biomolecular-Specific and Label-Free Multispectral Imaging Rapidly Detects Malignant Neoplasm in Surgically Excised Breast Tissue Specimens

The TumorMAP system relies on the intrinsic tissue AF of various endogenous fluorophores found in different tissue types. The TumorMAP system leverages changes in a matrix of these fluorophores that form the basis for tissue analysis and discerning cancerous tissue. It uses multispectral excitation and detection not only to increase the information content but also to produce differential and specific emission signals. This widefield system captures multispectral images and does not require any X-Y movement or scanning to evaluate the specimen. The multiexcitation in the UV region and multidetection strategy allows the acquisition of comprehensive and rich biomolecular and morphologic information from both cells and the extracellular matrix. The TumorMAP system is contact-free, fully automated, does not require sample preparation, and provides tissue imaging in quasi–real-time.

This preclinical research study was approved by the Institutional Review Board of Yale Smilow Cancer Center (New Haven, Connecticut). Under the study protocol, following tissue excision during breast lumpectomy surgery, the fresh breast tissue specimens are sent to the surgical pathology laboratory where they are inked and grossed by the pathology assistant or resident per the institution’s established surgical pathology process. After grossing into blocks and before tissue fixation, a selected set of the tissue blocks were imaged with the TumorMAP system. To maintain tissue integrity and to preserve the routine clinical workflow, no chemicals or other processing steps were applied to the tissue block. The total imaging time for a tissue block was only ∼32 seconds, ensuring that no undue drying of specimens took place during imaging and that the process did not impact the cycle time of the tissue grossing–to-fixation process. After the imaging, the tissue blocks were sent through the routine formalin-fixation and paraffin-embedding (FFPE) process. The corresponding H&E slides of the tissue blocks imaged by the TumorMAP system were digitized and used for histopathologic investigation and label creation for algorithm development. All patient-identifying data were de-identified for this study.

A subset of multispectral images was combined in a certain way to generate a composite image for registration with H&E images. The original H&E image of the tissue block was geometrically transformed to match with the AF composite image measured from the same block; 2 mm × 2 mm “tiles” containing areas of high tissue homogeneity on the H&E images were identified and transcribed onto the UV composite image. Each tile comprised 50 × 50 = 2500 image pixels, each of which represents a measure of the “biomolecular optical fingerprint” of the tissue at that spatial location. The selected homogeneous areas from the H&E images were then cropped and independently scored by 2 board-certified pathologists.

The primary inclusion criterion for malignant cases was that the 2 mm × 2 mm “tiles” should have a “cancer infiltration score” (CIS) of 80% or greater and no case selections were made on the basis of cancer type, grades or stages, receptor status, neoadjuvant status, or any other patient metadata. The benign category was represented by tiles that included different types of benign tissue, with homogeneity of 80% or greater as ground truth.

The 2 pathologists examined the targeted homogeneous areas (tiles) to identify the tissue types and scored the tiles for tissue identification. In the case of a cancer tile, a CIS was also noted. The CIS was defined as the percentage area of the tile that has cancer or is affected by the cancer field, including local cancer-associated desmoplasia. In case of a discrepancy between the 2 pathologists about the tissue-type assignment, a joint scoring session was performed to reach a consensus. Also, if the 2 pathologists’ scores were discordant by 15% or more, the tile was rescored by the same 2 pathologists to reach a consensus. A mean infiltration score was used for training the algorithm. The detailed tissue processing and algorithm training workflow is illustrated in Figure 1. The system was trained and tested by using a leave-one-patient-out validation process, and a CIS of 80% or greater was used for the training.

We measured multispectral AF images of breast tissue specimens, which resulted in 417 tiles obtained from 115 patients in the age range of 34 to 93 years. These specimens comprised both benign and cancerous tissues. The benign category included adipose, fibrous, and glandular tissues, along with different types of other nonmalignant tissues such as desmoplasia and inflammation (not associated with fat necrosis). Of these 417 tiles, the pathologists identified 156 tiles containing cancer from 55 patients. The Table shows the patient demographic details, cancer types, Nottingham score, cancer stage, estrogen receptors, human epidermal growth factor receptor 2 (HER2) scores, and neoadjuvant therapy status. Of the 55 patients’ cancer tiles, the pathologic read indicated N = 44 patients with primarily invasive ductal carcinoma (IDC), N = 7 with only ductal carcinoma in situ (DCIS), and N = 4 with invasive lobular carcinoma (ILC). Notably, this cancer type distinction is based on postoperative histopathology, not on the diagnosis before surgery. As evident in the Table, the patients had significant diversity in Nottingham scores, cancer stages, estrogen receptor and HER2 expression levels, and neoadjuvant therapy status.

We developed an ensemble classifier consisting of multiple base learners that were then combined through a meta learner for final classification. The classifier was tested by using leave-one-patient-out cross-validation. Figure 2 illustrates the confusion matrix reflecting the performance of the OncoSight AI classifier. Of a total of 417 labeled tiles, 156 tiles contained cancer, and the remaining 261 tiles were labeled as benign as per histologic ground truth. Our classifier yielded a sensitivity of 82%, specificity of 91%, positive predictive value of 84%, and negative predictive value of 89% with cancer prevalence of 38%. We also calculated the Matthews correlation coefficient (MCC), which is known as one of the most reliable and robust performance metrics of a binary classifier and has recently been recommended over diagnostic odds ratio.³⁸  A coefficient of +1 represents a perfect prediction, and our MCC value was 0.73.

We used the trained OncoSight AI classifier to create a heatmap at the full specimen level. This is performed by using a sliding window approach in which we selected a square patch of a specific pixel size. This patch is used as the input for our trained model, and the model calculates the probability of cancer in this patch. The probability score is assigned to the center pixel, and the window then slides across all the pixels in the test image to create a probability map. The final probability map is displayed as a heatmap to show cancer probability of 0% to 100% from blue to red.

Figure 3, A, shows a representative heatmap displaying the probability of cancer computed by using the above classifier on a tissue block that did not have any cancer. The corresponding ground truth H&E image is shown in Figure 3, B. As evident, the predicted heatmap is in good concordance with the H&E image devoid of identifiable cancer features. The small black points/areas within the cancer probability map represent background or dye correction artifacts.

Figure 4, A, and Figure 5, A, depict the cancer heatmap on histologically confirmed positive IDC blocks along with corresponding H&E images in Figure 4, B, and Figure 5, B. The tissue block in Figure 4 was from a 69-year-old woman with grade T2 cancer and known diabetes. The cancer heatmap in Figure 5, A, was generated from another patient’s tissue block with no diabetic history and with cancer grade T1. The heatmaps show the presence of cancer on the tissue blocks and spatially correlate well with the corresponding H&E slides.

Figure 6, A, shows the cancer heatmap on a predominantly ILC block with a corresponding H&E image (Figure 6, B). The heatmap indicates the presence of cancer on the tissue block and spatially correlates well with the corresponding H&E slide except for the top right area, which shows a false positive. This false positive can be attributed to deformation and distortion of the imaged tissue block during the FFPE process and microtoming.

Figure 7, A, shows a cancer heatmap on a tissue block containing DCIS compared with the corresponding H&E image from the same block in Figure 7, B. The heatmap shows the presence of cancer on the tissue block and spatially correlates with the corresponding H&E slide.

Figure 8 presents the receiver operating characteristic (ROC) curve for the ensemble classifier for neoplasm detection. The ROC curve plots sensitivity (true positive rate) versus false-positive rate (1 − specificity) for the decision algorithm as the discrimination threshold is varied. The area under the curve is computed to be 0.92.

We have developed a label-free optical imaging platform that exploits the endogenous multispectral AF contrast of tissue to discern neoplasms. The AF images reveal a different biomolecular composition at each location in the images, and quantitative analysis of this spatial-spectral content of tissue provides richer information than the fluorescence spectrum. The advent and availability of UVC LEDs, advancements in UV filter technology, and the emergence of AI have recently enabled full exploitation of the rich optical contrast of biomolecular fluorophores embedded in tissues. The underlying rationale for AF imaging–based tissue analysis is that multispectral AF images would provide a snapshot of the integral information of tissue fluorophores and, when combined with advanced ML, would capture latent biomolecular and morphologic differences that are encoded in the multispectral images.

In terms of clinical applications, fundus AF imaging is a routinely used diagnostic tool in ophthalmology.³⁹  Another AF-based device, the PTeye⁴⁰  system, has been cleared by the FDA to aid surgeons in the identification of parathyroid tissue during thyroid and parathyroid surgery. Further, the feasibility of tissue AF imaging has been established in clinical studies for real-time sampling and treatment guidance of bioburden in chronic wounds.⁴¹  Apart from the surgical space, AF microscopy has been used in the last few years in conjunction with a deep learning framework to create virtual histologic images.^42,43  However, prior work using AF to discriminate BCS has been limited, relying on fiber probes and using simpler data analysis tools. Our imaging platform effectively and optimally uses the comprehensive and rich biomolecular information embedded in the tissue matrix from both cells and the extracellular matrix and offers a potentially transformative assessment tool by using the prior rigor of research in AF of tissue and exploiting the very recent advancement in UV LEDs and AI/ML framework.

Other device-based intraoperative methods used in BCS include specimen imaging by radiography and by intraoperative ultrasonography. These devices provide some information about the tissue microstructure. Presently, the interpretation of these outputs requires trained personnel (a radiologist). A hand-held device called MarginProbe (Dilon Technologies, Inc.) that uses dielectric spectroscopy to characterize tissue has been marketed for BCS margin detection. In its pivotal clinical study, MarginProbe demonstrated margin level device sensitivity of 75% and margin level specificity of 46%, with a positive predictive value of 22% and a negative predictive value of 90%. The standard-of-care arm (ie, not aided by the device) recorded a sensitivity of 34%, a specificity of 83%, a positive predictive value of 30%, and a negative predictive value of 86%.⁴⁴  As reported by LeeVan et al,⁴⁵  the MarginProbe, when used as an adjunct to the standard operating procedure, does not significantly reduce re-excision rates in BCS.

There are 4 principal advantages of the TumorMAP system: First, it is widefield imaging and therefore allows near real-time detection. Second, it provides comprehensive multispectral information that is interpretable and attributable to the known biology and biochemical factors associated with normal and malignant tissue. Third, a successful demonstration of this approach will pave the way for the realization of an accurate and label-free device for neoplasm detection in the OR and in pathology. Fourth, its cost-effectiveness and ease of use have the potential to make effective BCS more accessible to people not served by major cancer centers.

The classification results shown in Figure 2 are very encouraging. These models are trained and tested on highly homogeneous data, and we achieved a classification accuracy of 88%. As illustrated in the confusion matrix, TumorMAP missed 28 tiles from 21 patients, and 1 patient was without a grade due to DCIS. Of 20 patients with histologic grades, 5 patients had grade I (Nottingham score 3–5) cancer, 11 patients had grade II cancer, and 4 patients had grade III cancer. We did not observe any marked association of false-negative rate with cancer grade in view of a total number of patients with various Nottingham scores as mentioned in the Table—grade I: 11, grade II: 29, grade III: 10 patients.

Several inherent and known errors are attributable to diminished classification performance. Error in the spatial registration of the AF image with the H&E image presents a significant challenge owing to distortion and deformation of tissue during the FFPE process and is a major source of error. Further, registration errors in the Z-axis due to microtoming is inevitable given the ∼100-mm penetration depth of UV light and the routine trimming of paraffin-embedded tissue blocks to obtain a flat tissue surface. Furthermore, while we had applied dye masks and corrected for the dye signals, either the residual uncorrected dye signal or artifacts due to correction could have introduced a confounding emission signal into the tissue AF data.

The cancer heatmaps shown in Figures 4 through 7 underscore that our TumorMAP system successfully detects different kinds of cancer. We have trained and tested 156 cancer tiles from 55 patients, and the classification accuracies for IDC, ILC, and DCIS were 84% [101 of 120 tiles], 72% [18 of 25], and 82% [9 of 11], respectively. The low classification performance for ILC can be attributed to its unique morphologic characteristics and growth pattern with dyscohesive cells dispersed diffusely in the fibroconnective stroma without eliciting a desmoplastic host stromal response, unlike IDC.

While these results showcase the capability of the TumorMAP system and show encouraging indications for its clinical utility in the pathology laboratory for triaging the sample, it is worth noting the limitations of the present proof-of-principle study for application in surgery. First, the data sets used for training and testing in this study were of a high homogeneity threshold, and it is yet to be seen how the OncoSight AI classifier performs on a mixed tissue sample. Second, we have measured only 2D tissue blocks in the grossing laboratory, and the system and algorithm results must be evaluated on freshly excised 3D tissue. Third, our malignant class did not include rare cancer types, such as mucinous carcinoma and sarcoma, owing to the prevalence and high cancer homogeneity threshold used in this study. We plan to address all the above issues in our planned intraoperative feasibility study.

Taken together, we have demonstrated that the TumorMAP system can accurately discern malignant neoplasm from different types of benign tissues without disruption to the clinical workflow. We are developing and moving to commercialize a cost-effective, intraoperative imaging system for the ex vivo examination of excised specimens. In particular, this label-free approach has unique advantages that allow the assessment of biomolecular and morphologic differences between benign and cancer tissue to assist surgeons in rapidly making intraoperative decisions. Combining the rich biomolecular and morphologic features derived from AF imaging with AI/ML would lead to the next generation of real-time intraoperative assessment, improve efficiencies, and potentially reduce the burden to the patient, surgeon, and pathologist in determining and knowing whether cancer has been left behind.

The authors acknowledge current and former CytoVeris employees for their help in the successful completion of this study. We are thankful to Heather Couture, PhD, for her guidance in classifier development. Finally, we express our appreciation to reviewers for their constructive critique of the manuscript.