Detection and Quantification of Myocardial Fibrosis Using Stain-Free Infrared Spectroscopic Imaging

Formalin-fixed, paraffin-embedded human heart specimens were obtained from 2 sources—the Oregon Health & Science University Knight Cardiovascular Institute's Cardiovascular Disease Tissue Bank and the Pacific Northwest Tissue Bank. Representative examples were taken from 3 classes of patients or tissues including (1) patients with decompensated heart failure undergoing VAD implantation; (2) patients with heart failure at the time of transplantation to receive a new heart; and (3) heart tissue from donor hearts, presumed to represent nonpathologic (control) cases. Tissue obtained from patients undergoing VAD implantation were derived from the “core” of left ventricular tissue removed at the time of implantation. Tissue from heart transplant patients and control cases were obtained from an area similar to the “VAD core” and by sampling all the layers of heart. The control samples obtained from the Pacific Northwest Tissue Bank were verified to have less than 60 minutes of warm ischemic time and less than 6 hours of cold ischemic time before tissue processing. Care was taken to select tissue that was free of gross pathology. This study was approved by the Oregon Health & Science University and Illinois institutional review boards.

Approximately 5-μm–thick tissue sections were created from the formalin-fixed, paraffin-embedded tissue blocks using a microtome, mounted on MirrIR low-emissivity slides (Kevley Technology, Chesterland, Ohio)³⁵  and were deparaffinized with repeated hexane washes for 24 hours before imaging. Parallel sections of each sample were stained with H&E stains for general histopathologic evaluation. Following DFIR imaging, the same tissue sections were subsequently stained with PSR stain, as detailed in the Supplementary Section (see the supplemental digital content, also containing a table and 3 figures at https://meridian.allenpress.com/aplm in the December 2021 table of contents). Discrete frequency IR imaging was conducted with the Agilent Hyperscan prototype system (Agilent, Santa Clara, California), sweeping through the spectral range (800–1800 cm⁻¹) in 4-cm⁻¹ steps and 5.5-μm pixel size. Hematoxylin-eosin–stained and PSR-stained tissue slides were imaged with the Axio full slide imaging system (Zeiss, Jena, Germany) with a pixel size of 1 μm. These data were processed using Environment for Visualization—Interactive Data Language, version 4.8 (Harris Geospatial Solutions, Boulder, Colorado) and with MATLAB (MathWorks, Natick, Massachusetts).

After constructing our network, we selected the regions of interest by consulting the PSR-stained images. Approximately 3000 pixels were selected with 40 wavenumber positions in the 1500 to 1800-cm⁻¹ range. The spectra were normalized using the amide I peak absorbance to account for tissue thickness variation effects. Seventy-five percent of the pixels were used for training and 25% for validation, with 2 samples used for identifying regions of interest. To prevent overfitting, the validation set was repeatedly checked, and, if loss of validation set began to increase, training was terminated.

Our deep-learning framework to generate digital PSR stain from IR data is adapted from our earlier work by Falahkheirkhah et al.³⁴  Briefly, we register both IR and PSR images (details in supplemental digital content). Then, we use conditional Generative Adversarial Networks for transferring the spatial details and the color scheme of PSR images to IR images. The probability map for cardiomyocytes and fibers generated from the IR data as described in the previous paragraph serves as the condition and 2 bands of IR image (amide I and II) are the inputs to the network and bright-field images of PSR stain of the same sections are the target of the network. Details of the generator and discriminator architecture are described in the supplementary section. The initialization of both generator and discriminator was random, and their parameters were optimized using Adam³⁸  with initial learning rate of 10⁻⁴ for generator and 10⁻⁵ for discriminator. The learning rate was multiplied by 0.95 after each 1000 iterations. To stabilize the training and prevent overfitting of discriminator, within each iteration of the discriminator, the generator iterated 5 times.

To increase the variety and length of the dataset, we performed data augmentation that included random affine transformation to both IR and PSR patches. All images in the dataset were originally 500 × 500 for each iteration, of which a region of 256 × 256 was randomly selected. The order of the images was random, and they were shuffled after each epoch. We used 400 images for training from 3 samples and test on 10 other samples. We trained the conditional Generative Adversarial Networks model using the resources of the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign with a single NVIDIA V100 GPU NVDIA and Intel Xeon CPU E5-2680 v4 at 2.4 GHz. The framework was implemented in PyTorch 1.3, CUDA 10.1, and Python 3.7.1. The imaging speed was approximately 15 seconds per wavelength band for a 4-mm² area scanned at 5-μm pixel size. Time for classification was approximately 1 second for a 1-megapixel image.

Figure 1 shows the overall schematic of this study. The typical site of a VAD “core” and outflow cannula implantation is shown in Figure 1, A. Figure 1, B shows a schematic of the histology of the heart wall. The endocardium, which is the innermost layer, consists of a thin layer of endothelial cells. The central and thickest layer is the myocardium, which comprises cardiomyocytes, extracellular matrix (collagen, elastin, etc), and vascular structures, such as arteries, veins, capillaries, arterioles, and venules. The outermost layer is the epicardium and is composed of adipocytes and connective tissues. Each of these cellular and acellular components are known to have distinct chemical compositions²²  and hence can be anticipated to be separated by IR spectral measurements, limited only by the resolution limit of IR and signal-to-noise. The segment of the specimen traversing the left ventricular wall of the heart, alongside the cell histology of interest, is shown upon thin-section preparation. On histologic sections, the endocardium, myocardium, and epicardium can be appreciated upon microscopic examination by a pathologist using H&E stains and an optical microscope. For example, Figure 1, C shows a histologic image for the left ventricular heart wall from a cardiac transplant sample. There is little to distinguish compartments of cardiac tissue from simple staining alone, however, and the structure seen in Figure 1B has to be locally recognized within the almost uniformly stained sample. In accordance with a previously published procedure,²⁶  the IR images were plotted as a ratio of the absorbance (1236 cm⁻¹:1652 cm⁻¹), which is known to be sensitive to differences in chemical composition among cardiac histologic classes in Figure 1, D. While IR images demonstrate a degree of contrast between structures based on chemical signature, just as for the simple H&E stain, this single chemical image is not sufficient to visualize differences in histologic subtypes either. As opposed to the red-green-blue channels of contrast in H&E images, however, IR data contain tens to hundreds of informative frequencies. Obviously, these cannot be analyzed by composite red-green-blue–type images and need sophisticated machine learning, and several studies have shown the potential to computationally delineate different histologic classes from high-bandwidth Fourier transform IR data.²⁶ 

These 3 components—pathologic knowledge desired, machine learning algorithm, and IR frequencies measured—can be optimized together for task-specific applications. For the current study, we wanted to develop a method for rapid detection and delineation of myocardial fibrosis subtypes, with accuracy comparable to the current gold standard diagnostic stains. Picrosirius red is the accepted gold standard special stain for detection of fibrosis and orientation of fibrous tissue. Going beyond this simple diagnostic test, we also wanted to convey information in a manner that was relevant to current practice. While testing different workflows to achieve these twin goals, we devised a two-step process. In the first step of the process, we use IR data to classify tissue into subtypes. Here, this model approximates the complex tissue heterogeneity into 2 classes—namely “fibers” and “cardiomyocytes.” In the second step of the process, we would turn IR information into a format that is clinically appreciated. Here, we sought to convey that information in terms of the PSR stain. The schematic workflow is shown in Figure 1, E. The subtype classification step is shown to the bottom left, where the first goal is to classify the image into a color-coded one (middle, bottom), where black denotes fibers and gray denotes cardiomyocytes. The PSR image (middle, top) of the same sections serves as ground truth for generating the classification probability maps and to validate the results. With the classification map acting as conditions and using the IR data again as input, we developed a computational approach to generating pathology images. The output of the conditional Generative Adversarial Networks is a “virtual” PSR image, which is generated using light and computational algorithms alone, does not need a dye and can be generated in seconds from the IR image.

We statistically validated the potential of the technology next. Figure 2, A and B show images from H&E-stained and PSR-stained serial sections, respectively. Before PSR staining, IR imaging was performed on the same section, as described in the method, and the classification result is presented in Figure 2, C. Comparing the images, the ease of identifying subtypes in the IR classified image is clear through the simple color code. While an image helps to understand the presence of a histologic-specific species, we demonstrate that a quantitative framework allows for higher accuracy or confidence in identifying any 1 pixel as being rich in that chemical species. This accuracy is judged by the likelihood of true identification balanced by the false-positive rate, typically assessed using a receiver-operator characteristic curve, as shown for separating collagen from cardiomyocytes in Figure 2, D. The high value of the area under the curve (0.998) indicates the high accuracy of the classification process. In classified images “fiber” pixels (here, ∼0.5 million) and “cardiomyocyte” pixels (∼1 million) were chosen, and the average normalized spectra are plotted in Figure 2, E. These characteristic spectra allow us to identify differences useful in classification as well as to understand the molecular origins of the differences. Here, the relative ratio of amide I band (∼1650 cm⁻¹) and amide II (∼1540 cm⁻¹) bands show excellent signal and differences between the subtypes that arise from protein composition and conformational differences. In addition to the obvious differences in intensity, features such as the shoulder peak at approximately 1640 cm⁻¹ in collagen, the relative ratio of amide III (∼1240 cm⁻¹) and amide I (1650 cm⁻¹), and the shoulder peak at 1204 cm⁻¹ all appear to provide information that is potentially useful. We especially focus on the amide I, II, and III regions that arise from the normal modes of the protein backbone and structure. Amide I consists mostly of C=O stretching vibrations, often making it the strongest peak in most biomedical IR images and providing a highly sensitive measure of the secondary structures of proteins through the transition dipole coupling mechanism.²⁴  The amide II and III regions are also sensitive to the secondary structure, but the mechanism is less understood.^40,41  We hypothesize that the difference in the relative ratio of the amide III/amide I and amide II/amide I likely originates from the different helical contents of collagen versus other protein constituents of the tissue. The shoulder peak at approximately 1640 cm⁻¹ has been assigned to both proteoglycans⁴²  and -polyproline II⁴³  in literature but cannot be unambiguously assigned. The shoulder peak at 1204 cm⁻¹ most likely originates from a bending mode of N-H.⁴⁴  The measurement of the absorption of tissue at these modes, rather than the full spectrum, affords the possibility of only recording discrete frequencies to decrease acquisition time, thereby making this method more rapid and clinically relevant.

Conventionally, the pathologic changes of fibrous heart tissue are characterized as (1) replacement fibrosis (concomitant with large cardiomyocyte loss), (2) interstitial fibrosis (smaller and diffuse deposits without significant cardio-myocyte loss), and (3) perivascular fibrosis (the deposits around blood vessels).⁴  These are descriptive morphologic differences, and the underlying chemical differences, if any, have not been explored. To do so, we examine the spectral differences of the different types. First, class images are used to isolate the “fiber pixels” from the hyperspectral images. Second, we measure the average spectra of the regions of interest for the 3 different types of fibrosis as described above as well as for the fibrous connective tissue around the epicardium. To minimize variations arising from different patients, the perivascular, interstitial, and epicardial areas were measured from the same patient (undergoing VAD implant). The replacement fibrosis was from the explanted heart of a heart transplant patient, however, because the former patient showed no measurable amount of replacement fibrosis. We used the class images of the fiber class as a mask to extract the images of the fibers (as shown in Figure 3, A through D) of the regions of interest corresponding to the different subtypes as detailed above. The average spectra of those pixels, normalized to the amide I band, are shown in Figure 3, E. Because the different types of fibrotic depositions largely originate from type I collagen, the average spectra are similar; however, differentiation can potentially be achieved by small but reproducible differences observed both in the fingerprint region from 1100 to 1450 cm⁻¹ and in the line shape of the amide I region probably originating from differences in secondary structural features. Inset second-derivative spectra enable easier appreciation of these small differences. The changes in the fingerprint region are relative intensities of amide III modes (1204–1282 cm⁻¹) and twin peaks at 1380 and 1344 cm⁻¹. The 1344 and 1388 cm⁻¹ likely originate from amino acid sidechains of collagens, with the 1344-cm⁻¹ peak possibly arising from CH₂ vibrations of proline residues of collagen⁴⁵  and the 1380-cm⁻¹ peak arising from bending CH₃ vibration modes of residues, like valine, leucine,²⁴  and so on. Collectively, the spectra show significant differences between the different types of fibers, which is particularly pronounced for replacement fibrosis versus the others and points to the differences in the underlying chemistry of the fibers. It is important to point out that precise peak frequencies tend to vary between samples because of the extent of solvation, fixation, and other issues. Furthermore, our imaging method uses polarized laser imaging and consequently the relative ratios between the different modes can vary from conventional Fourier transform IR imaging, because they are expected to depend on the relative orientations of the dipoles with respect to the laser polarization.^46,47  Currently, the precise chemical differences in the collagen network that gives rise to the spectral differences is not entirely understood. We hypothesize that they originate in differences in collagen subtypes and/or crosslinking and will be the purview of our future study. Notwithstanding, it clearly shows the ability of our technique to go beyond conventional pathology.

While the IR classified and color-coded images capture the underlying chemical signature and differences, generating “stainless” digital images that allow pathologists to make easy comparisons to conventional methods are also useful for diagnostic purposes. Because the diffraction limited spatial resolution of IR images is lower compared with visible images, we used deep-learning model to enhance the morphologic detail available in IR imaging. We call this process virtual PSR (VPSR). Accordingly, we generate PSR images from IR data as is shown schematically in Figure 1, E. Rendered VPSR examples are presented in Figure 4, A through F as image pairs for the following 3 classes of patient samples: tissue obtained from VAD implantation, transplant, and control. Also shown in Figure 4, G through R are selected regions of interest from each of the tissue fragments as image pairs. Although excellent visual agreement is seen in all the 3 patient samples, there are finer points of differences that deserve commentary. In the PSR-staining protocol used, a green counterstain is added to differentiate cardiomyocytes from collagen fibers, which appear bright red. However, the green counterstain was not uniformly taken up by all the samples in our data set. Furthermore, the counterstain was also deemed unnecessary for both qualitative assessment and subsequent quantification of fibrosis. We also observed small black dots (Figure 4, C), which we discovered arose from the PSR stain apparently reacting with the coating on the IR reflective glass slide used. To mitigate the effects of these, we restricted our training set for the deep neural network to samples, which had no overstaining from the green counterstain or black dots. Consequently, the digitally generated PSR stains has no green pixel or black dots. This illustrates the potential of our all-digital technique in not only dispensing with the staining steps, saving reagents, and effort, but also in overcoming variability that often arises from process differences or tissue staining and can greatly limit the use of computational methods for pathology.⁴⁸  It is also important to note that the merger of IR and optical data used can provide pathology-equivalent images, the spectral data are still localized to the diffraction limited spot size of IR wavelengths. Therefore, subtle differences in the shape and structure of cardiomyocytes will further benefit from emerging IR imaging techniques⁴⁹  that offer optical resolution of value in cardiomyopathies. In Figure 5, A through H, we demonstrate the robustness of this all-digital approach in detecting the 3 prevalent types of fibrosis—namely the interstitial, perivascular, and replacement fibrosis. Also presented is the connective tissue around the epicardium. The results show excellent agreement of the PSR-stained images with VPSR images and demonstrates the use of our combined IR, deep-learning approach in studying myocardial fibrosis. While the images provide a visualization of fibrosis, we sought to define quantitative metrics for the extent of fibrosis. The collagen volume fraction has been used in practice^50,51  and shown to be of prognostic value in heart failure. Collagen volume fraction is defined as the percentage fraction of the tissue that is fibrotic and we sought to compare results from VPSR with PSR images. To account for the heterogeneity of the fibrotic environment and simulate a diversity of pathologies, we compare the fibrotic fractions from 37 different regions of interest from the 3 different classes of tissues outside the training set of deep-learning network as detailed in the materials and methods section. Care was taken to include all the different morphologic types of fibrosis observed in the heart patients. Unsupervised K-means clustering of the colored images were performed to segregate the bright red pixels from the yellow/green background of cardiomyocytes for both the actual and virtually stained PSR images (detailed procedure in SI). The results are presented in Figure 6. The 4 points with highest absolute value of residuals in Figure 6 were determined to be the connective tissue around epicardium (first 3 points, VPSR) and nonspecific background staining (PSR; highlighted with asterisk). Notwithstanding, excellent correlation between the two sets were observed (PCC = 0.92, ρ = 7.76 × 10⁻¹⁵) attesting to diagnostic utility of the “virtual” stained images.

Myocardial collagen in human heart is mainly composed of type I (85%) and type III (11%) collagen.⁴  It is now understood that an imbalance in extracellular matrix turnover,⁵²  which results in cardiac fibrosis, is a common pathway associated with various forms of cardiac dysfunction.⁴  Whether owing to myocardial infarction, mechanical overloading, or unmet metabolic demands in nonischemic cardiomyopathy, alterations in the extracellular matrix affect the heart in the following 3 ways: decreased ventricular compliance preventing diastolic relaxation and filling (diastolic dysfunction),⁵³  contractile dysfunction causing reduced ejection (systolic dysfunction),⁵⁴  and aberrations in the conduction system⁵⁵  as a result of impaired intercellular signaling or contractile coordination. While there may be distinct patterns associated with particular diseases, a combination of all the aforementioned forms of fibrous tissue deposition may be seen in any form of cardiac pathology. While replacement fibrosis primarily originates as a tissue-repair response to myocardial infarction, interstitial and perivascular fibrosis originate from diverse pathologic processes. Interstitial fibrosis can be thought of as an attempt by the body to reinforce contractile elements of the heart, which has been exposed to unmitigated excessive pressures, as in the case of a patient with uncontrolled elevated blood pressure. Perivascular fibrosis, on the other hand, may occur as the result of inflammatory conditions, such as myocarditis, or as a consequence of myocardial infarction which triggers inflammatory repair mechanisms.⁵⁶  Perivascular fibrosis may carry specific implications on the delivery of blood to myocardial tissues, as thickening of connective tissues in and around the blood vessels of the heart prevent blood vessel dilation under increased metabolic demand, causing further insult to the failing heart.⁵⁷  Furthermore, there is a dynamic component to the extent and progression of fibrosis as evidenced by the survival benefit pharmacologic agents (eg, angiotensin-converting enzyme inhibitors) offer in preventing collagen remodeling after myocardial infarction in humans.⁵⁸  Bearing these universal maladaptive responses to insult in the heart and evidence to suggest that certain etiologies of fibrosis are reversible, fibrosis is an ample target for therapeutic intervention and developing better tools for assessing progression of fibrosis in situ. In our present work we demonstrate a label-free, nondestructive method of assessing and quantifying these different types of myocardial fibrosis. Beyond replicating the stained images that pathologist currently examine to assess morphologic features, the techniques we describe in IR spectroscopy opens up the possibility of detecting biochemical differences, such collagen subtypes and posttranslational modifications, like crosslinking, which could yield mechanistic insight into the progression of fibrotic diseases. Although we imaged 251 discrete frequencies for the spectra, we have demonstrated that an accurate classifier for fibrosis can be developed using only 40 discrete infrared frequencies in the amide I and II region. For DFIR imaging number of infrared bands chosen is a compromise between speed and the acquired spectral information. As discussed in the results, it should be possible to further reduce the number of infrared bands to make it of use in a near real-time diagnostic workflow schema (ie, intraoperatively).

Beyond simple identification of fibrosis are the possibilities of employing additional quantification methods to study distribution of fiber length, width, and other parameters analogous to what has been done with second harmonic generation imaging.⁵⁹  Collagen fiber orientation and its association with the so-called tumor microenvironment has been implicated in the pathogenesis and prognosis of various forms of cancer.^60–65  Similar work has been carried out in breast and prostate cancer using DFIR and Fourier transform IR imaging techniques to rapidly identify and subtype tumors by assessing the tumor microenvironment.^64,65  We believe a similar “fibrous microenvironment” may exist in the myocardium, and can provide similar insights into the state of disease and prognosis of patients suffering from heart failure. The quantification of fibrosis and its use in emerging artificial intelligence algorithms is critical to many decisions, such as the kind of mechanical support or allocation of organ donations for transplantation and related interventions. This work enables the analysis of fibrosis as a convenient and reagent-free method with minimal requirements for changing established protocols assess the state of disease in the failing myocardium. We have only used a simple model to demonstrate the principle of all-digital pathology for cardiovascular applications. Beyond the relatively simple insight into tissue histopathology, rapid assessment of capillary density, cellular and nuclear geometry, presence of and subtyping of lymphocytes, fibroblasts, and myofibroblasts may also provide additional clues for prognostic markers of heart disease.

This work uses resources provided by the Innovative Systems Laboratory at the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign. Dr Zimmermann would like to thank the National Institutes of Health, T32 training grant NIH NRSA T32HL094294 for support.