Dimensionality reduction for deep learning in infrared microscopy: a comparative computational survey.

While infrared microscopy provides molecular information at spatial resolution in a label-free manner, exploiting both spatial and molecular information for classifying the disease status of tissue samples constitutes a major challenge. One strategy to mitigate this problem is to embed high-dimensional pixel spectra in lower dimensions, aiming to preserve molecular information in a more compact manner, which reduces the amount of data and promises to make subsequent disease classification more accessible for machine learning procedures. In this study, we compare several dimensionality reduction approaches and their effect on identifying cancer in the context of a colon carcinoma study. We observe surprisingly small differences between convolutional neural networks trained on dimensionality reduced spectra compared to utilizing full spectra, indicating a clear tendency of the convolutional networks to focus on spatial rather than spectral information for classifying disease status.

Fast and label-free automated detection of microsatellite status in early colon cancer using artificial intelligence integrated infrared imaging.

PURPOSE: Microsatellite instability (MSI) due to mismatch repair (MMR) defects accounts for 15-20% of colon cancers (CC). MSI testing is currently standard of care in CC with immunohistochemistry of the four MMR proteins representing the gold standard. Instead, label-free quantum cascade laser (QCL) based infrared (IR) imaging combined with artificial intelligence (AI) may classify MSI/microsatellite stability (MSS) in unstained tissue sections user-independently and tissue preserving. METHODS: Paraffin-embedded unstained tissue sections of early CC from patients participating in the multicentre AIO ColoPredict Plus (CPP) 2.0 registry were analysed after dividing into three groups (training, test, and validation). IR images of tissue sections using QCL-IR microscopes were classified by AI (convolutional neural networks [CNN]) using a two-step approach. The first CNN (modified U-Net) detected areas of cancer while the second CNN (VGG-Net) classified MSI/MSS. End-points were area under receiver operating characteristic (AUROC) and area under precision recall curve (AUPRC). RESULTS: The cancer detection in the first step was based on 629 patients (train n = 273, test n = 138, and validation n = 218). Resulting classification AUROC was 1.0 for the validation dataset. The second step classifying MSI/MSS was performed on 547 patients (train n = 331, test n = 69, and validation n = 147) reaching AUROC and AUPRC of 0.9 and 0.74, respectively, for the validation cohort. CONCLUSION: Our novel label-free digital pathology approach accurately and rapidly classifies MSI vs. MSS. The tissue sections analysed were not processed leaving the sample unmodified for subsequent analyses. Our approach demonstrates an AI-based decision support tool potentially driving improved patient stratification and precision oncology in the future.

A framework for falsifiable explanations of machine learning models with an application in computational pathology.

In recent years, deep learning has been the key driver of breakthrough developments in computational pathology and other image based approaches that support medical diagnosis and treatment. The underlying neural networks as inherent black boxes lack transparency and are often accompanied by approaches to explain their output. However, formally defining explainability has been a notorious unsolved riddle. Here, we introduce a hypothesis-based framework for falsifiable explanations of machine learning models. A falsifiable explanation is a hypothesis that connects an intermediate space induced by the model with the sample from which the data originate. We instantiate this framework in a computational pathology setting using hyperspectral infrared microscopy. The intermediate space is an activation map, which is trained with an inductive bias to localize tumor. An explanation is constituted by hypothesizing that activation corresponds to tumor and associated structures, which we validate by histological staining as an independent secondary experiment.

Quantum Cascade Laser-Based Infrared Imaging as a Label-Free and Automated Approach to Determine Mutations in Lung Adenocarcinoma.

Therapeutic decisions in lung cancer critically depend on the determination of histologic types and oncogene mutations. Therefore, tumor samples are subjected to standard histologic and immunohistochemical analyses and examined for relevant mutations using comprehensive molecular diagnostics. In this study, an alternative diagnostic approach for automatic and label-free detection of mutations in lung adenocarcinoma tissue using quantum cascade laser-based infrared imaging is presented. For this purpose, a five-step supervised classification algorithm was developed, which was not only able to detect tissue types and tumor lesions, but also the tumor type and mutation status of adenocarcinomas. Tumor detection was verified on a data set of 214 patient samples with a specificity of 97% and a sensitivity of 95%. Furthermore, histology typing was verified on samples from 203 of the 214 patients with a specificity of 97% and a sensitivity of 94% for adenocarcinoma. The most frequently occurring mutations in adenocarcinoma (KRAS, EGFR, and TP53) were differentiated by this technique. Detection of mutations was verified in 60 patient samples from the data set with a sensitivity and specificity of 95% for each mutation. This demonstrates that quantum cascade laser infrared imaging can be used to analyze morphologic differences as well as molecular changes. Therefore, this single, one-step measurement provides comprehensive diagnostics of lung cancer histology types and most frequent mutations.

Application of Label-Free Proteomics for Quantitative Analysis of Urothelial Carcinoma and Cystitis Tissue.

A label-free approach based on a highly reproducible and stable workflow allows for quantitative proteome analysis . Due to advantages compared to labeling methods, the label-free approach has the potential to measure unlimited samples from clinical specimen monitoring and comparing thousands of proteins. The presented label-free workflow includes a new sample preparation technique depending on automatic annotation and tissue isolation via FTIR-guided laser microdissection, in-solution digestion, LC-MS/MS analyses, data evaluation by means of Proteome Discoverer and Progenesis software, and verification of differential proteins. We successfully applied this workflow in a proteomics study analyzing human cystitis and high-grade urothelial carcinoma tissue regarding the identification of a diagnostic tissue biomarker. The differential analysis of only 1 mm2 of isolated tissue cells led to 74 significantly differentially abundant proteins.

Advances in Digital Pathology: From Artificial Intelligence to Label-Free Imaging.

BACKGROUND: Digital pathology, in its primary meaning, describes the utilization of computer screens to view scanned histology slides. Digitized tissue sections can be easily shared for a second opinion. In addition, it allows tissue image analysis using specialized software to identify and measure events previously observed by a human observer. These tissue-based readouts were highly reproducible and precise. Digital pathology has developed over the years through new technologies. Currently, the most discussed development is the application of artificial intelligence to automatically analyze tissue images. However, even new label-free imaging technologies are being developed to allow imaging of tissues by means of their molecular composition. SUMMARY: This review provides a summary of the current state-of-the-art and future digital pathologies. Developments in the last few years have been presented and discussed. In particular, the review provides an outlook on interesting new technologies (e.g., infrared imaging), which would allow for deeper understanding and analysis of tissue thin sections beyond conventional histopathology. KEY MESSAGES: In digital pathology, mathematical methods are used to analyze images and draw conclusions about diseases and their progression. New innovative methods and techniques (e.g., label-free infrared imaging) will bring significant changes in the field in the coming years.

A representation learning approach for recovering scatter-corrected spectra from Fourier-transform infrared spectra of tissue samples.

Infrared spectra obtained from cell or tissue specimen have commonly been observed to involve a significant degree of scattering effects, often Mie scattering, which probably overshadows biochemically relevant spectral information by a nonlinear, nonadditive spectral component in Fourier transform infrared (FTIR) spectroscopic measurements. Correspondingly, many successful machine learning approaches for FTIR spectra have relied on preprocessing procedures that computationally remove the scattering components from an infrared spectrum. We propose an approach to approximate this complex preprocessing function using deep neural networks. As we demonstrate, the resulting model is not just several orders of magnitudes faster, which is important for real-time clinical applications, but also generalizes strongly across different tissue types. Using Bayesian machine learning approaches, our approach unveils model uncertainty that coincides with a band shift in the amide I region that occurs when scattering is removed computationally based on an established physical model. Furthermore, our proposed method overcomes the trade-off between computation time and the corrected spectrum being biased towards an artificial reference spectrum.

Label-free vibrational imaging of different Aß plaque types in Alzheimer's disease reveals sequential events in plaque development.

The neuropathology of Alzheimer's disease (AD) is characterized by hyperphosphorylated tau neurofibrillary tangles (NFTs) and amyloid-beta (Aß) plaques. Aß plaques are hypothesized to follow a development sequence starting with diffuse plaques, which evolve into more compact plaques and finally mature into the classic cored plaque type. A better molecular understanding of Aß pathology is crucial, as the role of Aß plaques in AD pathogenesis is under debate. Here, we studied the deposition and fibrillation of Aß in different plaque types with label-free infrared and Raman imaging. Fourier-transform infrared (FTIR) and Raman imaging was performed on native snap-frozen brain tissue sections from AD cases and non-demented control cases. Subsequently, the scanned tissue was stained against Aß and annotated for the different plaque types by an AD neuropathology expert. In total, 160 plaques (68 diffuse, 32 compact, and 60 classic cored plaques) were imaged with FTIR and the results of selected plaques were verified with Raman imaging. In diffuse plaques, we detect evidence of short antiparallel ß-sheets, suggesting the presence of Aß oligomers. Aß fibrillation significantly increases alongside the proposed plaque development sequence. In classic cored plaques, we spatially resolve cores containing predominantly large parallel ß-sheets, indicating Aß fibrils. Combining label-free vibrational imaging and immunohistochemistry on brain tissue samples of AD and non-demented cases provides novel insight into the spatial distribution of the Aß conformations in different plaque types. This way, we reconstruct the development process of Aß plaques in human brain tissue, provide insight into Aß fibrillation in the brain, and support the plaque development hypothesis.

Label-free, automated classification of microsatellite status in colorectal cancer by infrared imaging.

Challenging histopathological diagnostics in cancer include microsatellite instability-high (MSI-H) colorectal cancer (CRC), which occurs in 15% of early-stage CRC and is caused by a deficiency in the mismatch repair system. The diagnosis of MSI-H cannot be reliably achieved by visual inspection of a hematoxylin and eosin stained thin section alone, but additionally requires subsequent molecular analysis. Time- and sample-intensive immunohistochemistry with subsequent fragment length analysis is used. The aim of the presented feasibility study is to test the ability of quantum cascade laser (QCL)-based infrared (IR) imaging as an alternative diagnostic tool for MSI-H in CRC. We analyzed samples from 100 patients with sporadic CRC UICC stage II and III. Forty samples were used to develop the random forest classifier and 60 samples to verify the results on an independent blinded dataset. Specifically, 100% sensitivity and 93% specificity were achieved based on the independent 30 MSI-H- and 30 microsatellite stable (MSS)-patient validation cohort. This showed that QCL-based IR imaging is able to distinguish between MSI-H and MSS for sporadic CRC - a question that goes beyond morphological features - based on the use of spatially resolved infrared spectra used as biomolecular fingerprints.

Grayscale representation of infrared microscopy images by extended multiplicative signal correction for registration with histological images.

Fourier-transform infrared (FTIR) microspectroscopy is rounding the corner to become a label-free routine method for cancer diagnosis. In order to build infrared-spectral based classifiers, infrared images need to be registered with Hematoxylin and Eosin (H&E) stained histological images. While FTIR images have a deep spectral domain with thousands of channels carrying chemical and scatter information, the H&E images have only three color channels for each pixel and carry mainly morphological information. Therefore, image representations of infrared images are needed that match the morphological information in H&E images. In this paper, we propose a novel approach for representation of FTIR images based on extended multiplicative signal correction highlighting morphological features that showed to correlate well with morphological information in H&E images. Based on the obtained representations, we developed a strategy for global-to-local image registration for FTIR images and H&E stained histological images of parallel tissue sections.

Deep representation learning for domain adaptable classification of infrared spectral imaging data.

MOTIVATION: Applying infrared microscopy in the context of tissue diagnostics heavily relies on computationally preprocessing the infrared pixel spectra that constitute an infrared microscopic image. Existing approaches involve physical models, which are non-linear in nature and lead to classifiers that do not generalize well, e.g. across different types of tissue preparation. Furthermore, existing preprocessing approaches involve iterative procedures that are computationally demanding, so that computation time required for preprocessing does not keep pace with recent progress in infrared microscopes which can capture whole-slide images within minutes. RESULTS: We investigate the application of stacked contractive autoencoders as an unsupervised approach to preprocess infrared microscopic pixel spectra, followed by supervised fine-tuning to obtain neural networks that can reliably resolve tissue structure. To validate the robustness of the resulting classifier, we demonstrate that a network trained on embedded tissue can be transferred to classify fresh frozen tissue. The features obtained from unsupervised pretraining thus generalize across the large spectral differences between embedded and fresh frozen tissue, where under previous approaches separate classifiers had to be trained from scratch. AVAILABILITY AND IMPLEMENTATION: Our implementation can be downloaded from https://github.com/arnrau/SCAE_IR_Spectral_Imaging. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Integrated Fourier Transform Infrared Imaging and Proteomics for Identification of a Candidate Histochemical Biomarker in Bladder Cancer.

Histopathological differentiation between severe urocystitis with reactive urothelial atypia and carcinoma in situ (CIS) can be difficult, particularly after a treatment that deliberately induces an inflammatory reaction, such as intravesical instillation of Bacillus Calmette-Guèrin. However, precise grading in bladder cancer is critical for therapeutic decision making and thus requires reliable immunohistochemical biomarkers. Herein, an exemplary potential biomarker in bladder cancer was identified by the novel approach of Fourier transform infrared imaging for label-free tissue annotation of tissue thin sections. Identified regions of interest are collected by laser microdissection to provide homogeneous samples for liquid chromatography-tandem mass spectrometry-based proteomic analysis. This approach afforded label-free spatial classification with a high accuracy and without interobserver variability, along with the molecular resolution of the proteomic analysis. Cystitis and invasive high-grade urothelial carcinoma samples were analyzed. Three candidate biomarkers were identified and verified by immunohistochemistry in a small cohort, including low-grade urothelial carcinoma samples. The best-performing candidate AHNAK2 was further evaluated in a much larger independent verification cohort that also included CIS samples. Reactive urothelial atypia and CIS were distinguishable on the basis of the expression of this newly identified and verified immunohistochemical biomarker, with a sensitivity of 97% and a specificity of 69%. AHNAK2 can differentiate between reactive urothelial atypia in the setting of an acute or chronic cystitis and nonmuscle invasive-type CIS.

Quantum Cascade Laser-Based Infrared Microscopy for Label-Free and Automated Cancer Classification in Tissue Sections.

A feasibility study using a quantum cascade laser-based infrared microscope for the rapid and label-free classification of colorectal cancer tissues is presented. Infrared imaging is a reliable, robust, automated, and operator-independent tissue classification method that has been used for differential classification of tissue thin sections identifying tumorous regions. However, long acquisition time by the so far used FT-IR-based microscopes hampered the clinical translation of this technique. Here, the used quantum cascade laser-based microscope provides now infrared images for precise tissue classification within few minutes. We analyzed 110 patients with UICC-Stage II and III colorectal cancer, showing 96% sensitivity and 100% specificity of this label-free method as compared to histopathology, the gold standard in routine clinical diagnostics. The main hurdle for the clinical translation of IR-Imaging is overcome now by the short acquisition time for high quality diagnostic images, which is in the same time range as frozen sections by pathologists.

Spatial and molecular resolution of diffuse malignant mesothelioma heterogeneity by integrating label-free FTIR imaging, laser capture microdissection and proteomics.

Diffuse malignant mesothelioma (DMM) is a heterogeneous malignant neoplasia manifesting with three subtypes: epithelioid, sarcomatoid and biphasic. DMM exhibit a high degree of spatial heterogeneity that complicates a thorough understanding of the underlying different molecular processes in each subtype. We present a novel approach to spatially resolve the heterogeneity of a tumour in a label-free manner by integrating FTIR imaging and laser capture microdissection (LCM). Subsequent proteome analysis of the dissected homogenous samples provides in addition molecular resolution. FTIR imaging resolves tumour subtypes within tissue thin-sections in an automated and label-free manner with accuracy of about 85% for DMM subtypes. Even in highly heterogeneous tissue structures, our label-free approach can identify small regions of interest, which can be dissected as homogeneous samples using LCM. Subsequent proteome analysis provides a location specific molecular characterization. Applied to DMM subtypes, we identify 142 differentially expressed proteins, including five protein biomarkers commonly used in DMM immunohistochemistry panels. Thus, FTIR imaging resolves not only morphological alteration within tissue but it resolves even alterations at the level of single proteins in tumour subtypes. Our fully automated workflow FTIR-guided LCM opens new avenues collecting homogeneous samples for precise and predictive biomarkers from omics studies.

Marker-free automated histopathological annotation of lung tumour subtypes by FTIR imaging.

By integration of FTIR imaging and a novel trained random forest classifier, lung tumour classes and subtypes of adenocarcinoma are identified in fresh-frozen tissue slides automated and marker-free. The tissue slices are collected under standard operation procedures within our consortium and characterized by current gold standards in histopathology. In addition, meta data of the patients are taken. The improved standards on sample collection and characterization results in higher accuracy and reproducibility as compared to former studies and allows here for the first time the identification of adenocarcinoma subtypes by this approach. The differentiation of subtypes is especially important for prognosis and therapeutic decision.