Deep learning based histological classification of adnex tumors.

BACKGROUND: Cutaneous adnexal tumors are a diverse group of tumors arising from structures of the hair appendages. Although often benign, malignant entities occur which can metastasize and lead to patients´ death. Correct diagnosis is critical to ensure optimal treatment and best possible patient outcome. Artificial intelligence (AI) in the form of deep neural networks has recently shown enormous potential in the field of medicine including pathology, where we and others have found common cutaneous tumors can be detected with high sensitivity and specificity. To become a widely applied tool, AI approaches will also need to reliably detect and distinguish less common tumor entities including the diverse group of cutaneous adnexal tumors. METHODS: To assess the potential of AI to recognize cutaneous adnexal tumors, we selected a diverse set of these entities from five German centers. The algorithm was trained with samples from four centers and then tested on slides from the fifth center. RESULTS: The neural network was able to differentiate 14 different cutaneous adnexal tumors and distinguish them from more common cutaneous tumors (i.e. basal cell carcinoma and seborrheic keratosis). The total accuracy on the test set for classifying 248 samples into these 16 diagnoses was 89.92 %. Our findings support AI can distinguish rare tumors, for morphologically distinct entities even with very limited case numbers (< 50) for training. CONCLUSION: This study further underlines the enormous potential of AI in pathology which could become a standard tool to aid pathologists in routine diagnostics in the foreseeable future. The final diagnostic responsibility will remain with the pathologist.

Applying an artificial intelligence deep learning approach to routine dermatopathological diagnosis of basal cell carcinoma.

BACKGROUND: Institutes of dermatopathology are faced with considerable challenges including a continuously rising numbers of submitted specimens and a shortage of specialized health care practitioners. Basal cell carcinoma (BCC) is one of the most common tumors in the fair-skinned western population and represents a major part of samples submitted for histological evaluation. Digitalizing glass slides has enabled the application of artificial intelligence (AI)-based procedures. To date, these methods have found only limited application in routine diagnostics. The aim of this study was to establish an AI-based model for automated BCC detection. PATIENTS AND METHODS: In three dermatopathological centers, daily routine practice BCC cases were digitalized. The diagnosis was made both conventionally by analog microscope and digitally through an AI-supported algorithm based on a U-Net architecture neural network. RESULTS: In routine practice, the model achieved a sensitivity of 98.23% (center 1) and a specificity of 98.51%. The model generalized successfully without additional training to samples from the other centers, achieving similarly high accuracies in BCC detection (sensitivities of 97.67% and 98.57% and specificities of 96.77% and 98.73% in centers 2 and 3, respectively). In addition, automated AI-based basal cell carcinoma subtyping and tumor thickness measurement were established. CONCLUSIONS: AI-based methods can detect BCC with high accuracy in a routine clinical setting and significantly support dermatopathological work.

Deep learning detection of melanoma metastases in lymph nodes.

BACKGROUND: In melanoma patients, surgical excision of the first draining lymph node, the sentinel lymph node (SLN), is a routine procedure to evaluate lymphogenic metastases. Metastasis detection by histopathological analysis assesses multiple tissue levels with hematoxylin and eosin and immunohistochemically stained glass slides. Considering the amount of tissue to analyze, the detection of metastasis can be highly time-consuming for pathologists. The application of artificial intelligence in the clinical routine has constantly increased over the past few years. METHODS: In this multi-center study, a deep learning method was established on histological tissue sections of sentinel lymph nodes collected from the clinical routine. The algorithm was trained to highlight potential melanoma metastases for further review by pathologists, without relying on supplementary immunohistochemical stainings (e.g. anti-S100, anti-MelanA). RESULTS: The established method was able to detect the existence of metastasis on individual tissue cuts with an area under the curve of 0.9630 and 0.9856 respectively on two test cohorts from different laboratories. The method was able to accurately identify tumour deposits>0.1 mm and, by automatic tumour diameter measurement, classify these into 0.1 mm to -1.0 mm and>1.0 mm groups, thus identifying and classifying metastasis currently relevant for assessing prognosis and stratifying treatment. CONCLUSIONS: Our results demonstrate that AI-based SLN melanoma metastasis detection has great potential and could become a routinely applied aid for pathologists. Our current study focused on assessing established parameters; however, larger future AI-based studies could identify novel biomarkers potentially further improving SLN-based prognostic and therapeutic predictions for affected patients.

Multimodal Lung Cancer Subtyping Using Deep Learning Neural Networks on Whole Slide Tissue Images and MALDI MSI.

Artificial intelligence (AI) has shown potential for facilitating the detection and classification of tumors. In patients with non-small cell lung cancer, distinguishing between the most common subtypes, adenocarcinoma (ADC) and squamous cell carcinoma (SqCC), is crucial for the development of an effective treatment plan. This task, however, may still present challenges in clinical routine. We propose a two-modality, AI-based classification algorithm to detect and subtype tumor areas, which combines information from matrix-assisted laser desorption/ionization (MALDI) mass spectrometry imaging (MSI) data and digital microscopy whole slide images (WSIs) of lung tissue sections. The method consists of first detecting areas with high tumor cell content by performing a segmentation of the hematoxylin and eosin-stained (H&E-stained) WSIs, and subsequently classifying the tumor areas based on the corresponding MALDI MSI data. We trained the algorithm on six tissue microarrays (TMAs) with tumor samples from N = 232 patients and used 14 additional whole sections for validation and model selection. Classification accuracy was evaluated on a test dataset with another 16 whole sections. The algorithm accurately detected and classified tumor areas, yielding a test accuracy of 94.7% on spectrum level, and correctly classified 15 of 16 test sections. When an additional quality control criterion was introduced, a 100% test accuracy was achieved on sections that passed the quality control (14 of 16). The presented method provides a step further towards the inclusion of AI and MALDI MSI data into clinical routine and has the potential to reduce the pathologist's work load. A careful analysis of the results revealed specific challenges to be considered when training neural networks on data from lung cancer tissue.

Evaluation of a Deep Learning Approach to Differentiate Bowen's Disease and Seborrheic Keratosis.

Background: Some of the most common cutaneous neoplasms are Bowen's disease and seborrheic keratosis, a malignant and a benign proliferation, respectively. These entities represent a significant fraction of a dermatopathologists' workload, and in some cases, histological differentiation may be challenging. The potential of deep learning networks to distinguish these diseases is assessed. Methods: In total, 1935 whole-slide images from three institutions were scanned on two different slide scanners. A U-Net-based segmentation deep learning algorithm was trained on data from one of the centers to differentiate Bowen's disease, seborrheic keratosis, and normal tissue, learning from annotations performed by dermatopathologists. Optimal thresholds for the class distinction of diagnoses were extracted and assessed on a test set with data from all three institutions. Results: We aimed to diagnose Bowen's diseases with the highest sensitivity. A good performance was observed across all three centers, underlining the model's robustness. In one of the centers, the distinction between Bowen's disease and all other diagnoses was achieved with an AUC of 0.9858 and a sensitivity of 0.9511. Seborrheic keratosis was detected with an AUC of 0.9764 and a sensitivity of 0.9394. Nevertheless, distinguishing irritated seborrheic keratosis from Bowen's disease remained challenging. Conclusions: Bowen's disease and seborrheic keratosis could be correctly identified by the evaluated deep learning model on test sets from three different centers, two of which were not involved in training, and AUC scores > 0.97 were obtained. The method proved robust to changes in the staining solution and scanner model. We believe this demonstrates that deep learning algorithms can aid in clinical routine; however, the results should be confirmed by qualified histopathologists.

StainCUT: Stain Normalization with Contrastive Learning.

In recent years, numerous deep-learning approaches have been developed for the analysis of histopathology Whole Slide Images (WSI). A recurrent issue is the lack of generalization ability of a model that has been trained with images of one laboratory and then used to analyze images of a different laboratory. This occurs mainly due to the use of different scanners, laboratory procedures, and staining variations. This can produce strong color differences, which change not only the characteristics of the image, such as the contrast, brightness, and saturation, but also create more complex style variations. In this paper, we present a deep-learning solution based on contrastive learning to transfer from one staining style to another: StainCUT. This method eliminates the need to choose a reference frame and does not need paired images with different staining to learn the mapping between the stain distributions. Additionally, it does not rely on the CycleGAN approach, which makes the method efficient in terms of memory consumption and running time. We evaluate the model using two datasets that consist of the same specimens digitized with two different scanners. We also apply it as a preprocessing step for the semantic segmentation of metastases in lymph nodes. The model was trained on data from one of the laboratories and evaluated on data from another. The results validate the hypothesis that stain normalization indeed improves the performance of the model. Finally, we also investigate and compare the application of the stain normalization step during the training of the model and at inference.

Robust subtyping of non-small cell lung cancer whole sections through MALDI mass spectrometry imaging.

Subtyping of the most common non-small cell lung cancer (NSCLC) tumor types adenocarcinoma (ADC) and squamous cell carcinoma (SqCC) is still a challenge in the clinical routine and a correct diagnosis is crucial for an adequate therapy selection. Matrix-assisted laser desorption/ionization (MALDI) mass spectrometry imaging (MSI) has shown potential for NSCLC subtyping but is subject to strong technical variability and has only been applied to tissue samples assembled in tissue microarrays (TMAs). To our knowledge, a successful transfer of a classifier from TMAs to whole sections, which are generated in the standard clinical routine, has not been presented in the literature as of yet. We introduce a classification algorithm using extensive preprocessing and a classifier (either a neural network or a linear discriminant analysis (LDA)) to robustly classify whole sections of ADC and SqCC lung tissue. The classifiers were trained on TMAs and validated and tested on whole sections. Vital for a successful application on whole sections is the extensive preprocessing and the use of whole sections for hyperparameter selection. The classification system with the neural network/LDA results in 99.0%/98.3% test accuracy on spectra level and 100.0%/100.0% test accuracy on whole section level, respectively, and, therefore, provides a powerful tool to support the pathologist's decision making process. The presented method is a step further towards a clinical application of MALDI MSI and artificial intelligence for subtyping of NSCLC tissue sections.

Conditional Invertible Neural Networks for Medical Imaging.

Over recent years, deep learning methods have become an increasingly popular choice for solving tasks from the field of inverse problems. Many of these new data-driven methods have produced impressive results, although most only give point estimates for the reconstruction. However, especially in the analysis of ill-posed inverse problems, the study of uncertainties is essential. In our work, we apply generative flow-based models based on invertible neural networks to two challenging medical imaging tasks, i.e., low-dose computed tomography and accelerated medical resonance imaging. We test different architectures of invertible neural networks and provide extensive ablation studies. In most applications, a standard Gaussian is used as the base distribution for a flow-based model. Our results show that the choice of a radial distribution can improve the quality of reconstructions.

Deeply Supervised UNet for Semantic Segmentation to Assist Dermatopathological Assessment of Basal Cell Carcinoma.

Accurate and fast assessment of resection margins is an essential part of a dermatopathologist's clinical routine. In this work, we successfully develop a deep learning method to assist the dermatopathologists by marking critical regions that have a high probability of exhibiting pathological features in whole slide images (WSI). We focus on detecting basal cell carcinoma (BCC) through semantic segmentation using several models based on the UNet architecture. The study includes 650 WSI with 3443 tissue sections in total. Two clinical dermatopathologists annotated the data, marking tumor tissues' exact location on 100 WSI. The rest of the data, with ground-truth sectionwise labels, are used to further validate and test the models. We analyze two different encoders for the first part of the UNet network and two additional training strategies: (a) deep supervision, (b) linear combination of decoder outputs, and obtain some interpretations about what the network's decoder does in each case. The best model achieves over 96%, accuracy, sensitivity, and specificity on the Test set.

Quantitative Comparison of Deep Learning-Based Image Reconstruction Methods for Low-Dose and Sparse-Angle CT Applications.

The reconstruction of computed tomography (CT) images is an active area of research. Following the rise of deep learning methods, many data-driven models have been proposed in recent years. In this work, we present the results of a data challenge that we organized, bringing together algorithm experts from different institutes to jointly work on quantitative evaluation of several data-driven methods on two large, public datasets during a ten day sprint. We focus on two applications of CT, namely, low-dose CT and sparse-angle CT. This enables us to fairly compare different methods using standardized settings. As a general result, we observe that the deep learning-based methods are able to improve the reconstruction quality metrics in both CT applications while the top performing methods show only minor differences in terms of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). We further discuss a number of other important criteria that should be taken into account when selecting a method, such as the availability of training data, the knowledge of the physical measurement model and the reconstruction speed.

Cross-Normalization of MALDI Mass Spectrometry Imaging Data Improves Site-to-Site Reproducibility.

Matrix-assisted laser desorption/ionization mass spectrometry imaging (MALDI MSI) is an established tool for the investigation of formalin-fixed paraffin-embedded (FFPE) tissue samples and shows a high potential for applications in clinical research and histopathological tissue classification. However, the applicability of this method to serial clinical and pharmacological studies is often hampered by inevitable technical variation and limited reproducibility. We present a novel spectral cross-normalization algorithm that differs from the existing normalization methods in two aspects: (a) it is based on estimating the full statistical distribution of spectral intensities and (b) it involves applying a non-linear, mass-dependent intensity transformation to align this distribution with a reference distribution. This method is combined with a model-driven resampling step that is specifically designed for data from MALDI imaging of tryptic peptides. This method was performed on two sets of tissue samples: a single human teratoma sample and a collection of five tissue microarrays (TMAs) of breast and ovarian tumor tissue samples (N = 241 patients). The MALDI MSI data was acquired in two labs using multiple protocols, allowing us to investigate different inter-lab and cross-protocol scenarios, thus covering a wide range of technical variations. Our results suggest that the proposed cross-normalization significantly reduces such batch effects not only in inter-sample and inter-lab comparisons but also in cross-protocol scenarios. This demonstrates the feasibility of cross-normalization and joint data analysis even under conditions where preparation and acquisition protocols themselves are subject to variation.

LoDoPaB-CT, a benchmark dataset for low-dose computed tomography reconstruction.

Deep learning approaches for tomographic image reconstruction have become very effective and have been demonstrated to be competitive in the field. Comparing these approaches is a challenging task as they rely to a great extent on the data and setup used for training. With the Low-Dose Parallel Beam (LoDoPaB)-CT dataset, we provide a comprehensive, open-access database of computed tomography images and simulated low photon count measurements. It is suitable for training and comparing deep learning methods as well as classical reconstruction approaches. The dataset contains over 40000 scan slices from around 800 patients selected from the LIDC/IDRI database. The data selection and simulation setup are described in detail, and the generating script is publicly accessible. In addition, we provide a Python library for simplified access to the dataset and an online reconstruction challenge. Furthermore, the dataset can also be used for transfer learning as well as sparse and limited-angle reconstruction scenarios.

Using the Chemical Noise Background in MALDI Mass Spectrometry Imaging for Mass Alignment and Calibration.

Matrix-assisted laser desorption/ionization mass spectrometry imaging (MALDI MSI) is an established tool for the investigation of formalin fixed paraffin embedded (FFPE) tissue samples and shows a high potential for applications in clinical research and histopathological diagnosis. The applicability and accuracy of this method, however, heavily depends on the quality of the acquired data, and in particular mass misalignment in axial time-of-flight (TOF) MSI continues to be a serious issue. We present a mass alignment and recalibration method that is specifically designed to operate on MALDI peptide imaging data. The proposed method exploits statistical properties of the characteristic chemical noise background observed in peptide imaging experiments. By comparing these properties to a theoretical peptide mass model, the effective mass shift of each spectrum is estimated and corrected. The method was evaluated on a cohort of 31 FFPE tissue samples, pursuing a statistical validation approach to estimate both the reduction of relative misalignment, as well as the increase in absolute mass accuracy. Our results suggest that a relative mass precision of approximately 5 ppm and an absolute accuracy of approximately 20 ppm are achievable using our method.

Singular Values for ReLU Layers.

Despite their prevalence in neural networks, we still lack a thorough theoretical characterization of rectified linear unit (ReLU) layers. This article aims to further our understanding of ReLU layers by studying how the activation function ReLU interacts with the linear component of the layer and what role this interaction plays in the success of the neural network in achieving its intended task. To this end, we introduce two new tools: ReLU singular values of operators and the Gaussian mean width of operators. By presenting, on the one hand, theoretical justifications, results, and interpretations of these two concepts and, on the other hand, numerical experiments and results of the ReLU singular values and the Gaussian mean width being applied to trained neural networks, we hope to give a comprehensive, singular-value-centric view of ReLU layers. We find that ReLU singular values and the Gaussian mean width do not only enable theoretical insights but also provide one with metrics that seem promising for practical applications. In particular, these measures can be used to distinguish correctly and incorrectly classified data as it traverses the network. We conclude by introducing two tools based on our findings: double layers and harmonic pruning.

Supervised non-negative matrix factorization methods for MALDI imaging applications.

MOTIVATION: Non-negative matrix factorization (NMF) is a common tool for obtaining low-rank approximations of non-negative data matrices and has been widely used in machine learning, e.g. for supporting feature extraction in high-dimensional classification tasks. In its classical form, NMF is an unsupervised method, i.e. the class labels of the training data are not used when computing the NMF. However, incorporating the classification labels into the NMF algorithms allows to specifically guide them toward the extraction of data patterns relevant for discriminating the respective classes. This approach is particularly suited for the analysis of mass spectrometry imaging (MSI) data in clinical applications, such as tumor typing and classification, which are among the most challenging tasks in pathology. Thus, we investigate algorithms for extracting tumor-specific spectral patterns from MSI data by NMF methods. RESULTS: In this article, we incorporate a priori class labels into the NMF cost functional by adding appropriate supervised penalty terms. Numerical experiments on a MALDI imaging dataset confirm that the novel supervised NMF methods lead to significantly better classification accuracy and stability as compared with other standard approaches. AVAILABILITY AND IMPLEMENTATON: https://gitlab.informatik.uni-bremen.de/digipath/Supervised_NMF_Methods_for_MALDI.git. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Targeted Feature Extraction in MALDI Mass Spectrometry Imaging to Discriminate Proteomic Profiles of Breast and Ovarian Cancer.

PURPOSE: To develop a mass spectrometry imaging (MSI) based workflow for extracting m/z values related to putative protein biomarkers and using these for reliable tumor classification. EXPERIMENTAL DESIGN: Given a list of putative breast and ovarian cancer biomarker proteins, a set of related m/z values are extracted from heterogeneous MSI datasets derived from formalin-fixed paraffin-embedded tissue material. Based on these features, a linear discriminant analysis classification model is trained to discriminate the two tumor types. RESULTS: It is shown that the discriminative power of classification models based on the extracted features is increased compared to the automatic training approach, especially when classifiers are applied to spectral data acquired under different conditions (instrument, preparation, laboratory). CONCLUSIONS AND CLINICAL RELEVANCE: Robust classification models not confounded by technical variation between MSI measurements are obtained. This supports the assumption that the classification of the respective tumor types is based on biological rather than technical differences, and that the selected features are related to the proteomic profiles of the tumor types under consideration.

3D MALDI mass spectrometry imaging reveals specific localization of long-chain acylcarnitines within a 10-day time window of spinal cord injury.

We report, for the first time, the detection and specific localization of long-chain acylcarnitines (LC ACs) along the lesion margins in an experimental model of spinal cord injury (SCI) using 3D mass spectrometry imaging (MSI). Acylcarnitines palmitoylcarnitine (AC(16:0)), palmitoleoylcarnitine (AC(16:1)), elaidic carnitine (AC(18:1)) and tetradecanoylcarnitine (AC(14:1)) were detected as early as 3 days post injury, and were present along the lesion margins 7 and 10 days after SCI induced by balloon compression technique in the rat. 3D MSI revealed the heterogeneous distribution of these lipids across the injured spinal cord, appearing well-defined at the lesion margins rostral to the lesion center, and becoming widespread and less confined to the margins at the region located caudally. The assigned acylcarnitines co-localize with resident microglia/macrophages detected along the lesion margins by immunofluorescence. Given the reported pro-inflammatory role of these acylcarnitines, their specific spatial localization along the lesion margin could hint at their potential pathophysiological roles in the progression of SCI.

Deep learning for tumor classification in imaging mass spectrometry.

Motivation: Tumor classification using imaging mass spectrometry (IMS) data has a high potential for future applications in pathology. Due to the complexity and size of the data, automated feature extraction and classification steps are required to fully process the data. Since mass spectra exhibit certain structural similarities to image data, deep learning may offer a promising strategy for classification of IMS data as it has been successfully applied to image classification. Results: Methodologically, we propose an adapted architecture based on deep convolutional networks to handle the characteristics of mass spectrometry data, as well as a strategy to interpret the learned model in the spectral domain based on a sensitivity analysis. The proposed methods are evaluated on two algorithmically challenging tumor classification tasks and compared to a baseline approach. Competitiveness of the proposed methods is shown on both tasks by studying the performance via cross-validation. Moreover, the learned models are analyzed by the proposed sensitivity analysis revealing biologically plausible effects as well as confounding factors of the considered tasks. Thus, this study may serve as a starting point for further development of deep learning approaches in IMS classification tasks. Availability and implementation: https://gitlab.informatik.uni-bremen.de/digipath/Deep_Learning_for_Tumor_Classification_in_IMS. Contact: jbehrmann@uni-bremen.de or christianetmann@uni-bremen.de. Supplementary information: Supplementary data are available at Bioinformatics online.

A new classification method for MALDI imaging mass spectrometry data acquired on formalin-fixed paraffin-embedded tissue samples.

Matrix-assisted laser desorption/ionization imaging mass spectrometry (MALDI IMS) shows a high potential for applications in histopathological diagnosis, and in particular for supporting tumor typing and subtyping. The development of such applications requires the extraction of spectral fingerprints that are relevant for the given tissue and the identification of biomarkers associated with these spectral patterns. We propose a novel data analysis method based on the extraction of characteristic spectral patterns (CSPs) that allow automated generation of classification models for spectral data. Formalin-fixed paraffin embedded (FFPE) tissue samples from N=445 patients assembled on 12 tissue microarrays were analyzed. The method was applied to discriminate primary lung and pancreatic cancer, as well as adenocarcinoma and squamous cell carcinoma of the lung. A classification accuracy of 100% and 82.8%, resp., could be achieved on core level, assessed by cross-validation. The method outperformed the more conventional classification method based on the extraction of individual m/z values in the first application, while achieving a comparable accuracy in the second. LC-MS/MS peptide identification demonstrated that the spectral features present in selected CSPs correspond to peptides relevant for the respective classification. This article is part of a Special Issue entitled: MALDI Imaging, edited by Dr. Corinna Henkel and Prof. Peter Hoffmann.