An End-to-End Platform for Digital Pathology Using Hyperspectral Autofluorescence Microscopy and Deep Learning-Based Virtual Histology.

Conventional histopathology involves expensive and labor-intensive processes that often consume tissue samples, rendering them unavailable for other analyses. We present a novel end-to-end workflow for pathology powered by hyperspectral microscopy and deep learning. First, we developed a custom hyperspectral microscope to nondestructively image the autofluorescence of unstained tissue sections. We then trained a deep learning model to use autofluorescence to generate virtual histologic stains, which avoids the cost and variability of chemical staining procedures and conserves tissue samples. We showed that the virtual images reproduce the histologic features present in the real-stained images using a randomized nonalcoholic steatohepatitis (NASH) scoring comparison study, where both real and virtual stains are scored by pathologists (D.T., A.D.B., R.K.P.). The test showed moderate-to-good concordance between pathologists' scoring on corresponding real and virtual stains. Finally, we developed deep learning-based models for automated NASH Clinical Research Network score prediction. We showed that the end-to-end automated pathology platform is comparable with an independent panel of pathologists for NASH Clinical Research Network scoring when evaluated against the expert pathologist consensus scores. This study provides proof of concept for this virtual staining strategy, which could improve cost, efficiency, and reliability in pathology and enable novel approaches to spatial biology research.

Comparative analysis of machine learning approaches to classify tumor mutation burden in lung adenocarcinoma using histopathology images.

Both histologic subtypes and tumor mutation burden (TMB) represent important biomarkers in lung cancer, with implications for patient prognosis and treatment decisions. Typically, TMB is evaluated by comprehensive genomic profiling but this requires use of finite tissue specimens and costly, time-consuming laboratory processes. Histologic subtype classification represents an established component of lung adenocarcinoma histopathology, but can be challenging and is associated with substantial inter-pathologist variability. Here we developed a deep learning system to both classify histologic patterns in lung adenocarcinoma and predict TMB status using de-identified Hematoxylin and Eosin (H&E) stained whole slide images. We first trained a convolutional neural network to map histologic features across whole slide images of lung cancer resection specimens. On evaluation using an external data source, this model achieved patch-level area under the receiver operating characteristic curve (AUC) of 0.78-0.98 across nine histologic features. We then integrated the output of this model with clinico-demographic data to develop an interpretable model for TMB classification. The resulting end-to-end system was evaluated on 172 held out cases from TCGA, achieving an AUC of 0.71 (95% CI 0.63-0.80). The benefit of using histologic features in predicting TMB is highlighted by the significant improvement this approach offers over using the clinical features alone (AUC of 0.63 [95% CI 0.53-0.72], p = 0.002). Furthermore, we found that our histologic subtype-based approach achieved performance similar to that of a weakly supervised approach (AUC of 0.72 [95% CI 0.64-0.80]). Together these results underscore that incorporating histologic patterns in biomarker prediction for lung cancer provides informative signals, and that interpretable approaches utilizing these patterns perform comparably with less interpretable, weakly supervised approaches.

Multispectral open-air intraoperative fluorescence imaging.

Intraoperative fluorescence imaging informs decisions regarding surgical margins by detecting and localizing signals from fluorescent reporters, labeling targets such as malignant tissues. This guidance reduces the likelihood of undetected malignant tissue remaining after resection, eliminating the need for additional treatment or surgery. The primary challenges in performing open-air intraoperative fluorescence imaging come from the weak intensity of the fluorescence signal in the presence of strong surgical and ambient illumination, and the auto-fluorescence of non-target components, such as tissue, especially in the visible spectral window (400-650 nm). In this work, a multispectral open-air fluorescence imaging system is presented for translational image-guided intraoperative applications, which overcomes these challenges. The system is capable of imaging weak fluorescence signals with nanomolar sensitivity in the presence of surgical illumination. This is done using synchronized fluorescence excitation and image acquisition with real-time background subtraction. Additionally, the system uses a liquid crystal tunable filter for acquisition of multispectral images that are used to spectrally unmix target fluorescence from non-target auto-fluorescence. Results are validated by preclinical studies on murine models and translational canine oncology models.

Automated Quantitative Bone Analysis in In Vivo X-ray Micro-Computed Tomography.

Measurement and analysis of bone morphometry in 3D micro-computed tomography volumes using automated image processing and analysis improve the accuracy, consistency, reproducibility, and speed of preclinical osteological research studies. Automating segmentation and separation of individual bones in 3D micro-computed tomography volumes of murine models presents significant challenges considering partial volume effects and joints with thin spacing, i.e., 50 to [Formula: see text]. In this paper, novel hybrid splitting filters are presented to overcome the challenge of automated bone separation. This is achieved by enhancing joint contrast using rotationally invariant second-derivative operators. These filters generate split components that seed marker-controlled watershed segmentation. In addition, these filters can be used to separate metaphysis and epiphysis in long bones, e.g., femur, and remove the metaphyseal growth plate from the detected bone mask in morphometric measurements. Moreover, for slice-by-slice stereological measurements of long bones, particularly curved bones, such as tibia, the accuracy of the analysis can be improved if the planar measurements are guided to follow the longitudinal direction of the bone. In this paper, an approach is presented for characterizing the bone medial axis using morphological thinning and centerline operations. Building upon the medial axis, a novel framework is presented to automatically guide stereological measurements of long bones and enhance measurement accuracy and consistency. These image processing and analysis approaches are combined in an automated streamlined software workflow and applied to a range of in vivo micro-computed tomography studies for validation.

Hadamard multiplexed fluorescence tomography.

Depth-resolved three-dimensional (3D) reconstruction of fluorophore-tagged inclusions in fluorescence tomography (FT) poses a highly ill-conditioned problem as depth information must be extracted from boundary data. Due to the ill-posed nature of the FT inverse problem, noise and errors in the data can severely impair the accuracy of the 3D reconstructions. The signal-to-noise ratio (SNR) of the FT data strongly affects the quality of the reconstructions. Additionally, in FT scenarios where the fluorescent signal is weak, data acquisition requires lengthy integration times that result in excessive FT scan periods. Enhancing the SNR of FT data contributes to the robustness of the 3D reconstructions as well as the speed of FT scans. A major deciding factor in the SNR of the FT data is the power of the radiation illuminating the subject to excite the administered fluorescent reagents. In existing single-point illumination FT systems, the source power level is limited by the skin maximum radiation exposure levels. In this paper, we introduce and study the performance of a multiplexed fluorescence tomography system with orders-of-magnitude enhanced data SNR over existing systems. The proposed system allows for multi-point illumination of the subject without jeopardizing the information content of the FT measurements and results in highly robust reconstructions of fluorescent inclusions from noisy FT data. Improvements offered by the proposed system are validated by numerical and experimental studies.

Adaptive row-action inverse solver for fast noise-robust three-dimensional reconstructions in bioluminescence tomography: theory and dual-modality optical/computed tomography in vivo studies.

A novel approach is presented for obtaining fast robust three-dimensional (3-D) reconstructions of bioluminescent reporters buried deep inside animal subjects from multispectral images of surface bioluminescent photon densities. The proposed method iteratively acts upon the equations relating the multispectral data to the luminescent distribution with high computational efficiency to provide robust 3-D reconstructions. Unlike existing algebraic reconstruction techniques, the proposed method is designed to use adaptive projections that iteratively guide the updates to the solution with improved speed and robustness. Contrary to least-squares reconstruction methods, the proposed technique does not require parameter selection or optimization for optimal performance. Additionally, optimized schemes for thresholding, sampling, and ordering of the bioluminescence tomographic data used by the proposed method are presented. The performance of the proposed approach in reconstructing the shape, volume, flux, and depth of luminescent inclusions is evaluated in a multitude of phantom-based and dual-modality in vivo studies in which calibrated sources are implanted in animal subjects and imaged in a dual-modality optical/computed tomography platform. Statistical analysis of the errors in the depth and flux of the reconstructed inclusions and the convergence time of the proposed method is used to demonstrate its unbiased performance, low error variance, and computational efficiency.

High-contrast subcutaneous vein detection and localization using multispectral imaging.

Multispectral imaging has shown promise in subcutaneous vein detection and localization in human subjects. While many limitations of single-wavelength methods are addressed in multispectral vein detection methods, their performance is still limited by artifacts arising from background skin reflectance and optimality of postprocessing algorithms. We propose a background removal technique that enhances the contrast and performance of multispectral vein detection. We use images acquired at visible wavelengths as reference for removing skin reflectance background from subcutaneous structures in near-infrared images. Results are validated by experiments on human subjects.

Total variation regularization for 3D reconstruction in fluorescence tomography: experimental phantom studies.

Fluorescence tomography (FT) is depth-resolved three-dimensional (3D) localization and quantification of fluorescence distribution in biological tissue and entails a highly ill-conditioned problem as depth information must be extracted from boundary measurements. Conventionally, L2 regularization schemes that penalize the euclidean norm of the solution and possess smoothing effects are used for FT reconstruction. Oversmooth, continuous reconstructions lack high-frequency edge-type features of the original distribution and yield poor resolution. We propose an alternative regularization method for FT that penalizes the total variation (TV) norm of the solution to preserve sharp transitions in the reconstructed fluorescence map while overcoming ill-posedness. We have developed two iterative methods for fast 3D reconstruction in FT based on TV regularization inspired by Rudin-Osher-Fatemi and split Bregman algorithms. The performance of the proposed method is studied in a phantom-based experiment using a noncontact constant-wave trans-illumination FT system. It is observed that the proposed method performs better in resolving fluorescence inclusions at different depths.