GPU-Accelerated RSF Level Set Evolution for Large-Scale Microvascular Segmentation.

Microvascular networks are challenging to model because these structures are currently near the diffraction limit for most advanced three-dimensional imaging modalities, including confocal and light sheet microscopy. This makes semantic segmentation difficult, because individual components of these networks fluctuate within the confines of individual pixels. Level set methods are ideally suited to solve this problem by providing surface and topological constraints on the resulting model, however these active contour techniques are extremely time intensive and impractical for terabyte-scale images. We propose a reformulation and implementation of the region-scalable fitting (RSF) level set model that makes it amenable to three-dimensional evaluation using both single-instruction multiple data (SIMD) and single-program multiple-data (SPMD) parallel processing. This enables evaluation of the level set equation on independent regions of the data set using graphics processing units (GPUs), making large-scale segmentation of high-resolution networks practical and inexpensive. We tested this 3D parallel RSF approach on multiple data sets acquired using state-of-the-art imaging techniques to acquire microvascular data, including micro-CT, light sheet fluorescence microscopy (LSFM) and milling microscopy. To assess the performance and accuracy of the RSF model, we conducted a Monte-Carlo-based validation technique to compare results to other segmentation methods. We also provide a rigorous profiling to show the gains in processing speed leveraging parallel hardware. This study showcases the practical application of the RSF model, emphasizing its utility in the challenging domain of segmenting large-scale high-topology network structures with a particular focus on building microvascular models.

Rapid hyperspectral photothermal mid-infrared spectroscopic imaging from sparse data for gynecologic cancer tissue subtyping.

Ovarian cancer detection has traditionally relied on a multi-step process that includes biopsy, tissue staining, and morphological analysis by experienced pathologists. While widely practiced, this conventional approach suffers from several drawbacks: it is qualitative, time-intensive, and heavily dependent on the quality of staining. Mid-infrared (MIR) hyperspectral photothermal imaging is a label-free, biochemically quantitative technology that, when combined with machine learning algorithms, can eliminate the need for staining and provide quantitative results comparable to traditional histology. However, this technology is slow. This work presents a novel approach to MIR photothermal imaging that enhances its speed by an order of magnitude. Our method significantly accelerates data collection by capturing a combination of highresolution and interleaved, lower-resolution infrared band images and applying computational techniques for data interpolation. We effectively minimize data collection requirements by leveraging sparse data acquisition and employing curvelet-based reconstruction algorithms. This approach enhances imaging speed without compromising image quality and ensures robust tissue segmentation. This method resolves the longstanding trade-off between imaging resolution and data collection speed, enabling the reconstruction of high-quality, high-resolution images from undersampled datasets and achieving a 10X improvement in data acquisition time. We assessed the performance of our sparse imaging methodology using a variety of quantitative metrics, including mean squared error (MSE), structural similarity index (SSIM), and tissue subtype classification accuracies, employing both random forest and convolutional neural network (CNN) models, accompanied by Receiver Operating Characteristic (ROC) curves. Our statistically robust analysis, based on data from 100 ovarian cancer patient samples and over 65 million data points, demonstrates the method's capability to produce superior image quality and accurately distinguish between different gynecological tissue types with segmentation accuracy exceeding 95%. Our work demonstrates the feasibility of integrating rapid MIR hyperspectral photothermal imaging with machine learning in enhancing ovarian cancer tissue characterization, paving the way for quantitative, label-free, automated histopathology. It represents a significant leap forward from traditional histopathological methods, offering profound implications for cancer diagnostics and treatment decision-making.

Leveraging mid-infrared spectroscopic imaging and deep learning for tissue subtype classification in ovarian cancer.

Mid-infrared spectroscopic imaging (MIRSI) is an emerging class of label-free techniques being leveraged for digital histopathology. Modern histopathologic identification of ovarian cancer involves tissue staining followed by morphological pattern recognition. This process is time-consuming and subjective and requires extensive expertise. This paper presents the first label-free, quantitative, and automated histological recognition of ovarian tissue subtypes using a new MIRSI technique. This optical photothermal infrared (O-PTIR) imaging technique provides a 10× enhancement in spatial resolution relative to prior instruments. It enables sub-cellular spectroscopic investigation of tissue at biochemically important fingerprint wavelengths. We demonstrate that the enhanced resolution of sub-cellular features, combined with spectroscopic information, enables reliable classification of ovarian cell subtypes achieving a classification accuracy of 0.98. Moreover, we present a statistically robust analysis from 78 patient samples with over 60 million data points. We show that sub-cellular resolution from five wavenumbers is sufficient to outperform state-of-the-art diffraction-limited techniques with up to 235 wavenumbers. We also propose two quantitative biomarkers based on the relative quantities of epithelia and stroma that exhibit efficacy in early cancer diagnosis. This paper demonstrates that combining deep learning with intrinsic biochemical MIRSI measurements enables quantitative evaluation of cancerous tissue, improving the rigor and reproducibility of histopathology.

Polarization Sensitive Photothermal Mid-Infrared Spectroscopic Imaging of Human Bone Marrow Tissue.

Collagen quantity and integrity play an important role in understanding diseases such as myelofibrosis (MF). Label-free mid-infrared spectroscopic imaging (MIRSI) has the potential to quantify collagen while minimizing the subjective variance observed with conventional histopathology. Infrared (IR) spectroscopy with polarization sensitivity provides chemical information while also estimating tissue dichroism. This can potentially aid MF grading by revealing the structure and orientation of collagen fibers. Simultaneous measurement of collagen structure and biochemical properties can translate clinically into improved diagnosis and enhance our understanding of disease progression. In this paper, we present the first report of polarization-dependent spectroscopic variations in collagen from human bone marrow samples. We build on prior work with animal models and extend it to human clinical biopsies with a practical method for high-resolution chemical and structural imaging of bone marrow on clinical glass slides. This is done using a new polarization-sensitive photothermal mid-infrared spectroscopic imaging scheme that enables sample and source independent polarization control. This technology provides 0.5 µm spatial resolution, enabling the identification of thin (≈1 µm) collagen fibers that were not separable using Fourier Transform Infrared (FT-IR) imaging in the fingerprint region at diffraction-limited resolution ( ≈ 5 µm). Finally, we propose quantitative metrics to identify fiber orientation from discrete band images (amide I and amide II) measured under three polarizations. Previous studies have used a pair of orthogonal polarization measurements, which is insufficient for clinical samples since human bone biopsies contain collagen fibers with multiple orientations. Here, we address this challenge and demonstrate that three polarization measurements are necessary to resolve orientation ambiguity in clinical bone marrow samples. This is also the first study to demonstrate the ability to spectroscopically identify thin collagen fibers (≈1 µm diameter) and their orientations, which is critical for accurate grading of human bone marrow fibrosis.

Prostacyclin Promotes Degenerative Pathology in a Model of Alzheimer's Disease.

Alzheimer's disease (AD) is a progressive neurodegenerative disorder that is the most common form of dementia in aged populations. A substantial amount of data demonstrates that chronic neuroinflammation can accelerate neurodegenerative pathologies. In AD, chronic neuroinflammation results in the upregulation of cyclooxygenase and increased production of prostaglandin H2, a precursor for many vasoactive prostanoids. While it is well-established that many prostaglandins can modulate the progression of neurodegenerative disorders, the role of prostacyclin (PGI2) in the brain is poorly understood. We have conducted studies to assess the effect of elevated prostacyclin biosynthesis in a mouse model of AD. Upregulated prostacyclin expression significantly worsened multiple measures associated with amyloid-ß (Aß) disease pathologies. Mice overexpressing both Aß and PGI2 exhibited impaired learning and memory and increased anxiety-like behavior compared with non-transgenic and PGI2 control mice. PGI2 overexpression accelerated the development of Aß accumulation in the brain and selectively increased the production of soluble Aß42. PGI2 damaged the microvasculature through alterations in vascular length and branching; Aß expression exacerbated these effects. Our findings demonstrate that chronic prostacyclin expression plays a novel and unexpected role that hastens the development of the AD phenotype.

Computational insights on coronary artery function.

Collateral arteries may act as natural bypasses that reduce hypoperfusion after a coronary blockage. 3D imaging of neonatal and adult mouse hearts, plus human fetal and diseased adult hearts, is now used to computationally predict flow within the heart, and understand the cardioprotective role of collateral arteries in vivo.

Multimodal high-resolution embryonic imaging with light sheet fluorescence microscopy and optical coherence tomography.

A high-resolution imaging system combining optical coherence tomography (OCT) and light sheet fluorescence microscopy (LSFM) was developed. LSFM confined the excitation to only the focal plane, removing the out of plane fluorescence. This enabled imaging a murine embryo with higher speed and specificity than traditional fluorescence microscopy. OCT gives information about the structure of the embryo from the same plane illuminated by LSFM. The co-planar OCT and LSFM instrument was capable of performing co-registered functional and structural imaging of mouse embryos simultaneously.

Multi-modal image sharpening in fourier transform infrared (FTIR) microscopy.

Mid-infrared Spectroscopic Imaging (MIRSI) provides spatially-resolved molecular specificity by measuring wavelength-dependent mid-infrared absorbance. Infrared microscopes use large numerical aperture objectives to obtain high-resolution images of heterogeneous samples. However, the optical resolution is fundamentally diffraction-limited, and therefore wavelength-dependent. This significantly limits resolution in infrared microscopy, which relies on long wavelengths (2.5 µm to 12.5 µm) for molecular specificity. The resolution is particularly restrictive in biomedical and materials applications, where molecular information is encoded in the fingerprint region (6 µm to 12 µm), limiting the maximum resolving power to between 3 µm and 6 µm. We present an unsupervised curvelet-based image fusion method that overcomes limitations in spatial resolution by augmenting infrared images with label-free visible microscopy. We demonstrate the effectiveness of this approach by fusing images of breast and ovarian tumor biopsies acquired using both infrared and dark-field microscopy. The proposed fusion algorithm generates a hyperspectral dataset that has both high spatial resolution and good molecular contrast. We validate this technique using multiple standard approaches and through comparisons to super-resolved experimentally measured photothermal spectroscopic images. We also propose a novel comparison method based on tissue classification accuracy.

Artificial intelligence and machine learning in nephropathology.

Artificial intelligence (AI) for the purpose of this review is an umbrella term for technologies emulating a nephropathologist's ability to extract information on diagnosis, prognosis, and therapy responsiveness from native or transplant kidney biopsies. Although AI can be used to analyze a wide variety of biopsy-related data, this review focuses on whole slide images traditionally used in nephropathology. AI applications in nephropathology have recently become available through several advancing technologies, including (i) widespread introduction of glass slide scanners, (ii) data servers in pathology departments worldwide, and (iii) through greatly improved computer hardware to enable AI training. In this review, we explain how AI can enhance the reproducibility of nephropathology results for certain parameters in the context of precision medicine using advanced architectures, such as convolutional neural networks, that are currently the state of the art in machine learning software for this task. Because AI applications in nephropathology are still in their infancy, we show the power and potential of AI applications mostly in the example of oncopathology. Moreover, we discuss the technological obstacles as well as the current stakeholder and regulatory concerns about developing AI applications in nephropathology from the perspective of nephropathologists and the wider nephrology community. We expect the gradual introduction of these technologies into routine diagnostics and research for selective tasks, suggesting that this technology will enhance the performance of nephropathologists rather than making them redundant.

Multiplex protein-specific microscopy with ultraviolet surface excitation.

Immunohistochemical techniques, such as immunofluorescence (IF) staining, enable microscopic imaging of local protein expression within tissue samples. Molecular profiling enabled by IF is critical to understanding pathogenesis and is often involved in complex diagnoses. A recent innovation, known as microscopy with ultraviolet surface excitation (MUSE), uses deep ultraviolet (≈280 nm) illumination to excite labels at the tissue surface, providing equivalent images without fixation, embedding, and sectioning. However, MUSE has not yet been integrated into traditional IF pipelines. This limits its application in more complex diagnoses that rely on protein-specific markers. This paper aims to broaden the applicability of MUSE to multiplex immunohistochemistry using quantum dot nanoparticles. We demonstrate the advantages of quantum dot labels for protein-specific MUSE imaging on both paraffin-embedded and intact tissue, significantly expanding MUSE applicability to protein-specific applications. Furthermore, with recent innovations in three-dimensional ultraviolet fluorescence microscopy, this opens the door to three-dimensional IF imaging with quantum dots using ultraviolet excitation.

Automated Osteosclerosis Grading of Clinical Biopsies Using Infrared Spectroscopic Imaging.

Osteosclerosis and myefibrosis are complications of myeloproliferative neoplasms. These disorders result in excess growth of trabecular bone and collagen fibers that replace hematopoietic cells, resulting in abnormal bone marrow function. Treatments using imatinib and JAK2 pathway inhibitors can be effective on osteosclerosis and fibrosis; therefore, accurate grading is critical for tracking treatment effectiveness. Current grading standards use a four-class system based on analysis of biopsies stained with three histological stains: hematoxylin and eosin (H&E), Masson's trichrome, and reticulin. However, conventional grading can be subjective and imprecise, impacting the effectiveness of treatment. In this Article, we demonstrate that mid-infrared spectroscopic imaging may serve as a quantitative diagnostic tool for quantitatively tracking disease progression and response to treatment. The proposed approach is label-free and provides automated quantitative analysis of osteosclerosis and collagen fibrosis.

Three-Dimensional Microscopy by Milling with Ultraviolet Excitation.

Analysis of three-dimensional biological samples is critical to understanding tissue function and the mechanisms of disease. Many chronic conditions, like neurodegenerative diseases and cancers, correlate with complex tissue changes that are difficult to explore using two-dimensional histology. While three-dimensional techniques such as confocal and light-sheet microscopy are well-established, they are time consuming, require expensive instrumentation, and are limited to small tissue volumes. Three-dimensional microscopy is therefore impractical in clinical settings and often limited to core facilities at major research institutions. There would be a tremendous benefit to providing clinicians and researchers with the ability to routinely image large three-dimensional tissue volumes at cellular resolution. In this paper, we propose an imaging methodology that enables fast and inexpensive three-dimensional imaging that can be readily integrated into current histology pipelines. This method relies on block-face imaging of paraffin-embedded samples using deep-ultraviolet excitation. The imaged surface is then ablated to reveal the next tissue section for imaging. The final image stack is then aligned and reconstructed to provide tissue models that exceed the depth and resolution achievable with modern three-dimensional imaging systems.

Accurate flow in augmented networks (AFAN): an approach to generating three-dimensional biomimetic microfluidic networks with controlled flow.

In vivo, microvasculature provides oxygen, nutrients, and soluble factors necessary for cell survival and function. The highly tortuous, densely-packed, and interconnected three-dimensional (3D) architecture of microvasculature ensures that cells receive these crucial components. The ability to duplicate microvascular architecture in tissue-engineered models could provide a means to generate large-volume constructs as well as advanced microphysiological systems. Similarly, the ability to induce realistic flow in engineered microvasculature is crucial to recapitulating in vivo-like flow and transport. Advanced biofabrication techniques are capable of generating 3D, biomimetic microfluidic networks in hydrogels, however, these models can exhibit systemic aberrations in flow due to incorrect boundary conditions. To overcome this problem, we developed an automated method for generating synthetic augmented channels that induce the desired flow properties within three-dimensional microfluidic networks. These augmented inlets and outlets enforce the appropriate boundary conditions for achieving specified flow properties and create a three-dimensional output useful for image-guided fabrication techniques to create biomimetic microvascular networks.

Three-dimensional GPU-accelerated active contours for automated localization of cells in large images.

Cell segmentation in microscopy is a challenging problem, since cells are often asymmetric and densely packed. Successful cell segmentation algorithms rely identifying seed points, and are highly sensitive to variablility in cell size. In this paper, we present an efficient and highly parallel formulation for symmetric three-dimensional contour evolution that extends previous work on fast two-dimensional snakes. We provide a formulation for optimization on 3D images, as well as a strategy for accelerating computation on consumer graphics hardware. The proposed software takes advantage of Monte-Carlo sampling schemes in order to speed up convergence and reduce thread divergence. Experimental results show that this method provides superior performance for large 2D and 3D cell localization tasks when compared to existing methods on large 3D brain images.

Deep learning for FTIR histology: leveraging spatial and spectral features with convolutional neural networks.

Current methods for cancer detection rely on tissue biopsy, chemical labeling/staining, and examination of the tissue by a pathologist. Though these methods continue to remain the gold standard, they are non-quantitative and susceptible to human error. Fourier transform infrared (FTIR) spectroscopic imaging has shown potential as a quantitative alternative to traditional histology. However, identification of histological components requires reliable classification based on molecular spectra, which are susceptible to artifacts introduced by noise and scattering. Several tissue types, particularly in heterogeneous tissue regions, tend to confound traditional classification methods. Convolutional neural networks (CNNs) are the current state-of-the-art in image classification, providing the ability to learn spatial characteristics of images. In this paper, we demonstrate that CNNs with architectures designed to process both spectral and spatial information can significantly improve classifier performance over per-pixel spectral classification. We report classification results after applying CNNs to data from tissue microarrays (TMAs) to identify six major cellular and acellular constituents of tissue, namely adipocytes, blood, collagen, epithelium, necrosis, and myofibroblasts. Experimental results show that the use of spatial information in addition to the spectral information brings significant improvements in the classifier performance and allows classification of cellular subtypes, such as adipocytes, that exhibit minimal chemical information but have distinct spatial characteristics. This work demonstrates the application and efficiency of deep learning algorithms in improving the diagnostic techniques in clinical and research activities related to cancer.

Digital Staining of High-Definition Fourier Transform Infrared (FT-IR) Images Using Deep Learning.

Histological stains, such as hematoxylin and eosin (H&E), are routinely used in clinical diagnosis and research. While these labels offer a high degree of specificity, throughput is limited by the need for multiple samples. Traditional histology stains, such as immunohistochemical labels, also rely only on protein expression and cannot quantify small molecules and metabolites that may aid in diagnosis. Finally, chemical stains and dyes permanently alter the tissue, making downstream analysis impossible. Fourier transform infrared (FT-IR) spectroscopic imaging has shown promise for label-free characterization of important tissue phenotypes and can bypass the need for many chemical labels. Fourier transform infrared classification commonly leverages supervised learning, requiring human annotation that is tedious and prone to errors. One alternative is digital staining, which leverages machine learning to map IR spectra to a corresponding chemical stain. This replaces human annotation with computer-aided alignment. Previous work relies on alignment of adjacent serial tissue sections. Since the tissue samples are not identical at the cellular level, this technique cannot be applied to high-definition FT-IR images. In this paper, we demonstrate that cellular-level mapping can be accomplished using identical samples for both FT-IR and chemical labels. In addition, higher-resolution results can be achieved using a deep convolutional neural network that integrates spatial and spectral features.

Robust Tracing and Visualization of Heterogeneous Microvascular Networks.

Advances in high-throughput imaging allow researchers to collect three-dimensional images of whole organ microvascular networks. These extremely large images contain networks that are highly complex, time consuming to segment, and difficult to visualize. In this paper, we present a framework for segmenting and visualizing vascular networks from terabyte-sized three-dimensional images collected using high-throughput microscopy. While these images require terabytes of storage, the volume devoted to the fiber network is ≈ 4 percent of the total volume size. While the networks themselves are sparse, they are tremendously complex, interconnected, and vary widely in diameter. We describe a parallel GPU-based predictor-corrector method for tracing filaments that is robust to noise and sampling errors common in these data sets. We also propose a number of visualization techniques designed to convey the complex statistical descriptions of fibers across large tissue sections-including commonly studied microvascular characteristics, such as orientation and volume.

TIMING 2.0: high-throughput single-cell profiling of dynamic cell-cell interactions by time-lapse imaging microscopy in nanowell grids.

MOTIVATION: Automated profiling of cell-cell interactions from high-throughput time-lapse imaging microscopy data of cells in nanowell grids (TIMING) has led to fundamental insights into cell-cell interactions in immunotherapy. This application note aims to enable widespread adoption of TIMING by (i) enabling the computations to occur on a desktop computer with a graphical processing unit instead of a server; (ii) enabling image acquisition and analysis to occur in the laboratory avoiding network data transfers to/from a server and (iii) providing a comprehensive graphical user interface. RESULTS: On a desktop computer, TIMING 2.0 takes 5 s/block/image frame, four times faster than our previous method on the same computer, and twice as fast as our previous method (TIMING) running on a Dell PowerEdge server. The cell segmentation accuracy (f-number = 0.993) is superior to our previous method (f-number = 0.821). A graphical user interface provides the ability to inspect the video analysis results, make corrective edits efficiently (one-click editing of an entire nanowell video sequence in 5-10 s) and display a summary of the cell killing efficacy measurements. AVAILABILITY AND IMPLEMENTATION: Open source Python software (GPL v3 license), instruction manual, sample data and sample results are included with the Supplement (https://github.com/RoysamLab/TIMING2). SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Robust Cell Detection for Large-Scale 3D Microscopy Using GPU-Accelerated Iterative Voting.

High-throughput imaging techniques, such as Knife-Edge Scanning Microscopy (KESM),are capable of acquiring three-dimensional whole-organ images at sub-micrometer resolution. These images are challenging to segment since they can exceed several terabytes (TB) in size, requiring extremely fast and fully automated algorithms. Staining techniques are limited to contrast agents that can be applied to large samples and imaged in a single pass. This requires maximizing the number of structures labeled in a single channel, resulting in images that are densely packed with spatial features. In this paper, we propose a three-dimensional approach for locating cells based on iterative voting. Due to the computational complexity of this algorithm, a highly efficient GPU implementation is required to make it practical on large data sets. The proposed algorithm has a limited number of input parameters and is highly parallel.