Deep learning-based virtual H& E staining from label-free autofluorescence lifetime images.

Label-free autofluorescence lifetime is a unique feature of the inherent fluorescence signals emitted by natural fluorophores in biological samples. Fluorescence lifetime imaging microscopy (FLIM) can capture these signals enabling comprehensive analyses of biological samples. Despite the fundamental importance and wide application of FLIM in biomedical and clinical sciences, existing methods for analysing FLIM images often struggle to provide rapid and precise interpretations without reliable references, such as histology images, which are usually unavailable alongside FLIM images. To address this issue, we propose a deep learning (DL)-based approach for generating virtual Hematoxylin and Eosin (H&E) staining. By combining an advanced DL model with a contemporary image quality metric, we can generate clinical-grade virtual H&E-stained images from label-free FLIM images acquired on unstained tissue samples. Our experiments also show that the inclusion of lifetime information, an extra dimension beyond intensity, results in more accurate reconstructions of virtual staining when compared to using intensity-only images. This advancement allows for the instant and accurate interpretation of FLIM images at the cellular level without the complexities associated with co-registering FLIM and histology images. Consequently, we are able to identify distinct lifetime signatures of seven different cell types commonly found in the tumour microenvironment, opening up new opportunities towards biomarker-free tissue histology using FLIM across multiple cancer types.

Bayesian Statistical Analysis for Bacterial Detection in Pulmonary Endomicroscopic Fluorescence Lifetime Imaging.

Pneumonia, a respiratory disease often caused by bacterial infection in the distal lung, requires rapid and accurate identification, especially in settings such as critical care. Initiating or de-escalating antimicrobials should ideally be guided by the quantification of pathogenic bacteria for effective treatment. Optical endomicroscopy is an emerging technology with the potential to expedite bacterial detection in the distal lung by enabling in vivo and in situ optical tissue characterisation. With advancements in detector technology, optical endomicroscopy can utilize fluorescence lifetime imaging (FLIM) to help detect events that were previously challenging or impossible to identify using fluorescence intensity imaging. In this paper, we propose an iterative Bayesian approach for bacterial detection in FLIM. We model the FLIM image as a linear combination of background intensity, Gaussian noise, and additive outliers (labelled bacteria). While previous bacteria detection methods model anomalous pixels as bacteria, here the FLIM outliers are modelled as circularly symmetric Gaussian-shaped objects, based on their discrete shape observed through visual analysis and the physical nature of the imaging modality. A Hierarchical Bayesian model is used to solve the bacterial detection problem where prior distributions are assigned to unknown parameters. A Metropolis-Hastings within Gibbs sampler draws samples from the posterior distribution. The proposed method's detection performance is initially measured using synthetic images, and shows significant improvement over existing approaches. Further analysis is conducted on real optical endomicroscopy FLIM images annotated by trained personnel. The experiments show the proposed approach outperforms existing methods by a margin of +16.85% ( F1 ) for detection accuracy.

A Novel Fit-Flexible Fluorescence Soft Imager: Tri-Sensing of Intensity, Fall-Time, and Life Profile.

Time-resolved fluorescence imaging techniques, like confocal fluorescence lifetime imaging microscopy, are powerful photonic instrumentation tools of modern science with diverse applications, including: biology, medicine, and chemistry. However, complexities of the systems, both at specimen and device levels, cause difficulties in quantifying soft biomarkers. To address the problems, we first aim to understand and model the underlying photophysics of fluorescence decay curves. For this purpose, we provide a set of mathematical functions, called "life models", fittable with the real temporal recordings of histogram of photon counts. For each model, an equivalent electrical circuit, called a "life circuit", is derived for explaining the whole process. In confocal endomicroscopy, the components of excitation laser, specimen, and fluorescence-emission signal as the histogram of photon counts are modelled by a power source, network of resistor-inductor-capacitor circuitry, and multimetre, respectively. We then design a novel pixel-level temporal classification algorithm, called a "fit-flexible approach", where qualities of "intensity", "fall-time", and "life profile" are identified for each point. A model selection mechanism is used at each pixel to flexibly choose the best representative life model based on a proposed Misfit-percent metric. A two-dimensional arrangement of the quantified information detects some kind of structural information. This approach showed a potential of separating microbeads from lung tissue, distinguishing the tri-sensing from conventional methods. We alleviated by 7% the error of the Misfit-percent for recovering the histograms on real samples than the best state-of-the-art competitor. Codes are available online.

Applications of machine learning in time-domain fluorescence lifetime imaging: a review.

Many medical imaging modalities have benefited from recent advances in Machine Learning (ML), specifically in deep learning, such as neural networks. Computers can be trained to investigate and enhance medical imaging methods without using valuable human resources. In recent years, Fluorescence Lifetime Imaging (FLIm) has received increasing attention from the ML community. FLIm goes beyond conventional spectral imaging, providing additional lifetime information, and could lead to optical histopathology supporting real-time diagnostics. However, most current studies do not use the full potential of machine/deep learning models. As a developing image modality, FLIm data are not easily obtainable, which, coupled with an absence of standardisation, is pushing back the research to develop models which could advance automated diagnosis and help promote FLIm. In this paper, we describe recent developments that improve FLIm image quality, specifically time-domain systems, and we summarise sensing, signal-to-noise analysis and the advances in registration and low-level tracking. We review the two main applications of ML for FLIm: lifetime estimation and image analysis through classification and segmentation. We suggest a course of action to improve the quality of ML studies applied to FLIm. Our final goal is to promote FLIm and attract more ML practitioners to explore the potential of lifetime imaging.

Deep learning-assisted co-registration of full-spectral autofluorescence lifetime microscopic images with H&E-stained histology images.

Autofluorescence lifetime images reveal unique characteristics of endogenous fluorescence in biological samples. Comprehensive understanding and clinical diagnosis rely on co-registration with the gold standard, histology images, which is extremely challenging due to the difference of both images. Here, we show an unsupervised image-to-image translation network that significantly improves the success of the co-registration using a conventional optimisation-based regression network, applicable to autofluorescence lifetime images at different emission wavelengths. A preliminary blind comparison by experienced researchers shows the superiority of our method on co-registration. The results also indicate that the approach is applicable to various image formats, like fluorescence in-tensity images. With the registration, stitching outcomes illustrate the distinct differences of the spectral lifetime across an unstained tissue, enabling macro-level rapid visual identification of lung cancer and cellular-level characterisation of cell variants and common types. The approach could be effortlessly extended to lifetime images beyond this range and other staining technologies.

Multi-Scale Aggregated-Dilation Network for ex-vivo Lung Cancer Detection with Fluorescence Lifetime Imaging Endomicroscopy.

Multi-scale architectures at a granular level are characterised by separating input features into groups and applying multi-scale feature extractions to the split input features, and thus the correlations among the input features as global information are no longer retained. Moreover, they usually require more input features due to the separation, and therefore, more complexity is introduced. To retain the global information while utilising the advantages of feature-level hierarchical multi-scale architectures, we propose a multi-scale aggregated-dilation architecture (MSAD) to perform hierarchical fusion of features at a layer level, with the integration of dilated convolutions to overcome these issues. To evaluate the model, we integrate it into ResNet, and apply it to a unique dataset, containing over 60,000 fluorescence lifetime endomicroscopic images (FLIM) collected on ex-vivo lung normal/cancerous tissues from 14 patients, by a custom fibre-based FLIM system. To evaluate the performance of our proposal, we use accuracy, precision, recall, and AUC. We first compare our MSAD model with eight networks achieving a superiority over 6%. To illustrate the advantages and disadvantages of multi-scale architectures at layer and feature-level, we thoroughly compare our MSAD model with the state-of-the-art feature-level multiscale network, namely Res2Net, in terms of parameters, scales, and effective convolutions.

Deep Learning in ex-vivo Lung Cancer Discrimination using Fluorescence Lifetime Endomicroscopic Images.

Fluorescence lifetime is effective in discriminating cancerous tissue from normal tissue, but conventional discrimination methods are primarily based on statistical approaches in collaboration with prior knowledge. This paper investigates the application of deep convolutional neural networks (CNNs) for automatic differentiation of ex-vivo human lung cancer via fluorescence lifetime imaging. Around 70,000 fluorescence images from ex-vivo lung tissue of 14 patients were collected by a custom fibre-based fluorescence lifetime imaging endomicroscope. Five state-of-the-art CNN models, namely ResNet, ResNeXt, Inception, Xception, and DenseNet, were trained and tested to derive quantitative results using accuracy, precision, recall, and the area under receiver operating characteristic curve (AUC) as the metrics. The CNNs were firstly evaluated on lifetime images. Since fluorescence lifetime is independent of intensity, further experiments were conducted by stacking intensity and lifetime images together as the input to the CNNs. As the original CNNs were implemented for RGB images, two strategies were applied. One was retaining the CNNs by putting intensity and lifetime images in two different channels and leaving the remaining channel blank. The other was adapting the CNNs for two-channel input. Quantitative results demonstrate that the selected CNNs are considerably superior to conventional machine learning algorithms. Combining intensity and lifetime images introduces noticeable performance gain compared with using lifetime images alone. In addition, the CNNs with intensity-lifetime RGB image is comparable to the modified two-channel CNNs with intensity-lifetime two-channel input for accuracy and AUC, but significantly better for precision and recall.

Multi-Snapshot Imaging for Chromatographic Peak Analysis.

OBJECTIVE: Snapshot imaging has several advantages in automated gel electrophoresis compared with the finish-line method in capillary electrophoresis; this comes at the expense of resolution. A novel signal processing algorithm is proposed enabling a multisnapshot imaging (MSI) modality whose objective is to substantially improve resolution. MSI takes multiple-captures in time as macromolecules are electrophoresed. Peaks from latter snapshots have high resolution, but low signal-to-noise ratio (SNR), while earlier snapshots have low resolution, but high SNR. METHODS: Signals at different capture-times are related by a scale-in-separation, shift-in-separation, and amplitude gain. The proposed method realigns the multiple captures using least-squares and fuses them. The algorithm accounts for the partial waveforms observed as the chromatic peaks exit the sensor's field-of-view. RESULTS: MSI improves resolution by approximately [Formula: see text] on average per minute of additional electrophoresis. CONCLUSIONS: Comprehensive analysis of the resolution quantified on several data sets demonstrates the effectiveness of MSI. SIGNIFICANCE: MSI can double the resolution compared with traditional snap-shot imaging over a typical set of captures.

SERS as a tool for in vitro toxicology.

Measuring markers of stress such as pH and redox potential are important when studying toxicology in in vitro models because they are markers of oxidative stress, apoptosis and viability. While surface enhanced Raman spectroscopy is ideally suited to the measurement of redox potential and pH in live cells, the time-intensive nature and perceived difficulty in signal analysis and interpretation can be a barrier to its broad uptake by the biological community. In this paper we detail the development of signal processing and analysis algorithms that allow SERS spectra to be automatically processed so that the output of the processing is a pH or redox potential value. By automating signal processing we were able to carry out a comparative evaluation of the toxicology of silver and zinc oxide nanoparticles and correlate our findings with qPCR analysis. The combination of these two analytical techniques sheds light on the differences in toxicology between these two materials from the perspective of oxidative stress.

Analysis of echo signal from single ultrasound contrast microbubble using a reversible jump MCMC algorithm.

The understanding and exploitation of non-linear microbubble signals is an active research area that aims to advance contrast ultrasound into a high sensitivity and specificity diagnostic imaging modality. In order to discriminate the difference between echoes from tissue and contrast microbubbles, it is of particular interest to estimate the reflected signal pulse location in the time domain and its spectral content in the frequency domain. Therefore, a reversible jump Markov chain Monte Carlo (rjMCMC) algorithm, a robust statistical signal processing technique, is introduced in this paper for the analysis of echo signals from Ultrasound Contrast Agents (UCAs). This algorithm provides many advantages over conventional Fourier transform based techniques. Furthermore, our results also show that the frequency components and pulse location can be accurately estimated simultaneously, which assists in characterising the signal content and the design of transmit pulsing regimes in future work.