Deep Learning-Based Spectral Unmixing for Optoacoustic Imaging of Tissue Oxygen Saturation.

Label free imaging of oxygenation distribution in tissues is highly desired in numerous biomedical applications, but is still elusive, in particular in sub-epidermal measurements. Eigenspectra multispectral optoacoustic tomography (eMSOT) and its Bayesian-based implementation have been introduced to offer accurate label-free blood oxygen saturation (sO2) maps in tissues. The method uses the eigenspectra model of light fluence in tissue to account for the spectral changes due to the wavelength dependent attenuation of light with tissue depth. eMSOT relies on the solution of an inverse problem bounded by a number of ad hoc hand-engineered constraints. Despite the quantitative advantage offered by eMSOT, both the non-convex nature of the optimization problem and the possible sub-optimality of the constraints may lead to reduced accuracy. We present herein a neural network architecture that is able to learn how to solve the inverse problem of eMSOT by directly regressing from a set of input spectra to the desired fluence values. The architecture is composed of a combination of recurrent and convolutional layers and uses both spectral and spatial features for inference. We train an ensemble of such networks using solely simulated data and demonstrate how this approach can improve the accuracy of sO2 computation over the original eMSOT, not only in simulations but also in experimental datasets obtained from blood phantoms and small animals (mice) in vivo. The use of a deep-learning approach in optoacoustic sO2 imaging is confirmed herein for the first time on ground truth sO2 values experimentally obtained in vivo and ex vivo.

Longitudinal imaging of T cell-based immunotherapy with multi-spectral, multi-scale optoacoustic tomography.

Most imaging studies of immunotherapy have focused on tracking labeled T cell biodistribution in vivo for understanding trafficking and homing parameters and predicting therapeutic efficacy by the presence of transferred T cells at or in the tumour mass. Conversely, we investigate here a novel concept for longitudinally elucidating anatomical and pathophysiological changes of solid tumours after adoptive T cell transfer in a preclinical set up, using previously unexplored in-tandem macroscopic and mesoscopic optoacoustic (photoacoustic) imaging. We show non-invasive in vivo observations of vessel collapse during tumour rejection across entire tumours and observe for the first time longitudinal tumour rejection in a label-free manner based on optical absorption changes in the tumour mass due to cellular decline. We complement these observations with high resolution episcopic fluorescence imaging of T cell biodistribution using optimized T cell labeling based on two near-infrared dyes targeting the cell membrane and the cytoplasm. We discuss how optoacoustic macroscopy and mesoscopy offer unique contrast and immunotherapy insights, allowing label-free and longitudinal observations of tumour therapy. The results demonstrate optoacoustic imaging as an invaluable tool in understanding and optimizing T cell therapy.

Synthetic data framework to estimate the minimum detectable concentration of contrast agents for multispectral optoacoustic imaging of small animals.

The concentrations of contrast agents for optoacoustic imaging of small animals must usually be optimized through extensive pilot experiments on a case-by-case basis. The present work describes a streamlined approach for determining the minimum detectable concentration (MDC) of a contrast agent given experimental conditions and imaging system parameters. The developed Synthetic Data Framework (SDF) allows estimation of MDCs of various contrast agents under different tissue conditions without extensive animal experiments. The SDF combines simulated optoacoustic signals from exogenously administered contrast agents with in vivo experimental signals from background tissue to generate realistic synthetic multispectral optoacoustic images. In this paper, the SDF is validated with in vivo measurements and demonstrates close agreement between SDF synthetic data and experimental data in terms of both image intensity and MDCs. Use of the SDF to estimate MDCs for fluorescent dyes and nanoparticles at different tissue depths and for imaging lesions of different sizes is illustrated.

Generalized Adaptive Gaussian Markov Random Field for X-Ray Luminescence Computed Tomography.

OBJECTIVE: X-ray luminescence computed tomography (XLCT) is an emerging and promising modality, but suffers from inferior reconstructions and smoothed target shapes. This work aims to improve the image quality with new mathematical framework. METHODS: We present a Bayesian local regularization framework to tackle the ill-conditioness of XLCT. Different from traditional overall regularization strategies, the proposed method utilizes correlations of neighboring voxels to regularize the solution locally based on generalized adaptive Gaussian Markov random field (GAGMRF), and provides an adjustable parameter to facilitate the edge-preserving property. RESULTS: Numerical simulations and phantom experiments show that the GAGMRF method yields both high image quality and accurate target shapes. CONCLUSION: Compared to conventional L2 and L1 regularizations, GAGMRF provides a new and efficient model for high quality imaging based on the Bayesian framework. SIGNIFICANCE: The GAGMRF method offers a flexible regularization framework to adapt to a wide range of biomedical applications.

Coded-Aperture Compressed Sensing X-Ray Luminescence Tomography.

OBJECTIVE: The purpose of this work is to introduce and study a novel imaging geometry for X-ray luminescence computed tomography (XLCT), termed coded aperture compressive X-ray luminescence tomography (CAC-XLCT). METHODS: CAC-XLCT is studied through simulations of X-ray and diffuse light propagation and the implementation of a compressed sensing image reconstruction algorithm. RESULTS: CAC-XLCT is compared against cone beam XLCT considering simulated targets with varying complexity, and it is found to offer a remarkable enhancement in spatial resolution and image quality with only a small overhead in image acquisition time. CONCLUSIONS AND SIGNIFICANCE: XLCT has been mainly investigated so far in pencil beam and cone beam excitation geometries which suffer from either very long image acquisition time or low spatial resolution and accuracy. CAC-XLCT presents a very promising alternative, which can offer simultaneously high spatial resolution, high image quality, and fast image acquisition, appropriate for in vivo imaging.

A Bayesian Approach to Eigenspectra Optoacoustic Tomography.

The quantification of hemoglobin oxygen saturation (sO2) with multispectral optoacoustic (OA) (photoacoustic) tomography (MSOT) is a complex spectral unmixing problem, since the OA spectra of hemoglobin are modified with tissue depth due to depth (location) and wavelength dependencies of optical fluence in tissue. In a recent work, a method termed eigenspectra MSOT (eMSOT) was proposed for addressing the dependence of spectra on fluence and quantifying blood sO2 in deep tissue. While eMSOT offers enhanced sO2 quantification accuracy over conventional unmixing methods, its performance may be compromised by noise and image reconstruction artifacts. In this paper, we propose a novel Bayesian method to improve eMSOT performance in noisy environments. We introduce a spectral reliability map, i.e., a method that can estimate the level of noise superimposed onto the recorded OA spectra. Using this noise estimate, we formulate eMSOT as a Bayesian inverse problem where the inversion constraints are based on probabilistic graphical models. Results based on numerical simulations indicate that the proposed method offers improved accuracy and robustness under high noise levels due the adaptive nature of the Bayesian method.

Polarized x-ray excitation for scatter reduction in x-ray fluorescence computed tomography.

PURPOSE: X-ray fluorescence computer tomography (XFCT) is a new molecular imaging modality which uses x-ray excitation to stimulate the emission of fluorescent photons in high atomic number contrast agents. Scatter contamination is one of the main challenges in XFCT imaging which limits the molecular sensitivity. When polarized x rays are used, it is possible to reduce the scatter contamination significantly by placing detectors perpendicular to the polarization direction. This study quantifies scatter contamination for polarized and unpolarized x-ray excitation and determines the advantages of scatter reduction. METHODS: The amount of scatter in preclinical XFCT is quantified in Monte Carlo simulations. The fluorescent x rays are emitted isotropically, while scattered x rays propagate in polarization direction. The magnitude of scatter contamination is studied in XFCT simulations of a mouse phantom. In this study, the contrast agent gold is examined as an example, but a scatter reduction from polarized excitation is also expected for other elements. The scatter reduction capability is examined for different polarization intensities with a monoenergetic x-ray excitation energy of 82 keV. The study evaluates two different geometrical shapes of CZT detectors which are modeled with an energy resolution of 1 keV FWHM at an x-ray energy of 80 keV. Benefits of a detector placement perpendicular to the polarization direction are shown in iterative and analytic image reconstruction including scatter correction. The contrast to noise ratio (CNR) and the normalized mean square error (NMSE) are analyzed and compared for the reconstructed images. RESULTS: A substantial scatter reduction for common detector sizes was achieved for 100% and 80% linear polarization while lower polarization intensities provide a decreased scatter reduction. By placing the detector perpendicular to the polarization direction, a scatter reduction by factor up to 5.5 can be achieved for common detector sizes. The image reconstruction showed that for a scatter magnitude decrease by a factor of 2.4, the molecular sensitivity could almost be doubled. CONCLUSION: Scatter reduction lowers the amount of noise in the projection datasets and reconstructed images which enhance molecular sensitivity at equal dose. The results support the use of linear polarized x rays to reduce scatter in XFCT imaging.

Spectral unmixing techniques for optoacoustic imaging of tissue pathophysiology.

A key feature of optoacoustic imaging is the ability to illuminate tissue at multiple wavelengths and therefore record images with a spectral dimension. While optoacoustic images at single wavelengths reveal morphological features, in analogy to ultrasound imaging or X-ray imaging, spectral imaging concedes sensing of intrinsic chromophores and externally administered agents that can reveal physiological, cellular and subcellular functions. Nevertheless, identification of spectral moieties within images obtained at multiple wavelengths requires spectral unmixing techniques, which present a unique mathematical problem given the three-dimensional nature of the optoacoustic images. Herein we discuss progress with spectral unmixing techniques developed for multispectral optoacoustic tomography. We explain how different techniques are required for accurate sensing of intrinsic tissue chromophores such as oxygenated and deoxygenated haemoglobin versus extrinsically administered photo-absorbing agents and nanoparticles. Finally, we review recent developments that allow accurate quantification of blood oxygen saturation (sO2) by transforming and solving the sO2 estimation problem from the spatial to the spectral domain.This article is part of the themed issue 'Challenges for chemistry in molecular imaging'.

Eigenspectra optoacoustic tomography achieves quantitative blood oxygenation imaging deep in tissues.

Light propagating in tissue attains a spectrum that varies with location due to wavelength-dependent fluence attenuation, an effect that causes spectral corruption. Spectral corruption has limited the quantification accuracy of optical and optoacoustic spectroscopic methods, and impeded the goal of imaging blood oxygen saturation (sO2) deep in tissues; a critical goal for the assessment of oxygenation in physiological processes and disease. Here we describe light fluence in the spectral domain and introduce eigenspectra multispectral optoacoustic tomography (eMSOT) to account for wavelength-dependent light attenuation, and estimate blood sO2 within deep tissue. We validate eMSOT in simulations, phantoms and animal measurements and spatially resolve sO2 in muscle and tumours, validating our measurements with histology data. eMSOT shows substantial sO2 accuracy enhancement over previous optoacoustic methods, potentially serving as a valuable tool for imaging tissue pathophysiology.

Statistical Molecular Target Detection Framework for Multispectral Optoacoustic Tomography.

Statistical sub-pixel detection via the adaptive matched filter (AMF) has been shown to improve the molecular imaging sensitivity and specificity of optoacoustic (photoacoustic) imaging. Applied to multispectral optoacoustic tomography (MSOT), AMF assumes that the spatially-varying tissue spectra follow a multivariate Gaussian distribution, that the spectrum of the target molecule is precisely known and that the molecular target lies in "low probability" within the data. However, when these assumptions are violated, AMF may result in considerable performance degradation. The objective of this work is to develop a robust statistical detection framework that is appropriately suited to the characteristics of MSOT molecular imaging. Using experimental imaging data, we perform a statistical characterization of MSOT tissue images and conclude to a detector that is based on the t-distribution. More importantly, we introduce a method for estimating the covariance matrix of the background-tissue statistical distribution, which enables robust detection performance independently of the molecular target size or intensity. The performance of the statistical detection framework is assessed through simulations and experimental in vivo measurements and compared to previously used methods.

High-Throughput Sparsity-Based Inversion Scheme for Optoacoustic Tomography.

The concept of sparsity is extensively exploited in the fields of data acquisition and image processing, contributing to better signal-to-noise and spatio-temporal performance of the various imaging methods. In the field of optoacoustic tomography, the image reconstruction problem is often characterized by computationally extensive inversion of very large datasets, for instance when acquiring volumetric multispectral data with high temporal resolution. In this article we seek to accelerate accurate model-based optoacoustic inversions by identifying various sources of sparsity in the forward and inverse models as well as in the single- and multi-frame representation of the projection data. These sources of sparsity are revealed through appropriate transformations in the signal, model and image domains and are subsequently exploited for expediting image reconstruction. The sparsity-based inversion scheme was tested with experimental data, offering reconstruction speed enhancement by a factor of 40 to 700 times as compared with the conventional iterative model-based inversions while preserving similar image quality. The demonstrated results pave the way for achieving real-time performance of model-based reconstruction in multi-dimensional optoacoustic imaging.

Sparsity-based acoustic inversion in cross-sectional multiscale optoacoustic imaging.

PURPOSE: With recent advancement in hardware of optoacoustic imaging systems, highly detailed cross-sectional images may be acquired at a single laser shot, thus eliminating motion artifacts. Nonetheless, other sources of artifacts remain due to signal distortion or out-of-plane signals. The purpose of image reconstruction algorithms is to obtain the most accurate images from noisy, distorted projection data. METHODS: In this paper, the authors use the model-based approach for acoustic inversion, combined with a sparsity-based inversion procedure. Specifically, a cost function is used that includes the L1 norm of the image in sparse representation and a total variation (TV) term. The optimization problem is solved by a numerically efficient implementation of a nonlinear gradient descent algorithm. TV-L1 model-based inversion is tested in the cross section geometry for numerically generated data as well as for in vivo experimental data from an adult mouse. RESULTS: In all cases, model-based TV-L1 inversion showed a better performance over the conventional Tikhonov regularization, TV inversion, and L1 inversion. In the numerical examples, the images reconstructed with TV-L1 inversion were quantitatively more similar to the originating images. In the experimental examples, TV-L1 inversion yielded sharper images and weaker streak artifact. CONCLUSIONS: The results herein show that TV-L1 inversion is capable of improving the quality of highly detailed, multiscale optoacoustic images obtained in vivo using cross-sectional imaging systems. As a result of its high fidelity, model-based TV-L1 inversion may be considered as the new standard for image reconstruction in cross-sectional imaging.

Effects of multispectral excitation on the sensitivity of molecular optoacoustic imaging.

Molecular optoacoustic (photoacoustic) imaging typically relies on the spectral identification of absorption signatures from molecules of interest. To achieve this, two or more excitation wavelengths are employed to sequentially illuminate tissue. Due to depth-related spectral dependencies and detection related effects, the multispectral optoacoustic tomography (MSOT) spectral unmixing problem presents a complex non-linear inversion operation. So far, different studies have showcased the spectral capacity of optoacoustic imaging, without however relating the performance achieved to the number of wavelengths employed. Overall, the dependence of the sensitivity and accuracy of optoacoustic imaging as a function of the number of illumination wavelengths has not been so far comprehensively studied. In this paper we study the impact of the number of excitation wavelengths employed on the sensitivity and accuracy achieved by molecular optoacoustic tomography. We present a quantitative analysis, based on synthetic MSOT datasets and observe a trend of sensitivity increase for up to 20 wavelengths. Importantly we quantify this relation and demonstrate an up to an order of magnitude sensitivity increase of multi-wavelength illumination vs. single or dual wavelength optoacoustic imaging. Examples from experimental animal studies are finally utilized to support the findings. In vivo MSOT imaging of a mouse brain bearing a tumor that is expressing a near-infrared fluorescent protein. (a) Monochromatic optoacoustic imaging at the peak excitation wavelength of the fluorescent protein. (b) Overlay of the detected bio-distribution of the protein (red pseudocolor) on the monochromatic optoacoustic image. (c) Ex vivo validation by means of cryoslicing fluorescence imaging.

Spatiospectral denoising framework for multispectral optoacoustic imaging based on sparse signal representation.

PURPOSE: One of the major challenges in dynamic multispectral optoacoustic imaging is its relatively low signal-to-noise ratio which often requires repetitive signal acquisition and averaging, thus limiting imaging rate. The development of denoising methods which prevent the need for signal averaging in time presents an important goal for advancing the dynamic capabilities of the technology. METHODS: In this paper, a denoising method is developed for multispectral optoacoustic imaging which exploits the implicit sparsity of multispectral optoacoustic signals both in space and in spectrum. Noise suppression is achieved by applying thresholding on a combined wavelet-Karhunen-Loève representation in which multispectral optoacoustic signals appear particularly sparse. The method is based on inherent characteristics of multispectral optoacoustic signals of tissues, offering promise for general application in different incarnations of multispectral optoacoustic systems. RESULTS: The performance of the proposed method is demonstrated on mouse images acquired in vivo for two common additive noise sources: time-varying parasitic signals and white noise. In both cases, the proposed method shows considerable improvement in image quality in comparison to previously published denoising strategies that do not consider multispectral information. CONCLUSIONS: The suggested denoising methodology can achieve noise suppression with minimal signal loss and considerably outperforms previously proposed denoising strategies, holding promise for advancing the dynamic capabilities of multispectral optoacoustic imaging while retaining image quality.

Immune cell imaging using multi-spectral optoacoustic tomography.

Multispectral optoacoustic tomography (MSOT) offers the potential to image in high-resolution cells tagged with optical labels. In contrast to single wavelength imaging, multispectral excitation and spectral unmixing can differentiate labeled moieties over tissue absorption in the absence of background measurements. This feature can enable longitudinal cellular biology studies well beyond the depths reached by optical microscopy. However, the relation between spectrally resolved fluorescently labeled cells and optoacoustic detection has not been systematically investigated. Herein, we measured titrations of fluorescently labeled cells and establish the optoacoustic signal generated by these cells as a function of cell number and across different cell types. We then assess the MSOT sensitivity to resolve cells implanted in animals.

FMT-PCCT: hybrid fluorescence molecular tomography-x-ray phase-contrast CT imaging of mouse models.

The implementation of hybrid fluorescence molecular tomography (FMT) and X-ray computed tomography (CT) has been shown to be a necessary development, not only for combining anatomical with functional and molecular contrast, but also for generating optical images of high accuracy. FMT affords highly sensitive 3-D imaging of fluorescence bio-distribution, but in stand-alone form it offers images of low resolution. It was shown that FMT accuracy significantly improves by considering anatomical priors from CT. Conversely, CT generally suffers from low soft tissue contrast. Therefore utilization of CT data as prior information in FMT inversion is challenging when different internal organs are not clearly differentiated. Instead, we combined herein FMT with emerging X-ray phase-contrast CT (PCCT). PCCT relies on phase shift differences in tissue to achieve soft tissue contrast superior to conventional CT. We demonstrate for the first time FMT-PCCT imaging of different animal models, where FMT and PCCT scans were performed in vivo and ex vivo, respectively. The results show that FMT-PCCT expands the potential of FMT in imaging lesions with otherwise low or no CT contrast, while retaining the cost benefits of CT and simplicity of hybrid device realizations. The results point to the most accurate FMT performance to date.

Unmixing Molecular Agents From Absorbing Tissue in Multispectral Optoacoustic Tomography.

Detection of intrinsic or extrinsically administered chromophores and photo-absorbing nanoparticles has been achieved by multi-spectral optoacoustic tomography (MSOT). The detection sensitivity of MSOT depends not only on the signal to noise ratio considerations, as in conventional optoacoustic (photoacoustic) tomography implementations, but also on the ability to resolve the molecular targets of interest from the absorbing tissue background by means of spectral unmixing or sub-pixel detection methods. However, it is not known which unmixing methods are optimally suited for the characteristics of multispectral optoacoustic images. In this work we investigated the performance of different sub-pixel detection methods, typically used in remote sensing hyperspectral imaging, within the context of MSOT. A quantitative comparison of the different algorithmic approaches was carried out in an effort to identify methods that operate optimally under the particulars of molecular imaging applications. We find that statistical sub-pixel detection methods can demonstrate a unique detection performance with up to five times enhanced sensitivity as compared to linear unmixing approximations, under the condition that the optical agent of interest is sparsely present within the tissue volume, as common when using targeted agents and reporter genes.