Highly robust reconstruction framework for three-dimensional optical imaging based on physical model constrained neural networks.

Objective. The reconstruction of three-dimensional optical imaging that can quantitatively acquire the target distribution from surface measurements is a serious ill-posed problem. Traditional regularization-based reconstruction can solve such ill-posed problem to a certain extent, but its accuracy is highly dependent ona priorinformation, resulting in a less stable and adaptable method. Data-driven deep learning-based reconstruction avoids the errors of light propagation models and the reliance on experience and a prior by learning the mapping relationship between the surface light distribution and the target directly from the dataset. However, the acquisition of the training dataset and the training of the network itself are time consuming, and the high dependence of the network performance on the training dataset results in a low generalization ability. The objective of this work is to develop a highly robust reconstruction framework to solve the existing problems.Approach. This paper proposes a physical model constrained neural networks-based reconstruction framework. In the framework, the neural networks are to generate a target distribution from surface measurements, while the physical model is used to calculate the surface light distribution based on this target distribution. The mean square error between the calculated surface light distribution and the surface measurements is then used as a loss function to optimize the neural network. To further reduce the dependence ona prioriinformation, a movable region is randomly selected and then traverses the entire solution interval. We reconstruct the target distribution in this movable region and the results are used as the basis for its next movement.Main Results. The performance of the proposed framework is evaluated with a series of simulations andin vivoexperiment, including accuracy robustness of different target distributions, noise immunity, depth robustness, and spatial resolution. The results collectively demonstrate that the framework can reconstruct targets with a high accuracy, stability and versatility.Significance. The proposed framework has high accuracy and robustness, as well as good generalizability. Compared with traditional regularization-based reconstruction methods, it eliminates the need to manually delineate feasible regions and adjust regularization parameters. Compared with emerging deep learning assisted methods, it does not require any training dataset, thus saving a lot of time and resources and solving the problem of poor generalization and robustness of deep learning methods. Thus, the framework opens up a new perspective for the reconstruction of three-dimension optical imaging.

Classification of unlabeled cells using lensless digital holographic images and deep neural networks.

BACKGROUND: Image-based cell analytic methodologies offer a relatively simple and economical way to analyze and understand cell heterogeneities and developments. Owing to developments in high-resolution image sensors and high-performance computation processors, the emerging lensless digital holography technique enables a simple and cost-effective approach to obtain label-free cell images with a large field of view and microscopic spatial resolution. METHODS: The holograms of three types of cells, including MCF-10A, EC-109, and MDA-MB-231 cells, were recorded using a lensless digital holography system composed of a laser diode, a sample stage, an image sensor, and a laptop computer. The amplitude images were reconstructed using the angular spectrum method, and the sample to sensor distance was determined using the autofocusing criteria based on the sparsity of image edges and corner points. Four convolutional neural networks (CNNs) were used to classify the cell types based on the recovered holographic images. RESULTS: Classification of two cell types and three cell types achieved an accuracy of higher than 91% by all the networks used. The ResNet and the DenseNet models had similar classification accuracy of 95% or greater, outperforming the GoogLeNet and the CNN-5 models. CONCLUSIONS: These experiments demonstrated that the CNNs were effective at classifying two or three types of tumor cells. The lensless holography combined with machine learning holds great promise in the application of stainless cell imaging and classification, such as in cancer diagnosis and cancer biology research, where distinguishing normal cells from cancer cells and recognizing different cancer cell types will be greatly beneficial.

Kinetic modeling and analysis of dynamic bioluminescence imaging of substrates administered by intraperitoneal injection.

BACKGROUND: Bioluminescence imaging (BLI) has been found to have diverse applications in the life sciences and medical research due to its ease of use and high sensitivity. From kinetics analysis, dynamic imaging studies have significant advantages for diagnosis when compared to traditional static imaging studies. This work focuses on modeling and quantitatively analyzing the dynamic data produced from the intraperitoneal (IP) injection of D-luciferin in longitudinal BLI, aiming to provide a powerful tool for monitoring the growth of tumors. METHODS: We constructed a three-compartment pharmacokinetic (PK) model and employed the standard Michaelis-Menten (M-M) kinetics to investigate the dynamic BLI data produced from the IP injection of D-luciferin. The 3 compartments were the plasma compartment, the non-specific compartment, and the specific compartment. The validity of this PK model was tested by the dynamic BLI data of MKN28M-luc xenograft mice, along with the published longitudinal dynamic BLI data of B16F10-luc xenograft mice. RESULTS: The R-squares between the simulated lines and the measurement were 1 and 0.99, respectively, for the mice data and the published data. In addition, the 2 kinetic macroparameters obtained reflected the rate of tumor growth in vivo. In particular, the values of macroparameters A showed a significant dependence on tumor surface area. CONCLUSIONS: The proposed PK model may be an effective tool for use in drug development programs and for monitoring the response of tumors to treatment.

Removal of random-valued impulse noise from Cerenkov luminescence images.

Cerenkov luminescence imaging(CLI) is an emerging molecular imaging technology able to optically visualize radioactive decay signals from medical isotopes and has found wide application in tumor diagnose, cancer therapy, drug development, intraoperative guidance, and so on. When Cerenkov luminescence data are collected, the high-energy particles from the radioactive nucleus will be detected by the sensitive CCD camera and lead to impulse noise. To suppress the impulse noise and improve the contrast of the useful signal to the background, the detection-based fuzzy switching median filtering framework is proposed in this paper. Several experiments were conducted respectively to investigate the statistical feature of the noise and to evaluate the performance of the proposed noise removal framework. The results show that the signal-to-noise ratio is improved after noise elimination. The proposed filtering framework outperforms the classical median filter in terms of root mean squared error and the structural similarity index. It also preserves the maximum value and the mean value in the regions of interest better than the median filter does. In addition, compared with the FLICMCDD algorithm, the proposed method works much faster while getting similar results. Graphical abstract.

Intravenous Administration-Oriented Pharmacokinetic Model for Dynamic Bioluminescence Imaging.

OBJECTIVE: In vivo bioluminescence imaging (BLI) is a promising tool for monitoring the growth and metastasis of tumors. However, quantitative BLI research based on intravenous (IV) injection is limited, which greatly restricts its further application. To address this problem, we designed a pharmacokinetic (PK) model which is suitable for applying on IV administration of small amounts of D-Luciferin. METHODS: After three weeks of postimplantation, mkn28-luc xenografted mice were subjected to 40-min dynamic BLI immediately following D-Luciferin intravenous injection on days 1, 3, 5, 7, and 9. Furthermore, the PK model was applied on dynamic BLI data to obtain the sum of kinetic rate constants (SKRC). RESULTS: Results showed that the SKRC values decreased rapidly with the growth of the tumor. There was a statistical difference between the SKRC values measured at different time points, while the time point of luminous intensity peak was unaffected by the growth of the tumor. CONCLUSION: In short, our results imply that dynamic BLI combined with our PK model can predict tumor growth earlier and with higher sensitivity compared to the conventional method, which is crucial for improving drug evaluation efficacy. In addition, the dynamic BLI may provide a valuable reference for the noninvasive acquiring arterial input function, which may also provide a new application prospect for hybrid PET-optical imaging.

Reconstruction for Limited-Projection Fluorescence Molecular Tomography Based on a Double-Mesh Strategy.

Limited-projection fluorescence molecular tomography (FMT) has short data acquisition time that allows fast resolving of the three-dimensional visualization of fluorophore within small animal in vivo. However, limited-projection FMT reconstruction suffers from severe ill-posedness because only limited projections are used for reconstruction. To alleviate the ill-posedness, a feasible region extraction strategy based on a double mesh is presented for limited-projection FMT. First, an initial result is rapidly recovered using a coarse discretization mesh. Then, the reconstructed fluorophore area in the initial result is selected as a feasible region to guide the reconstruction using a fine discretization mesh. Simulation experiments on a digital mouse and small animal experiment in vivo are performed to validate the proposed strategy. It demonstrates that the presented strategy provides a good distribution of fluorophore with limited projections of fluorescence measurements. Hence, it is suitable for reconstruction of limited-projection FMT.

Temporal Unmixing of Dynamic Fluorescent Images by Blind Source Separation Method with a Convex Framework.

By recording a time series of tomographic images, dynamic fluorescence molecular tomography (FMT) allows exploring perfusion, biodistribution, and pharmacokinetics of labeled substances in vivo. Usually, dynamic tomographic images are first reconstructed frame by frame, and then unmixing based on principle component analysis (PCA) or independent component analysis (ICA) is performed to detect and visualize functional structures with different kinetic patterns. PCA and ICA assume sources are statistically uncorrelated or independent and don't perform well when correlated sources are present. In this paper, we deduce the relationship between the measured imaging data and the kinetic patterns and present a temporal unmixing approach, which is based on nonnegative blind source separation (BSS) method with a convex analysis framework to separate the measured data. The presented method requires no assumption on source independence or zero correlations. Several numerical simulations and phantom experiments are conducted to investigate the performance of the proposed temporal unmixing method. The results indicate that it is feasible to unmix the measured data before the tomographic reconstruction and the BSS based method provides better unmixing quality compared with PCA and ICA.

Incorporating MRI structural information into bioluminescence tomography: system, heterogeneous reconstruction and in vivo quantification.

Combining two or more imaging modalities to provide complementary information has become commonplace in clinical practice and in preclinical and basic biomedical research. By incorporating the structural information provided by computed tomography (CT) or magnetic resonance imaging (MRI), the ill poseness nature of bioluminescence tomography (BLT) can be reduced significantly, thus improve the accuracies of reconstruction and in vivo quantification. In this paper, we present a small animal imaging system combining multi-view and multi-spectral BLT with MRI. The independent MRI-compatible optical device is placed at the end of the clinical MRI scanner. The small animal is transferred between the light tight chamber of the optical device and the animal coil of MRI via a guide rail during the experiment. After the optical imaging and MRI scanning procedures are finished, the optical images are mapped onto the MRI surface by interactive registration between boundary of optical images and silhouette of MRI. Then, incorporating the MRI structural information, a heterogeneous reconstruction algorithm based on finite element method (FEM) with L 1 normalization is used to reconstruct the position, power and region of the light source. In order to validate the feasibility of the system, we conducted experiments of nude mice model implanted with artificial light source and quantitative analysis of tumor inoculation model with MDA-231-GFP-luc. Preliminary results suggest the feasibility and effectiveness of the prototype system.

Reconstruction algorithms based on l1-norm and l2-norm for two imaging models of fluorescence molecular tomography: a comparative study.

Fluorescence molecular tomography (FMT) is an important imaging technique of optical imaging. The major challenge of the reconstruction method for FMT is the ill-posed and underdetermined nature of the inverse problem. In past years, various regularization methods have been employed for fluorescence target reconstruction. A comparative study between the reconstruction algorithms based on l1-norm and l2-norm for two imaging models of FMT is presented. The first imaging model is adopted by most researchers, where the fluorescent target is of small size to mimic small tissue with fluorescent substance, as demonstrated by the early detection of a tumor. The second model is the reconstruction of distribution of the fluorescent substance in organs, which is essential to drug pharmacokinetics. Apart from numerical experiments, in vivo experiments were conducted on a dual-modality FMT/micro-computed tomography imaging system. The experimental results indicated that l1-norm regularization is more suitable for reconstructing the small fluorescent target, while l2-norm regularization performs better for the reconstruction of the distribution of fluorescent substance.

Cone beam x-ray luminescence computed tomography: a feasibility study.

PURPOSE: The appearance of x-ray luminescence computed tomography (XLCT) opens new possibilities to perform molecular imaging by x ray. In the previous XLCT system, the sample was irradiated by a sequence of narrow x-ray beams and the x-ray luminescence was measured by a highly sensitive charge coupled device (CCD) camera. This resulted in a relatively long sampling time and relatively low utilization of the x-ray beam. In this paper, a novel cone beam x-ray luminescence computed tomography strategy is proposed, which can fully utilize the x-ray dose and shorten the scanning time. The imaging model and reconstruction method are described. The validity of the imaging strategy has been studied in this paper. METHODS: In the cone beam XLCT system, the cone beam x ray was adopted to illuminate the sample and a highly sensitive CCD camera was utilized to acquire luminescent photons emitted from the sample. Photons scattering in biological tissues makes it an ill-posed problem to reconstruct the 3D distribution of the x-ray luminescent sample in the cone beam XLCT. In order to overcome this issue, the authors used the diffusion approximation model to describe the photon propagation in tissues, and employed the sparse regularization method for reconstruction. An incomplete variables truncated conjugate gradient method and permissible region strategy were used for reconstruction. Meanwhile, traditional x-ray CT imaging could also be performed in this system. The x-ray attenuation effect has been considered in their imaging model, which is helpful in improving the reconstruction accuracy. RESULTS: First, simulation experiments with cylinder phantoms were carried out to illustrate the validity of the proposed compensated method. The experimental results showed that the location error of the compensated algorithm was smaller than that of the uncompensated method. The permissible region strategy was applied and reduced the reconstruction error to less than 2 mm. The robustness and stability were then evaluated from different view numbers, different regularization parameters, different measurement noise levels, and optical parameters mismatch. The reconstruction results showed that the settings had a small effect on the reconstruction. The nonhomogeneous phantom simulation was also carried out to simulate a more complex experimental situation and evaluated their proposed method. Second, the physical cylinder phantom experiments further showed similar results in their prototype XLCT system. With the discussion of the above experiments, it was shown that the proposed method is feasible to the general case and actual experiments. CONCLUSIONS: Utilizing numerical simulation and physical experiments, the authors demonstrated the validity of the new cone beam XLCT method. Furthermore, compared with the previous narrow beam XLCT, the cone beam XLCT could more fully utilize the x-ray dose and the scanning time would be shortened greatly. The study of both simulation experiments and physical phantom experiments indicated that the proposed method was feasible to the general case and actual experiments.

Extended finite element method with simplified spherical harmonics approximation for the forward model of optical molecular imaging.

An extended finite element method (XFEM) for the forward model of 3D optical molecular imaging is developed with simplified spherical harmonics approximation (SP(N)). In XFEM scheme of SP(N) equations, the signed distance function is employed to accurately represent the internal tissue boundary, and then it is used to construct the enriched basis function of the finite element scheme. Therefore, the finite element calculation can be carried out without the time-consuming internal boundary mesh generation. Moreover, the required overly fine mesh conforming to the complex tissue boundary which leads to excess time cost can be avoided. XFEM conveniences its application to tissues with complex internal structure and improves the computational efficiency. Phantom and digital mouse experiments were carried out to validate the efficiency of the proposed method. Compared with standard finite element method and classical Monte Carlo (MC) method, the validation results show the merits and potential of the XFEM for optical imaging.

Mapping of bioluminescent images onto CT volume surface for dual-modality BLT and CT imaging.

We present a method for mapping the two-dimensional (2D) bioluminescent images (BLIs) onto a three-dimensional (3D) body surface derived from the computed tomography (CT) volume data. This mapping includes two closely-related steps, the spatial registration of the 2D BLIs into the coordinate system of the CT volume data and the light flux recovering on the body surface from BLIs. By labeling markers on the body surface, we proposed an effective registration method to achieve the spatial position alignment. The subsequent light flux recovering is presented based on the inverse process of the free-space light transport model and taking the influence of the camera lens diaphragm into account. Incorporating the mapping procedure into the bioluminescence tomography (BLT) reconstruction, we developed a dual-modality BLT and CT imaging framework to provide both optical and anatomical information. The accuracy of the registration and the light flux recovering methods were evaluated via physical phantom experiments. The registration method was found to have a mean error of 0.41 mm and 0.35 mm in horizontal and vertical direction, and the accuracy of the light flux recovering method was below 5%. Furthermore, we evaluated the performance of the dual-modality BLT/CT imaging framework using a mouse phantom. Preliminary results revealed the potential and feasibility of the dual-modality imaging framework.

Multilevel, hybrid regularization method for reconstruction of fluorescent molecular tomography.

In this paper, a multilevel, hybrid regularization method is presented for fluorescent molecular tomography (FMT) based on the hp-finite element method (hp-FEM) with a continuous wave. The hybrid regularization method combines sparsity regularization and Landweber iterative regularization to improve the stability of the solution of the ill-posed inverse problem. In the first coarse mesh level, considering the fact that the fluorescent probes are sparsely distributed in the entire reconstruction region in most FMT applications, the sparse regularization method is employed to take full advantage of this sparsity. In the subsequent refined mesh levels, since the reconstruction region is reduced and the initial value of the unknown parameters is provided from the previous mesh, these mesh levels seem to be different from the first level. As a result, the Landweber iterative regularization method is applied for reconstruction. Simulation experiments on a 3D digital mouse atlas and physical experiments on a phantom are conducted to evaluate the performance of our method. The reconstructed results show the potential and feasibility of the proposed approach.

Sparse regularization-based reconstruction for bioluminescence tomography using a multilevel adaptive finite element method.

Bioluminescence tomography (BLT) is a promising tool for studying physiological and pathological processes at cellular and molecular levels. In most clinical or preclinical practices, fine discretization is needed for recovering sources with acceptable resolution when solving BLT with finite element method (FEM). Nevertheless, uniformly fine meshes would cause large dataset and overfine meshes might aggravate the ill-posedness of BLT. Additionally, accurately quantitative information of density and power has not been simultaneously obtained so far. In this paper, we present a novel multilevel sparse reconstruction method based on adaptive FEM framework. In this method, permissible source region gradually reduces with adaptive local mesh refinement. By using sparse reconstruction with l(1) regularization on multilevel adaptive meshes, simultaneous recovery of density and power as well as accurate source location can be achieved. Experimental results for heterogeneous phantom and mouse atlas model demonstrate its effectiveness and potentiality in the application of quantitative BLT.

Sparse reconstruction for quantitative bioluminescence tomography based on the incomplete variables truncated conjugate gradient method.

In this paper, we present an incomplete variables truncated conjugate gradient (IVTCG) method for bioluminescence tomography (BLT). Considering the sparse characteristic of the light source and insufficient surface measurement in the BLT scenarios, we combine a sparseness-inducing (ℓ1 norm) regularization term with a quadratic error term in the IVTCG-based framework for solving the inverse problem. By limiting the number of variables updated at each iterative and combining a variable splitting strategy to find the search direction more efficiently, it obtains fast and stable source reconstruction, even without a priori information of the permissible source region and multispectral measurements. Numerical experiments on a mouse atlas validate the effectiveness of the method. In vivo mouse experimental results further indicate its potential for a practical BLT system.

3D reconstruction of light flux distribution on arbitrary surfaces from 2D multi-photographic images.

Optical tomography can demonstrate accurate three-dimensional (3D) imaging that recovers the 3D spatial distribution and concentration of the luminescent probes in biological tissues, compared with planar imaging. However, the tomographic approach is extremely difficult to implement due to the complexity in the reconstruction of 3D surface flux distribution from multi-view two dimensional (2D) measurements on the subject surface. To handle this problem, a novel and effective method is proposed in this paper to determine the surface flux distribution from multi-view 2D photographic images acquired by a set of non-contact detectors. The method is validated with comparison experiments involving both regular and irregular surfaces. Reconstruction of the inside probes based on the reconstructed surface flux distribution further demonstrates the potential of the proposed method in its application in optical tomography.

Generalized free-space diffuse photon transport model based on the influence analysis of a camera lens diaphragm.

The camera lens diaphragm is an important component in a noncontact optical imaging system and has a crucial influence on the images registered on the CCD camera. However, this influence has not been taken into account in the existing free-space photon transport models. To model the photon transport process more accurately, a generalized free-space photon transport model is proposed. It combines Lambertian source theory with analysis of the influence of the camera lens diaphragm to simulate photon transport process in free space. In addition, the radiance theorem is also adopted to establish the energy relationship between the virtual detector and the CCD camera. The accuracy and feasibility of the proposed model is validated with a Monte-Carlo-based free-space photon transport model and physical phantom experiment. A comparison study with our previous hybrid radiosity-radiance theorem based model demonstrates the improvement performance and potential of the proposed model for simulating photon transport process in free space.

Qualitative simulation of photon transport in free space based on monte carlo method and its parallel implementation.

During the past decade, Monte Carlo method has obtained wide applications in optical imaging to simulate photon transport process inside tissues. However, this method has not been effectively extended to the simulation of free-space photon transport at present. In this paper, a uniform framework for noncontact optical imaging is proposed based on Monte Carlo method, which consists of the simulation of photon transport both in tissues and in free space. Specifically, the simplification theory of lens system is utilized to model the camera lens equipped in the optical imaging system, and Monte Carlo method is employed to describe the energy transformation from the tissue surface to the CCD camera. Also, the focusing effect of camera lens is considered to establish the relationship of corresponding points between tissue surface and CCD camera. Furthermore, a parallel version of the framework is realized, making the simulation much more convenient and effective. The feasibility of the uniform framework and the effectiveness of the parallel version are demonstrated with a cylindrical phantom based on real experimental results.

In vivo quantitative bioluminescence tomography using heterogeneous and homogeneous mouse models.

Bioluminescence tomography (BLT) is a new optical molecular imaging modality, which can monitor both physiological and pathological processes by using bioluminescent light-emitting probes in small living animal. Especially, this technology possesses great potential in drug development, early detection, and therapy monitoring in preclinical settings. In the present study, we developed a dual modality BLT prototype system with Micro-computed tomography (MicroCT) registration approach, and improved the quantitative reconstruction algorithm based on adaptive hp finite element method (hp-FEM). Detailed comparisons of source reconstruction between the heterogeneous and homogeneous mouse models were performed. The models include mice with implanted luminescence source and tumor-bearing mice with firefly luciferase report gene. Our data suggest that the reconstruction based on heterogeneous mouse model is more accurate in localization and quantification than the homogeneous mouse model with appropriate optical parameters and that BLT allows super-early tumor detection in vivo based on tomographic reconstruction of heterogeneous mouse model signal.

In vivo quantitative reconstruction studies of bioluminescence tomography: effects of peak-wavelength shift and model deviation.

Bioluminescence tomography is a novel optical molecular imaging technology. The corresponding system, theory, and algorithmic frames have been set up. In the present study, we concentrated on the analysis of quantitative reconstruction deviation from peak-wavelength shift of luminescent source and the deviation of heterogeneous mouse model. The findings suggest that the reconstruction results are significantly affected by the peak-wavelength shift and deviation of anatomical structure animal models. Furthermore, the model deviations exhibit much more influence than the wavelength shift on the reconstruction results.