Assessing cell viability with dynamic optical coherence microscopy.

Assessing cell viability is important in many fields of research. Current optical methods to assess cell viability typically involve fluorescent dyes, which are often less reliable and have poor permeability in primary tissues. Dynamic optical coherence microscopy (dOCM) is an emerging tool that provides label-free contrast reflecting changes in cellular metabolism. In this work, we compare the live contrast obtained from dOCM to viability dyes, and for the first time to our knowledge, demonstrate that dOCM can distinguish live cells from dead cells in murine syngeneic tumors. We further demonstrate a strong correlation between dOCM live contrast and optical redox ratio by metabolic imaging in primary mouse liver tissue. The dOCM technique opens a new avenue to apply label-free imaging to assess the effects of immuno-oncology agents, targeted therapies, chemotherapy, and cell therapies using live tumor tissues.

Fluorescence Lifetime Imaging for Quantification of Targeted Drug Delivery in Varying Tumor Microenvironments.

Rationale: Trastuzumab (TZM) is a monoclonal antibody that targets the human epidermal growth factor receptor (HER2) and is clinically used for the treatment of HER2-positive breast tumors. However, the tumor microenvironment can limit the access of TZM to the HER2 targets across the whole tumor and thereby compromise TZM's therapeutic efficacy. An imaging methodology that can non-invasively quantify the binding of TZM-HER2, which is required for therapeutic action, and distribution within tumors with varying tumor microenvironments is much needed. Methods: We performed near-infrared (NIR) fluorescence lifetime (FLI) Forster Resonance Energy Transfer (FRET) to measure TZM-HER2 binding, using in vitro microscopy and in vivo widefield macroscopy, in HER2 overexpressing breast and ovarian cancer cells and tumor xenografts, respectively. Immunohistochemistry was used to validate in vivo imaging results. Results: NIR FLI FRET in vitro microscopy data show variations in intracellular distribution of bound TZM in HER2-positive breast AU565 and AU565 tumor-passaged XTM cell lines in comparison to SKOV-3 ovarian cancer cells. Macroscopy FLI (MFLI) FRET in vivo imaging data show that SKOV-3 tumors display reduced TZM binding compared to AU565 and XTM tumors, as validated by ex vivo immunohistochemistry. Moreover, AU565/XTM and SKOV-3 tumor xenografts display different amounts and distributions of TME components, such as collagen and vascularity. Therefore, these results suggest that SKOV-3 tumors are refractory to TZM delivery due to their disrupted vasculature and increased collagen content. Conclusion: Our study demonstrates that FLI is a powerful analytical tool to monitor the delivery of antibody drug tumor both in cell cultures and in vivo live systems. Especially, MFLI FRET is a unique imaging modality that can directly quantify target engagement with potential to elucidate the role of the TME in drug delivery efficacy in intact live tumor xenografts.

In vivo quantitative FRET small animal imaging: Intensity versus lifetime-based FRET.

Förster resonance energy transfer (FRET) microscopy is used in numerous biophysical and biomedical applications to monitor inter- and intramolecular interactions and conformational changes in the 2-10 nm range. FRET is currently being extended to in vivo optical imaging, its main application being in quantifying drug-target engagement or drug release in animal models of cancer using organic dye or nanoparticle-labeled probes. Herein, we compared FRET quantification using intensity-based FRET (sensitized emission FRET analysis with the three-cube approach using an IVIS imager) and macroscopic fluorescence lifetime (MFLI) FRET using a custom system using a time-gated-intensified charge-coupled device, for small animal optical in vivo imaging. The analytical expressions and experimental protocols required to quantify the product fDE of the FRET efficiency E and the fraction of donor molecules involved in FRET, fD, are described in detail for both methodologies. Dynamic in vivo FRET quantification of transferrin receptor-transferrin binding was acquired in live intact nude mice upon intravenous injection of a near-infrared-labeled transferrin FRET pair and benchmarked against in vitro FRET using hybridized oligonucleotides. Even though both in vivo imaging techniques provided similar dynamic trends for receptor-ligand engagement, we demonstrate that MFLI-FRET has significant advantages. Whereas the sensitized emission FRET approach using the IVIS imager required nine measurements (six of which are used for calibration) acquired from three mice, MFLI-FRET needed only one measurement collected from a single mouse, although a control mouse might be needed in a more general situation. Based on our study, MFLI therefore represents the method of choice for longitudinal preclinical FRET studies such as that of targeted drug delivery in intact, live mice.

Deep learning-based fusion of widefield diffuse optical tomography and micro-CT structural priors for accurate 3D reconstructions.

Widefield illumination and detection strategies leveraging structured light have enabled fast and robust probing of tissue properties over large surface areas and volumes. However, when applied to diffuse optical tomography (DOT) applications, they still require a time-consuming and expert-centric solving of an ill-posed inverse problem. Deep learning (DL) models have been recently proposed to facilitate this challenging step. Herein, we expand on a previously reported deep neural network (DNN) -based architecture (modified AUTOMAP - ModAM) for accurate and fast reconstructions of the absorption coefficient in 3D DOT based on a structured light illumination and detection scheme. Furthermore, we evaluate the improved performances when incorporating a micro-CT structural prior in the DNN-based workflow, named Z-AUTOMAP. This Z-AUTOMAP significantly improves the widefield imaging process's spatial resolution, especially in the transverse direction. The reported DL-based strategies are validated both in silico and in experimental phantom studies using spectral micro-CT priors. Overall, this is the first successful demonstration of micro-CT and DOT fusion using deep learning, greatly enhancing the prospect of rapid data-integration strategies, often demanded in challenging pre-clinical scenarios.

Roadmap on Deep Learning for Microscopy.

Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences.

in vivo quantitative FRET small animal imaging: intensity versus lifetime-based FRET.

Förster Resonance Energy Transfer (FRET) microscopy is used in numerous biophysical and biomedical applications to monitor inter- and intramolecular interactions and conformational changes in the 2-10 nm range. FRET is currently being extended to in vivo optical imaging, its main application being in quantifying drug-target engagement or drug release in animal models of cancer using organic dye or nanoparticle-labeled probes. Herein, we compared FRET quantification using intensity-based FRET (sensitized emission FRET analysis with the 3-cube approach using an IVIS imager) and macroscopic fluorescence lifetime (MFLI) FRET using a custom system using a time-gated ICCD, for small animal optical in vivo imaging. The analytical expressions and experimental protocols required to quantify the product f D E of the FRET efficiency E and the fraction of donor molecules involved in FRET, f D , are described in detail for both methodologies. Dynamic in vivo FRET quantification of transferrin receptor-transferrin binding was acquired in live intact nude mice upon intravenous injection of near infrared-labeled transferrin FRET pair and benchmarked against in vitro FRET using hybridized oligonucleotides. Even though both in vivo imaging techniques provided similar dynamic trends for receptor-ligand engagement, we demonstrate that MFLI FRET has significant advantages. Whereas the sensitized emission FRET approach using the IVIS imager required 9 measurements (6 of which are used for calibration) acquired from three mice, MFLI FRET needed only one measurement collected from a single mouse, although a control mouse might be needed in a more general situation. Based on our study, MFLI therefore represents the method of choice for longitudinal preclinical FRET studies such as that of targeted drug delivery in intact, live mice.

Computational macroscopic lifetime imaging and concentration unmixing of autofluorescence.

Single-pixel computational imaging can leverage highly sensitive detectors that concurrently acquire data across spectral and temporal domains. For molecular imaging, such methodology enables to collect rich intensity and lifetime multiplexed fluorescence datasets. Herein we report on the application of a single-pixel structured light-based platform for macroscopic imaging of tissue autofluorescence. The super-continuum visible excitation and hyperspectral single-pixel detection allow for parallel characterization of autofluorescence intensity and lifetime. Furthermore, we exploit a deep learning based data processing pipeline, to perform autofluorescence unmixing while yielding the autofluorophores' concentrations. The full scheme (setup and processing) is validated in silico and in vitro with clinically relevant autofluorophores flavin adenine dinucleotide, riboflavin, and protoporphyrin. The presented results demonstrate the potential of the methodology for macroscopically quantifying the intensity and lifetime of autofluorophores, with higher specificity for cases of mixed emissions, which are ubiquitous in autofluorescence and multiplexed in vivo imaging.

Design and characterization of a time-domain optical tomography platform for mesoscopic lifetime imaging.

We report on the system design and instrumental characteristics of a novel time-domain mesoscopic fluorescence molecular tomography (TD-MFMT) system for multiplexed molecular imaging in turbid media. The system is equipped with a supercontinuum pulsed laser for broad spectral excitation, based on a high-density descanned raster scanning intensity-based acquisition for 2D and 3D imaging and augmented with a high-dynamical range linear time-resolved single-photon avalanche diode (SPAD) array for lifetime quantification. We report on the system's spatio-temporal and spectral characteristics and its sensitivity and specificity in controlled experimental settings. Also, a phantom study is undertaken to test the performance of the system to image deeply-seated fluorescence inclusions in tissue-like media. In addition, ex vivo tumor xenograft imaging is performed to validate the system's applicability to the biological sample. The characterization results manifest the capability to sense small fluorescence concentrations (on the order of nanomolar) while quantifying fluorescence lifetimes and lifetime-based parameters at high resolution. The phantom results demonstrate the system's potential to perform 3D multiplexed imaging thanks to spectral and lifetime contrast in the mesoscopic range (at millimeters depth). The ex vivo imaging exhibits the prospect of TD-MFMT to resolve intra-tumoral heterogeneity in a depth-dependent manner.

In vitro and in vivo NIR fluorescence lifetime imaging with a time-gated SPAD camera.

Near-infrared (NIR) fluorescence lifetime imaging (FLI) provides a unique contrast mechanism to monitor biological parameters and molecular events in vivo. Single-photon avalanche diode (SPAD) cameras have been recently demonstrated in FLI microscopy (FLIM) applications, but their suitability for in vivo macroscopic FLI (MFLI) in deep tissues remains to be demonstrated. Herein, we report in vivo NIR MFLI measurement with SwissSPAD2, a large time-gated SPAD camera. We first benchmark its performance in well-controlled in vitro experiments, ranging from monitoring environmental effects on fluorescence lifetime, to quantifying Förster resonant energy transfer (FRET) between dyes. Next, we use it for in vivo studies of target-drug engagement in live and intact tumor xenografts using FRET. Information obtained with SwissSPAD2 was successfully compared to that obtained with a gated intensified charge-coupled device (ICCD) camera, using two different approaches. Our results demonstrate that SPAD cameras offer a powerful technology for in vivo preclinical applications in the NIR window.

Monte Carlo-based data generation for efficient deep learning reconstruction of macroscopic diffuse optical tomography and topography applications.

SIGNIFICANCE: Deep learning (DL) models are being increasingly developed to map sensor data to the image domain directly. However, DL methodologies are data-driven and require large and diverse data sets to provide robust and accurate image formation performances. For research modalities such as 2D/3D diffuse optical imaging, the lack of large publicly available data sets and the wide variety of instrumentation designs, data types, and applications leads to unique challenges in obtaining well-controlled data sets for training and validation. Meanwhile, great efforts over the last four decades have focused on developing accurate and computationally efficient light propagation models that are flexible enough to simulate a wide variety of experimental conditions. AIM: Recent developments in Monte Carlo (MC)-based modeling offer the unique advantage of simulating accurately light propagation spatially, temporally, and over an extensive range of optical parameters, including minimally to highly scattering tissue within a computationally efficient platform. Herein, we demonstrate how such MC platforms, namely "Monte Carlo eXtreme" and "Mesh-based Monte Carlo," can be leveraged to generate large and representative data sets for training the DL model efficiently. APPROACH: We propose data generator pipeline strategies using these platforms and demonstrate their potential in fluorescence optical topography, fluorescence optical tomography, and single-pixel diffuse optical tomography. These applications represent a large variety in instrumentation design, sample properties, and contrast function. RESULTS: DL models trained using the MC-based in silico datasets, validated further with experimental data not used during training, show accurate and promising results. CONCLUSION: Overall, these MC-based data generation pipelines are expected to support the development of DL models for rapid, robust, and user-friendly image formation in a wide variety of applications.

3D k-space reflectance fluorescence tomography via deep learning.

We report on the potential to perform image reconstruction in 3D k-space reflectance fluorescence tomography (FT) using deep learning (DL). Herein, we adopt a modified AUTOMAP architecture and develop a training methodology that leverages an open-source Monte-Carlo-based simulator to generate a large dataset. Using an enhanced EMNIST (EEMNIST) dataset as an embedded contrast function allows us to train the network efficiently. The optical strategy utilizes k-space illumination in a reflectance configuration to probe tissue in the mesoscopic regime with high sensitivity and resolution. The proposed DL model training and validation is performed with both in silico data and a phantom experiment. Overall, our results indicate that the approach can correctly reconstruct both single and multiple fluorescent embedding(s) in a 3D volume. Furthermore, the presented technique is shown to outperform the traditional approaches [least-squares (LSQ) and total-variation minimization (TVAL)], especially at higher depths. We, therefore, expect the proposed computational technique to have future implications in preclinical studies.

Deep learning in macroscopic diffuse optical imaging.

SIGNIFICANCE: Biomedical optics system design, image formation, and image analysis have primarily been guided by classical physical modeling and signal processing methodologies. Recently, however, deep learning (DL) has become a major paradigm in computational modeling and has demonstrated utility in numerous scientific domains and various forms of data analysis. AIM: We aim to comprehensively review the use of DL applied to macroscopic diffuse optical imaging (DOI). APPROACH: First, we provide a layman introduction to DL. Then, the review summarizes current DL work in some of the most active areas of this field, including optical properties retrieval, fluorescence lifetime imaging, and diffuse optical tomography. RESULTS: The advantages of using DL for DOI versus conventional inverse solvers cited in the literature reviewed herein are numerous. These include, among others, a decrease in analysis time (often by many orders of magnitude), increased quantitative reconstruction quality, robustness to noise, and the unique capability to learn complex end-to-end relationships. CONCLUSIONS: The heavily validated capability of DL's use across a wide range of complex inverse solving methodologies has enormous potential to bring novel DOI modalities, otherwise deemed impractical for clinical translation, to the patient's bedside.

Macroscopic Fluorescence Lifetime Imaging for Monitoring of Drug-Target Engagement.

Precision medicine promises to improve therapeutic efficacy while reducing adverse effects, especially in oncology. However, despite great progresses in recent years, precision medicine for cancer treatment is not always part of routine care. Indeed, the ability to specifically tailor therapies to distinct patient profiles requires still significant improvements in targeted therapy development as well as decreases in drug treatment failures. In this regard, preclinical animal research is fundamental to advance our understanding of tumor biology, and diagnostic and therapeutic response. Most importantly, the ability to measure drug-target engagement accurately in live and intact animals is critical in guiding the development and optimization of targeted therapy. However, a major limitation of preclinical molecular imaging modalities is their lack of capability to directly and quantitatively discriminate between drug accumulation and drug-target engagement at the pathological site. Recently, we have developed Macroscopic Fluorescence Lifetime Imaging (MFLI) as a unique feature of optical imaging to quantitate in vivo drug-target engagement. MFLI quantitatively reports on nanoscale interactions via lifetime-sensing of Förster Resonance Energy Transfer (FRET) in live, intact animals. Hence, MFLI FRET acts as a direct reporter of receptor dimerization and target engagement via the measurement of the fraction of labeled-donor entity undergoing binding to its respective receptor. MFLI is expected to greatly impact preclinical imaging and also adjacent fields such as image-guided surgery and drug development.

Deep Learning in Biomedical Optics.

This article reviews deep learning applications in biomedical optics with a particular emphasis on image formation. The review is organized by imaging domains within biomedical optics and includes microscopy, fluorescence lifetime imaging, in vivo microscopy, widefield endoscopy, optical coherence tomography, photoacoustic imaging, diffuse tomography, and functional optical brain imaging. For each of these domains, we summarize how deep learning has been applied and highlight methods by which deep learning can enable new capabilities for optics in medicine. Challenges and opportunities to improve translation and adoption of deep learning in biomedical optics are also summarized. Lasers Surg. Med. © 2021 Wiley Periodicals LLC.

Quantification of Trastuzumab-HER2 Engagement In Vitro and In Vivo.

Human EGF Receptor 2 (HER2) is an important oncogene driving aggressive metastatic growth in up to 20% of breast cancer tumors. At the same time, it presents a target for passive immunotherapy such as trastuzumab (TZM). Although TZM has been widely used clinically since 1998, not all eligible patients benefit from this therapy due to primary and acquired drug resistance as well as potentially lack of drug exposure. Hence, it is critical to directly quantify TZM-HER2 binding dynamics, also known as cellular target engagement, in undisturbed tumor environments in live, intact tumor xenograft models. Herein, we report the direct measurement of TZM-HER2 binding in HER2-positive human breast cancer cells and tumor xenografts using fluorescence lifetime Forster Resonance Energy Transfer (FLI-FRET) via near-infrared (NIR) microscopy (FLIM-FRET) as well as macroscopy (MFLI-FRET) approaches. By sensing the reduction of fluorescence lifetime of donor-labeled TZM in the presence of acceptor-labeled TZM, we successfully quantified the fraction of HER2-bound and internalized TZM immunoconjugate both in cell culture and tumor xenografts in live animals. Ex vivo immunohistological analysis of tumors confirmed the binding and internalization of TZM-HER2 complex in breast cancer cells. Thus, FLI-FRET imaging presents a powerful analytical tool to monitor and quantify cellular target engagement and subsequent intracellular drug delivery in live HER2-positive tumor xenografts.

Real-time, wide-field and high-quality single snapshot imaging of optical properties with profile correction using deep learning.

The development of real-time, wide-field and quantitative diffuse optical imaging methods to visualize functional and structural biomarkers of living tissues is a pressing need for numerous clinical applications including image-guided surgery. In this context, Spatial Frequency Domain Imaging (SFDI) is an attractive method allowing for the fast estimation of optical properties using the Single Snapshot of Optical Properties (SSOP) approach. Herein, we present a novel implementation of SSOP based on a combination of deep learning network at the filtering stage and Graphics Processing Units (GPU) capable of simultaneous high visual quality image reconstruction, surface profile correction and accurate optical property (OP) extraction in real-time across large fields of view. In the most optimal implementation, the presented methodology demonstrates megapixel profile-corrected OP imaging with results comparable to that of profile-corrected SFDI, with a processing time of 18 ms and errors relative to SFDI method less than 10% in both profilometry and profile-corrected OPs. This novel processing framework lays the foundation for real-time multispectral quantitative diffuse optical imaging for surgical guidance and healthcare applications. All code and data used for this work is publicly available at www.healthphotonics.org under the resources tab.

UNMIX-ME: spectral and lifetime fluorescence unmixing via deep learning.

Hyperspectral fluorescence lifetime imaging allows for the simultaneous acquisition of spectrally resolved temporal fluorescence emission decays. In turn, the acquired rich multidimensional data set enables simultaneous imaging of multiple fluorescent species for a comprehensive molecular assessment of biotissues. However, to enable quantitative imaging, inherent spectral overlap between the considered fluorescent probes and potential bleed-through must be considered. Such a task is performed via either spectral or lifetime unmixing, typically independently. Herein, we present "UNMIX-ME" (unmix multiple emissions), a deep learning-based fluorescence unmixing routine, capable of quantitative fluorophore unmixing by simultaneously using both spectral and temporal signatures. UNMIX-ME was trained and validated using an in silico framework replicating the data acquisition process of a compressive hyperspectral fluorescent lifetime imaging platform (HMFLI). It was benchmarked against a conventional LSQ method for tri and quadri-exponential simulated samples. Last, UNMIX-ME's potential was assessed for NIR FRET in vitro and in vivo preclinical applications.

Macroscopic fluorescence lifetime topography enhanced via spatial frequency domain imaging.

We report on a macroscopic fluorescence lifetime imaging (MFLI) topography computational framework based around machine learning with the main goal of retrieving the depth of fluorescent inclusions deeply seated in bio-tissues. This approach leverages the depth-resolved information inherent to time-resolved fluorescence data sets coupled with the retrieval of in situ optical properties as obtained via spatial frequency domain imaging (SFDI). Specifically, a Siamese network architecture is proposed with optical properties (OPs) and time-resolved fluorescence decays as input followed by simultaneous retrieval of lifetime maps and depth profiles. We validate our approach using comprehensive in silico data sets as well as with a phantom experiment. Overall, our results demonstrate that our approach can retrieve the depth of fluorescence inclusions, especially when coupled with optical properties estimation, with high accuracy. We expect the presented computational approach to find great utility in applications such as optical-guided surgery.

Fast fit-free analysis of fluorescence lifetime imaging via deep learning.

Fluorescence lifetime imaging (FLI) provides unique quantitative information in biomedical and molecular biology studies but relies on complex data-fitting techniques to derive the quantities of interest. Herein, we propose a fit-free approach in FLI image formation that is based on deep learning (DL) to quantify fluorescence decays simultaneously over a whole image and at fast speeds. We report on a deep neural network (DNN) architecture, named fluorescence lifetime imaging network (FLI-Net) that is designed and trained for different classes of experiments, including visible FLI and near-infrared (NIR) FLI microscopy (FLIM) and NIR gated macroscopy FLI (MFLI). FLI-Net outputs quantitatively the spatially resolved lifetime-based parameters that are typically employed in the field. We validate the utility of the FLI-Net framework by performing quantitative microscopic and preclinical lifetime-based studies across the visible and NIR spectra, as well as across the 2 main data acquisition technologies. These results demonstrate that FLI-Net is well suited to accurately quantify complex fluorescence lifetimes in cells and, in real time, in intact animals without any parameter settings. Hence, FLI-Net paves the way to reproducible and quantitative lifetime studies at unprecedented speeds, for improved dissemination and impact of FLI in many important biomedical applications ranging from fundamental discoveries in molecular and cellular biology to clinical translation.

Novel small RNA (sRNA) landscape of the starvation-stress response transcriptome of Salmonella enterica serovar typhimurium.

Small RNAs (sRNAs) are short (∼50-200 nucleotides) noncoding RNAs that regulate cellular activities across bacteria. Salmonella enterica starved of a carbon-energy (C) source experience a host of genetic and physiological changes broadly referred to as the starvation-stress response (SSR). In an attempt to identify novel sRNAs contributing to SSR control, we grew log-phase, 5-h C-starved and 24-h C-starved cultures of the virulent Salmonella enterica subspecies enterica serovar Typhimurium strain SL1344 and comprehensively sequenced their small RNA transcriptomes. Strikingly, after employing a novel strategy for sRNA discovery based on identifying dynamic transcripts arising from "gene-empty" regions, we identify 58 wholly undescribed Salmonella sRNA genes potentially regulating SSR averaging an ∼1,000-fold change in expression between log-phase and C-starved cells. Importantly, the expressions of individual sRNA loci were confirmed by both comprehensive transcriptome analyses and northern blotting of select candidates. Of note, we find 43 candidate sRNAs share significant sequence identity to characterized sRNAs in other bacteria, and ∼70% of our sRNAs likely assume characteristic sRNA structural conformations. In addition, we find 53 of our 58 candidate sRNAs either overlap neighboring mRNA loci or share significant sequence complementarity to mRNAs transcribed elsewhere in the SL1344 genome strongly suggesting they regulate the expression of transcripts via antisense base-pairing. Finally, in addition to this work resulting in the identification of 58 entirely novel Salmonella enterica genes likely participating in the SSR, we also find evidence suggesting that sRNAs are significantly more prevalent than currently appreciated and that Salmonella sRNAs may actually number in the thousands.