Deep learning based coherence holography reconstruction of 3D objects.

We propose a reconstruction method for coherence holography using deep neural networks. cGAN and U-NET models were developed to reconstruct 3D complex objects from recorded interferograms. Our proposed methods, dubbed deep coherence holography (DCH), predict the non-diffracted fields or the sub-objects included in the 3D object from the captured interferograms, yielding better reconstructed objects than the traditional analytical imaging methods in terms of accuracy, resolution, and time. The DCH needs one image per sub-object as opposed to N images for the traditional sin-fit algorithm, and hence the total reconstruction time is reduced by N×. Furthermore, with noisy interferograms the DCH amplitude mean square reconstruction error (MSE) is 5×104× and 104× and phase MSE is 102× and 3×103× better than Fourier fringe and sin-fit algorithms, respectively. The amplitude peak signal to noise ratio (PSNR) is 3× and 2× and phase PSNR is 5× and 3× better than Fourier fringe and sin-fit algorithms, respectively. The reconstruction resolution is the same as sin-fit but 2× better than the Fourier fringe analysis technique.

Three-dimensional refractive index estimation based on deep-inverse non-interferometric optical diffraction tomography (ODT-Deep).

Optical diffraction tomography (ODT) solves an inverse scattering problem to obtain label-free, 3D refractive index (RI) estimation of biological specimens. This work demonstrates 3D RI retrieval methods suitable for partially-coherent ODT systems supported by intensity-only measurements consisting of axial and angular illumination scanning. This framework allows for access to 3D quantitative RI contrast using a simplified non-interferometric technique. We consider a traditional iterative tomographic solver based on a multiple in-plane representation of the optical scattering process and gradient descent optimization adapted for focus-scanning systems, as well as an approach that relies solely on 3D convolutional neural networks (CNNs) to invert the scattering process. The approaches are validated using simulations of the 3D scattering potential for weak phase 3D biological samples.

Digital Holographic Microscopy to Assess Cell Behavior.

Digital holographic microscopy is an imaging technique particularly well suited to the study of living cells in culture, as no labeling is required and computed phase maps produce high contrast, quantitative pixel information. A full experiment involves instrument calibration, cell culture quality checks, selection and setup of imaging chambers, a sampling plan, image acquisition, phase and amplitude map reconstruction, and parameter map post-processing to extract information about cell morphology and/or motility. Each step is described below, focusing on results from imaging four human cell lines. Several post-processing approaches are detailed, with an aim of tracking individual cells and dynamics of cell populations.

Dual-modality digital holographic and polarization microscope to quantify phase and birefringence signals in biospecimens with a complex microstructure.

Optical phase and birefringence signals occur in cells and thin, semi-transparent biomaterials. A dual-modality quantitative phase and polarization microscope was designed to study the interaction of cells with extracellular matrix networks and to relate optical pathlength and birefringence signals within structurally anisotropic biomaterial constructs. The design was based on an existing, custom-built digital holographic microscope, to which was added a polarization microscope utilizing liquid crystal variable retarders. Phase and birefringence channels were calibrated, and data was acquired sequentially from cell-seeded collagen hydrogels and electrofabricated chitosan membranes. Computed phase height and retardance from standard targets were accurate within 99.7% and 99.8%, respectively. Phase height and retardance channel background standard deviations were 35 nm and 0.6 nm, respectively. Human fibroblasts, visible in the phase channel, aligned with collagen network microstructure, with retardance and azimuth visible in the polarization channel. Electrofabricated chitosan membranes formed in 40 µm tall microfluidic channels possessed optical retardance ranging from 7 to 11 nm, and phase height from 37 to 39 µm. These results demonstrate co-registered dual-channel acquisition of phase and birefringence parameter maps from microstructurally-complex biospecimens using a novel imaging system combining digital holographic microscopy with voltage-controlled polarization microscopy.

Computational multi-wavelength phase synthesis using convolutional neural networks [Invited].

Multi-wavelength digital holographic microscopy (MWDHM) provides indirect measurements of the refractive index for non-dispersive samples. Successive-shot MWDHM is not appropriate for dynamic samples and single-shot MWDHM significantly increases the complexity of the optical setup due to the need for multiple lasers or a wavelength tunable source. Here we consider deep learning convolutional neural networks for computational phase synthesis to obtain high-speed simultaneous phase estimates on different wavelengths and thus single-shot estimates of the integral refractive index without increased experimental complexity. This novel, to the best of our knowledge, computational concept is validated using cell phantoms consisting of internal refractive index variations representing cytoplasm and membrane-bound organelles, respectively, and a simulation of a realistic holographic recording process. Specifically, in this work we employed data-driven computational techniques to perform accurate dual-wavelength hologram synthesis (hologram-to-hologram prediction), dual-wavelength phase synthesis (unwrapped phase-to-phase prediction), direct phase-to-index prediction using a single wavelength, hologram-to-phase prediction, and 2D phase unwrapping with sharp discontinuities (wrapped-to-unwrapped phase prediction).

Mueller matrix polarimetry and polar decomposition of articular cartilage imaged in reflectance.

Articular cartilage birefringence relates to zonal architecture primarily of type II collagen, which has been assessed extensively in transmission, through thin tissue sections, to evaluate cartilage repair and degeneration. Mueller matrix imaging of articular cartilage in reflection is of potential utility for non-destructive imaging in clinical and research applications. Therefore, such an imaging system was constructed to measure laser reflectance signals, calibrated, and tested with optical standards. Polar decomposition was chosen as a method to extract fundamental optical parameters from the experimental Mueller matrices, with performance confirmed by simulations. Adult bovine articular cartilage from the patellofemoral groove was found to have ∼0.93 radians retardance, low diattenuation of ∼0.2, and moderately high depolarization of 0.66. Simulations showed that variation in depolarization drives inaccuracy of depolarization and retardance maps derived by polar decomposition. These results create a basis for further investigation of the clinical utility of polarized signals from knee tissue and suggest potential approaches for improving the accuracy of polar decomposition maps.

Simulation of digital holographic recording and reconstruction using a generalized matrix method.

In recent years, research efforts in the field of digital holography have expanded significantly, due to the ability to obtain high-resolution intensity and phase images. The information contained in these images have become of great interest to the machine learning community, with applications spanning a wide portfolio of research areas, including bioengineering. In this work, we seek to demonstrate a high-fidelity simulation of holographic recording. By accurately and numerically simulating the propagation of a coherent light source through a series of optical elements and the object itself, we accurately predict the optical interference of the object and reference wave at the recording plane, including diffraction effects, aberrations, and speckle. We show that the optical transformation that predicts the complex field at the recording plane can be generalized for arbitrary holographic recording configurations using a matrix method. In addition, we provide a detailed description of digital phase reconstruction and aberration compensation for a variety of off-axis holographic configurations. Reconstruction errors are presented for the various holographic recording geometries and complex field objects. While the primary objective of this work is not to evaluate phase reconstruction approaches, the reconstruction of simulated holograms provides validation of the generalized simulation method. The long-term goal of this work is that the generalized holographic simulation motivates the use of phase reconstruction of the simulated holograms to populate databases for training machine-learning algorithms aimed at classifying relevant objects recorded through a variety of holographic setups.

Quantitative scoring of epithelial and mesenchymal qualities of cancer cells using machine learning and quantitative phase imaging.

SIGNIFICANCE: We introduce an application of machine learning trained on optical phase features of epithelial and mesenchymal cells to grade cancer cells' morphologies, relevant to evaluation of cancer phenotype in screening assays and clinical biopsies. AIM: Our objective was to determine quantitative epithelial and mesenchymal qualities of breast cancer cells through an unbiased, generalizable, and linear score covering the range of observed morphologies. APPROACH: Digital holographic microscopy was used to generate phase height maps of noncancerous epithelial (Gie-No3B11) and fibroblast (human gingival) cell lines, as well as MDA-MB-231 and MCF-7 breast cancer cell lines. Several machine learning algorithms were evaluated as binary classifiers of the noncancerous cells that graded the cancer cells by transfer learning. RESULTS: Epithelial and mesenchymal cells were classified with 96% to 100% accuracy. Breast cancer cells had scores in between the noncancer scores, indicating both epithelial and mesenchymal morphological qualities. The MCF-7 cells skewed toward epithelial scores, while MDA-MB-231 cells skewed toward mesenchymal scores. Linear support vector machines (SVMs) produced the most distinct score distributions for each cell line. CONCLUSIONS: The proposed epithelial-mesenchymal score, derived from linear SVM learning, is a sensitive and quantitative approach for detecting epithelial and mesenchymal characteristics of unknown cells based on well-characterized cell lines. We establish a framework for rapid and accurate morphological evaluation of single cells and subtle phenotypic shifts in imaged cell populations.

Experimentally measuring a detectability index of a computational imaging system.

Computational imaging (CI) systems are an enabling technology for multifunctional cameras capable of performing a wide variety of imaging tasks. However, given the complexity of CI systems, it is often difficult to characterize their performance. In this research, a novel measurement technique is proposed and tested to evaluate the performance of complex non-shift invariant linear CI systems performing a detection task at the system level. The performance is characterized using detectability indexes such as an average Hotelling's statistic (t2). The proposed measurement technique relies on a previously developed general CI system framework. The detectability predicts the upper-bounded signal-to-noise ratio of a linear algorithm through evaluation of a matched filter. The experimental results are compared with theoretical expected values through the Night Vision Integrated Performance Model (NV-IPM) and Monte Carlo simulations. We demonstrate the experimental results for a variety of target sizes, colors, and brightnesses on different colored flat backgrounds. Our results demonstrate how the detectability indexes can provide valuable insight into the final system performance. Finally, the measurement technique is used to compare the detection performance of two different cameras.

Machine Learning with Optical Phase Signatures for Phenotypic Profiling of Cell Lines.

Robust and reproducible profiling of cell lines is essential for phenotypic screening assays. The goals of this study were to determine robust and reproducible optical phase signatures of cell lines for classification with machine learning and to correlate optical phase parameters to motile behavior. Digital holographic microscopy (DHM) reconstructed phase maps of cells from two pairs of cancer and non-cancer cell lines. Seventeen image parameters were extracted from each cell's phase map, used for linear support vector machine learning, and correlated to scratch wound closure and Boyden chamber chemotaxis. The classification accuracy was between 90% and 100% for the six pairwise cell line comparisons. Several phase parameters correlated with wound closure rate and chemotaxis across the four cell lines. The level of cell confluence in culture affected phase parameters in all cell lines tested. Results indicate that optical phase features of cell lines are a robust set of quantitative data of potential utility for phenotypic screening and prediction of motile behavior. © 2019 International Society for Advancement of Cytometry.

Deep learning approach for Fourier ptychography microscopy.

Convolutional neural networks (CNNs) have gained tremendous success in solving complex inverse problems. The aim of this work is to develop a novel CNN framework to reconstruct video sequences of dynamic live cells captured using a computational microscopy technique, Fourier ptychographic microscopy (FPM). The unique feature of the FPM is its capability to reconstruct images with both wide field-of-view (FOV) and high resolution, i.e. a large space-bandwidth-product (SBP), by taking a series of low resolution intensity images. For live cell imaging, a single FPM frame contains thousands of cell samples with different morphological features. Our idea is to fully exploit the statistical information provided by these large spatial ensembles so as to make predictions in a sequential measurement, without using any additional temporal dataset. Specifically, we show that it is possible to reconstruct high-SBP dynamic cell videos by a CNN trained only on the first FPM dataset captured at the beginning of a time-series experiment. Our CNN approach reconstructs a 12800×10800 pixel phase image using only ∼25 seconds, a 50× speedup compared to the model-based FPM algorithm. In addition, the CNN further reduces the required number of images in each time frame by ∼ 6×. Overall, this significantly improves the imaging throughput by reducing both the acquisition and computational times. The proposed CNN is based on the conditional generative adversarial network (cGAN) framework. We further propose a mixed loss function that combines the standard image domain loss and a weighted Fourier domain loss, which leads to improved reconstruction of the high frequency information. Additionally, we also exploit transfer learning so that our pre-trained CNN can be further optimized to image other cell types. Our technique demonstrates a promising deep learning approach to continuously monitor large live-cell populations over an extended time and gather useful spatial and temporal information with sub-cellular resolution.

Phase-noise model for actively linearized frequency-modulated continuous-wave ladar.

Understanding the effects of laser phase noise on frequency-modulated continuous-wave distance measurements is important in evaluating ranging accuracy. The standard white-frequency-noise assumption is commonly used to predict the ranging performance. However, other noise sources are typically present that can further degrade the heterodyne beat signal and make this assumption invalid. In addition, many ranging systems employ active sweep linearization techniques that can impact the phase noise. Here, we present a phase-noise model for assessing the accuracy of a phase-locked swept laser source.

Multiaxis heterodyne vibrometer for simultaneous observation of 5 degrees of dynamic freedom from a single beam.

A multiaxis heterodyne interferometer concept is under development for observations of 5 deg of dynamic freedom using a single illumination source. This Letter presents a laboratory system that combines elements of heterodyne Doppler vibrometry, holography, and digital image correlation to simultaneously quantify in-plane translation, out-of-plane rotation, and out-of-plane displacement. The sensor concept observes a dynamic object by mixing a single optical field with heterodyne reference beams and collecting these combined fields at the image and Fourier planes, simultaneously. Polarization and frequency multiplexing are applied to separate two segments of a receive Mach-Zehnder interferometer. Different optical configurations are utilized; one segment produces a focused image of the optical field scattered off the object while the other segment produces an optical Fourier transform of the optical field scattered off the object. Utilizing the amplitude and phase from each plane allows quantification of multiple components of transient motion using a single, orthogonal beam.

Large-volume, low-cost, high-precision FMCW tomography using stitched DFBs.

Optical frequency-modulated continuous-wave (FMCW) reflectometry is a ranging technique that allows for high-resolution distance measurements over long ranges. Similarly, swept-source optical coherence tomography (SS-OCT) provides high-resolution depth imaging over typically shorter distances and higher scan speeds. In this work, we demonstrate a low-cost, low-bandwidth 3D imaging system that provides the high axial resolution imaging capability normally associated with SS-OCT over typical FMCW ranging depths. The imaging system combines 12 distributed feedback laser (DFB) elements from a single butterfly module to provide an axial resolution of 27.1 µm over 6 m of depth and up to 14 cubic meters of volume. Active sweep linearization is used, greatly reducing the signal processing overhead. Various sub-surface, OCT-style tomograms of semi-transparent objects are shown, as well as 3D maps of various objects over depths ranging from sub-millimeter to several meters. Such imaging capability would make long-distance, high-resolution surface interrogation possible in a low-cost, compact package.

Quantitative assessment of cancer cell morphology and motility using telecentric digital holographic microscopy and machine learning.

The noninvasive, fast acquisition of quantitative phase maps using digital holographic microscopy (DHM) allows tracking of rapid cellular motility on transparent substrates. On two-dimensional surfaces in vitro, MDA-MB-231 cancer cells assume several morphologies related to the mode of migration and substrate stiffness, relevant to mechanisms of cancer invasiveness in vivo. The quantitative phase information from DHM may accurately classify adhesive cancer cell subpopulations with clinical relevance. To test this, cells from the invasive breast cancer MDA-MB-231 cell line were cultured on glass, tissue-culture treated polystyrene, and collagen hydrogels, and imaged with DHM followed by epifluorescence microscopy after staining F-actin and nuclei. Trends in cell phase parameters were tracked on the different substrates, during cell division, and during matrix adhesion, relating them to F-actin features. Support vector machine learning algorithms were trained and tested using parameters from holographic phase reconstructions and cell geometric features from conventional phase images, and used to distinguish between elongated and rounded cell morphologies. DHM was able to distinguish between elongated and rounded morphologies of MDA-MB-231 cells with 94% accuracy, compared to 83% accuracy using cell geometric features from conventional brightfield microscopy. This finding indicates the potential of DHM to detect and monitor cancer cell morphologies relevant to cell cycle phase status, substrate adhesion, and motility. © 2017 International Society for Advancement of Cytometry.

Standardized target-specific detectivity metric for computational imaging systems.

Multifunctional cameras capable of performing a wide variety of nearly simultaneous imaging tasks are expected to play a major role in the near future. Computational imaging (CI) systems will serve as one of the main enabling technologies for multifunctional cameras, especially due to the abundance of low-cost, high-speed computational processing available today. An important aspect of these systems is to be able to quantify their performance with respect to specific imaging tasks. However, the non-traditional design of CI systems, both available and proposed, presents a considerable challenge to modeling, comparing, specifying, and measuring their performance. To solve this problem, this paper presents a standardized detection signal-to-noise ratio, referred to as a detectivity metric, along with a general CI system framework. This metric has the flexibility to handle various types of CI systems and specific targets while minimizing the complexity and assumptions needed. The detectivity metric is designed to assess the performance of a CI system searching for a specific known target or signal of interest. An analytical version of the detectivity metric is also presented for a compressive sensing CI system. Special considerations for standardization are also discussed.

Automatic phase aberration compensation for digital holographic microscopy based on deep learning background detection.

We propose a fully automatic technique to obtain aberration free quantitative phase imaging in digital holographic microscopy (DHM) based on deep learning. The traditional DHM solves the phase aberration compensation problem by manually detecting the background for quantitative measurement. This would be a drawback in real time implementation and for dynamic processes such as cell migration phenomena. A recent automatic aberration compensation approach using principle component analysis (PCA) in DHM avoids human intervention regardless of the cells' motion. However, it corrects spherical/elliptical aberration only and disregards the higher order aberrations. Traditional image segmentation techniques can be employed to spatially detect cell locations. Ideally, automatic image segmentation techniques make real time measurement possible. However, existing automatic unsupervised segmentation techniques have poor performance when applied to DHM phase images because of aberrations and speckle noise. In this paper, we propose a novel method that combines a supervised deep learning technique with convolutional neural network (CNN) and Zernike polynomial fitting (ZPF). The deep learning CNN is implemented to perform automatic background region detection that allows for ZPF to compute the self-conjugated phase to compensate for most aberrations.

Noninvasive assessment of articular cartilage surface damage using reflected polarized light microscopy.

Articular surface damage occurs to cartilage during normal aging, osteoarthritis, and in trauma. A noninvasive assessment of cartilage microstructural alterations is useful for studies involving cartilage explants. This study evaluates polarized reflectance microscopy as a tool to assess surface damage to cartilage explants caused by mechanical scraping and enzymatic degradation. Adult bovine articular cartilage explants were scraped, incubated in collagenase, or underwent scrape and collagenase treatments. In an additional experiment, cartilage explants were subject to scrapes at graduated levels of severity. Polarized reflectance parameters were compared with India ink surface staining, features of histological sections, changes in explant wet weight and thickness, and chondrocyte viability. The polarized reflectance signal was sensitive to surface scrape damage and revealed individual scrape features consistent with India ink marks. Following surface treatments, the reflectance contrast parameter was elevated and correlated with image area fraction of India ink. After extensive scraping, polarized reflectance contrast and chondrocyte viability were lower than that from untreated explants. As part of this work, a mathematical model was developed and confirmed the trend in the reflectance signal due to changes in surface scattering and subsurface birefringence. These results demonstrate the effectiveness of polarized reflectance microscopy to sensitively assess surface microstructural alterations in articular cartilage explants.

Digital Holography and 3D Imaging: introduction to the joint feature issue in Applied Optics and Journal of the Optical Society of America B.

The OSA Topical Meeting on Digital Holography and 3D Imaging (DH) was held 25-28 July 2016 in Heidelberg, Germany, as part of the Imaging Congress. Feature issues based on the DH meeting series have been released by Applied Optics (AO) since 2007. This year, AO and the Journal of the Optical Society of America B (JOSA B) jointly decided to have one such feature issue in each journal. This feature issue includes 31 papers in AO and 11 in JOSA B, and covers a large range of topics, reflecting the rapidly expanding techniques and applications of digital holography and 3D imaging. The upcoming DH meeting (DH 2017) will be held from 29 May to 1 June in Jeju Island, South Korea.