Efficient compressed sensing reconstruction for 3D fluorescence microscopy using OptoMechanical Modulation Tomography (OMMT) with a 1+2D regularization.

OptoMechanical Modulation Tomography (OMMT) exploits compressed sensing to reconstruct high resolution microscopy volumes from fewer measurement images compared to exhaustive section sampling in conventional light sheet microscopy. Nevertheless, the volumetric reconstruction process is computationally expensive, making it impractically slow to use on large-size images, and prone to generating visual artefacts. Here, we propose a reconstruction approach that uses a 1+2D Total Variation (TV1+2) regularization that does not generate such artefacts and is amenable to efficient implementation using parallel computing. We evaluate our method for accuracy and scaleability on simulated and experimental data. Using a high quality, but computationally expensive, Plug-and-Play (PnP) method that uses the BM4D denoiser as a benchmark, we observe that our approach offers an advantageous trade-off between speed and accuracy.

PAAQ: Paired Alternating AcQuisitions for virtual high frame rate multichannel cardiac fluorescence microscopy.

In vivo fluorescence microscopy is a powerful tool to image the beating heart in its early development stages. A high acquisition frame rate is necessary to study its fast contractions, but the limited fluorescence intensity requires sensitive cameras that are often too slow. Moreover, the problem is even more complex when imaging distinct tissues in the same sample using different fluorophores. We present Paired Alternating AcQuisitions, a method to image cyclic processes in multiple channels, which requires only a single (possibly slow) camera. We generate variable temporal illumination patterns in each frame, alternating between channel-specific illuminations (fluorescence) in odd frames and a motion-encoding brightfield pattern as a common reference in even frames. Starting from the image pairs, we find the position of each reference frame in the cardiac cycle through a combination of image-based sorting and regularized curve fitting. Thanks to these estimated reference positions, we assemble multichannel videos whose frame rate is virtually increased. We characterize our method on synthetic and experimental images collected in zebrafish embryos, showing quantitative and visual improvements in the reconstructed videos over existing nongated sorting-based alternatives. Using a 15 Hz camera, we showcase a reconstructed video containing two fluorescence channels at 100 fps.

Aliasing mitigation in optical microscopy of dynamic biological samples by use of temporally modulated color illumination and a standard RGB camera.

SIGNIFICANCE: Despite recent developments in microscopy, temporal aliasing can arise when imaging dynamic samples. Modern sampling frameworks, such as generalized sampling, mitigate aliasing but require measurement of temporally overlapping and potentially negative-valued inner products. Conventional cameras cannot collect these directly as they operate sequentially and are only sensitive to light intensity. AIM: We aim to mitigate aliasing in microscopy of dynamic monochrome samples by implementing generalized sampling via the use of a color camera and modulated color illumination. APPROACH: We solve the overlap problem by spectrally multiplexing the acquisitions and using (positive) B-spline segments as projection kernels. Reconstruction involves spectral unmixing and inverse filtering. We implemented this method using a color LED illuminator. We evaluated its performance by imaging a rotating grid and its applicability by imaging the beating zebrafish embryo heart in transmission and light-sheet microscopes. RESULTS: Compared to stroboscopic imaging, our method mitigates aliasing with performance improving as the projection order increases. The approach can be implemented in conventional microscopes but is limited by the number of available LED colors and camera channels. CONCLUSIONS: Generalized sampling can be implemented via color modulation in microscopy to mitigate temporal aliasing. The simple hardware requirements could make it applicable to other optical imaging modalities.

Spatially-Variant CNN-based Point Spread Function Estimation for Blind Deconvolution and Depth Estimation in Optical Microscopy.

Optical microscopy is an essential tool in biology and medicine. Imaging thin, yet non-flat objects in a single shot (without relying on more sophisticated sectioning setups) remains challenging as the shallow depth of field that comes with highresolution microscopes leads to unsharp image regions and makes depth localization and quantitative image interpretation difficult. Here, we present a method that improves the resolution of light microscopy images of such objects by locally estimating image distortion while jointly estimating object distance to the focal plane. Specifically, we estimate the parameters of a spatiallyvariant Point Spread Function (PSF) model using a Convolutional Neural Network (CNN), which does not require instrument- or object-specific calibration. Our method recovers PSF parameters from the image itself with up to a squared Pearson correlation coefficient of 0.99 in ideal conditions, while remaining robust to object rotation, illumination variations, or photon noise. When the recovered PSFs are used with a spatially-variant and regularized Richardson-Lucy (RL) deconvolution algorithm, we observed up to 2.1 dB better Signal-to-Noise Ratio (SNR) compared to other Blind Deconvolution (BD) techniques. Following microscope-specific calibration, we further demonstrate that the recovered PSF model parameters permit estimating surface depth with a precision of 2 micrometers and over an extended range when using engineered PSFs. Our method opens up multiple possibilities for enhancing images of non-flat objects with minimal need for a priori knowledge about the optical setup.

Temporal super-resolution microscopy using a hue-encoded shutter.

Limited time-resolution in microscopy is an obstacle to many biological studies. Despite recent advances in hardware, digital cameras have limited operation modes that constrain frame-rate, integration time, and color sensing patterns. In this paper, we propose an approach to extend the temporal resolution of a conventional digital color camera by leveraging a multi-color illumination source. Our method allows for the imaging of single-hue objects at an increased frame-rate by trading spectral for temporal information (while retaining the ability to measure base hue). It also allows rapid switching to standard RGB acquisition. We evaluated the feasibility and performance of our method via experiments with mobile resolution targets. We observed a time-resolution increase by a factor 2.8 with a three-fold increase in temporal sampling rate. We further illustrate the use of our method to image the beating heart of a zebrafish larva, allowing the display of color or fast grayscale images. Our method is particularly well-suited to extend the capabilities of imaging systems where the flexibility of rapidly switching between high frame rate and color imaging are necessary.

Direct inversion algorithm for focal plane scanning optical projection tomography.

To achieve approximately parallel projection geometry, traditional optical projection tomography (OPT) requires the use of low numerical aperture (NA) objectives, which have a long depth-of-field at the expense of poor lateral resolution. Particularly promising methods to improve spatial resolution include ad-hoc post-processing filters that limit the effect of the system's MTF and focal-plane-scanning OPT (FPS-OPT), an alternative acquisition procedure that allows the use of higher NA objectives by limiting the effect of their shallow depth of field yet still assumes parallel projection rays during reconstruction. Here, we provide a detailed derivation that establishes the existence of a direct inversion formula for FPS-OPT. Based on this formula, we propose a point spread function-aware algorithm that is similar in form and complexity to traditional filtered backprojection (FBP). With simulations, we demonstrate that our point-spread-function aware FBP for FPS-OPT leads to more accurate images than both traditional OPT with deconvolution and FPS-OPT with naive FBP reconstruction. We further illustrate the technique on experimental zebrafish data, which shows that our approach reduces out-of-focus blur compared to a direct FBP reconstruction with FPS-OPT.

Development of the cardiac conduction system in zebrafish.

The cardiac conduction system (CCS) propagates and coordinates the electrical excitation that originates from the pacemaker cells, throughout the heart, resulting in rhythmic heartbeat. Its defects result in life-threatening arrhythmias and sudden cardiac death. Understanding of the factors involved in the formation and function of the CCS remains incomplete. By transposon assisted transgenesis, we have developed enhancer trap (ET) lines of zebrafish that express fluorescent protein in the pacemaker cells at the sino-atrial node (SAN) and the atrio-ventricular region (AVR), termed CCS transgenics. This expression pattern begins at the stage when the heart undergoes looping morphogenesis at 36 h post fertilization (hpf) and is maintained into adulthood. Using the CCS transgenics, we investigated the effects of perturbation of cardiac function, as simulated by either the absence of endothelium or hemodynamic stimulation, on the cardiac conduction cells, which resulted in abnormal compaction of the SAN. To uncover the identity of the gene represented by the EGFP expression in the CCS transgenics, we mapped the transposon integration sites on the zebrafish genome to positions in close proximity to the gene encoding fibroblast growth homologous factor 2a (fhf2a). Fhf2a is represented by three transcripts, one of which is expressed in the developing heart. These transgenics are useful tools for studies of development of the CCS and cardiac disease.

Dynamic structure and protein expression of the live embryonic heart captured by 2-photon light sheet microscopy and retrospective registration.

We present an imaging and image reconstruction pipeline that captures the dynamic three-dimensional beating motion of the live embryonic zebrafish heart at subcellular resolution. Live, intact zebrafish embryos were imaged using 2-photon light sheet microscopy, which offers deep and fast imaging at 70 frames per second, and the individual optical sections were assembled into a full 4D reconstruction of the beating heart using an optimized retrospective image registration algorithm. This imaging and reconstruction platform permitted us to visualize protein expression patterns at endogenous concentrations in zebrafish gene trap lines.

Developmental Alterations in Heart Biomechanics and Skeletal Muscle Function in Desmin Mutants Suggest an Early Pathological Root for Desminopathies.

Desminopathies belong to a family of muscle disorders called myofibrillar myopathies that are caused by Desmin mutations and lead to protein aggregates in muscle fibers. To date, the initial pathological steps of desminopathies and the impact of desmin aggregates in the genesis of the disease are unclear. Using live, high-resolution microscopy, we show that Desmin loss of function and Desmin aggregates promote skeletal muscle defects and alter heart biomechanics. In addition, we show that the calcium dynamics associated with heart contraction are impaired and are associated with sarcoplasmic reticulum dilatation as well as abnormal subcellular distribution of Ryanodine receptors. Our results demonstrate that desminopathies are associated with perturbed excitation-contraction coupling machinery and that aggregates are more detrimental than Desmin loss of function. Additionally, we show that pharmacological inhibition of aggregate formation and Desmin knockdown revert these phenotypes. Our data suggest alternative therapeutic approaches and further our understanding of the molecular determinants modulating Desmin aggregate formation.

Endothelial cilia mediate low flow sensing during zebrafish vascular development.

VIDEO ABSTRACT: The pattern of blood flow has long been thought to play a significant role in vascular morphogenesis, yet the flow-sensing mechanism that is involved at early embryonic stages, when flow forces are low, remains unclear. It has been proposed that endothelial cells use primary cilia to sense flow, but this has never been tested in vivo. Here we show, by noninvasive, high-resolution imaging of live zebrafish embryos, that endothelial cilia progressively deflect at the onset of blood flow and that the deflection angle correlates with calcium levels in endothelial cells. We demonstrate that alterations in shear stress, ciliogenesis, or expression of the calcium channel PKD2 impair the endothelial calcium level and both increase and perturb vascular morphogenesis. Altogether, these results demonstrate that endothelial cilia constitute a highly sensitive structure that permits the detection of low shear forces during vascular morphogenesis.

High-resolution imaging of cardiomyocyte behavior reveals two distinct steps in ventricular trabeculation.

Over the course of development, the vertebrate heart undergoes a series of complex morphogenetic processes that transforms it from a simple myocardial epithelium to the complex 3D structure required for its function. One of these processes leads to the formation of trabeculae to optimize the internal structure of the ventricle for efficient conduction and contraction. Despite the important role of trabeculae in the development and physiology of the heart, little is known about their mechanism of formation. Using 3D time-lapse imaging of beating zebrafish hearts, we observed that the initiation of cardiac trabeculation can be divided into two processes. Before any myocardial cell bodies have entered the trabecular layer, cardiomyocytes extend protrusions that invade luminally along neighboring cell-cell junctions. These protrusions can interact within the trabecular layer to form new cell-cell contacts. Subsequently, cardiomyocytes constrict their abluminal surface, moving their cell bodies into the trabecular layer while elaborating more protrusions. We also examined the formation of these protrusions in trabeculation-deficient animals, including erbb2 mutants, tnnt2a morphants, which lack cardiac contractions and flow, and myh6 morphants, which lack atrial contraction and exhibit reduced flow. We found that, compared with cardiomyocytes in wild-type hearts, those in erbb2 mutants were less likely to form protrusions, those in tnnt2a morphants formed less stable protrusions, and those in myh6 morphants extended fewer protrusions per cell. Thus, through detailed 4D imaging of beating hearts, we have identified novel cellular behaviors underlying cardiac trabeculation.

Discretization of continuous convolution operators for accurate modeling of wave propagation in digital holography.

Discretization of continuous (analog) convolution operators by direct sampling of the convolution kernel and use of fast Fourier transforms is highly efficient. However, it assumes the input and output signals are band-limited, a condition rarely met in practice, where signals have finite support or abrupt edges and sampling is nonideal. Here, we propose to approximate signals in analog, shift-invariant function spaces, which do not need to be band-limited, resulting in discrete coefficients for which we derive discrete convolution kernels that accurately model the analog convolution operator while taking into account nonideal sampling devices (such as finite fill-factor cameras). This approach retains the efficiency of direct sampling but not its limiting assumption. We propose fast forward and inverse algorithms that handle finite-length, periodic, and mirror-symmetric signals with rational sampling rates. We provide explicit convolution kernels for computing coherent wave propagation in the context of digital holography. When compared to band-limited methods in simulations, our method leads to fewer reconstruction artifacts when signals have sharp edges or when using nonideal sampling devices.

Pulse propagation by a capacitive mechanism drives embryonic blood flow.

Pulsatile flow is a universal feature of the blood circulatory system in vertebrates and can lead to diseases when abnormal. In the embryo, blood flow forces stimulate vessel remodeling and stem cell proliferation. At these early stages, when vessels lack muscle cells, the heart is valveless and the Reynolds number (Re) is low, few details are available regarding the mechanisms controlling pulses propagation in the developing vascular network. Making use of the recent advances in optical-tweezing flow probing approaches, fast imaging and elastic-network viscous flow modeling, we investigated the blood-flow mechanics in the zebrafish main artery and show how it modifies the heart pumping input to the network. The movement of blood cells in the embryonic artery suggests that elasticity of the network is an essential factor mediating the flow. Based on these observations, we propose a model for embryonic blood flow where arteries act like a capacitor in a way that reduces heart effort. These results demonstrate that biomechanics is key in controlling early flow propagation and argue that intravascular elasticity has a role in determining embryonic vascular function.

4D reconstruction of the beating embryonic heart from two orthogonal sets of parallel optical coherence tomography slice-sequences.

Current methods to build dynamic optical coherence tomography (OCT) volumes of the beating embryonic heart involve synchronization of 2D+time slice-sequences acquired over separate heartbeats. Temporal registration of these sequences is performed either through gating or postprocessing. While synchronization algorithms that exclusively rely on image- intrinsic signals allow forgoing external gating hardware, they are prone to error accumulation, require operator-supervised correction, or lead to nonisotropic resolution. Here, we propose an image-based, retrospective reconstruction technique that uses two sets of parallel 2D+T slice-sequences, acquired perpendicularly to each other, to yield accurate and automatic reconstructions with isotropic resolution. The method utilizes the similarity of the data at the slice intersections to spatio-temporally register the two sets of slice sequences and fuse them into a high-resolution 4D volume. We characterize our method by using 1) simulated heart phantom datasets and 2) OCT datasets acquired from the beating heart of live cultured E9.5 mouse and E10.5 rat embryos. We demonstrate that while our method requires greater acquisition and reconstruction time compared to methods that use slices from a single direction, it produces more accurate and self-validating reconstructions since each set of reconstructed slices acts as a reference for the slices in the perpendicular set.

Motion-based structure separation for label-free, high-speed, 3D cardiac microscopy.

Capturing the dynamics of individual structures in the embryonic heart is an essential step for studying its function and development. Label-free brightfield (BF) microscopy allows for higher acquisition frame-rates than techniques requiring molecular labeling, without interfering with embryo viability or needing complex equipment. However, since different structures contribute similarly to image contrast, label-free microscopy lacks specificity. Here we mitigate this problem by separating a single-channel image series into multiple channels specific to different cardio-vascular structures, based only on their motion patterns. The technique combines images from multiple cardiac cycles and z-sections after non-uniform temporal registration to produce 3D+time image volumes of one full cardiac cycle with separate channels for static, transient and periodically moving structures. The resulting data is well-suited for velocity analysis and 3D-visualization. We characterize the separating capabilities of our technique on a synthetic cardiac dataset and demonstrate its practical applicability, by reconstructing three-channel views of the beating embryonic zebrafish heart with an effective frame rate of 1000 volumes (256×256×20 voxels each) per second. This technique enables quantitative characterization of dynamic heart function during cardiogenesis.

Sequential Turning Acquisition and Reconstruction (STAR) method for four-dimensional imaging of cyclically moving structures.

Optical coherence tomography allows for dynamic, three-dimensional (3D+T) imaging of the heart within animal embryos. However, direct 3D+T imaging frame rates remain insufficient for cardiodynamic analysis. Previously, this limitation has been addressed by reconstructing 3D+T representations of the beating heart based on sets of two-dimensional image sequences (2D+T) acquired sequentially at high frame rate and in fixed (and parallel) planes throughout the heart. These methods either require additional hardware to trigger the acquisition of each 2D+T series to the same phase of the cardiac cycle or accumulate registration errors as the slices are synchronized retrospectively by pairs, without a gating signal. Here, we present a sequential turning acquisition and reconstruction (STAR) method for 3D+T imaging of periodically moving structures, which does not require any additional gating signal and is not prone to registration error accumulation. Similarly to other sequential cardiac imaging methods, multiple fast image series are consecutively acquired for different sections but in between acquisitions, the imaging plane is rotated around the center line instead of shifted along the direction perpendicular to the slices. As the central lines of all image-sequences coincide and represent measurements of the same spatial position, they can be used to accurately synchronize all the slices to a single inherent reference signal. We characterized the accuracy of our method on a simulated dynamic phantom and successfully imaged a beating embryonic rat heart. Potentially, this method can be applied for structural or Doppler imaging approaches with any direct space imaging modality such as computed tomography, ultrasound, or light microscopy.