Pixel-wise programmability enables dynamic high-SNR cameras for high-speed microscopy.

High-speed wide-field fluorescence microscopy has the potential to capture biological processes with exceptional spatiotemporal resolution. However, conventional cameras suffer from low signal-to-noise ratio at high frame rates, limiting their ability to detect faint fluorescent events. Here, we introduce an image sensor where each pixel has individually programmable sampling speed and phase, so that pixels can be arranged to simultaneously sample at high speed with a high signal-to-noise ratio. In high-speed voltage imaging experiments, our image sensor significantly increases the output signal-to-noise ratio compared to a low-noise scientific CMOS camera (~2-3 folds). This signal-to-noise ratio gain enables the detection of weak neuronal action potentials and subthreshold activities missed by the standard scientific CMOS cameras. Our camera with flexible pixel exposure configurations offers versatile sampling strategies to improve signal quality in various experimental conditions.

Fast light-field 3D microscopy with out-of-distribution detection and adaptation through conditional normalizing flows.

Real-time 3D fluorescence microscopy is crucial for the spatiotemporal analysis of live organisms, such as neural activity monitoring. The eXtended field-of-view light field microscope (XLFM), also known as Fourier light field microscope, is a straightforward, single snapshot solution to achieve this. The XLFM acquires spatial-angular information in a single camera exposure. In a subsequent step, a 3D volume can be algorithmically reconstructed, making it exceptionally well-suited for real-time 3D acquisition and potential analysis. Unfortunately, traditional reconstruction methods (like deconvolution) require lengthy processing times (0.0220 Hz), hampering the speed advantages of the XLFM. Neural network architectures can overcome the speed constraints but do not automatically provide a way to certify the realism of their reconstructions, which is essential in the biomedical realm. To address these shortcomings, this work proposes a novel architecture to perform fast 3D reconstructions of live immobilized zebrafish neural activity based on a conditional normalizing flow. It reconstructs volumes at 8 Hz spanning 512x512x96 voxels, and it can be trained in under two hours due to the small dataset requirements (50 image-volume pairs). Furthermore, normalizing flows provides a way to compute the exact likelihood of a sample. This allows us to certify whether the predicted output is in- or ood, and retrain the system when a novel sample is detected. We evaluate the proposed method on a cross-validation approach involving multiple in-distribution samples (genetically identical zebrafish) and various out-of-distribution ones.

Temporally multiplexed imaging of dynamic signaling networks in living cells.

Molecular signals interact in networks to mediate biological processes. To analyze these networks, it would be useful to image many signals at once, in the same living cell, using standard microscopes and genetically encoded fluorescent reporters. Here, we report temporally multiplexed imaging (TMI), which uses genetically encoded fluorescent proteins with different clocklike properties-such as reversibly photoswitchable fluorescent proteins with different switching kinetics-to represent different cellular signals. We linearly decompose a brief (few-second-long) trace of the fluorescence fluctuations, at each point in a cell, into a weighted sum of the traces exhibited by each fluorophore expressed in the cell. The weights then represent the signal amplitudes. We use TMI to analyze relationships between different kinase activities in individual cells, as well as between different cell-cycle signals, pointing toward broad utility throughout biology in the analysis of signal transduction cascades in living systems.

Pixel-wise programmability enables dynamic high-SNR cameras for high-speed microscopy.

High-speed widefield fluorescence microscopy has the potential to capture biological processes with exceptional spatiotemporal resolution. However, conventional cameras suffer from low signal-to-noise ratio (SNR) at high frame rates, limiting their ability to detect faint fluorescent events. Here we introduce an image sensor where each pixel has individually programmable sampling speed and phase, so that pixels can be arranged to simultaneously sample at high speed with a high SNR. In high-speed voltage imaging experiments, our image sensor significantly increases the output SNR compared to a low-noise scientific CMOS camera (~2-3 folds). This SNR gain enables the detection of weak neuronal action potentials and subthreshold activities missed by the standard scientific CMOS cameras. Our proposed camera with flexible pixel exposure configurations offers versatile sampling strategies to improve signal quality in various experimental conditions.

Fast light-field 3D microscopy with out-of-distribution detection and adaptation through Conditional Normalizing Flows.

Real-time 3D fluorescence microscopy is crucial for the spatiotemporal analysis of live organisms, such as neural activity monitoring. The eXtended field-of-view light field microscope (XLFM), also known as Fourier light field microscope, is a straightforward, single snapshot solution to achieve this. The XLFM acquires spatial-angular information in a single camera exposure. In a subsequent step, a 3D volume can be algorithmically reconstructed, making it exceptionally well-suited for real-time 3D acquisition and potential analysis. Unfortunately, traditional reconstruction methods (like deconvolution) require lengthy processing times (0.0220 Hz), hampering the speed advantages of the XLFM. Neural network architectures can overcome the speed constraints at the expense of lacking certainty metrics, which renders them untrustworthy for the biomedical realm. This work proposes a novel architecture to perform fast 3D reconstructions of live immobilized zebrafish neural activity based on a conditional normalizing flow. It reconstructs volumes at 8 Hz spanning 512 × 512 × 96 voxels, and it can be trained in under two hours due to the small dataset requirements (10 image-volume pairs). Furthermore, normalizing flows allow for exact Likelihood computation, enabling distribution monitoring, followed by out-of-distribution detection and retraining of the system when a novel sample is detected. We evaluate the proposed method on a cross-validation approach involving multiple in-distribution samples (genetically identical zebrafish) and various out-of-distribution ones.

Imaging the voltage of neurons distributed across entire brains of larval zebrafish.

Neurons interact in networks distributed throughout the brain. Although much effort has focused on whole-brain calcium imaging, recent advances in genetically encoded voltage indicators (GEVIs) raise the possibility of imaging voltage of neurons distributed across brains. To achieve this, a microscope must image at high volumetric rate and signal-to-noise ratio. We present a remote scanning light-sheet microscope capable of imaging GEVI-expressing neurons distributed throughout entire brains of larval zebrafish at a volumetric rate of 200.8 Hz. We measured voltage of ∼1/3 of the neurons of the brain, distributed throughout. We observed that neurons firing at different times during a sequence were located at different brain locations, for sequences elicited by a visual stimulus, which mapped onto locations throughout the optic tectum, as well as during stimulus-independent bursts, which mapped onto locations in the cerebellum and medulla. Whole-brain voltage imaging may open up frontiers in the fundamental operation of neural systems.

Optical volumetric projection for fast 3D imaging through circularly symmetric pupil engineering.

Monitoring and manipulating neuronal activities with optical microscopy desires a method where light can be focused or projected over a long axial range so that large brain tissues (>100 [Formula: see text] thick) can be simultaneously imaged, and specific brain regions can be optogenetically stimulated without the need for slow optical refocusing. However, the micron-scale resolution required in neuronal imaging yields a depth of field of less than 10 [Formula: see text] in conventional imaging systems. We propose to use a circularly symmetric phase mask to extend the depth of field. A numerical study shows that our method maintains both the peak and the shape of the point spread function vs the axial position better than current methods. Imaging of a 3D bead suspension and sparsely labelled thick brain tissue confirms the feasibility of the system for fast volumetric imaging.

Rapid whole brain imaging of neural activity in freely behaving larval zebrafish (Danio rerio).

The internal brain dynamics that link sensation and action are arguably better studied during natural animal behaviors. Here, we report on a novel volume imaging and 3D tracking technique that monitors whole brain neural activity in freely swimming larval zebrafish (Danio rerio). We demonstrated the capability of our system through functional imaging of neural activity during visually evoked and prey capture behaviors in larval zebrafish.