High-resolution and high-speed 3D tracking of microrobots using a fluorescent light field microscope.

Background: Imaging and tracking are crucial for microrobots which navigate through complex 3D environments. Fluorescent imaging (FI) by microscope offers a high-resolution and high-sensitive imaging method to study the property of microrobots. However, conventional microscope suffers from shallow depth of field (DOF) and lacks 3D imaging capability. Methods: We proposed a high-resolution and high-speed 3D tracking method for microrobots based on a fluorescent light field microscope (FLFM). We designed the FLFM system according to the size of a representative helical microrobot (150 µm body length, 50 µm screw diameter), and studied the system's performance. We also proposed a 3D tracking algorithm for microrobots using digital refocusing. Results: We validated the method by simulations and built an FLFM system to perform the tracking experiments of microrobots with representative size. Our 3D tracking method achieves a 30 fps data acquisition rate, 10 µm lateral resolution and approximately 40 µm axial resolution over a volume of 1,200×1,200×326 µm3. Results indicate that the accuracy of the method can reach about 9 µm. Conclusions: Compared with the FI by a conventional microscope, the FLFM-based method gains wider DOF and 3D imaging capability with a single-shot image. The tracking method succeeds in providing the trajectory of the microrobot with a good lateral resolution.

Multiplexed phase-space imaging for 3D fluorescence microscopy.

Optical phase-space functions describe spatial and angular information simultaneously; examples of optical phase-space functions include light fields in ray optics and Wigner functions in wave optics. Measurement of phase-space enables digital refocusing, aberration removal and 3D reconstruction. High-resolution capture of 4D phase-space datasets is, however, challenging. Previous scanning approaches are slow, light inefficient and do not achieve diffraction-limited resolution. Here, we propose a multiplexed method that solves these problems. We use a spatial light modulator (SLM) in the pupil plane of a microscope in order to sequentially pattern multiplexed coded apertures while capturing images in real space. Then, we reconstruct the 3D fluorescence distribution of our sample by solving an inverse problem via regularized least squares with a proximal accelerated gradient descent solver. We experimentally reconstruct a 101 Megavoxel 3D volume (1010×510×500µm with NA 0.4), demonstrating improved acquisition time, light throughput and resolution compared to scanning aperture methods. Our flexible patterning scheme further allows sparsity in the sample to be exploited for reduced data capture.

3D imaging in volumetric scattering media using phase-space measurements.

We demonstrate the use of phase-space imaging for 3D localization of multiple point sources inside scattering material. The effect of scattering is to spread angular (spatial frequency) information, which can be measured by phase space imaging. We derive a multi-slice forward model for homogenous volumetric scattering, then develop a reconstruction algorithm that exploits sparsity in order to further constrain the problem. By using 4D measurements for 3D reconstruction, the dimensionality mismatch provides significant robustness to multiple scattering, with either static or dynamic diffusers. Experimentally, our high-resolution 4D phase-space data is collected by a spectrogram setup, with results successfully recovering the 3D positions of multiple LEDs embedded in turbid scattering media.

Experimental robustness of Fourier ptychography phase retrieval algorithms.

Fourier ptychography is a new computational microscopy technique that provides gigapixel-scale intensity and phase images with both wide field-of-view and high resolution. By capturing a stack of low-resolution images under different illumination angles, an inverse algorithm can be used to computationally reconstruct the high-resolution complex field. Here, we compare and classify multiple proposed inverse algorithms in terms of experimental robustness. We find that the main sources of error are noise, aberrations and mis-calibration (i.e. model mis-match). Using simulations and experiments, we demonstrate that the choice of cost function plays a critical role, with amplitude-based cost functions performing better than intensity-based ones. The reason for this is that Fourier ptychography datasets consist of images from both brightfield and darkfield illumination, representing a large range of measured intensities. Both noise (e.g. Poisson noise) and model mis-match errors are shown to scale with intensity. Hence, algorithms that use an appropriate cost function will be more tolerant to both noise and model mis-match. Given these insights, we propose a global Newton's method algorithm which is robust and accurate. Finally, we discuss the impact of procedures for algorithmic correction of aberrations and mis-calibration.