SmartEM: machine-learning guided electron microscopy.

Connectomics provides essential nanometer-resolution, synapse-level maps of neural circuits to understand brain activity and behavior. However, few researchers have access to the high-throughput electron microscopes necessary to generate enough data for whole circuit or brain reconstruction. To date, machine-learning methods have been used after the collection of images by electron microscopy (EM) to accelerate and improve neuronal segmentation, synapse reconstruction and other data analysis. With the computational improvements in processing EM images, acquiring EM images has now become the rate-limiting step. Here, in order to speed up EM imaging, we integrate machine-learning into real-time image acquisition in a singlebeam scanning electron microscope. This SmartEM approach allows an electron microscope to perform intelligent, data-aware imaging of specimens. SmartEM allocates the proper imaging time for each region of interest - scanning all pixels equally rapidly, then re-scanning small subareas more slowly where a higher quality signal is required to achieve accurate segmentability, in significantly less time. We demonstrate that this pipeline achieves a 7-fold acceleration of image acquisition time for connectomics using a commercial single-beam SEM. We apply SmartEM to reconstruct a portion of mouse cortex with the same accuracy as traditional microscopy but in less time.

Sparse Scanning Electron Microscopy Data Acquisition and Deep Neural Networks for Automated Segmentation in Connectomics.

With the growing importance of three-dimensional and very large field of view imaging, acquisition time becomes a serious bottleneck. Additionally, dose reduction is of importance when imaging material like biological tissue that is sensitive to electron radiation. Random sparse scanning can be used in the combination with image reconstruction techniques to reduce the acquisition time or electron dose in scanning electron microscopy. In this study, we demonstrate a workflow that includes data acquisition on a scanning electron microscope, followed by a sparse image reconstruction based on compressive sensing or alternatively using neural networks. Neuron structures are automatically segmented from the reconstructed images using deep learning techniques. We show that the average dwell time per pixel can be reduced by a factor of 2-3, thereby providing a real-life confirmation of previous results on simulated data in one of the key segmentation applications in connectomics and thus demonstrating the feasibility and benefit of random sparse scanning techniques for a specific real-world scenario.

How should a fixed budget of dwell time be spent in scanning electron microscopy to optimize image quality?

In scanning electron microscopy, the achievable image quality is often limited by a maximum feasible acquisition time per dataset. Particularly with regard to three-dimensional or large field-of-view imaging, a compromise must be found between a high amount of shot noise, which leads to a low signal-to-noise ratio, and excessive acquisition times. Assuming a fixed acquisition time per frame, we compared three different strategies for algorithm-assisted image acquisition in scanning electron microscopy. We evaluated (1) raster scanning with a reduced dwell time per pixel followed by a state-of-the-art Denoising algorithm, (2) raster scanning with a decreased resolution in conjunction with a state-of-the-art Super Resolution algorithm, and (3) a sparse scanning approach where a fixed percentage of pixels is visited by the beam in combination with state-of-the-art inpainting algorithms. Additionally, we considered increased beam currents for each of the strategies. The experiments showed that sparse scanning using an appropriate reconstruction technique was superior to the other strategies.