Pillar data-acquisition strategies for cryo-electron tomography of beam-sensitive biological samples.

For cryo-electron tomography (cryo-ET) of beam-sensitive biological specimens, a planar sample geometry is typically used. As the sample is tilted, the effective thickness of the sample along the direction of the electron beam increases and the signal-to-noise ratio concomitantly decreases, limiting the transfer of information at high tilt angles. In addition, the tilt range where data can be collected is limited by a combination of various sample-environment constraints, including the limited space in the objective lens pole piece and the possible use of fixed conductive braids to cool the specimen. Consequently, most tilt series are limited to a maximum of ±70°, leading to the presence of a missing wedge in Fourier space. The acquisition of cryo-ET data without a missing wedge, for example using a cylindrical sample geometry, is hence attractive for volumetric analysis of low-symmetry structures such as organelles or vesicles, lysis events, pore formation or filaments for which the missing information cannot be compensated by averaging techniques. Irrespective of the geometry, electron-beam damage to the specimen is an issue and the first images acquired will transfer more high-resolution information than those acquired last. There is also an inherent trade-off between higher sampling in Fourier space and avoiding beam damage to the sample. Finally, the necessity of using a sufficient electron fluence to align the tilt images means that this fluence needs to be fractionated across a small number of images; therefore, the order of data acquisition is also a factor to consider. Here, an n-helix tilt scheme is described and simulated which uses overlapping and interleaved tilt series to maximize the use of a pillar geometry, allowing the entire pillar volume to be reconstructed as a single unit. Three related tilt schemes are also evaluated that extend the continuous and classic dose-symmetric tilt schemes for cryo-ET to pillar samples to enable the collection of isotropic information across all spatial frequencies. A fourfold dose-symmetric scheme is proposed which provides a practical compromise between uniform information transfer and complexity of data acquisition.

High-speed 4-dimensional scanning transmission electron microscopy using compressive sensing techniques.

Here we show that compressive sensing allows 4-dimensional (4-D) STEM data to be obtained and accurately reconstructed with both high-speed and reduced electron fluence. The methodology needed to achieve these results compared to conventional 4-D approaches requires only that a random subset of probe locations is acquired from the typical regular scanning grid, which immediately generates both higher speed and the lower fluence experimentally. We also consider downsampling of the detector, showing that oversampling is inherent within convergent beam electron diffraction (CBED) patterns and that detector downsampling does not reduce precision but allows faster experimental data acquisition. Analysis of an experimental atomic resolution yttrium silicide dataset shows that it is possible to recover over 25 dB peak signal-to-noise ratio in the recovered phase using 0.3% of the total data. Lay abstract: Four-dimensional scanning transmission electron microscopy (4-D STEM) is a powerful technique for characterizing complex nanoscale structures. In this method, a convergent beam electron diffraction pattern (CBED) is acquired at each probe location during the scan of the sample. This means that a 2-dimensional signal is acquired at each 2-D probe location, equating to a 4-D dataset. Despite the recent development of fast direct electron detectors, some capable of 100kHz frame rates, the limiting factor for 4-D STEM is acquisition times in the majority of cases, where cameras will typically operate on the order of 2kHz. This means that a raster scan containing 256^2 probe locations can take on the order of 30s, approximately 100-1000 times longer than a conventional STEM imaging technique using monolithic radial detectors. As a result, 4-D STEM acquisitions can be subject to adverse effects such as drift, beam damage, and sample contamination. Recent advances in computational imaging techniques for STEM have allowed for faster acquisition speeds by way of acquiring only a random subset of probe locations from the field of view. By doing this, the acquisition time is significantly reduced, in some cases by a factor of 10-100 times. The acquired data is then processed to fill-in or inpaint the missing data, taking advantage of the inherently low-complex signals which can be linearly combined to recover the information. In this work, similar methods are demonstrated for the acquisition of 4-D STEM data, where only a random subset of CBED patterns are acquired over the raster scan. We simulate the compressive sensing acquisition method for 4-D STEM and present our findings for a variety of analysis techniques such as ptychography and differential phase contrast. Our results show that acquisition times can be significantly reduced on the order of 100-300 times, therefore improving existing frame rates, as well as further reducing the electron fluence beyond just using a faster camera.

The Potential of Subsampling and Inpainting for Fast Low-Dose Cryo FIB-SEM Imaging.

Traditional image acquisition for cryo focused ion-beam scanning electron microscopy (FIB-SEM) tomography often sees thousands of images being captured over a period of many hours, with immense data sets being produced. When imaging beam sensitive materials, these images are often compromised by additional constraints related to beam damage and the devitrification of the material during imaging, which renders data acquisition both costly and unreliable. Subsampling and inpainting are proposed as solutions for both of these aspects, allowing fast and low-dose imaging to take place in the Focused ion-beam scanning electron microscopy FIB-SEM without an appreciable loss in image quality. In this work, experimental data are presented which validate subsampling and inpainting as a useful tool for convenient and reliable data acquisition in a FIB-SEM, with new methods of handling three-dimensional data being employed in the context of dictionary learning and inpainting algorithms using a newly developed microscope control software and data recovery algorithm.

Computational models of amorphous ice for accurate simulation of cryo-EM images of biological samples.

Simulations of cryo-electron microscopy (cryo-EM) images of biological samples can be used to produce test datasets to support the development of instrumentation, methods, and software, as well as to assess data acquisition and analysis strategies. To be useful, these simulations need to be based on physically realistic models which include large volumes of amorphous ice. The gold standard model for EM image simulation is a physical atom-based ice model produced using molecular dynamics simulations. Although practical for small sample volumes; for simulation of cryo-EM data from large sample volumes, this can be too computationally expensive. We have evaluated a Gaussian Random Field (GRF) ice model which is shown to be more computationally efficient for large sample volumes. The simulated EM images are compared with the gold standard atom-based ice model approach and shown to be directly comparable. Comparison with experimentally acquired data shows the Gaussian random field ice model produces realistic simulations. The software required has been implemented in the Parakeet software package and the underlying atomic models are available online for use by the wider community.

Cryo-electron ptychography: Applications and potential in biological characterisation.

There is a clear need for developments in characterisation techniques that provide detailed information about structure-function relationships in biology. Using electron microscopy to achieve high resolution while maintaining a broad field of view remains a challenge, particularly for radiation-sensitive specimens where the signal-to-noise ratio required to maintain structural integrity is limited by low electron fluence. In this review, we explore the potential of cryogenic electron ptychography as an alternative method for characterising biological systems under low-fluence conditions. Using this method with increased information content from multiple sampled regions of interest potentially allows 3D reconstruction with far fewer particles than required in conventional cryo-electron microscopy. This is important for achieving higher resolution in systems where distributions of homogeneous single particles are difficult to obtain. We discuss the progress, limitations, and potential areas for future development of this approach for both single particle analysis and applications to heterogeneous large objects.

Mapping Nanoscale Electrostatic Field Fluctuations around Graphene Dislocation Cores Using Four-Dimensional Scanning Transmission Electron Microscopy (4D-STEM).

Defects in crystalline lattices cause modulation of the atomic density, and this leads to variations in the associated electrostatics at the nanoscale. Mapping these spatially varying charge fluctuations using transmission electron microscopy has typically been challenging due to complicated contrast transfer inherent to conventional phase contrast imaging. To overcome this, we used four-dimensional scanning transmission electron microscopy (4D-STEM) to measure electrostatic fields near point dislocations in a monolayer. The asymmetry of the atomic density in a (1,0) edge dislocation core in graphene yields a local enhancement of the electric field in part of the dislocation core. Through experiment and simulation, the increased electric field magnitude is shown to arise from "long-range" interactions from beyond the nearest atomic neighbor. These results provide insights into the use of 4D-STEM to quantify electrostatics in thin materials and map out the lateral potential variations that are important for molecular and atomic bonding through Coulombic interactions.

Phosphinecarboxamide based InZnP QDs - an air tolerant route to luminescent III-V semiconductors.

We describe a new synthetic methodology for the preparation of high quality, emission tuneable InP-based quantum dots (QDs) using a solid, air- and moisture-tolerant primary phosphine as a group-V precursor. This presents a significantly simpler synthetic pathway compared to the state-of-the-art precursors currently employed in phosphide quantum dot synthesis which are volatile, dangerous and air-sensitive, e.g. P(Si(CH3)3)3.

Cryogenic electron ptychographic single particle analysis with wide bandwidth information transfer.

Advances in cryogenic transmission electron microscopy have revolutionised the determination of many macromolecular structures at atomic or near-atomic resolution. This method is based on conventional defocused phase contrast imaging. However, it has limitations of weaker contrast for small biological molecules embedded in vitreous ice, in comparison with cryo-ptychography, which shows increased contrast. Here we report a single-particle analysis based on the use of ptychographic reconstruction data, demonstrating that three dimensional reconstructions with a wide information transfer bandwidth can be recovered by Fourier domain synthesis. Our work suggests future applications in otherwise challenging single particle analyses, including small macromolecules and heterogeneous or flexible particles. In addition structure determination in situ within cells without the requirement for protein purification and expression may be possible.

Multi-synaptic boutons are a feature of CA1 hippocampal connections in the stratum oriens.

Excitatory synapses are typically described as single synaptic boutons (SSBs), where one presynaptic bouton contacts a single postsynaptic spine. Using serial section block-face scanning electron microscopy, we found that this textbook definition of the synapse does not fully apply to the CA1 region of the hippocampus. Roughly half of all excitatory synapses in the stratum oriens involved multi-synaptic boutons (MSBs), where a single presynaptic bouton containing multiple active zones contacted many postsynaptic spines (from 2 to 7) on the basal dendrites of different cells. The fraction of MSBs increased during development (from postnatal day 22 [P22] to P100) and decreased with distance from the soma. Curiously, synaptic properties such as active zone (AZ) or postsynaptic density (PSD) size exhibited less within-MSB variation when compared with neighboring SSBs, features that were confirmed by super-resolution light microscopy. Computer simulations suggest that these properties favor synchronous activity in CA1 networks.

Towards real-time STEM simulations through targeted subsampling strategies.

Scanning transmission electron microscopy images can be complex to interpret on the atomic scale as the contrast is sensitive to multiple factors such as sample thickness, composition, defects and aberrations. Simulations are commonly used to validate or interpret real experimental images, but they come at a cost of either long computation times or specialist hardware such as graphics processing units. Recent works in compressive sensing for experimental STEM images have shown that it is possible to significantly reduce the amount of acquired signal and still recover the full image without significant loss of image quality, and therefore it is proposed here that similar methods can be applied to STEM simulations. In this paper, we demonstrate a method that can significantly increase the efficiency of STEM simulations through a targeted sampling strategy, along with a new approach to independently subsample each frozen phonon layer. We show the effectiveness of this method by simulating a SrTiO3 grain boundary and monolayer 2H-MoS2 containing a sulphur vacancy using the abTEM software. We also show how this method is not limited to only traditional multislice methods, but also increases the speed of the PRISM simulation method. Furthermore, we discuss the possibility for STEM simulations to seed the acquisition of real data, to potentially lead the way to self-driving (correcting) STEM.

SIM-STEM Lab: Incorporating Compressed Sensing Theory for Fast STEM Simulation.

Recently it has been shown that precise dose control and an increase in the overall acquisition speed of atomic resolution scanning transmission electron microscope (STEM) images can be achieved by acquiring only a small fraction of the pixels in the image experimentally and then reconstructing the full image using an inpainting algorithm. In this paper, we apply the same inpainting approach (a form of compressed sensing) to simulated, sub-sampled atomic resolution STEM images. We find that it is possible to significantly sub-sample the area that is simulated, the number of g-vectors contributing the image, and the number of frozen phonon configurations contributing to the final image while still producing an acceptable fit to a fully sampled simulation. Here we discuss the parameters that we use and how the resulting simulations can be quantifiably compared to the full simulations. As with any Compressed Sensing methodology, care must be taken to ensure that isolated events are not excluded from the process, but the observed increase in simulation speed provides significant opportunities for real time simulations, image classification and analytics to be performed as a supplement to experiments on a microscope to be developed in the future.

Three-dimensional electron ptychography of organic-inorganic hybrid nanostructures.

Three dimensional scaffolded DNA origami with inorganic nanoparticles has been used to create tailored multidimensional nanostructures. However, the image contrast of DNA is poorer than those of the heavy nanoparticles in conventional transmission electron microscopy at high defocus so that the biological and non-biological components in 3D scaffolds cannot be simultaneously resolved using tomography of samples in a native state. We demonstrate the use of electron ptychography to recover high contrast phase information from all components in a DNA origami scaffold without staining. We further quantitatively evaluate the enhancement of contrast in comparison with conventional transmission electron microscopy. In addition, We show that for ptychography post-reconstruction focusing simplifies the workflow and reduces electron dose and beam damage.

Trainable segmentation for transmission electron microscope images of inorganic nanoparticles.

We present a trainable segmentation method implemented within the python package ParticleSpy. The method takes user labelled pixels, which are used to train a classifier and segment images of inorganic nanoparticles from transmission electron microscope images. This implementation is based on the trainable Waikato Environment for Knowledge Analysis (WEKA) segmentation, but is written in python, allowing a large degree of flexibility and meaning it can be easily expanded using other python packages. We find that trainable segmentation offers better accuracy than global or local thresholding methods and requires as few as 100 user-labelled pixels to produce an accurate segmentation. Trainable segmentation presents a balance of accuracy and training time between global/local thresholding and neural networks, when used on transmission electron microscope images of nanoparticles. We also quantitatively investigate the effectiveness of the components of trainable segmentation, its filter kernels and classifiers, in order to demonstrate the use cases for the different filter kernels in ParticleSpy and the most accurate classifiers for different data types. A set of filter kernels is identified that are effective in distinguishing particles from background but that retain dissimilar features. In terms of classifiers, we find that different classifiers perform optimally for different image contrast; specifically, a random forest classifier performs best for high-contrast ADF images, but that QDA and Gaussian Naïve Bayes classifiers perform better for low-contrast TEM images.

Measurement of the size, shape and composition of nanoparticles is routinely performed using transmission electron microscopy and related techniques. Typically, distinguishing particles from the background in an image is performed using the intensity of each pixel, creating two sets of pixels to separate particles from background. However, this separation of intensity can be difficult if the contrast in an image is low, or if the intensity of the background varies significantly. In this study, an approach that takes into account additional image features (such as boundaries and texture) was investigated to study electron microscope images of metallic nanoparticles. In this 'trainable segmentation' approach, the user labels examples of particle and background pixels in order to train a machine learning algorithm to distinguish between particles and background. The performance of different machine learning algorithms was investigated, in addition to the effect of using different features to aid the segmentation. Overall, a trainable segmentation approach was found to perform better than use of an intensity threshold to distinguish between particles and background in electron microscope images.

Direct Visualisation of the Surface Atomic Active Sites of Carbon-Supported Co3 O4 Nanocrystals via High-Resolution Phase Restoration.

The atomic arrangement of the terminating facets on spinel Co3 O4 nanocrystals is strongly linked to their catalytic performance. However, the spinel crystal structure offers multiple possible surface terminations depending on the synthesis. Thus, understanding the terminating surface atomic structure is essential in developing high-performance Co3 O4 nanocrystals. In this work, we present direct atomic-scale observation of the surface terminations of Co3 O4 nanoparticles supported on hollow carbon spheres (HCSs) using exit wavefunction reconstruction from aberration-corrected transmission electron microscopy focal-series. The restored high-resolution phases show distinct resolved oxygen and cobalt atomic columns. The data show that the structure of {100}, {110}, and {111} facets of spinel Co3 O4 exhibit characteristic active sites for carbon monoxide (CO) adsorption, in agreement with density functional theory calculations. Of these facets, the {100} and {110} surface terminations are better suited for CO adsorption than the {111}. However, the presence of oxygen on the {111} surface termination indicates this facet also plays an essential role in CO adsorption. Our results demonstrate direct evidence of the surface termination atomic structure beyond the assumed stoichiometry of the surface.

Applications of deep learning in electron microscopy.

We review the growing use of machine learning in electron microscopy (EM) driven in part by the availability of fast detectors operating at kiloHertz frame rates leading to large data sets that cannot be processed using manually implemented algorithms. We summarize the various network architectures and error metrics that have been applied to a range of EM-related problems including denoising and inpainting. We then provide a review of the application of these in both physical and life sciences, highlighting how conventional networks and training data have been specifically modified for EM.