Content-aware frame interpolation (CAFI): deep learning-based temporal super-resolution for fast bioimaging.

The development of high-resolution microscopes has made it possible to investigate cellular processes in 3D and over time. However, observing fast cellular dynamics remains challenging because of photobleaching and phototoxicity. Here we report the implementation of two content-aware frame interpolation (CAFI) deep learning networks, Zooming SlowMo and Depth-Aware Video Frame Interpolation, that are highly suited for accurately predicting images in between image pairs, therefore improving the temporal resolution of image series post-acquisition. We show that CAFI is capable of understanding the motion context of biological structures and can perform better than standard interpolation methods. We benchmark CAFI's performance on 12 different datasets, obtained from four different microscopy modalities, and demonstrate its capabilities for single-particle tracking and nuclear segmentation. CAFI potentially allows for reduced light exposure and phototoxicity on the sample for improved long-term live-cell imaging. The models and the training and testing data are available via the ZeroCostDL4Mic platform.

High-fidelity 3D live-cell nanoscopy through data-driven enhanced super-resolution radial fluctuation.

Live-cell super-resolution microscopy enables the imaging of biological structure dynamics below the diffraction limit. Here we present enhanced super-resolution radial fluctuations (eSRRF), substantially improving image fidelity and resolution compared to the original SRRF method. eSRRF incorporates automated parameter optimization based on the data itself, giving insight into the trade-off between resolution and fidelity. We demonstrate eSRRF across a range of imaging modalities and biological systems. Notably, we extend eSRRF to three dimensions by combining it with multifocus microscopy. This realizes live-cell volumetric super-resolution imaging with an acquisition speed of ~1 volume per second. eSRRF provides an accessible super-resolution approach, maximizing information extraction across varied experimental conditions while minimizing artifacts. Its optimal parameter prediction strategy is generalizable, moving toward unbiased and optimized analyses in super-resolution microscopy.

Fast4DReg - fast registration of 4D microscopy datasets.

Unwanted sample drift is a common issue that plagues microscopy experiments, preventing accurate temporal visualization and quantification of biological processes. Although multiple methods and tools exist to correct images post acquisition, performing drift correction of three-dimensional (3D) videos using open-source solutions remains challenging and time consuming. Here, we present a new tool developed for ImageJ or Fiji called Fast4DReg that can quickly correct axial and lateral drift in 3D video-microscopy datasets. Fast4DReg works by creating intensity projections along multiple axes and estimating the drift between frames using two-dimensional cross-correlations. Using synthetic and acquired datasets, we demonstrate that Fast4DReg can perform better than other state-of-the-art open-source drift-correction tools and significantly outperforms them in speed. We also demonstrate that Fast4DReg can be used to register misaligned channels in 3D using either calibration slides or misaligned images directly. Altogether, Fast4DReg provides a quick and easy-to-use method to correct 3D imaging data before further visualization and analysis.

TissUExM enables quantitative ultrastructural analysis in whole vertebrate embryos by expansion microscopy.

Super-resolution microscopy reveals the molecular organization of biological structures down to the nanoscale. While it allows the study of protein complexes in single cells, small organisms, or thin tissue sections, there is currently no versatile approach for ultrastructural analysis compatible with whole vertebrate embryos. Here, we present tissue ultrastructure expansion microscopy (TissUExM), a method to expand millimeter-scale and mechanically heterogeneous whole embryonic tissues, including Drosophila wing discs, whole zebrafish, and mouse embryos. TissUExM is designed for the observation of endogenous proteins. It permits quantitative characterization of protein complexes in various organelles at super-resolution in a range of ∼3 mm-sized tissues using conventional microscopes. We demonstrate its strength by investigating tissue-specific ciliary architecture heterogeneity and ultrastructural defects observed upon ciliary protein overexpression. Overall, TissUExM is ideal for performing ultrastructural studies and molecular mapping in situ in whole embryos.

DeepBacs for multi-task bacterial image analysis using open-source deep learning approaches.

This work demonstrates and guides how to use a range of state-of-the-art artificial neural-networks to analyse bacterial microscopy images using the recently developed ZeroCostDL4Mic platform. We generated a database of image datasets used to train networks for various image analysis tasks and present strategies for data acquisition and curation, as well as model training. We showcase different deep learning (DL) approaches for segmenting bright field and fluorescence images of different bacterial species, use object detection to classify different growth stages in time-lapse imaging data, and carry out DL-assisted phenotypic profiling of antibiotic-treated cells. To also demonstrate the ability of DL to enhance low-phototoxicity live-cell microscopy, we showcase how image denoising can allow researchers to attain high-fidelity data in faster and longer imaging. Finally, artificial labelling of cell membranes and predictions of super-resolution images allow for accurate mapping of cell shape and intracellular targets. Our purposefully-built database of training and testing data aids in novice users' training, enabling them to quickly explore how to analyse their data through DL. We hope this lays a fertile ground for the efficient application of DL in microbiology and fosters the creation of tools for bacterial cell biology and antibiotic research.

TrackMate 7: integrating state-of-the-art segmentation algorithms into tracking pipelines.

TrackMate is an automated tracking software used to analyze bioimages and is distributed as a Fiji plugin. Here, we introduce a new version of TrackMate. TrackMate 7 is built to address the broad spectrum of modern challenges researchers face by integrating state-of-the-art segmentation algorithms into tracking pipelines. We illustrate qualitatively and quantitatively that these new capabilities function effectively across a wide range of bio-imaging experiments.

A method for the fast and photon-efficient analysis of time-domain fluorescence lifetime image data over large dynamic ranges.

Fluorescence lifetime imaging (FLIM) allows the quantification of sub-cellular processes in situ, in living cells. A number of approaches have been developed to extract the lifetime from time-domain FLIM data, but they are often limited in terms of speed, photon efficiency, precision or the dynamic range of lifetimes they can measure. Here, we focus on one of the best performing methods in the field, the centre-of-mass method (CMM), that conveys advantages in terms of speed and photon efficiency over others. In this paper, however, we identify a loss of photon efficiency of CMM for short lifetimes when background noise is present. We subsequently present a new development and generalization of CMM that provides for the rapid and accurate extraction of fluorescence lifetime over a large lifetime dynamic range. We provide software tools to simulate, validate and analyse FLIM data sets and compare the performance of our approach against the standard CMM and the commonly employed least-square minimization (LSM) methods. Our method features a better photon efficiency than standard CMM and LSM and is robust in the presence of background noise. The algorithm is applicable to any time-domain FLIM data set.

ALFA-PRF: a novel approach to detect murine perforin release from CTLs into the immune synapse.

When killing through the granule exocytosis pathway, cytotoxic lymphocytes release key effector molecules into the immune synapse, perforin and granzymes, to initiate target cell killing. The pore-forming perforin is essential for the function of cytotoxic lymphocytes, as its pores disrupt the target cell membrane and allow diffusion of pro-apoptotic serine proteases, granzyme, into the target cell, where they initiate various cell death cascades. Unlike human perforin, the detection of its murine counterpart in a live cell system has been problematic due its relatively low expression level and the lack of sensitive antibodies. The lack of a suitable methodology to visualise murine perforin secretion into the synapse hinders the study of the cytotoxic lymphocyte secretory machinery in murine models of human disease. Here, we describe a novel recombinant technology, whereby a short ALFA-tag sequence has been fused with the amino-terminus of a mature murine perforin, and this allowed its detection by the highly specific FluoTag®-X2 anti-ALFA nanobodies using both Total Internal Reflection Fluorescence (TIRF) microscopy of an artificial synapse, and confocal microscopy of the physiological immune synapse with a target cell. This methodology can have broad application in the field of cytotoxic lymphocyte biology and for the many models of human disease.

Imaging in focus: An introduction to denoising bioimages in the era of deep learning.

Fluorescence microscopy enables the direct observation of previously hidden dynamic processes of life, allowing profound insights into mechanisms of health and disease. However, imaging of live samples is fundamentally limited by the toxicity of the illuminating light and images are often acquired using low light conditions. As a consequence, images can become very noisy which severely complicates their interpretation. In recent years, deep learning (DL) has emerged as a very successful approach to remove this noise while retaining the useful signal. Unlike classical algorithms which use well-defined mathematical functions to remove noise, DL methods learn to denoise from example data, providing a powerful content-aware approach. In this review, we first describe the different types of noise that typically corrupt fluorescence microscopy images and introduce the denoising task. We then present the main DL-based denoising methods and their relative advantages and disadvantages. We aim to provide insights into how DL-based denoising methods operate and help users choose the most appropriate tools for their applications.

Comparative Studies in the A30P and A53T α-Synuclein C. elegans Strains to Investigate the Molecular Origins of Parkinson's Disease.

The aggregation of α-synuclein is a hallmark of Parkinson's disease (PD) and a variety of related neurological disorders. A number of mutations in this protein, including A30P and A53T, are associated with familial forms of the disease. Patients carrying the A30P mutation typically exhibit a similar age of onset and symptoms as sporadic PD, while those carrying the A53T mutation generally have an earlier age of onset and an accelerated progression. We report two C. elegans models of PD (PDA30P and PDA53T), which express these mutational variants in the muscle cells, and probed their behavior relative to animals expressing the wild-type protein (PDWT). PDA30P worms showed a reduced speed of movement and an increased paralysis rate, control worms, but no change in the frequency of body bends. By contrast, in PDA53T worms both speed and frequency of body bends were significantly decreased, and paralysis rate was increased. α-Synuclein was also observed to be less well localized into aggregates in PDA30P worms compared to PDA53T and PDWT worms, and amyloid-like features were evident later in the life of the animals, despite comparable levels of expression of α-synuclein. Furthermore, squalamine, a natural product currently in clinical trials for treating symptomatic aspects of PD, was found to reduce significantly the aggregation of α-synuclein and its associated toxicity in PDA53T and PDWT worms, but had less marked effects in PDA30P. In addition, using an antibody that targets the N-terminal region of α-synuclein, we observed a suppression of toxicity in PDA30P, PDA53T and PDWT worms. These results illustrate the use of these two C. elegans models in fundamental and applied PD research.

Democratising deep learning for microscopy with ZeroCostDL4Mic.

Deep Learning (DL) methods are powerful analytical tools for microscopy and can outperform conventional image processing pipelines. Despite the enthusiasm and innovations fuelled by DL technology, the need to access powerful and compatible resources to train DL networks leads to an accessibility barrier that novice users often find difficult to overcome. Here, we present ZeroCostDL4Mic, an entry-level platform simplifying DL access by leveraging the free, cloud-based computational resources of Google Colab. ZeroCostDL4Mic allows researchers with no coding expertise to train and apply key DL networks to perform tasks including segmentation (using U-Net and StarDist), object detection (using YOLOv2), denoising (using CARE and Noise2Void), super-resolution microscopy (using Deep-STORM), and image-to-image translation (using Label-free prediction - fnet, pix2pix and CycleGAN). Importantly, we provide suitable quantitative tools for each network to evaluate model performance, allowing model optimisation. We demonstrate the application of the platform to study multiple biological processes.

Application of Super-Resolution and Advanced Quantitative Microscopy to the Spatio-Temporal Analysis of Influenza Virus Replication.

With an estimated three to five million human cases annually and the potential to infect domestic and wild animal populations, influenza viruses are one of the greatest health and economic burdens to our society, and pose an ongoing threat of large-scale pandemics. Despite our knowledge of many important aspects of influenza virus biology, there is still much to learn about how influenza viruses replicate in infected cells, for instance, how they use entry receptors or exploit host cell trafficking pathways. These gaps in our knowledge are due, in part, to the difficulty of directly observing viruses in living cells. In recent years, advances in light microscopy, including super-resolution microscopy and single-molecule imaging, have enabled many viral replication steps to be visualised dynamically in living cells. In particular, the ability to track single virions and their components, in real time, now allows specific pathways to be interrogated, providing new insights to various aspects of the virus-host cell interaction. In this review, we discuss how state-of-the-art imaging technologies, notably quantitative live-cell and super-resolution microscopy, are providing new nanoscale and molecular insights into influenza virus replication and revealing new opportunities for developing antiviral strategies.

Single-Molecule Super-Resolution Imaging of T-Cell Plasma Membrane CD4 Redistribution upon HIV-1 Binding.

The first step of cellular entry for the human immunodeficiency virus type-1 (HIV-1) occurs through the binding of its envelope protein (Env) with the plasma membrane receptor CD4 and co-receptor CCR5 or CXCR4 on susceptible cells, primarily CD4+ T cells and macrophages. Although there is considerable knowledge of the molecular interactions between Env and host cell receptors that lead to successful fusion, the precise way in which HIV-1 receptors redistribute to sites of virus binding at the nanoscale remains unknown. Here, we quantitatively examine changes in the nanoscale organisation of CD4 on the surface of CD4+ T cells following HIV-1 binding. Using single-molecule super-resolution imaging, we show that CD4 molecules are distributed mostly as either individual molecules or small clusters of up to 4 molecules. Following virus binding, we observe a local 3-to-10-fold increase in cluster diameter and molecule number for virus-associated CD4 clusters. Moreover, a similar but smaller magnitude reorganisation of CD4 was also observed with recombinant gp120. For one of the first times, our results quantify the nanoscale CD4 reorganisation triggered by HIV-1 on host CD4+ T cells. Our quantitative approach provides a robust methodology for characterising the nanoscale organisation of plasma membrane receptors in general with the potential to link spatial organisation to function.

Automated cell tracking using StarDist and TrackMate.

The ability of cells to migrate is a fundamental physiological process involved in embryonic development, tissue homeostasis, immune surveillance, and wound healing. Therefore, the mechanisms governing cellular locomotion have been under intense scrutiny over the last 50 years. One of the main tools of this scrutiny is live-cell quantitative imaging, where researchers image cells over time to study their migration and quantitatively analyze their dynamics by tracking them using the recorded images. Despite the availability of computational tools, manual tracking remains widely used among researchers due to the difficulty setting up robust automated cell tracking and large-scale analysis. Here we provide a detailed analysis pipeline illustrating how the deep learning network StarDist can be combined with the popular tracking software TrackMate to perform 2D automated cell tracking and provide fully quantitative readouts. Our proposed protocol is compatible with both fluorescent and widefield images. It only requires freely available and open-source software (ZeroCostDL4Mic and Fiji), and does not require any coding knowledge from the users, making it a versatile and powerful tool for the field. We demonstrate this pipeline's usability by automatically tracking cancer cells and T cells using fluorescent and brightfield images. Importantly, we provide, as supplementary information, a detailed step-by-step protocol to allow researchers to implement it with their images.

Fluctuation-Based Super-Resolution Traction Force Microscopy.

Cellular mechanics play a crucial role in tissue homeostasis and are often misregulated in disease. Traction force microscopy is one of the key methods that has enabled researchers to study fundamental aspects of mechanobiology; however, traction force microscopy is limited by poor resolution. Here, we propose a simplified protocol and imaging strategy that enhances the output of traction force microscopy by increasing i) achievable bead density and ii) the accuracy of bead tracking. Our approach relies on super-resolution microscopy, enabled by fluorescence fluctuation analysis. Our pipeline can be used on spinning-disk confocal or widefield microscopes and is compatible with available analysis software. In addition, we demonstrate that our workflow can be used to gain biologically relevant information and is suitable for fast long-term live measurement of traction forces even in light-sensitive cells. Finally, using fluctuation-based traction force microscopy, we observe that filopodia align to the force field generated by focal adhesions.

OptiJ: Open-source optical projection tomography of large organ samples.

The three-dimensional imaging of mesoscopic samples with Optical Projection Tomography (OPT) has become a powerful tool for biomedical phenotyping studies. OPT uses visible light to visualize the 3D morphology of large transparent samples. To enable a wider application of OPT, we present OptiJ, a low-cost, fully open-source OPT system capable of imaging large transparent specimens up to 13 mm tall and 8 mm deep with 50 µm resolution. OptiJ is based on off-the-shelf, easy-to-assemble optical components and an ImageJ plugin library for OPT data reconstruction. The software includes novel correction routines for uneven illumination and sample jitter in addition to CPU/GPU accelerated reconstruction for large datasets. We demonstrate the use of OptiJ to image and reconstruct cleared lung lobes from adult mice. We provide a detailed set of instructions to set up and use the OptiJ framework. Our hardware and software design are modular and easy to implement, allowing for further open microscopy developments for imaging large organ samples.

Artificial intelligence for microscopy: what you should know.

Artificial Intelligence based on Deep Learning (DL) is opening new horizons in biomedical research and promises to revolutionize the microscopy field. It is now transitioning from the hands of experts in computer sciences to biomedical researchers. Here, we introduce recent developments in DL applied to microscopy, in a manner accessible to non-experts. We give an overview of its concepts, capabilities and limitations, presenting applications in image segmentation, classification and restoration. We discuss how DL shows an outstanding potential to push the limits of microscopy, enhancing resolution, signal and information content in acquired data. Its pitfalls are discussed, along with the future directions expected in this field.

Fast Fluorescence Lifetime Imaging Reveals the Aggregation Processes of α-Synuclein and Polyglutamine in Aging Caenorhabditis elegans.

The nematode worm Caenorhabditis elegans has emerged as an important model organism in the study of the molecular mechanisms of protein misfolding diseases associated with amyloid formation because of its small size, ease of genetic manipulation, and optical transparency. Obtaining a reliable and quantitative read-out of protein aggregation in this system, however, remains a challenge. To address this problem, we here present a fast time-gated fluorescence lifetime imaging (TG-FLIM) method and show that it provides functional insights into the process of protein aggregation in living animals by enabling the rapid characterization of different types of aggregates. Specifically, in longitudinal studies of C. elegans models of Parkinson's and Huntington's diseases, we observed marked differences in the aggregation kinetics and the nature of the protein inclusions formed by α-synuclein and polyglutamine. In particular, we found that α-synuclein inclusions do not display amyloid-like features until late in the life of the worms, whereas polyglutamine forms amyloid characteristics rapidly in early adulthood. Furthermore, we show that the TG-FLIM method is capable of imaging live and non-anaesthetized worms moving in specially designed agarose microchambers. Taken together, our results show that the TG-FLIM method enables high-throughput functional imaging of living C. elegans that can be used to study in vivo mechanisms of protein aggregation and that has the potential to aid the search for therapeutic modifiers of protein aggregation and toxicity.

Intrinsically aggregation-prone proteins form amyloid-like aggregates and contribute to tissue aging in Caenorhabditis elegans.

Reduced protein homeostasis leading to increased protein instability is a common molecular feature of aging, but it remains unclear whether this is a cause or consequence of the aging process. In neurodegenerative diseases and other amyloidoses, specific proteins self-assemble into amyloid fibrils and accumulate as pathological aggregates in different tissues. More recently, widespread protein aggregation has been described during normal aging. Until now, an extensive characterization of the nature of age-dependent protein aggregation has been lacking. Here, we show that age-dependent aggregates are rapidly formed by newly synthesized proteins and have an amyloid-like structure resembling that of protein aggregates observed in disease. We then demonstrate that age-dependent protein aggregation accelerates the functional decline of different tissues in C. elegans. Together, these findings imply that amyloid-like aggregates contribute to the aging process and therefore could be important targets for strategies designed to maintain physiological functions in the late stages of life.