Resolution enhancement with a task-assisted GAN to guide optical nanoscopy image analysis and acquisition.

Super-resolution fluorescence microscopy methods enable the characterization of nanostructures in living and fixed biological tissues. However, they require the adjustment of multiple imaging parameters while attempting to satisfy conflicting objectives, such as maximizing spatial and temporal resolution while minimizing light exposure. To overcome the limitations imposed by these trade-offs, post-acquisition algorithmic approaches have been proposed for resolution enhancement and image-quality improvement. Here we introduce the task-assisted generative adversarial network (TA-GAN), which incorporates an auxiliary task (for example, segmentation, localization) closely related to the observed biological nanostructure characterization. We evaluate how the TA-GAN improves generative accuracy over unassisted methods, using images acquired with different modalities such as confocal, bright-field, stimulated emission depletion and structured illumination microscopy. The TA-GAN is incorporated directly into the acquisition pipeline of the microscope to predict the nanometric content of the field of view without requiring the acquisition of a super-resolved image. This information is used to automatically select the imaging modality and regions of interest, optimizing the acquisition sequence by reducing light exposure. Data-driven microscopy methods like the TA-GAN will enable the observation of dynamic molecular processes with spatial and temporal resolutions that surpass the limits currently imposed by the trade-offs constraining super-resolution microscopy.

Automated Microscopy Image Segmentation and Analysis with Machine Learning.

The development of automated quantitative image analysis pipelines requires thoughtful considerations to extract meaningful information. Commonly, extraction rules for quantitative parameters are defined and agreed beforehand to ensure repeatability between annotators. Machine/Deep Learning (ML/DL) now provides tools to automatically extract the set of rules to obtain quantitative information from the images (e.g. segmentation, enumeration, classification, etc.). Many parameters must be considered in the development of proper ML/DL pipelines. We herein present the important vocabulary, the necessary steps to create a thorough image segmentation pipeline, and also discuss technical aspects that should be considered in the development of automated image analysis pipelines through ML/DL.

Activity-Dependent Remodeling of Synaptic Protein Organization Revealed by High Throughput Analysis of STED Nanoscopy Images.

The organization of proteins in the apposed nanodomains of pre- and postsynaptic compartments is thought to play a pivotal role in synaptic strength and plasticity. As such, the alignment between pre- and postsynaptic proteins may regulate, for example, the rate of presynaptic release or the strength of postsynaptic signaling. However, the analysis of these structures has mainly been restricted to subsets of synapses, providing a limited view of the diversity of synaptic protein cluster remodeling during synaptic plasticity. To characterize changes in the organization of synaptic nanodomains during synaptic plasticity over a large population of synapses, we combined STimulated Emission Depletion (STED) nanoscopy with a Python-based statistical object distance analysis (pySODA), in dissociated cultured hippocampal circuits exposed to treatments driving different forms of synaptic plasticity. The nanoscale organization, characterized in terms of coupling properties, of presynaptic (Bassoon, RIM1/2) and postsynaptic (PSD95, Homer1c) scaffold proteins was differently altered in response to plasticity-inducing stimuli. For the Bassoon - PSD95 pair, treatments driving synaptic potentiation caused an increase in their coupling probability, whereas a stimulus driving synaptic depression had an opposite effect. To enrich the characterization of the synaptic cluster remodeling at the population level, we applied unsupervised machine learning approaches to include selected morphological features into a multidimensional analysis. This combined analysis revealed a large diversity of synaptic protein cluster subtypes exhibiting differential activity-dependent remodeling, yet with common features depending on the expected direction of plasticity. The expanded palette of synaptic features revealed by our unbiased approach should provide a basis to further explore the widely diverse molecular mechanisms of synaptic plasticity.

Neuronal activity remodels the F-actin based submembrane lattice in dendrites but not axons of hippocampal neurons.

The nanoscale organization of the F-actin cytoskeleton in neurons comprises membrane-associated periodical rings, bundles, and longitudinal fibers. The F-actin rings have been observed predominantly in axons but only sporadically in dendrites, where fluorescence nanoscopy reveals various patterns of F-actin arranged in mixed patches. These complex dendritic F-actin patterns pose a challenge for investigating quantitatively their regulatory mechanisms. We developed here a weakly supervised deep learning segmentation approach of fluorescence nanoscopy images of F-actin in cultured hippocampal neurons. This approach enabled the quantitative assessment of F-actin remodeling, revealing the disappearance of the rings during neuronal activity in dendrites, but not in axons. The dendritic F-actin cytoskeleton of activated neurons remodeled into longitudinal fibers. We show that this activity-dependent remodeling involves [Formula: see text] and NMDA receptor-dependent mechanisms. This highly dynamic restructuring of dendritic F-actin based submembrane lattice into longitudinal fibers may serve to support activity-dependent membrane remodeling, protein trafficking and neuronal plasticity.

A machine learning approach for online automated optimization of super-resolution optical microscopy.

Traditional approaches for finding well-performing parameterizations of complex imaging systems, such as super-resolution microscopes rely on an extensive exploration phase over the illumination and acquisition settings, prior to the imaging task. This strategy suffers from several issues: it requires a large amount of parameter configurations to be evaluated, it leads to discrepancies between well-performing parameters in the exploration phase and imaging task, and it results in a waste of time and resources given that optimization and final imaging tasks are conducted separately. Here we show that a fully automated, machine learning-based system can conduct imaging parameter optimization toward a trade-off between several objectives, simultaneously to the imaging task. Its potential is highlighted on various imaging tasks, such as live-cell and multicolor imaging and multimodal optimization. This online optimization routine can be integrated to various imaging systems to increase accessibility, optimize performance and improve overall imaging quality.