Normalized level set model for segmentation of low-contrast objects in 2- and 3- dimensional images.

Analyses of biomedical images often rely on accurate segmentation of structures of interest. Traditional segmentation methods based on thresholding, watershed, fast marching, and level set perform well in high-contrast images containing structures of similar intensities. However, such methods can under-segment or miss entirely low-intensity objects on noisy backgrounds. Machine learning segmentation methods promise superior performance but require large training datasets of labeled images which are difficult to create, particularly in 3D. Here, we propose an algorithm based on the Local Binary Fitting (LBF) level set method, specifically designed to improve the segmentation of low-contrast structures.

Correction of topological errors in automated traces of neurites.

Our understanding of synaptic connectivity in the brain relies on the ability to accurately trace sparsely labeled neurons from 3D optical microscopy stacks of images. A variety of automated algorithms and software tools have been developed for this task. These algorithms can capture the general layout of neurites with high fidelity, but the resulting traces often contain topological errors such as broken and incorrectly merged branches. Even a small number of isolated topological errors can drastically alter the connectivity, and therefore, their detection and correction are paramount for connectomics studies. Here, we describe an automated trace proofreading approach that utilizes machine learning to correct trace topology. Multiple stacks of neuron images were traced by two users to create a labeled dataset and assess the baseline of inter-user variability. All traces were then disconnected at branch points and a deep neural network was trained to detect the correct way of reconnecting the branches. Custom morphological features were generated for each cluster of branch points, in a way that is dependent on a merging scenario but invariant to translations, rotations, and reflections of the cluster in the imaging plane. The features and image volume centered at the branch point were used for training a neural network that concatenates these input streams and outputs the confidence measure for different branch merging scenarios. The designed method significantly reduces the number of topological errors in automated traces and comes close to the accuracy achieved by expert users which is the gold standard in the field.

Noise in Neurons and Synapses Enables Reliable Associative Memory Storage in Local Cortical Circuits.

Neural networks in the brain can function reliably despite various sources of errors and noise present at every step of signal transmission. These sources include errors in the presynaptic inputs to the neurons, noise in synaptic transmission, and fluctuations in the neurons' postsynaptic potentials (PSPs). Collectively they lead to errors in the neurons' outputs which are, in turn, injected into the network. Does unreliable network activity hinder fundamental functions of the brain, such as learning and memory retrieval? To explore this question, this article examines the effects of errors and noise on the properties of model networks of inhibitory and excitatory neurons involved in associative sequence learning. The associative learning problem is solved analytically and numerically, and it is also shown how memory sequences can be loaded into the network with a biologically more plausible perceptron-type learning rule. Interestingly, the results reveal that errors and noise during learning increase the probability of memory recall. There is a trade-off between the capacity and reliability of stored memories, and, noise during learning is required for optimal retrieval of stored information. What is more, networks loaded with associative memories to capacity display many structural and dynamical features observed in local cortical circuits in mammals. Based on the similarities between the associative and cortical networks, this article predicts that connections originating from more unreliable neurons or neuron classes in the cortex are more likely to be depressed or eliminated during learning, while connections onto noisier neurons or neuron classes have lower probabilities and higher weights.

Robust Associative Learning Is Sufficient to Explain the Structural and Dynamical Properties of Local Cortical Circuits.

The ability of neural networks to associate successive states of network activity lies at the basis of many cognitive functions. Hence, we hypothesized that many ubiquitous structural and dynamical properties of local cortical networks result from associative learning. To test this hypothesis, we trained recurrent networks of excitatory and inhibitory neurons on memories composed of varying numbers of associations and compared the resulting network properties with those observed experimentally. We show that, when the network is robustly loaded with near-maximum amount of associations it can support, it develops properties that are consistent with the observed probabilities of excitatory and inhibitory connections, shapes of connection weight distributions, overexpression of specific 2- and 3-neuron motifs, distributions of connection numbers in clusters of 3-8 neurons, sustained, irregular, and asynchronous firing activity, and balance of excitation and inhibition. In addition, memories loaded into the network can be retrieved, even in the presence of noise that is comparable with the baseline variations in the postsynaptic potential. The confluence of these results suggests that many structural and dynamical properties of local cortical networks are simply a byproduct of associative learning. We predict that overexpression of excitatory-excitatory bidirectional connections observed in many cortical systems must be accompanied with underexpression of bidirectionally connected inhibitory-excitatory neuron pairs.SIGNIFICANCE STATEMENT Many structural and dynamical properties of local cortical networks are ubiquitously present across areas and species. Because synaptic connectivity is shaped by experience, we wondered whether continual learning, rather than genetic control, is responsible for producing such features. To answer this question, we developed a biologically constrained recurrent network of excitatory and inhibitory neurons capable of learning predefined sequences of network states. Embedding such associative memories into the network revealed that, when individual neurons are robustly loaded with a near-maximum amount of memories they can support, the network develops many properties that are consistent with experimental observations. Our findings suggest that basic structural and dynamical properties of local networks in the brain are simply a byproduct of learning and memory storage.

Accurate registration of in vivo time-lapse images.

In vivo imaging experiments often require automated detection and tracking of changes in the specimen. These tasks can be hindered by variations in the position and orientation of the specimen relative to the microscope, as well as by linear and nonlinear tissue deformations. We propose a feature-based registration method, coupled with optimal transformations, designed to address these problems in 3D time-lapse microscopy images. Features are detected as local regions of maximum intensity in source and target image stacks, and their bipartite intensity dissimilarity matrix is used as an input to the Hungarian algorithm to establish initial correspondences. A random sampling refinement method is employed to eliminate outliers, and the resulting set of corresponding features is used to determine an optimal translation, rigid, affine, or B-spline transformation for the registration of the source and target images. Accuracy of the proposed algorithm was tested on fluorescently labeled axons imaged over a 68-day period with a two-photon laser scanning microscope. To that end, multiple axons in individual stacks of images were traced semi-manually and optimized in 3D, and the distances between the corresponding traces were measured before and after the registration. The results show that there is a progressive improvement in the registration accuracy with increasing complexity of the transformations. In particular, sub-micrometer accuracy (2-3 voxels) was achieved with the regularized affine and B-spline transformations.

Artificial neural network filters for enhancing 3D optical microscopy images of neurites.

The ability to extract accurate morphology of labeled neurons from microscopy images is crucial for mapping brain connectivity and for understanding changes in connectivity that underlie learning and neurological disorders. There are, however, two problems, specific to optical microscopy imaging of neurons, which make accurate neuron tracing exceedingly challenging: (i) neurites can appear broken due to inhomogeneous labeling and (ii) neurites can appear fused in 3D due to limited resolution. Here, we propose and evaluate several artificial neural network (ANN) architectures and conventional image enhancement filters with the aim of alleviating both problems. We developed four image quality metrics to evaluate the effects of the proposed filters: normalized intensity in the cross-over regions between neurites, effective radius of neurites, coefficient of variation of intensity along neurites, and local background to neurite intensity ratio. Our results show that ANN-based filters, trained on optimized semi-manual traces of neurites, can significantly outperform conventional filters. In particular, U-Net based filtering can virtually eliminate background intensity, while also reducing the effective radius of neurites to nearly 1 voxel. In addition, this filter significantly decreases intensity in the cross-over regions between neurites and reduces fluctuations of intensity on neurites' centerlines. These results suggest that including an ANN-based filtering step, which does not require substantial extra time or computing power, can be beneficial for automated neuron tracing projects.

Computer assisted detection of axonal bouton structural plasticity in in vivo time-lapse images.

The ability to measure minute structural changes in neural circuits is essential for long-term in vivo imaging studies. Here, we propose a methodology for detection and measurement of structural changes in axonal boutons imaged with time-lapse two-photon laser scanning microscopy (2PLSM). Correlative 2PLSM and 3D electron microscopy (EM) analysis, performed in mouse barrel cortex, showed that the proposed method has low fractions of false positive/negative bouton detections (2/0 out of 18), and that 2PLSM-based bouton weights are correlated with their volumes measured in EM (r = 0.93). Next, the method was applied to a set of axons imaged in quick succession to characterize measurement uncertainty. The results were used to construct a statistical model in which bouton addition, elimination, and size changes are described probabilistically, rather than being treated as deterministic events. Finally, we demonstrate that the model can be used to quantify significant structural changes in boutons in long-term imaging experiments.

Effects of homeostatic constraints on associative memory storage and synaptic connectivity of cortical circuits.

The impact of learning and long-term memory storage on synaptic connectivity is not completely understood. In this study, we examine the effects of associative learning on synaptic connectivity in adult cortical circuits by hypothesizing that these circuits function in a steady-state, in which the memory capacity of a circuit is maximal and learning must be accompanied by forgetting. Steady-state circuits should be characterized by unique connectivity features. To uncover such features we developed a biologically constrained, exactly solvable model of associative memory storage. The model is applicable to networks of multiple excitatory and inhibitory neuron classes and can account for homeostatic constraints on the number and the overall weight of functional connections received by each neuron. The results show that in spite of a large number of neuron classes, functional connections between potentially connected cells are realized with less than 50% probability if the presynaptic cell is excitatory and generally a much greater probability if it is inhibitory. We also find that constraining the overall weight of presynaptic connections leads to Gaussian connection weight distributions that are truncated at zero. In contrast, constraining the total number of functional presynaptic connections leads to non-Gaussian distributions, in which weak connections are absent. These theoretical predictions are compared with a large dataset of published experimental studies reporting amplitudes of unitary postsynaptic potentials and probabilities of connections between various classes of excitatory and inhibitory neurons in the cerebellum, neocortex, and hippocampus.

DNA bridging and looping by HMO1 provides a mechanism for stabilizing nucleosome-free chromatin.

The regulation of chromatin structure in eukaryotic cells involves abundant architectural factors such as high mobility group B (HMGB) proteins. It is not understood how these factors control the interplay between genome accessibility and compaction. In vivo, HMO1 binds the promoter and coding regions of most ribosomal RNA genes, facilitating transcription and possibly stabilizing chromatin in the absence of histones. To understand how HMO1 performs these functions, we combine single molecule stretching and atomic force microscopy (AFM). By stretching HMO1-bound DNA, we demonstrate a hierarchical organization of interactions, in which HMO1 initially compacts DNA on a timescale of seconds, followed by bridge formation and stabilization of DNA loops on a timescale of minutes. AFM experiments demonstrate DNA bridging between strands as well as looping by HMO1. Our results support a model in which HMO1 maintains the stability of nucleosome-free chromatin regions by forming complex and dynamic DNA structures mediated by protein-protein interactions.

Active learning of neuron morphology for accurate automated tracing of neurites.

Automating the process of neurite tracing from light microscopy stacks of images is essential for large-scale or high-throughput quantitative studies of neural circuits. While the general layout of labeled neurites can be captured by many automated tracing algorithms, it is often not possible to differentiate reliably between the processes belonging to different cells. The reason is that some neurites in the stack may appear broken due to imperfect labeling, while others may appear fused due to the limited resolution of optical microscopy. Trained neuroanatomists routinely resolve such topological ambiguities during manual tracing tasks by combining information about distances between branches, branch orientations, intensities, calibers, tortuosities, colors, as well as the presence of spines or boutons. Likewise, to evaluate different topological scenarios automatically, we developed a machine learning approach that combines many of the above mentioned features. A specifically designed confidence measure was used to actively train the algorithm during user-assisted tracing procedure. Active learning significantly reduces the training time and makes it possible to obtain less than 1% generalization error rates by providing few training examples. To evaluate the overall performance of the algorithm a number of image stacks were reconstructed automatically, as well as manually by several trained users, making it possible to compare the automated traces to the baseline inter-user variability. Several geometrical and topological features of the traces were selected for the comparisons. These features include the total trace length, the total numbers of branch and terminal points, the affinity of corresponding traces, and the distances between corresponding branch and terminal points. Our results show that when the density of labeled neurites is sufficiently low, automated traces are not significantly different from manual reconstructions obtained by trained users.

Efficient associative memory storage in cortical circuits of inhibitory and excitatory neurons.

Many features of synaptic connectivity are ubiquitous among cortical systems. Cortical networks are dominated by excitatory neurons and synapses, are sparsely connected, and function with stereotypically distributed connection weights. We show that these basic structural and functional features of synaptic connectivity arise readily from the requirement of efficient associative memory storage. Our theory makes two fundamental predictions. First, we predict that, despite a large number of neuron classes, functional connections between potentially connected cells must be realized with <50% probability if the presynaptic cell is excitatory and >50% probability if the presynaptic cell is inhibitory. Second, we establish a unique relation between probability of connection and coefficient of variation in connection weights. These predictions are consistent with a dataset of 74 published experiments reporting connection probabilities and distributions of postsynaptic potential amplitudes in various cortical systems. What is more, our theory explains the shapes of the distributions obtained in these experiments.

Automated Reconstruction of Neural Trees Using Front Re-initialization.

This paper proposes a greedy algorithm for automated reconstruction of neural arbors from light microscopy stacks of images. The algorithm is based on the minimum cost path method. While the minimum cost path, obtained using the Fast Marching Method, results in a trace with the least cumulative cost between the start and the end points, it is not sufficient for the reconstruction of neural trees. This is because sections of the minimum cost path can erroneously travel through the image background with undetectable detriment to the cumulative cost. To circumvent this problem we propose an algorithm that grows a neural tree from a specified root by iteratively re-initializing the Fast Marching fronts. The speed image used in the Fast Marching Method is generated by computing the average outward flux of the gradient vector flow field. Each iteration of the algorithm produces a candidate extension by allowing the front to travel a specified distance and then tracking from the farthest point of the front back to the tree. Robust likelihood ratio test is used to evaluate the quality of the candidate extension by comparing voxel intensities along the extension to those in the foreground and the background. The qualified extensions are appended to the current tree, the front is re-initialized, and Fast Marching is continued until the stopping criterion is met. To evaluate the performance of the algorithm we reconstructed 6 stacks of two-photon microscopy images and compared the results to the ground truth reconstructions by using the DIADEM metric. The average comparison score was 0.82 out of 1.0, which is on par with the performance achieved by expert manual tracers.

Statistical traces of long-term memories stored in strengths and patterns of synaptic connections.

Learning and long-term memory rely on plasticity of neural circuits. In adult cerebral cortex, plasticity can result from potentiation and depression of synaptic strengths and structural reorganization of circuits through growth and retraction of dendritic spines. By analyzing 166 distributions of spine head volumes and spine lengths from mouse, rat, monkey, and human brains, we determine the "generalized cost" of dendritic spines. This cost universally depends on spine shape, i.e., the dependence is the same in all the analyzed systems. We show that, in adult, synaptic strength and structural synaptic plasticity mechanisms are in statistical equilibrium, the numbers of dendritic spines in different cortical areas are nearly optimally chosen for memory storage, and the distributions of spine lengths and head volumes are governed by a single parameter--the effective temperature. We suggest that the effective temperature may be viewed as a measure of circuit stability or longevity of stored memories.

Automated tracing of neurites from light microscopy stacks of images.

Automating the process of neural circuit reconstruction on a large-scale is one of the foremost challenges in the field of neuroscience. In this study we examine the methodology for circuit reconstruction from three-dimensional light microscopy (LM) stacks of images. We show how the minimal error-rate of an ideal reconstruction procedure depends on the density of labeled neurites, giving rise to the fundamental limitation of an LM based approach for neural circuit research. Circuit reconstruction procedures typically involve steps related to neuron labeling and imaging, and subsequent image pre-processing and tracing of neurites. In this study, we focus on the last step--detection of traces of neurites from already pre-processed stacks of images. Our automated tracing algorithm, implemented as part of the Neural Circuit Tracer software package, consists of the following main steps. First, image stack is filtered to enhance labeled neurites. Second, centerline of the neurites is detected and optimized. Finally, individual branches of the optimal trace are merged into trees based on a cost minimization approach. The cost function accounts for branch orientations, distances between their end-points, curvature of the merged structure, and its intensity. The algorithm is capable of connecting branches which appear broken due to imperfect labeling and can resolve situations where branches appear to be fused due the limited resolution of light microscopy. The Neural Circuit Tracer software is designed to automatically incorporate ImageJ plug-ins and functions written in MatLab and provides roughly a 10-fold increases in speed in comparison to manual tracing.

Cooperative synapse formation in the neocortex.

Neuron morphology plays an important role in defining synaptic connectivity. Clearly, only pairs of neurons with closely positioned axonal and dendritic branches can be synaptically coupled. For excitatory neurons in the cerebral cortex, such axo-dendritic oppositions, termed potential synapses, must be bridged by dendritic spines to form synaptic connections. To explore the rules by which synaptic connections are formed within the constraints imposed by neuron morphology, we compared the distributions of the numbers of actual and potential synapses between pre- and postsynaptic neurons forming different laminar projections in rat barrel cortex. Quantitative comparison explicitly ruled out the hypothesis that individual synapses between neurons are formed independently of each other. Instead, the data are consistent with a cooperative scheme of synapse formation where multiple-synaptic connections between neurons are stabilized while neurons that do not establish a critical number of synapses are not likely to remain synaptically coupled.

Maximization of the connectivity repertoire as a statistical principle governing the shapes of dendritic arbors.

The shapes of dendritic arbors are fascinating and important, yet the principles underlying these complex and diverse structures remain unclear. Here, we analyzed basal dendritic arbors of 2,171 pyramidal neurons sampled from mammalian brains and discovered 3 statistical properties: the dendritic arbor size scales with the total dendritic length, the spatial correlation of dendritic branches within an arbor has a universal functional form, and small parts of an arbor are self-similar. We proposed that these properties result from maximizing the repertoire of possible connectivity patterns between dendrites and surrounding axons while keeping the cost of dendrites low. We solved this optimization problem by drawing an analogy with maximization of the entropy for a given energy in statistical physics. The solution is consistent with the above observations and predicts scaling relations that can be tested experimentally. In addition, our theory explains why dendritic branches of pyramidal cells are distributed more sparsely than those of Purkinje cells. Our results represent a step toward a unifying view of the relationship between neuronal morphology and function.

The fractions of short- and long-range connections in the visual cortex.

When analyzing synaptic connectivity in a brain tissue slice, it is difficult to discern between synapses made by local neurons and those arising from long-range axonal projections. We analyzed a data set of excitatory neurons and inhibitory basket cells reconstructed from cat primary visual cortex in an attempt to provide a quantitative answer to the question: What fraction of cortical synapses is local, and what fraction is mediated by long-range projections? We found an unexpectedly high proportion of nonlocal synapses. For example, 92% of excitatory synapses near the axis of a 200-microm-diameter iso-orientation column come from neurons located outside the column, and this fraction remains high--76%--even for an 800-micromocular dominance column. The long-range nature of connectivity has dramatic implications for experiments in cortical tissue slices. Our estimate indicates that in a 300-microm-thick section cut perpendicularly to the cortical surface, the number of viable excitatory synapses is reduced to about 10%, and the number of synapses made by inhibitory basket cell axons is reduced to 38%. This uneven reduction in the numbers of excitatory and inhibitory synapses changes the excitation-inhibition balance by a factor of 3.8 toward inhibition, and may result in cortical tissue that is less excitable than in vivo. We found that electrophysiological studies conducted in tissue sections may significantly underestimate the extent of cortical connectivity; for example, for some projections, the reported probabilities of finding connected nearby neuron pairs in slices could understate the in vivo probabilities by a factor of 3.

Detection of the optimal neuron traces in confocal microscopy images.

Quantitative methods of analysis of neural circuits rely on large datasets of neurons reconstructed accurately in three dimensions (3D). Due to the complexity of neuronal arbors, large datasets of reconstructed neurons must be generated with automated algorithms. Here, we attempted to automate the process of neuron tracing from sparsely labeled 3D stacks of confocal microscopy images. Our algorithm involves two steps. In the first step, the segmented image of neurites in the stack is voxel-coded. Centers of intensity of consecutively coded wave fronts are connected into a branched structure, which represents a coarse trace of the neurites. In the second step, this trace is optimized with the modified active contour method, which tends to maximize the intensity along the trace while keeping it under tension. To assess the performance of the algorithm we used manual reconstructions of neurons and converted them into artificial stacks of intensity images. These images were traced using the developed algorithm and quantitatively compared to the corresponding manual traces. The optimal traces were on average 6.0% shorter than the manual traces. This reduction in length resulted from the smoothness of the optimal traces, which, in comparison to the manual ones, were built out of shorter segments, and, as a result, were 3.3% less tortuous. The average distance between the optimal and the manual traces was 0.14 microm, and the average distance between their corresponding branch-points was 2.2 microm, illustrating good agreement between the traces.

Structural plasticity of circuits in cortical neuropil.

Learning and memory formation in the brain depend on the plasticity of neural circuits. In the adult and developing cerebral cortex, this plasticity can result from the formation and elimination of dendritic spines. New synaptic contacts appear in the neuropil where the gaps between axonal and dendritic branches can be bridged by dendritic spines. Such sites are termed potential synapses. Here, we describe a theoretical framework for the analysis of spine remodeling plasticity. We provide a quantitative description of two models of spine remodeling in which the presence of a bouton is either required or not for the formation of a new synapse. We derive expressions for the density of potential synapses in the neuropil, the connectivity fraction, which is the ratio of actual to potential synapses, and the number of structurally different circuits attainable with spine remodeling. We calculate these parameters in mouse occipital cortex, rat CA1, monkey V1, and human temporal cortex. We find that, on average, a dendritic spine can choose among 4-7 potential targets in rodents and 10-20 potential targets in primates. The potential of neuropil for structural circuit remodeling is highest in rat CA1 (7.1-8.6 bits/mum(3)) and lowest in monkey V1 (1.3-1.5 bits/mum(3)). We also evaluate the lower bound of neuron selectivity in the choice of synaptic partners. Postsynaptic excitatory neurons in rodents make synaptic contacts with >21-30% of presynaptic axons encountered with new spine growth. Primate neurons appear to be more selective, making synaptic connections with >7-15% of encountered axons.