Versatile multiple object tracking in sparse 2D/3D videos via deformable image registration.

Tracking body parts in behaving animals, extracting fluorescence signals from cells embedded in deforming tissue, and analyzing cell migration patterns during development all require tracking objects with partially correlated motion. As dataset sizes increase, manual tracking of objects becomes prohibitively inefficient and slow, necessitating automated and semi-automated computational tools. Unfortunately, existing methods for multiple object tracking (MOT) are either developed for specific datasets and hence do not generalize well to other datasets, or require large amounts of training data that are not readily available. This is further exacerbated when tracking fluorescent sources in moving and deforming tissues, where the lack of unique features and sparsely populated images create a challenging environment, especially for modern deep learning techniques. By leveraging technology recently developed for spatial transformer networks, we propose ZephIR, an image registration framework for semi-supervised MOT in 2D and 3D videos. ZephIR can generalize to a wide range of biological systems by incorporating adjustable parameters that encode spatial (sparsity, texture, rigidity) and temporal priors of a given data class. We demonstrate the accuracy and versatility of our approach in a variety of applications, including tracking the body parts of a behaving mouse and neurons in the brain of a freely moving C. elegans. We provide an open-source package along with a web-based graphical user interface that allows users to provide small numbers of annotations to interactively improve tracking results.

Learning Probabilistic Piecewise Rigid Atlases of Model Organisms via Generative Deep Networks.

Atlases are crucial to imaging statistics as they enable the standardization of inter-subject and inter-population analyses. While existing atlas estimation methods based on fluid/elastic/diffusion registration yield high-quality results for the human brain, these deformation models do not extend to a variety of other challenging areas of neuroscience such as the anatomy of C. elegans worms and fruit flies. To this end, this work presents a general probabilistic deep network-based framework for atlas estimation and registration which can flexibly incorporate various deformation models and levels of keypoint supervision that can be applied to a wide class of model organisms. Of particular relevance, it also develops a deformable piecewise rigid atlas model which is regularized to preserve inter-observation distances between neighbors. These modeling considerations are shown to improve atlas construction and key-point alignment across a diversity of datasets with small sample sizes including neuron positions in C. elegans hermaphrodites, fluorescence microscopy of male C. elegans, and images of fruit fly wings. Code is accessible at https://github.com/amin-nejat/Deformable-Atlas.

Toward a more accurate 3D atlas of C. elegans neurons.

BACKGROUND: Determining cell identity in volumetric images of tagged neuronal nuclei is an ongoing challenge in contemporary neuroscience. Frequently, cell identity is determined by aligning and matching tags to an "atlas" of labeled neuronal positions and other identifying characteristics. Previous analyses of such C. elegans datasets have been hampered by the limited accuracy of such atlases, especially for neurons present in the ventral nerve cord, and also by time-consuming manual elements of the alignment process. RESULTS: We present a novel automated alignment method for sparse and incomplete point clouds of the sort resulting from typical C. elegans fluorescence microscopy datasets. This method involves a tunable learning parameter and a kernel that enforces biologically realistic deformation. We also present a pipeline for creating alignment atlases from datasets of the recently developed NeuroPAL transgene. In combination, these advances allow us to label neurons in volumetric images with confidence much higher than previous methods. CONCLUSIONS: We release, to the best of our knowledge, the most complete full-body C. elegans 3D positional neuron atlas, incorporating positional variability derived from at least 7 animals per neuron, for the purposes of cell-type identity prediction for myriad applications (e.g., imaging neuronal activity, gene expression, and cell-fate).

Visualizing the organization and differentiation of the male-specific nervous system of C. elegans.

Sex differences in the brain are prevalent throughout the animal kingdom and particularly well appreciated in the nematode Caenorhabditis elegans, where male animals contain a little-studied set of 93 male-specific neurons. To make these neurons amenable for future study, we describe here how a multicolor reporter transgene, NeuroPAL, is capable of visualizing the distinct identities of all male-specific neurons. We used NeuroPAL to visualize and characterize a number of features of the male-specific nervous system. We provide several proofs of concept for using NeuroPAL to identify the sites of expression of gfp-tagged reporter genes and for cellular fate analysis by analyzing the effect of removal of several developmental patterning genes on neuronal identity acquisition. We use NeuroPAL and its intrinsic cohort of more than 40 distinct differentiation markers to show that, even though male-specific neurons are generated throughout all four larval stages, they execute their terminal differentiation program in a coordinated manner in the fourth larval stage. This coordinated wave of differentiation, which we call 'just-in-time' differentiation, couples neuronal maturation programs with the appearance of sexual organs.

Estimating the Unique Information of Continuous Variables.

The integration and transfer of information from multiple sources to multiple targets is a core motive of neural systems. The emerging field of partial information decomposition (PID) provides a novel information-theoretic lens into these mechanisms by identifying synergistic, redundant, and unique contributions to the mutual information between one and several variables. While many works have studied aspects of PID for Gaussian and discrete distributions, the case of general continuous distributions is still uncharted territory. In this work we present a method for estimating the unique information in continuous distributions, for the case of one versus two variables. Our method solves the associated optimization problem over the space of distributions with fixed bivariate marginals by combining copula decompositions and techniques developed to optimize variational autoencoders. We obtain excellent agreement with known analytic results for Gaussians, and illustrate the power of our new approach in several brain-inspired neural models. Our method is capable of recovering the effective connectivity of a chaotic network of rate neurons, and uncovers a complex trade-off between redundancy, synergy and unique information in recurrent networks trained to solve a generalized XOR task.

NeuroPAL: A Multicolor Atlas for Whole-Brain Neuronal Identification in C. elegans.

Comprehensively resolving neuronal identities in whole-brain images is a major challenge. We achieve this in C. elegans by engineering a multicolor transgene called NeuroPAL (a neuronal polychromatic atlas of landmarks). NeuroPAL worms share a stereotypical multicolor fluorescence map for the entire hermaphrodite nervous system that resolves all neuronal identities. Neurons labeled with NeuroPAL do not exhibit fluorescence in the green, cyan, or yellow emission channels, allowing the transgene to be used with numerous reporters of gene expression or neuronal dynamics. We showcase three applications that leverage NeuroPAL for nervous-system-wide neuronal identification. First, we determine the brainwide expression patterns of all metabotropic receptors for acetylcholine, GABA, and glutamate, completing a map of this communication network. Second, we uncover changes in cell fate caused by transcription factor mutations. Third, we record brainwide activity in response to attractive and repulsive chemosensory cues, characterizing multimodal coding for these stimuli.