Mapping model units to visual neurons reveals population code for social behaviour.

The rich variety of behaviours observed in animals arises through the interplay between sensory processing and motor control. To understand these sensorimotor transformations, it is useful to build models that predict not only neural responses to sensory input1-5 but also how each neuron causally contributes to behaviour6,7. Here we demonstrate a novel modelling approach to identify a one-to-one mapping between internal units in a deep neural network and real neurons by predicting the behavioural changes that arise from systematic perturbations of more than a dozen neuronal cell types. A key ingredient that we introduce is 'knockout training', which involves perturbing the network during training to match the perturbations of the real neurons during behavioural experiments. We apply this approach to model the sensorimotor transformations of Drosophila melanogaster males during a complex, visually guided social behaviour8-11. The visual projection neurons at the interface between the optic lobe and central brain form a set of discrete channels12, and prior work indicates that each channel encodes a specific visual feature to drive a particular behaviour13,14. Our model reaches a different conclusion: combinations of visual projection neurons, including those involved in non-social behaviours, drive male interactions with the female, forming a rich population code for behaviour. Overall, our framework consolidates behavioural effects elicited from various neural perturbations into a single, unified model, providing a map from stimulus to neuronal cell type to behaviour, and enabling future incorporation of wiring diagrams of the brain15 into the model.

Neurotransmitter classification from electron microscopy images at synaptic sites in Drosophila melanogaster.

High-resolution electron microscopy of nervous systems has enabled the reconstruction of synaptic connectomes. However, we do not know the synaptic sign for each connection (i.e., whether a connection is excitatory or inhibitory), which is implied by the released transmitter. We demonstrate that artificial neural networks can predict transmitter types for presynapses from electron micrographs: a network trained to predict six transmitters (acetylcholine, glutamate, GABA, serotonin, dopamine, octopamine) achieves an accuracy of 87% for individual synapses, 94% for neurons, and 91% for known cell types across a D. melanogaster whole brain. We visualize the ultrastructural features used for prediction, discovering subtle but significant differences between transmitter phenotypes. We also analyze transmitter distributions across the brain and find that neurons that develop together largely express only one fast-acting transmitter (acetylcholine, glutamate, or GABA). We hope that our publicly available predictions act as an accelerant for neuroscientific hypothesis generation for the fly.

The role of fruitless in specifying courtship behaviors across divergent Drosophila species.

Sex-specific behaviors are critical for reproduction and species survival. The sex-specifically spliced transcription factor fruitless (fru) helps establish male courtship behaviors in invertebrates. Forcing male-specific fru (fruM) splicing in Drosophila melanogaster females produces male-typical behaviors while disrupting female-specific behaviors. However, whether fru's joint role in specifying male and inhibiting female behaviors is conserved across species is unknown. We used CRISPR-Cas9 to force FruM expression in female Drosophila virilis, a species in which males and females produce sex-specific songs. In contrast to D. melanogaster, in which one fruM allele is sufficient to generate male behaviors in females, two alleles are needed in D. virilis females. D. virilis females expressing FruM maintain the ability to sing female-typical song as well as lay eggs, whereas D. melanogaster FruM females cannot lay eggs. These results reveal potential differences in fru function between divergent species and underscore the importance of studying diverse behaviors and species for understanding the genetic basis of sex differences.

Inferring neural dynamics of memory during naturalistic social communication.

Memory processes in complex behaviors like social communication require forming representations of the past that grow with time. The neural mechanisms that support such continually growing memory remain unknown. We address this gap in the context of fly courtship, a natural social behavior involving the production and perception of long, complex song sequences. To study female memory for male song history in unrestrained courtship, we present 'Natural Continuation' (NC)-a general, simulation-based model comparison procedure to evaluate candidate neural codes for complex stimuli using naturalistic behavioral data. Applying NC to fly courtship revealed strong evidence for an adaptive population mechanism for how female auditory neural dynamics could convert long song histories into a rich mnemonic format. Song temporal patterning is continually transformed by heterogeneous nonlinear adaptation dynamics, then integrated into persistent activity, enabling common neural mechanisms to retain continuously unfolding information over long periods and yielding state-of-the-art predictions of female courtship behavior. At a population level this coding model produces multi-dimensional advection-diffusion-like responses that separate songs over a continuum of timescales and can be linearly transformed into flexible output signals, illustrating its potential to create a generic, scalable mnemonic format for extended input signals poised to drive complex behavioral responses. This work thus shows how naturalistic behavior can directly inform neural population coding models, revealing here a novel process for memory formation.

From connectome to effectome: learning the causal interaction map of the fly brain.

A long-standing goal of neuroscience is to obtain a causal model of the nervous system. This would allow neuroscientists to explain animal behavior in terms of the dynamic interactions between neurons. The recently reported whole-brain fly connectome [1-7] specifies the synaptic paths by which neurons can affect each other but not whether, or how, they do affect each other in vivo. To overcome this limitation, we introduce a novel combined experimental and statistical strategy for efficiently learning a causal model of the fly brain, which we refer to as the "effectome". Specifically, we propose an estimator for a dynamical systems model of the fly brain that uses stochastic optogenetic perturbation data to accurately estimate causal effects and the connectome as a prior to drastically improve estimation efficiency. We then analyze the connectome to propose circuits that have the greatest total effect on the dynamics of the fly nervous system. We discover that, fortunately, the dominant circuits significantly involve only relatively small populations of neurons-thus imaging, stimulation, and neuronal identification are feasible. Intriguingly, we find that this approach also re-discovers known circuits and generates testable hypotheses about their dynamics. Overall, our analyses of the connectome provide evidence that global dynamics of the fly brain are generated by a large collection of small and often anatomically localized circuits operating, largely, independently of each other. This in turn implies that a causal model of a brain, a principal goal of systems neuroscience, can be feasibly obtained in the fly.

Network Statistics of the Whole-Brain Connectome of Drosophila.

Brains comprise complex networks of neurons and connections. Network analysis applied to the wiring diagrams of brains can offer insights into how brains support computations and regulate information flow. The completion of the first whole-brain connectome of an adult Drosophila, the largest connectome to date, containing 130,000 neurons and millions of connections, offers an unprecedented opportunity to analyze its network properties and topological features. To gain insights into local connectivity, we computed the prevalence of two- and three-node network motifs, examined their strengths and neurotransmitter compositions, and compared these topological metrics with wiring diagrams of other animals. We discovered that the network of the fly brain displays rich club organization, with a large population (30% percent of the connectome) of highly connected neurons. We identified subsets of rich club neurons that may serve as integrators or broadcasters of signals. Finally, we examined subnetworks based on 78 anatomically defined brain regions or neuropils. These data products are shared within the FlyWire Codex and will serve as a foundation for models and experiments exploring the relationship between neural activity and anatomical structure.

Transcriptional profiling of Drosophila male-specific P1 (pC1) neurons.

In Drosophila melanogaster, the P1 (pC1) cluster of male-specific neurons both integrates sensory cues and drives or modulates behavioral programs such as courtship, in addition to contributing to a social arousal state. The behavioral function of these neurons is linked to the genes they express, which underpin their capacity for synaptic signaling, neuromodulation, and physiology. Yet, P1 (pC1) neurons have not been fully characterized at the transcriptome level. Moreover, it is unknown how the molecular landscape of P1 (pC1) neurons acutely changes after flies engage in social behaviors, where baseline P1 (pC1) neural activity is expected to increase. To address these two gaps, we use single cell-type RNA sequencing to profile and compare the transcriptomes of P1 (pC1) neurons harvested from socially paired versus solitary male flies. Compared to control transcriptome datasets, we find that P1 (pC1) neurons are enriched in 2,665 genes, including those encoding receptors, neuropeptides, and cell-adhesion molecules (dprs/DIPs). Furthermore, courtship is characterized by changes in ~300 genes, including those previously implicated in regulating behavior (e.g. DopEcR, Octß3R, Fife, kairos, rad). Finally, we identify a suite of genes that link conspecific courtship with the innate immune system. Together, these data serve as a molecular map for future studies of an important set of higher-order and sexually-dimorphic neurons.

Neuronal "parts list" and wiring diagram for a visual system.

A catalog of neuronal cell types has often been called a "parts list" of the brain, and regarded as a prerequisite for understanding brain function. In the optic lobe of Drosophila, rules of connectivity between cell types have already proven essential for understanding fly vision. Here we analyze the fly connectome to complete the list of cell types intrinsic to the optic lobe, as well as the rules governing their connectivity. We more than double the list of known types. Most new cell types contain between 10 and 100 cells, and integrate information over medium distances in the visual field. Some existing type families (transmedullary, lobula intrinsic, and lobula plate intrinsic) at least double in number of types, with implications for perception of color, motion, and form. We introduce a new family, serpentine medulla intrinsic, which has more types than any other, and three new families of types that span multiple neuropils. We demonstrate self-consistency of our cell types through automatic assignment of cells by distance in high-dimensional feature space, and provide further validation by selection of small subsets of discriminative features. Our work showcases the advantages of connectomic cell typing: complete and unbiased sampling, a rich array of features based on connectivity, and reduction of the connectome to a drastically simpler wiring diagram of cell types, with immediate relevance for brain function and development.

Flexible circuit mechanisms for context-dependent song sequencing.

Sequenced behaviours, including locomotion, reaching and vocalization, are patterned differently in different contexts, enabling animals to adjust to their environments. How contextual information shapes neural activity to flexibly alter the patterning of actions is not fully understood. Previous work has indicated that this could be achieved via parallel motor circuits, with differing sensitivities to context1,2. Here we demonstrate that a single pathway operates in two regimes dependent on recent sensory history. We leverage the Drosophila song production system3 to investigate the role of several neuron types4-7 in song patterning near versus far from the female fly. Male flies sing 'simple' trains of only one mode far from the female fly but complex song sequences comprising alternations between modes when near her. We find that ventral nerve cord (VNC) circuits are shaped by mutual inhibition and rebound excitability8 between nodes driving the two song modes. Brief sensory input to a direct brain-to-VNC excitatory pathway drives simple song far from the female, whereas prolonged input enables complex song production via simultaneous recruitment of functional disinhibition of VNC circuitry. Thus, female proximity unlocks motor circuit dynamics in the correct context. We construct a compact circuit model to demonstrate that the identified mechanisms suffice to replicate natural song dynamics. These results highlight how canonical circuit motifs8,9 can be combined to enable circuit flexibility required for dynamic communication.

Whole-brain annotation and multi-connectome cell typing quantifies circuit stereotypy in Drosophila.

The fruit fly Drosophila melanogaster combines surprisingly sophisticated behaviour with a highly tractable nervous system. A large part of the fly's success as a model organism in modern neuroscience stems from the concentration of collaboratively generated molecular genetic and digital resources. As presented in our FlyWire companion paper 1 , this now includes the first full brain connectome of an adult animal. Here we report the systematic and hierarchical annotation of this ~130,000-neuron connectome including neuronal classes, cell types and developmental units (hemilineages). This enables any researcher to navigate this huge dataset and find systems and neurons of interest, linked to the literature through the Virtual Fly Brain database 2 . Crucially, this resource includes 4,552 cell types. 3,094 are rigorous consensus validations of cell types previously proposed in the hemibrain connectome 3 . In addition, we propose 1,458 new cell types, arising mostly from the fact that the FlyWire connectome spans the whole brain, whereas the hemibrain derives from a subvolume. Comparison of FlyWire and the hemibrain showed that cell type counts and strong connections were largely stable, but connection weights were surprisingly variable within and across animals. Further analysis defined simple heuristics for connectome interpretation: connections stronger than 10 unitary synapses or providing >1% of the input to a target cell are highly conserved. Some cell types showed increased variability across connectomes: the most common cell type in the mushroom body, required for learning and memory, is almost twice as numerous in FlyWire as the hemibrain. We find evidence for functional homeostasis through adjustments of the absolute amount of excitatory input while maintaining the excitation-inhibition ratio. Finally, and surprisingly, about one third of the cell types proposed in the hemibrain connectome could not yet be reliably identified in the FlyWire connectome. We therefore suggest that cell types should be defined to be robust to inter-individual variation, namely as groups of cells that are quantitatively more similar to cells in a different brain than to any other cell in the same brain. Joint analysis of the FlyWire and hemibrain connectomes demonstrates the viability and utility of this new definition. Our work defines a consensus cell type atlas for the fly brain and provides both an intellectual framework and open source toolchain for brain-scale comparative connectomics.

Neuronal wiring diagram of an adult brain.

Connections between neurons can be mapped by acquiring and analyzing electron microscopic (EM) brain images. In recent years, this approach has been applied to chunks of brains to reconstruct local connectivity maps that are highly informative, yet inadequate for understanding brain function more globally. Here, we present the first neuronal wiring diagram of a whole adult brain, containing 5×107 chemical synapses between ~130,000 neurons reconstructed from a female Drosophila melanogaster. The resource also incorporates annotations of cell classes and types, nerves, hemilineages, and predictions of neurotransmitter identities. Data products are available by download, programmatic access, and interactive browsing and made interoperable with other fly data resources. We show how to derive a projectome, a map of projections between regions, from the connectome. We demonstrate the tracing of synaptic pathways and the analysis of information flow from inputs (sensory and ascending neurons) to outputs (motor, endocrine, and descending neurons), across both hemispheres, and between the central brain and the optic lobes. Tracing from a subset of photoreceptors all the way to descending motor pathways illustrates how structure can uncover putative circuit mechanisms underlying sensorimotor behaviors. The technologies and open ecosystem of the FlyWire Consortium set the stage for future large-scale connectome projects in other species.

BIFROST: a method for registering diverse imaging datasets.

Quantitative comparison of brain-wide neural dynamics across different experimental conditions often requires precise alignment to a common set of anatomical coordinates. While such approaches are routinely applied in functional magnetic resonance imaging (fMRI), registering in vivo fluorescence imaging data to ex vivo-derived reference atlases is challenging, given the many differences in imaging modality, microscope specification, and sample preparation. Moreover, in many systems, animal to animal variation in brain structure limits registration precision. Using the highly stereotyped architecture of the fruit fly brain as a model, we overcome these challenges by building a reference atlas based directly on in vivo multiphoton-imaged brains, called the Functional Drosophila Atlas (FDA). We then develop a novel two-step pipeline, BrIdge For Registering Over Statistical Templates (BIFROST), for transforming neural imaging data into this common space, and for importing ex vivo resources, such as connectomes. Using genetically labeled cell types to provide ground truth, we demonstrate that this method allows voxel registration with micron precision. Thus, this method provides a generalizable pipeline for registering neural activity datasets to one another, allowing quantitative comparisons across experiments, microscopes, genotypes, and anatomical atlases, including connectomes.

A leaky integrate-and-fire computational model based on the connectome of the entire adult Drosophila brain reveals insights into sensorimotor processing.

The forthcoming assembly of the adult Drosophila melanogaster central brain connectome, containing over 125,000 neurons and 50 million synaptic connections, provides a template for examining sensory processing throughout the brain. Here, we create a leaky integrate-and-fire computational model of the entire Drosophila brain, based on neural connectivity and neurotransmitter identity, to study circuit properties of feeding and grooming behaviors. We show that activation of sugar-sensing or water-sensing gustatory neurons in the computational model accurately predicts neurons that respond to tastes and are required for feeding initiation. Computational activation of neurons in the feeding region of the Drosophila brain predicts those that elicit motor neuron firing, a testable hypothesis that we validate by optogenetic activation and behavioral studies. Moreover, computational activation of different classes of gustatory neurons makes accurate predictions of how multiple taste modalities interact, providing circuit-level insight into aversive and appetitive taste processing. Our computational model predicts that the sugar and water pathways form a partially shared appetitive feeding initiation pathway, which our calcium imaging and behavioral experiments confirm. Additionally, we applied this model to mechanosensory circuits and found that computational activation of mechanosensory neurons predicts activation of a small set of neurons comprising the antennal grooming circuit that do not overlap with gustatory circuits, and accurately describes the circuit response upon activation of different mechanosensory subtypes. Our results demonstrate that modeling brain circuits purely from connectivity and predicted neurotransmitter identity generates experimentally testable hypotheses and can accurately describe complete sensorimotor transformations.

Neural network organization for courtship-song feature detection in Drosophila.

Animals communicate using sounds in a wide range of contexts, and auditory systems must encode behaviorally relevant acoustic features to drive appropriate reactions. How feature detection emerges along auditory pathways has been difficult to solve due to challenges in mapping the underlying circuits and characterizing responses to behaviorally relevant features. Here, we study auditory activity in the Drosophila melanogaster brain and investigate feature selectivity for the two main modes of fly courtship song, sinusoids and pulse trains. We identify 24 new cell types of the intermediate layers of the auditory pathway, and using a new connectomic resource, FlyWire, we map all synaptic connections between these cell types, in addition to connections to known early and higher-order auditory neurons-this represents the first circuit-level map of the auditory pathway. We additionally determine the sign (excitatory or inhibitory) of most synapses in this auditory connectome. We find that auditory neurons display a continuum of preferences for courtship song modes and that neurons with different song-mode preferences and response timescales are highly interconnected in a network that lacks hierarchical structure. Nonetheless, we find that the response properties of individual cell types within the connectome are predictable from their inputs. Our study thus provides new insights into the organization of auditory coding within the Drosophila brain.

SLEAP: A deep learning system for multi-animal pose tracking.

The desire to understand how the brain generates and patterns behavior has driven rapid methodological innovation in tools to quantify natural animal behavior. While advances in deep learning and computer vision have enabled markerless pose estimation in individual animals, extending these to multiple animals presents unique challenges for studies of social behaviors or animals in their natural environments. Here we present Social LEAP Estimates Animal Poses (SLEAP), a machine learning system for multi-animal pose tracking. This system enables versatile workflows for data labeling, model training and inference on previously unseen data. SLEAP features an accessible graphical user interface, a standardized data model, a reproducible configuration system, over 30 model architectures, two approaches to part grouping and two approaches to identity tracking. We applied SLEAP to seven datasets across flies, bees, mice and gerbils to systematically evaluate each approach and architecture, and we compare it with other existing approaches. SLEAP achieves greater accuracy and speeds of more than 800 frames per second, with latencies of less than 3.5 ms at full 1,024 × 1,024 image resolution. This makes SLEAP usable for real-time applications, which we demonstrate by controlling the behavior of one animal on the basis of the tracking and detection of social interactions with another animal.

FlyWire: online community for whole-brain connectomics.

Due to advances in automated image acquisition and analysis, whole-brain connectomes with 100,000 or more neurons are on the horizon. Proofreading of whole-brain automated reconstructions will require many person-years of effort, due to the huge volumes of data involved. Here we present FlyWire, an online community for proofreading neural circuits in a Drosophila melanogaster brain and explain how its computational and social structures are organized to scale up to whole-brain connectomics. Browser-based three-dimensional interactive segmentation by collaborative editing of a spatially chunked supervoxel graph makes it possible to distribute proofreading to individuals located virtually anywhere in the world. Information in the edit history is programmatically accessible for a variety of uses such as estimating proofreading accuracy or building incentive systems. An open community accelerates proofreading by recruiting more participants and accelerates scientific discovery by requiring information sharing. We demonstrate how FlyWire enables circuit analysis by reconstructing and analyzing the connectome of mechanosensory neurons.

Auditory activity is diverse and widespread throughout the central brain of Drosophila.

Sensory pathways are typically studied by starting at receptor neurons and following postsynaptic neurons into the brain. However, this leads to a bias in analyses of activity toward the earliest layers of processing. Here, we present new methods for volumetric neural imaging with precise across-brain registration to characterize auditory activity throughout the entire central brain of Drosophila and make comparisons across trials, individuals and sexes. We discover that auditory activity is present in most central brain regions and in neurons responsive to other modalities. Auditory responses are temporally diverse, but the majority of activity is tuned to courtship song features. Auditory responses are stereotyped across trials and animals in early mechanosensory regions, becoming more variable at higher layers of the putative pathway, and this variability is largely independent of ongoing movements. This study highlights the power of using an unbiased, brain-wide approach for mapping the functional organization of sensory activity.