Divisive normalization processors in the early visual system of the Drosophila brain.

Divisive normalization is a model of canonical computation of brain circuits. We demonstrate that two cascaded divisive normalization processors (DNPs), carrying out intensity/contrast gain control and elementary motion detection, respectively, can model the robust motion detection realized by the early visual system of the fruit fly. We first introduce a model of elementary motion detection and rewrite its underlying phase-based motion detection algorithm as a feedforward divisive normalization processor. We then cascade the DNP modeling the photoreceptor/amacrine cell layer with the motion detection DNP. We extensively evaluate the DNP for motion detection in dynamic environments where light intensity varies by orders of magnitude. The results are compared to other bio-inspired motion detectors as well as state-of-the-art optic flow algorithms under natural conditions. Our results demonstrate the potential of DNPs as canonical building blocks modeling the analog processing of early visual systems. The model highlights analog processing for accurately detecting visual motion, in both vertebrates and invertebrates. The results presented here shed new light on employing DNP-based algorithms in computer vision.

The functional logic of odor information processing in the Drosophila antennal lobe.

Recent advances in molecular transduction of odorants in the Olfactory Sensory Neurons (OSNs) of the Drosophila Antenna have shown that the odorant object identity is multiplicatively coupled with the odorant concentration waveform. The resulting combinatorial neural code is a confounding representation of odorant semantic information (identity) and syntactic information (concentration). To distill the functional logic of odor information processing in the Antennal Lobe (AL) a number of challenges need to be addressed including 1) how is the odorant semantic information decoupled from the syntactic information at the level of the AL, 2) how are these two information streams processed by the diverse AL Local Neurons (LNs) and 3) what is the end-to-end functional logic of the AL? By analyzing single-channel physiology recordings at the output of the AL, we found that the Projection Neuron responses can be decomposed into a concentration-invariant component, and two transient components boosting the positive/negative concentration contrast that indicate onset/offset timing information of the odorant object. We hypothesized that the concentration-invariant component, in the multi-channel context, is the recovered odorant identity vector presented between onset/offset timing events. We developed a model of LN pathways in the Antennal Lobe termed the differential Divisive Normalization Processors (DNPs), which robustly extract the semantics (the identity of the odorant object) and the ON/OFF semantic timing events indicating the presence/absence of an odorant object. For real-time processing with spiking PN models, we showed that the phase-space of the biological spike generator of the PN offers an intuit perspective for the representation of recovered odorant semantics and examined the dynamics induced by the odorant semantic timing events. Finally, we provided theoretical and computational evidence for the functional logic of the AL as a robust ON-OFF odorant object identity recovery processor across odorant identities, concentration amplitudes and waveform profiles.

A Programmable Ontology Encompassing the Functional Logic of the Drosophila Brain.

The Drosophila brain has only a fraction of the number of neurons of higher organisms such as mice and humans. Yet the sheer complexity of its neural circuits recently revealed by large connectomics datasets suggests that computationally modeling the function of fruit fly brain circuits at this scale poses significant challenges. To address these challenges, we present here a programmable ontology that expands the scope of the current Drosophila brain anatomy ontologies to encompass the functional logic of the fly brain. The programmable ontology provides a language not only for modeling circuit motifs but also for programmatically exploring their functional logic. To achieve this goal, we tightly integrated the programmable ontology with the workflow of the interactive FlyBrainLab computing platform. As part of the programmable ontology, we developed NeuroNLP++, a web application that supports free-form English queries for constructing functional brain circuits fully anchored on the available connectome/synaptome datasets, and the published worldwide literature. In addition, we present a methodology for including a model of the space of odorants into the programmable ontology, and for modeling olfactory sensory circuits of the antenna of the fruit fly brain that detect odorant sources. Furthermore, we describe a methodology for modeling the functional logic of the antennal lobe circuit consisting of a massive number of local feedback loops, a characteristic feature observed across Drosophila brain regions. Finally, using a circuit library, we demonstrate the power of our methodology for interactively exploring the functional logic of the massive number of feedback loops in the antennal lobe.

Connectomic analysis of the Drosophila lateral neuron clock cells reveals the synaptic basis of functional pacemaker classes.

The circadian clock orchestrates daily changes in physiology and behavior to ensure internal temporal order and optimal timing across the day. In animals, a central brain clock coordinates circadian rhythms throughout the body and is characterized by a remarkable robustness that depends on synaptic connections between constituent neurons. The clock neuron network of Drosophila, which shares network motifs with clock networks in the mammalian brain yet is built of many fewer neurons, offers a powerful model for understanding the network properties of circadian timekeeping. Here, we report an assessment of synaptic connectivity within a clock network, focusing on the critical lateral neuron (LN) clock neuron classes within the Janelia hemibrain dataset. Our results reveal that previously identified anatomical and functional subclasses of LNs represent distinct connectomic types. Moreover, we identify a small number of non-clock cell subtypes representing highly synaptically coupled nodes within the clock neuron network. This suggests that neurons lacking molecular timekeeping likely play integral roles within the circadian timekeeping network. To our knowledge, this represents the first comprehensive connectomic analysis of a circadian neuronal network.

Most organisms on Earth possess an internal timekeeping system which ensures that bodily processes such as sleep, wakefulness or digestion take place at the right time. These precise daily rhythms are kept in check by a master clock in the brain. There, thousands of neurons ­ some of which carrying an internal 'molecular clock' ­ connect to each other through structures known as synapses. Exactly how the resulting network is organised to support circadian timekeeping remains unclear. To explore this question, Shafer, Gutierrez et al. focused on fruit flies, as recent efforts have systematically mapped every neuron and synaptic connection in the brain of this model organism. Analysing available data from the hemibrain connectome project at Janelia revealed that that the neurons with the most important timekeeping roles were in fact forming the fewest synapses within the network. In addition, neurons without internal molecular clocks mediated strong synaptic connections between those that did, suggesting that 'clockless' cells still play an integral role in circadian timekeeping. With this research, Shafer, Gutierrez et al. provide unexpected insights into the organisation of the master body clock. Better understanding the networks that underpin circadian rhythms will help to grasp how and why these are disrupted in obesity, depression and Alzheimer's disease.

Accelerating with FlyBrainLab the discovery of the functional logic of the Drosophila brain in the connectomic and synaptomic era.

In recent years, a wealth of Drosophila neuroscience data have become available including cell type and connectome/synaptome datasets for both the larva and adult fly. To facilitate integration across data modalities and to accelerate the understanding of the functional logic of the fruit fly brain, we have developed FlyBrainLab, a unique open-source computing platform that integrates 3D exploration and visualization of diverse datasets with interactive exploration of the functional logic of modeled executable brain circuits. FlyBrainLab's User Interface, Utilities Libraries and Circuit Libraries bring together neuroanatomical, neurogenetic and electrophysiological datasets with computational models of different researchers for validation and comparison within the same platform. Seeking to transcend the limitations of the connectome/synaptome, FlyBrainLab also provides libraries for molecular transduction arising in sensory coding in vision/olfaction. Together with sensory neuron activity data, these libraries serve as entry points for the exploration, analysis, comparison, and evaluation of circuit functions of the fruit fly brain.

A molecular odorant transduction model and the complexity of spatio-temporal encoding in the Drosophila antenna.

Over the past two decades, substantial amount of work has been conducted to characterize different odorant receptors, neuroanatomy and odorant response properties of the early olfactory system of Drosophila melanogaster. Yet many odorant receptors remain only partially characterized, and the odorant transduction process and the axon hillock spiking mechanism of the olfactory sensory neurons (OSNs) have yet to be fully determined. Identity and concentration, two key characteristics of the space of odorants, are encoded by the odorant transduction process. Detailed molecular models of the odorant transduction process are, however, scarce for fruit flies. To address these challenges we advance a comprehensive model of fruit fly OSNs as a cascade consisting of an odorant transduction process (OTP) and a biophysical spike generator (BSG). We model odorant identity and concentration using an odorant-receptor binding rate tensor, modulated by the odorant concentration profile, and an odorant-receptor dissociation rate tensor, and quantitatively describe the mechanics of the molecular ligand binding/dissociation of the OTP. We model the BSG as a Connor-Stevens point neuron. The resulting spatio-temporal encoding model of the Drosophila antenna provides a theoretical foundation for understanding the neural code of both odorant identity and odorant concentration and advances the state-of-the-art in a number of ways. First, it quantifies on the molecular level the spatio-temporal level of complexity of the transformation taking place in the antennae. The concentration-dependent spatio-temporal code at the output of the antenna circuits determines the level of complexity of olfactory processing in the downstream neuropils, such as odorant recognition and olfactory associative learning. Second, the model is biologically validated using multiple electrophysiological recordings. Third, the model demonstrates that the currently available data for odorant-receptor responses only enable the estimation of the affinity of the odorant-receptor pairs. The odorant-dissociation rate is only available for a few odorant-receptor pairs. Finally, our model calls for new experiments for massively identifying the odorant-receptor dissociation rates of relevance to flies.

Sparse identification of contrast gain control in the fruit fly photoreceptor and amacrine cell layer.

The fruit fly's natural visual environment is often characterized by light intensities ranging across several orders of magnitude and by rapidly varying contrast across space and time. Fruit fly photoreceptors robustly transduce and, in conjunction with amacrine cells, process visual scenes and provide the resulting signal to downstream targets. Here, we model the first step of visual processing in the photoreceptor-amacrine cell layer. We propose a novel divisive normalization processor (DNP) for modeling the computation taking place in the photoreceptor-amacrine cell layer. The DNP explicitly models the photoreceptor feedforward and temporal feedback processing paths and the spatio-temporal feedback path of the amacrine cells. We then formally characterize the contrast gain control of the DNP and provide sparse identification algorithms that can efficiently identify each the feedforward and feedback DNP components. The algorithms presented here are the first demonstration of tractable and robust identification of the components of a divisive normalization processor. The sparse identification algorithms can be readily employed in experimental settings, and their effectiveness is demonstrated with several examples.

Sparse Functional Identification of Complex Cells from Spike Times and the Decoding of Visual Stimuli.

We investigate the sparse functional identification of complex cells and the decoding of spatio-temporal visual stimuli encoded by an ensemble of complex cells. The reconstruction algorithm is formulated as a rank minimization problem that significantly reduces the number of sampling measurements (spikes) required for decoding. We also establish the duality between sparse decoding and functional identification and provide algorithms for identification of low-rank dendritic stimulus processors. The duality enables us to efficiently evaluate our functional identification algorithms by reconstructing novel stimuli in the input space. Finally, we demonstrate that our identification algorithms substantially outperform the generalized quadratic model, the nonlinear input model, and the widely used spike-triggered covariance algorithm.

Generating Executable Models of the Drosophila Central Complex.

The central complex (CX) is a set of neuropils in the center of the fly brain that have been implicated as playing an important role in vision-mediated behavior and integration of spatial information with locomotor control. In contrast to currently available data regarding the neural circuitry of neuropils in the fly's vision and olfactory systems, comparable data for the CX neuropils is relatively incomplete; many categories of neurons remain only partly characterized, and the synaptic connectivity between CX neurons has yet to be fully determined. Successful modeling of the information processing functions of the CX neuropils therefore requires a means of easily constructing and testing a range of hypotheses regarding both the high-level structure of their neural circuitry and the properties of their constituent neurons and synapses. To this end, we have created a web application that enables simultaneous graphical querying and construction of executable models of the CX neural circuitry based upon currently available information regarding the geometry and polarity of the arborizations of identified local and projection neurons in the CX. The application's novel functionality is made possible by the Fruit Fly Brain Observatory, a platform for collaborative study and development of fruit fly brain models.

A Motion Detection Algorithm Using Local Phase Information.

Previous research demonstrated that global phase alone can be used to faithfully represent visual scenes. Here we provide a reconstruction algorithm by using only local phase information. We also demonstrate that local phase alone can be effectively used to detect local motion. The local phase-based motion detector is akin to models employed to detect motion in biological vision, for example, the Reichardt detector. The local phase-based motion detection algorithm introduced here consists of two building blocks. The first building block measures/evaluates the temporal change of the local phase. The temporal derivative of the local phase is shown to exhibit the structure of a second order Volterra kernel with two normalized inputs. We provide an efficient, FFT-based algorithm for implementing the change of the local phase. The second processing building block implements the detector; it compares the maximum of the Radon transform of the local phase derivative with a chosen threshold. We demonstrate examples of applying the local phase-based motion detection algorithm on several video sequences. We also show how the locally detected motion can be used for segmenting moving objects in video scenes and compare our local phase-based algorithm to segmentation achieved with a widely used optic flow algorithm.

Neurokernel: An Open Source Platform for Emulating the Fruit Fly Brain.

We have developed an open software platform called Neurokernel for collaborative development of comprehensive models of the brain of the fruit fly Drosophila melanogaster and their execution and testing on multiple Graphics Processing Units (GPUs). Neurokernel provides a programming model that capitalizes upon the structural organization of the fly brain into a fixed number of functional modules to distinguish between these modules' local information processing capabilities and the connectivity patterns that link them. By defining mandatory communication interfaces that specify how data is transmitted between models of each of these modules regardless of their internal design, Neurokernel explicitly enables multiple researchers to collaboratively model the fruit fly's entire brain by integration of their independently developed models of its constituent processing units. We demonstrate the power of Neurokernel's model integration by combining independently developed models of the retina and lamina neuropils in the fly's visual system and by demonstrating their neuroinformation processing capability. We also illustrate Neurokernel's ability to take advantage of direct GPU-to-GPU data transfers with benchmarks that demonstrate scaling of Neurokernel's communication performance both over the number of interface ports exposed by an emulation's constituent modules and the total number of modules comprised by an emulation.

Projection neurons in Drosophila antennal lobes signal the acceleration of odor concentrations.

Temporal experience of odor gradients is important in spatial orientation of animals. The fruit fly Drosophila melanogaster exhibits robust odor-guided behaviors in an odor gradient field. In order to investigate how early olfactory circuits process temporal variation of olfactory stimuli, we subjected flies to precisely defined odor concentration waveforms and examined spike patterns of olfactory sensory neurons (OSNs) and projection neurons (PNs). We found a significant temporal transformation between OSN and PN spike patterns, manifested by the PN output strongly signaling the OSN spike rate and its rate of change. A simple two-dimensional model admitting the OSN spike rate and its rate of change as inputs closely predicted the PN output. When cascaded with the rate-of-change encoding by OSNs, PNs primarily signal the acceleration and the rate of change of dynamic odor stimuli to higher brain centers, thereby enabling animals to reliably respond to the onsets of odor concentrations.

Massively parallel neural circuits for stereoscopic color vision: encoding, decoding and identification.

Past work demonstrated how monochromatic visual stimuli could be faithfully encoded and decoded under Nyquist-type rate conditions. Color visual stimuli were then traditionally encoded and decoded in multiple separate monochromatic channels. The brain, however, appears to mix information about color channels at the earliest stages of the visual system, including the retina itself. If information about color is mixed and encoded by a common pool of neurons, how can colors be demixed and perceived? We present Color Video Time Encoding Machines (Color Video TEMs) for encoding color visual stimuli that take into account a variety of color representations within a single neural circuit. We then derive a Color Video Time Decoding Machine (Color Video TDM) algorithm for color demixing and reconstruction of color visual scenes from spikes produced by a population of visual neurons. In addition, we formulate Color Video Channel Identification Machines (Color Video CIMs) for functionally identifying color visual processing performed by a spiking neural circuit. Furthermore, we derive a duality between TDMs and CIMs that unifies the two and leads to a general theory of neural information representation for stereoscopic color vision. We provide examples demonstrating that a massively parallel color visual neural circuit can be first identified with arbitrary precision and its spike trains can be subsequently used to reconstruct the encoded stimuli. We argue that evaluation of the functional identification methodology can be effectively and intuitively performed in the stimulus space. In this space, a signal reconstructed from spike trains generated by the identified neural circuit can be compared to the original stimulus.

Spiking neural circuits with dendritic stimulus processors : encoding, decoding, and identification in reproducing kernel Hilbert spaces.

We present a multi-input multi-output neural circuit architecture for nonlinear processing and encoding of stimuli in the spike domain. In this architecture a bank of dendritic stimulus processors implements nonlinear transformations of multiple temporal or spatio-temporal signals such as spike trains or auditory and visual stimuli in the analog domain. Dendritic stimulus processors may act on both individual stimuli and on groups of stimuli, thereby executing complex computations that arise as a result of interactions between concurrently received signals. The results of the analog-domain computations are then encoded into a multi-dimensional spike train by a population of spiking neurons modeled as nonlinear dynamical systems. We investigate general conditions under which such circuits faithfully represent stimuli and demonstrate algorithms for (i) stimulus recovery, or decoding, and (ii) identification of dendritic stimulus processors from the observed spikes. Taken together, our results demonstrate a fundamental duality between the identification of the dendritic stimulus processor of a single neuron and the decoding of stimuli encoded by a population of neurons with a bank of dendritic stimulus processors. This duality result enabled us to derive lower bounds on the number of experiments to be performed and the total number of spikes that need to be recorded for identifying a neural circuit.

Channel identification machines for multidimensional receptive fields.

We present algorithms for identifying multidimensional receptive fields directly from spike trains produced by biophysically-grounded neuron models. We demonstrate that only the projection of a receptive field onto the input stimulus space may be perfectly identified and derive conditions under which this identification is possible. We also provide detailed examples of identification of neural circuits incorporating spatiotemporal and spectrotemporal receptive fields.

Volterra dendritic stimulus processors and biophysical spike generators with intrinsic noise sources.

We consider a class of neural circuit models with internal noise sources arising in sensory systems. The basic neuron model in these circuits consists of a dendritic stimulus processor (DSP) cascaded with a biophysical spike generator (BSG). The dendritic stimulus processor is modeled as a set of nonlinear operators that are assumed to have a Volterra series representation. Biophysical point neuron models, such as the Hodgkin-Huxley neuron, are used to model the spike generator. We address the question of how intrinsic noise sources affect the precision in encoding and decoding of sensory stimuli and the functional identification of its sensory circuits. We investigate two intrinsic noise sources arising (i) in the active dendritic trees underlying the DSPs, and (ii) in the ion channels of the BSGs. Noise in dendritic stimulus processing arises from a combined effect of variability in synaptic transmission and dendritic interactions. Channel noise arises in the BSGs due to the fluctuation of the number of the active ion channels. Using a stochastic differential equations formalism we show that encoding with a neuron model consisting of a nonlinear DSP cascaded with a BSG with intrinsic noise sources can be treated as generalized sampling with noisy measurements. For single-input multi-output neural circuit models with feedforward, feedback and cross-feedback DSPs cascaded with BSGs we theoretically analyze the effect of noise sources on stimulus decoding. Building on a key duality property, the effect of noise parameters on the precision of the functional identification of the complete neural circuit with DSP/BSG neuron models is given. We demonstrate through extensive simulations the effects of noise on encoding stimuli with circuits that include neuron models that are akin to those commonly seen in sensory systems, e.g., complex cells in V1.

Functional identification of spike-processing neural circuits.

We introduce a novel approach for a complete functional identification of biophysical spike-processing neural circuits. The circuits considered accept multidimensional spike trains as their input and comprise a multitude of temporal receptive fields and conductance-based models of action potential generation. Each temporal receptive field describes the spatiotemporal contribution of all synapses between any two neurons and incorporates the (passive) processing carried out by the dendritic tree. The aggregate dendritic current produced by a multitude of temporal receptive fields is encoded into a sequence of action potentials by a spike generator modeled as a nonlinear dynamical system. Our approach builds on the observation that during any experiment, an entire neural circuit, including its receptive fields and biophysical spike generators, is projected onto the space of stimuli used to identify the circuit. Employing the reproducing kernel Hilbert space (RKHS) of trigonometric polynomials to describe input stimuli, we quantitatively describe the relationship between underlying circuit parameters and their projections. We also derive experimental conditions under which these projections converge to the true parameters. In doing so, we achieve the mathematical tractability needed to characterize the biophysical spike generator and identify the multitude of receptive fields. The algorithms obviate the need to repeat experiments in order to compute the neurons' rate of response, rendering our methodology of interest to both experimental and theoretical neuroscientists.

The power of connectivity: identity preserving transformations on visual streams in the spike domain.

We investigate neural architectures for identity preserving transformations (IPTs) on visual stimuli in the spike domain. The stimuli are encoded with a population of spiking neurons; the resulting spikes are processed and finally decoded. A number of IPTs are demonstrated including faithful stimulus recovery, as well as simple transformations on the original visual stimulus such as translations, rotations and zoomings. We show that if the set of receptive fields satisfies certain symmetry properties, then IPTs can easily be realized and additionally, the same basic stimulus decoding algorithm can be employed to recover the transformed input stimulus. Using group theoretic methods we advance two different neural encoding architectures and discuss the realization of exact and approximate IPTs. These are realized in the spike domain processing block by a "switching matrix" that regulates the input/output connectivity between the stimulus encoding and decoding blocks. For example, for a particular connectivity setting of the switching matrix, the original stimulus is faithfully recovered. For other settings, translations, rotations and dilations (or combinations of these operations) of the original video stream are obtained. We evaluate our theoretical derivations through extensive simulations on natural video scenes, and discuss implications of our results on the problem of invariant object recognition in the spike domain.

Channel identification machines.

We present a formal methodology for identifying a channel in a system consisting of a communication channel in cascade with an asynchronous sampler. The channel is modeled as a multidimensional filter, while models of asynchronous samplers are taken from neuroscience and communications and include integrate-and-fire neurons, asynchronous sigma/delta modulators and general oscillators in cascade with zero-crossing detectors. We devise channel identification algorithms that recover a projection of the filter(s) onto a space of input signals loss-free for both scalar and vector-valued test signals. The test signals are modeled as elements of a reproducing kernel Hilbert space (RKHS) with a Dirichlet kernel. Under appropriate limiting conditions on the bandwidth and the order of the test signal space, the filter projection converges to the impulse response of the filter. We show that our results hold for a wide class of RKHSs, including the space of finite-energy bandlimited signals. We also extend our channel identification results to noisy circuits.

Massively parallel neural encoding and decoding of visual stimuli.

The massively parallel nature of video Time Encoding Machines (TEMs) calls for scalable, massively parallel decoders that are implemented with neural components. The current generation of decoding algorithms is based on computing the pseudo-inverse of a matrix and does not satisfy these requirements. Here we consider video TEMs with an architecture built using Gabor receptive fields and a population of Integrate-and-Fire neurons. We show how to build a scalable architecture for video Time Decoding Machines using recurrent neural networks. Furthermore, we extend our architecture to handle the reconstruction of visual stimuli encoded with massively parallel video TEMs having neurons with random thresholds. Finally, we discuss in detail our algorithms and demonstrate their scalability and performance on a large scale GPU cluster.