A deep-learning strategy to identify cell types across species from high-density extracellular recordings.

High-density probes allow electrophysiological recordings from many neurons simultaneously across entire brain circuits but don't reveal cell type. Here, we develop a strategy to identify cell types from extracellular recordings in awake animals, revealing the computational roles of neurons with distinct functional, molecular, and anatomical properties. We combine optogenetic activation and pharmacology using the cerebellum as a testbed to generate a curated ground-truth library of electrophysiological properties for Purkinje cells, molecular layer interneurons, Golgi cells, and mossy fibers. We train a semi-supervised deep-learning classifier that predicts cell types with greater than 95% accuracy based on waveform, discharge statistics, and layer of the recorded neuron. The classifier's predictions agree with expert classification on recordings using different probes, in different laboratories, from functionally distinct cerebellar regions, and across animal species. Our classifier extends the power of modern dynamical systems analyses by revealing the unique contributions of simultaneously-recorded cell types during behavior.

A sensory-motor decoder that transforms neural responses in extrastriate area MT into smooth pursuit eye movements.

Visual motion drives smooth pursuit eye movements through a sensory-motor decoder that uses multiple parallel neural pathways to transform the population response in extrastriate area MT into movement. We evaluated the decoder by challenging pursuit in monkeys with reduced motion reliability created by reducing coherence of motion in patches of dots. Our strategy was to determine how reduced dot coherence changes the population response in MT. We then predicted the properties of a decoder that transforms the MT population response into dot coherence-induced deficits in the initiation of pursuit and steady-state tracking. During pursuit initiation, decreased dot coherence reduces MT population response amplitude without changing the preferred speed at its peak. The successful decoder reproduces the measured eye movements by multiplication of 1) the estimate of target speed from the peak of the population response with 2) visual-motor gain based on the amplitude of the population response. During steady-state tracking, the decoder that worked for pursuit initiation failed to reproduce the paradox that steady-state eye speeds do not accelerate to the target speed despite persistent image motion. It predicted eye acceleration to target speed even when monkeys' eye speeds were steady at well below the target speed. To account for the effect of dot coherence on steady-state eye speed, we postulate that the decoder uses sensory-motor gain to modulate the eye velocity positive feedback that normally sustains perfect steady-state tracking. Then, poor steady-state tracking persists because of balance between eye deceleration caused by low positive feedback gain and acceleration driven by MT.NEW & NOTEWORTHY By challenging a sensory-motor system with degraded sensory stimuli, we reveal how the sensory-motor decoder transforms the population response in extrastriate area MT into commands for the initiation and steady-state behavior of smooth pursuit eye movements. Conclusions are based on measuring population responses in MT for multiple target speeds and different levels of motion reliability and evaluating a decoder with a biologically motivated architecture to determine the decoder properties that create the measured eye movements.

Rate versus synchrony codes for cerebellar control of motor behavior.

Information transmission between neural populations could occur through either coordinated changes in firing rates or the precise transmission of spike timing. We investigate the code for information transmission from a part of the cerebellar cortex that is crucial for the accurate execution of a quantifiable motor behavior. Simultaneous recordings from Purkinje cell pairs in the cerebellum of rhesus macaques reveal how these cells coordinate their activity to drive smooth pursuit eye movements. Purkinje cells show millisecond-scale coordination of spikes (synchrony), but the level of synchrony is small and insufficient to impact the firing of downstream vestibular nucleus neurons. Analysis of previous metrics that purported to reveal Purkinje cell synchrony demonstrates that these metrics conflate changes in firing rate and neuron-neuron covariance. We conclude that the output of the cerebellar cortex uses primarily a rate rather than a synchrony code to drive the activity of downstream neurons and thus control motor behavior.

A sensory-motor decoder that transforms neural responses in extrastriate area MT into smooth pursuit eye movements.

Visual motion drives smooth pursuit eye movements through a sensory-motor decoder that uses multiple parallel components and neural pathways to transform the population response in extrastriate area MT into movement. We evaluated the decoder by challenging pursuit in monkeys with reduced motion reliability created by reducing coherence of motion in patches of dots. Reduced dot coherence caused deficits in both the initiation of pursuit and steady-state tracking, revealing the paradox of steady-state eye speeds that fail to accelerate to target speed in spite of persistent image motion. We recorded neural responses to reduced dot coherence in MT and found a decoder that transforms MT population responses into eye movements. During pursuit initiation, decreased dot coherence reduces MT population response amplitude without changing the preferred speed at the peak of the population response. The successful decoder reproduces the measured eye movements by multiplication of (i) the estimate of target speed from the peak of the population response with (ii) visual-motor gain based on the amplitude of the population response. During steady-state tracking, the decoder that worked for pursuit initiation failed. It predicted eye acceleration to target speed even when monkeys' eye speeds were steady at a level well below target speed. We can account for the effect of dot coherence on steady-state eye speed if sensorymotor gain also modulates the eye velocity positive feedback that normally sustains perfect steadystate tracking. Then, poor steady-state tracking persists because of balance between deceleration caused by low positive feedback gain and acceleration driven by MT.

Rate versus synchrony codes for cerebellar control of motor behavior.

Control of movement requires the coordination of multiple brain areas, each containing populations of neurons that receive inputs, process these inputs via recurrent dynamics, and then relay the processed information to downstream populations. Information transmission between neural populations could occur through either coordinated changes in firing rates or the precise transmission of spike timing. We investigate the nature of the code for transmission of signals to downstream areas from a part of the cerebellar cortex that is crucial for the accurate execution of a quantifiable motor behavior. Simultaneous recordings from Purkinje cell pairs in the cerebellar flocculus of rhesus macaques revealed how these cells coordinate their activity to drive smooth pursuit eye movements. Purkinje cells show millisecond-scale coordination of spikes (synchrony), but the level of synchrony is small and likely insufficient to impact the firing of downstream neurons in the vestibular nucleus. Further, analysis of previous metrics for assaying Purkinje cell synchrony demonstrates that these metrics conflate changes in firing rate and neuron-neuron covariance. We conclude that the output of the cerebellar cortex uses primarily a rate code rather than synchrony code to drive activity of downstream neurons and thus control motor behavior. Impact statement: Information transmission in the brain can occur via changes in firing rate or via the precise timing of spikes. Simultaneous recordings from pairs of Purkinje cells in the floccular complex reveals that information transmission out of the cerebellar cortex relies almost exclusively on changes in firing rates rather than millisecond-scale coordination of spike timing across the Purkinje cell population.

Toward a Biomimetic Neural Circuit Model of Sensory-Motor Processing.

Computational models have been a mainstay of research on smooth pursuit eye movements in monkeys. Pursuit is a sensory-motor system that is driven by the visual motion of small targets. It creates a smooth eye movement that accelerates up to target speed and tracks the moving target essentially perfectly. In this review of my laboratory's research, I trace the development of computational models of pursuit eye movements from the early control-theory models to the most recent neural circuit models. I outline a combined experimental and computational plan to move the models to the next level. Finally, I explain why research on nonhuman primates is so critical to the development of the neural circuit models I think we need.

Neural structure of a sensory decoder for motor control.

The transformation of sensory input to motor output is often conceived as a decoder operating on neural representations. We seek a mechanistic understanding of sensory decoding by mimicking neural circuitry in the decoder's design. The results of a simple experiment shape our approach. Changing the size of a target for smooth pursuit eye movements changes the relationship between the variance and mean of the evoked behavior in a way that contradicts the regime of "signal-dependent noise" and defies traditional decoding approaches. A theoretical analysis leads us to propose a circuit for pursuit that includes multiple parallel pathways and multiple sources of variation. Behavioral and neural responses with biomimetic statistics emerge from a biologically-motivated circuit model with noise in the pathway that is dedicated to flexibly adjusting the strength of visual-motor transmission. Our results demonstrate the power of re-imagining decoding as processing through the parallel pathways of neural systems.

Evaluation and resolution of many challenges of neural spike sorting: a new sorter.

We evaluate existing spike sorters and present a new one that resolves many sorting challenges. The new sorter, called "full binary pursuit" or FBP, comprises multiple steps. First, it thresholds and clusters to identify the waveforms of all unique neurons in the recording. Second, it uses greedy binary pursuit to optimally assign all the spike events in the original voltages to separable neurons. Third, it resolves spike events that are described more accurately as the superposition of spikes from two other neurons. Fourth, it resolves situations where the recorded neurons drift in amplitude or across electrode contacts during a long recording session. Comparison with other sorters on ground-truth data sets reveals many of the failure modes of spike sorting. We examine overall spike sorter performance in ground-truth data sets and suggest postsorting analyses that can improve the veracity of neural analyses by minimizing the intrusion of failure modes into analysis and interpretation of neural data. Our analysis reveals the tradeoff between the number of channels a sorter can process, speed of sorting, and some of the failure modes of spike sorting. FBP works best on data from 32 channels or fewer. It trades speed and number of channels for avoidance of specific failure modes that would be challenges for some use cases. We conclude that all spike sorting algorithms studied have advantages and shortcomings, and the appropriate use of a spike sorter requires a detailed assessment of the data being sorted and the experimental goals for analyses.NEW & NOTEWORTHY Electrophysiological recordings from multiple neurons across multiple channels pose great difficulty for spike sorting of single neurons. We propose methods that improve the ability to determine the number of individual neurons present in a recording and resolve near-simultaneous spike events from single neurons. We use ground-truth data sets to demonstrate the pros and cons of several current sorting algorithms and suggest strategies for determining the accuracy of spike sorting when ground-truth data are not available.

The Rules of Cerebellar Learning: Around the Ito Hypothesis.

As a tribute to Masao Ito, we propose a model of cerebellar learning that incorporates and extends his original model. We suggest four principles that align well with conclusions from multiple cerebellar learning systems. (1) Climbing fiber inputs to the cerebellum drive early, fast, poorly-retained learning in the parallel fiber to Purkinje cell synapse. (2) Learned Purkinje cell outputs drive late, slow, well-retained learning in non-Purkinje cell inputs to neurons in the cerebellar nucleus, transferring learning from the cortex to the nucleus. (3) Recurrent feedback from Purkinje cells to the inferior olive, through interneurons in the cerebellar nucleus, limits the magnitude of fast, early learning in the cerebellar cortex. (4) Functionally different inputs are subjected to plasticity in the cerebellar cortex versus the cerebellar nucleus. A computational neural circuit model that is based on these principles mimics a large amount of neural and behavioral data obtained from the smooth pursuit eye movements of monkeys.

Diversity and dynamism in the cerebellum.

The past several years have brought revelations and paradigm shifts in research on the cerebellum. Historically viewed as a simple sensorimotor controller with homogeneous architecture, the cerebellum is increasingly implicated in cognitive functions. It possesses an impressive diversity of molecular, cellular and circuit mechanisms, embedded in a dynamic, recurrent circuit architecture. Recent insights about the diversity and dynamism of the cerebellum provide a roadmap for the next decade of cerebellar research, challenging some old concepts, reinvigorating others and defining major new research directions.

Principles of operation of a cerebellar learning circuit.

We provide behavioral evidence using monkey smooth pursuit eye movements for four principles of cerebellar learning. Using a circuit-level model of the cerebellum, we link behavioral data to learning's neural implementation. The four principles are: (1) early, fast, acquisition driven by climbing fiber inputs to the cerebellar cortex, with poor retention; (2) learned responses of Purkinje cells guide transfer of learning from the cerebellar cortex to the deep cerebellar nucleus, with excellent retention; (3) functionally different neural signals are subject to learning in the cerebellar cortex versus the deep cerebellar nuclei; and (4) negative feedback from the cerebellum to the inferior olive reduces the magnitude of the teaching signal in climbing fibers and limits learning. Our circuit-level model, based on these four principles, explains behavioral data obtained by strategically manipulating the signals responsible for acquisition and recall of direction learning in smooth pursuit eye movements across multiple timescales.

The human brain can do many things, from reading and remembering the words written on a page to adapting and improving movements. When a movement misses its goal, the strength of the connections between cells in a part of the brain known as the cerebellum changes. The cerebellum is important for coordinating movements, including eye movements. When the connections between the cells in the cerebellum ­ known as neurons ­ strengthen or weaken, the cerebellum changes how it will respond in the future, leading to more accurate movements. However, the speed of the changes in the connections and how the connections between different neurons evolve and coordinate were unknown. Herzfeld et al. have now combined eye-tracking studies in monkeys with computer modeling based on what is known about the neural circuits in the cerebellum to learn more about the changes in these connections. Monkeys watched a moving target that would abruptly change direction. In the next movement, the eye-tracking equipment monitored how well the monkey's eyes anticipated the unexpected change in the target's direction ­ a form of motor learning. Using the experimental data, Herzfeld et al. produced a model that outlines general principles of how the cerebellum might manage this process. The model suggested that neurons in one region in the cerebellum, known as Purkinje cells, learn from mistakes quickly, but have poor long-term retention. If the movement is repeated, Purkinje cells teach another area of the cerebellum, the cerebellar nucleus, which takes longer to learn but has much better retention. Although these findings are based on a simple motor learning task, they are the first step to understanding how the brain forms memories and how we might learn more complex behaviors.

Different mechanisms for modulation of the initiation and steady-state of smooth pursuit eye movements.

Smooth pursuit eye movements are used by primates to track moving objects. They are initiated by sensory estimates of target speed represented in the middle temporal (MT) area of extrastriate visual cortex and then supported by motor feedback to maintain steady-state eye speed at target speed. Here, we show that reducing the coherence in a patch of dots for a tracking target degrades the eye speed both at the initiation of pursuit and during steady-state tracking, when eye speed reaches an asymptote well below target speed. The deficits are quantitatively different between the motor-supported steady-state of pursuit and the sensory-driven initiation of pursuit, suggesting separate mechanisms. The deficit in visually guided pursuit initiation could not explain the deficit in steady-state tracking. Pulses of target speed during steady-state tracking revealed lower sensitivities to image motion across the retina for lower values of dot coherence. However, sensitivity was not zero, implying that visual motion should still be driving eye velocity toward target velocity. When we changed dot coherence from 100% to lower values during accurate steady-state pursuit, we observed larger eye decelerations for lower coherences, as expected if motor feedback was reduced in gain. A simple pursuit model accounts for our data based on separate modulation of the strength of visual-motor transmission and motor feedback. We suggest that reduced dot coherence allows us to observe evidence for separate modulations of the gain of visual-motor transmission during pursuit initiation and of the motor corollary discharges that comprise eye velocity memory and support steady-state tracking.NEW & NOTEWORTHY We exploit low-coherence patches of dots to control the initiation and steady state of smooth pursuit eye movements and show that these two phases of movement are modulated separately by the reliability of visual motion signals. We conclude that the neural circuit for pursuit includes separate modulation of the strength of visual-motor transmission for movement initiation and of eye velocity positive feedback to support steady-state tracking.

Mechanisms that allow cortical preparatory activity without inappropriate movement.

We reveal a novel mechanism that explains how preparatory activity can evolve in motor-related cortical areas without prematurely inducing movement. The smooth eye movement region of the frontal eye fields (FEFSEM) is a critical node in the neural circuit controlling smooth pursuit eye movement. Preparatory activity evolves in the monkey FEFSEM during fixation in parallel with an objective measure of visual-motor gain. We propose that the use of FEFSEM output as a gain signal rather than a movement command allows for preparation to progress in pursuit without causing movement. We also show that preparatory modulation of firing rate in FEFSEM predicts movement, providing evidence against the 'movement-null' space hypothesis as an explanation of how preparatory activity can progress without movement. Finally, there is a partial reorganization of FEFSEM population activity between preparation and movement that would allow for a directionally non-specific component of preparatory visual-motor gain enhancement in pursuit.

The Neural Basis for Response Latency in a Sensory-Motor Behavior.

We seek a neural circuit explanation for sensory-motor reaction times. In the smooth eye movement region of the frontal eye fields (FEFSEM), the latencies of pairs of neurons show trial-by-trial correlations that cause trial-by-trial correlations in neural and behavioral latency. These correlations can account for two-third of the observed variation in behavioral latency. The amplitude of preparatory activity also could contribute, but the responses of many FEFSEM neurons fail to support predictions of the traditional "ramp-to-threshold" model. As a correlate of neural processing that determines reaction time, the local field potential in FEFSEM includes a brief wave in the 5-15-Hz frequency range that precedes pursuit initiation and whose phase is correlated with the latency of pursuit in individual trials. We suggest that the latency of the incoming visual motion signals combines with the state of preparatory activity to determine the latency of the transient response that controls eye movement. IMPACT STATEMENT: The motor cortex for smooth pursuit eye movements contributes to sensory-motor reaction time through the amplitude of preparatory activity and the latency of transient, visually driven responses.

Neural implementation of Bayesian inference in a sensorimotor behavior.

Actions are guided by a Bayesian-like interaction between priors based on experience and current sensory evidence. Here we unveil a complete neural implementation of Bayesian-like behavior, including adaptation of a prior. We recorded the spiking of single neurons in the smooth eye-movement region of the frontal eye fields (FEFSEM), a region that is causally involved in smooth-pursuit eye movements. Monkeys tracked moving targets in contexts that set different priors for target speed. Before the onset of target motion, preparatory activity encodes and adapts in parallel with the behavioral adaptation of the prior. During the initiation of pursuit, FEFSEM output encodes a maximum a posteriori estimate of target speed based on a reliability-weighted combination of the prior and sensory evidence. FEFSEM responses during pursuit are sufficient both to adapt a prior that may be stored in FEFSEM and, through known downstream pathways, to cause Bayesian-like behavior in pursuit.

Multiple components in direction learning in smooth pursuit eye movements of monkeys.

We analyzed behavioral features of smooth pursuit eye movements to characterize the course of acquisition and expression of multiple neural components of motor learning. Monkeys tracked a target that began to move in an initial "pursuit" direction and suddenly, but predictably, changed direction after a fixed interval of 250 ms. As the trial is repeated, monkeys learn to make eye movements that predict the change in target direction. Quantitative analysis of the learned response revealed evidence for multiple, dynamic, parallel processes at work during learning. 1) The overall learning followed at least two trial courses: a fast component grew and saturated rapidly over tens of trials, and a slow component grew steadily over up to 1,000 trials. 2) The temporal specificity of the learned response within each trial was crude during the first 100 trials but then improved gradually over the remaining trials. 3) External influences on the gain of pursuit initiation modulate the expression but probably not the acquisition of learning. The gain of pursuit initiation and the expression of the learned response decreased in parallel, both gradually through a 1,000-trial learning block and immediately between learning trials with different gains in the initiation of pursuit. We conclude that at least two distinct neural mechanisms drive the acquisition of pursuit learning over 100 to 1,000 trials (3 to 30 min). Both mechanisms generate underlying memory traces that are modulated in relation to the gain of pursuit initiation before expression in the final motor output. NEW & NOTEWORTHY We show that cerebellum-dependent direction learning in smooth pursuit eye movements grows in at least two components over 1,100 behavioral learning repetitions. One component grows over tens of trials and the other over hundreds. Within trials, learned temporal specificity gradually improves over hundreds of trials. The expression of each learning component on a given trial can be modified by external factors that do not affect the underlying memory trace.

Modulation of Complex-Spike Duration and Probability during Cerebellar Motor Learning in Visually Guided Smooth-Pursuit Eye Movements of Monkeys.

Activation of an inferior olivary neuron powerfully excites Purkinje cells via its climbing fiber input and triggers a characteristic high-frequency burst, known as the complex spike (CS). The theory of cerebellar learning postulates that the CS induces long-lasting depression of the strength of synapses from active parallel fibers onto Purkinje cells, and that synaptic depression leads to changes in behavior. Prior reports showed that a CS on one learning trial is linked to a properly timed depression of simple spikes on the subsequent trial, as well as a learned change in pursuit eye movement. Further, the duration of a CS is a graded instruction for single-trial plasticity and behavioral learning. We now show across multiple learning paradigms that both the probability and duration of CS responses are correlated with the magnitudes of neural and behavioral learning in awake behaving monkeys. When the direction of the instruction for learning repeatedly was in the same direction or alternated directions, the duration and probability of CS responses decreased over a learning block along with the magnitude of trial-over-trial neural learning. When the direction of the instruction was randomized, CS duration, CS probability, and neural and behavioral learning remained stable across time. In contrast to depression, potentiation of simple-spike firing rate for ON-direction learning instructions follows a longer time course and plays a larger role as depression wanes. Computational analysis provides a model that accounts fully for the detailed statistics of a complex set of data.

Control of the strength of visual-motor transmission as the mechanism of rapid adaptation of priors for Bayesian inference in smooth pursuit eye movements.

Bayesian inference provides a cogent account of how the brain combines sensory information with "priors" based on past experience to guide many behaviors, including smooth pursuit eye movements. We now demonstrate very rapid adaptation of the pursuit system's priors for target direction and speed. We go on to leverage that adaptation to outline possible neural mechanisms that could cause pursuit to show features consistent with Bayesian inference. Adaptation of the prior causes changes in the eye speed and direction at the initiation of pursuit. The adaptation appears after a single trial and accumulates over repeated exposure to a given history of target speeds and directions. The influence of the priors depends on the reliability of visual motion signals: priors are more effective against the visual motion signals provided by low-contrast vs. high-contrast targets. Adaptation of the direction prior generalizes to eye speed and vice versa, suggesting that both priors could be controlled by a single neural mechanism. We conclude that the pursuit system can learn the statistics of visual motion rapidly and use those statistics to guide future behavior. Furthermore, a model that adjusts the gain of visual-motor transmission predicts the effects of recent experience on pursuit direction and speed, as well as the specifics of the generalization between the priors for speed and direction. We suggest that Bayesian inference in pursuit behavior is implemented by distinctly non-Bayesian internal mechanisms that use the smooth eye movement region of the frontal eye fields to control of the gain of visual-motor transmission.NEW & NOTEWORTHY Bayesian inference can account for the interaction between sensory data and past experience in many behaviors. Here, we show, using smooth pursuit eye movements, that the priors based on past experience can be adapted over a very short time frame. We also show that a single model based on direction-specific adaptation of the strength of visual-motor transmission can explain the implementation and adaptation of priors for both target direction and target speed.

Responses of Purkinje cells in the oculomotor vermis of monkeys during smooth pursuit eye movements and saccades: comparison with floccular complex.

We recorded the responses of Purkinje cells in the oculomotor vermis during smooth pursuit and saccadic eye movements. Our goal was to characterize the responses in the vermis using approaches that would allow direct comparisons with responses of Purkinje cells in another cerebellar area for pursuit, the floccular complex. Simple-spike firing of vermis Purkinje cells is direction selective during both pursuit and saccades, but the preferred directions are sufficiently independent so that downstream circuits could decode signals to drive pursuit and saccades separately. Complex spikes also were direction selective during pursuit, and almost all Purkinje cells showed a peak in the probability of complex spikes during the initiation of pursuit in at least one direction. Unlike the floccular complex, the preferred directions for simple spikes and complex spikes were not opposite. The kinematics of smooth eye movement described the simple-spike responses of vermis Purkinje cells well. Sensitivities were similar to those in the floccular complex for eye position and considerably lower for eye velocity and acceleration. The kinematic relations were quite different for saccades vs. pursuit, supporting the idea that the contributions from the vermis to each kind of movement could contribute independently in downstream areas. Finally, neither the complex-spike nor the simple-spike responses of vermis Purkinje cells were appropriate to support direction learning in pursuit. Complex spikes were not triggered reliably by an instructive change in target direction; simple-spike responses showed very small amounts of learning. We conclude that the vermis plays a different role in pursuit eye movements compared with the floccular complex.NEW & NOTEWORTHY The midline oculomotor cerebellum plays a different role in smooth pursuit eye movements compared with the lateral, floccular complex and appears to be much less involved in direction learning in pursuit. The output from the oculomotor vermis during pursuit lies along a null-axis for saccades and vice versa. Thus the vermis can play independent roles in the two kinds of eye movement.