Selective encoding of reward predictions and prediction errors by globus pallidus subpopulations.

Basal ganglia (BG) circuits help guide and invigorate actions using predictions of future rewards (values). Within the BG, the globus pallidus pars externa (GPe) may play an essential role in aggregating and distributing value information. We recorded from the GPe in unrestrained rats performing both Pavlovian and instrumental tasks to obtain rewards and distinguished neuronal subtypes by their firing properties across the wake/sleep cycle and optogenetic tagging. In both tasks, the parvalbumin-positive (PV+), faster-firing "prototypical" neurons showed strong, sustained modulation by value, unlike other subtypes, including the "arkypallidal" cells that project back to striatum. Furthermore, we discovered that a distinct minority (7%) of GP cells display slower, pacemaker-like firing and encode reward prediction errors (RPEs) almost identically to midbrain dopamine neurons. These cell-specific forms of GPe value representation help define the circuit mechanisms by which the BG contribute to motivation and reinforcement learning.

An emergent temporal basis set robustly supports cerebellar time-series learning.

The cerebellum is considered a "learning machine" essential for time interval estimation underlying motor coordination and other behaviors. Theoretical work has proposed that the cerebellum's input recipient structure, the granule cell layer (GCL), performs pattern separation of inputs that facilitates learning in Purkinje cells (P-cells). However, the relationship between input reformatting and learning has remained debated, with roles emphasized for pattern separation features from sparsification to decorrelation. We took a novel approach by training a minimalist model of the cerebellar cortex to learn complex time-series data from time-varying inputs, typical during movements. The model robustly produced temporal basis sets from these inputs, and the resultant GCL output supported better learning of temporally complex target functions than mossy fibers alone. Learning was optimized at intermediate threshold levels, supporting relatively dense granule cell activity, yet the key statistical features in GCL population activity that drove learning differed from those seen previously for classification tasks. These findings advance testable hypotheses for mechanisms of temporal basis set formation and predict that moderately dense population activity optimizes learning.NEW & NOTEWORTHY During movement, mossy fiber inputs to the cerebellum relay time-varying information with strong intrinsic relationships to ongoing movement. Are such mossy fibers signals sufficient to support Purkinje signals and learning? In a model, we show how the GCL greatly improves Purkinje learning of complex, temporally dynamic signals relative to mossy fibers alone. Learning-optimized GCL population activity was moderately dense, which retained intrinsic input variance while also performing pattern separation.

Predicting the responses of repetitively firing neurons to current noise.

We used phase resetting methods to predict firing patterns of rat subthalamic nucleus (STN) neurons when their rhythmic firing was densely perturbed by noise. We applied sequences of contiguous brief (0.5-2 ms) current pulses with amplitudes drawn from a Gaussian distribution (10-100 pA standard deviation) to autonomously firing STN neurons in slices. Current noise sequences increased the variability of spike times with little or no effect on the average firing rate. We measured the infinitesimal phase resetting curve (PRC) for each neuron using a noise-based method. A phase model consisting of only a firing rate and PRC was very accurate at predicting spike timing, accounting for more than 80% of spike time variance and reliably reproducing the spike-to-spike pattern of irregular firing. An approximation for the evolution of phase was used to predict the effect of firing rate and noise parameters on spike timing variability. It quantitatively predicted changes in variability of interspike intervals with variation in noise amplitude, pulse duration and firing rate over the normal range of STN spontaneous rates. When constant current was used to drive the cells to higher rates, the PRC was altered in size and shape and accurate predictions of the effects of noise relied on incorporating these changes into the prediction. Application of rate-neutral changes in conductance showed that changes in PRC shape arise from conductance changes known to accompany rate increases in STN neurons, rather than the rate increases themselves. Our results show that firing patterns of densely perturbed oscillators cannot readily be distinguished from those of neurons randomly excited to fire from the rest state. The spike timing of repetitively firing neurons may be quantitatively predicted from the input and their PRCs, even when they are so densely perturbed that they no longer fire rhythmically.

Basal ganglia output to the thalamus: still a paradox.

The basal ganglia (BG)-recipient thalamus controls motor output but it remains unclear how its activity is regulated. Several studies report that thalamic activation occurs via disinhibition during pauses in the firing of inhibitory pallidal inputs from the BG. Other studies indicate that thalamic spiking is triggered by pallidal inputs via post-inhibitory 'rebound' calcium spikes. Finally excitatory cortical inputs can drive thalamic activity, which becomes entrained, or time-locked, to pallidal spikes. We present a unifying framework where these seemingly distinct results arise from a continuum of thalamic firing 'modes' controlled by excitatory inputs. We provide a mechanistic explanation for paradoxical pallidothalamic coactivations observed during behavior that raises new questions about what information is integrated in the thalamus to control behavior.

Phase response curves of subthalamic neurons measured with synaptic input and current injection.

Infinitesimal phase response curves (iPRCs) provide a simple description of the response of repetitively firing neurons and may be used to predict responses to any pattern of synaptic input. Their simplicity makes them useful for understanding the dynamics of neurons when certain conditions are met. For example, the sizes of evoked phase shifts should scale linearly with stimulus strength, and the form of the iPRC should remain relatively constant as firing rate varies. We measured the PRCs of rat subthalamic neurons in brain slices using corticosubthalamic excitatory postsynaptic potentials (EPSPs; mediated by both AMPA- and NMDA-type receptors) and injected current pulses and used them to calculate the iPRC. These were relatively insensitive to both the size of the stimulus and the cell's firing rate, suggesting that the iPRC can predict the response of subthalamic nucleus cells to extrinsic inputs. However, the iPRC calculated using EPSPs differed from that obtained using current pulses. EPSPs (normalized for charge) were much more effective at altering the phase of subthalamic neurons than current pulses. The difference was not attributable to the extended time course of NMDA receptor-mediated currents, being unaffected by blockade of NMDA receptors. The iPRC provides a good description of subthalamic neurons' response to input, but iPRCs are best estimated using synaptic inputs rather than somatic current injection.

Biophysical basis of the phase response curve of subthalamic neurons with generalization to other cell types.

Experimental evidence indicates that the response of subthalamic neurons to excitatory postsynaptic potentials (EPSPs) is well described by their infinitesimal phase response curves (iPRC). However, the factors controlling the shape of that iPRC, and hence controlling the way subthalamic neurons respond to synaptic input, are unclear. We developed a biophysical model of subthalamic neurons to aid in the understanding of their iPRCs; this model exhibited an iPRC type common to many subthalamic cells. We devised a method for deriving its iPRC from its biophysical properties that clarifies how these different properties interact to shape the iPRC. This method revealed why the response of subthalamic neurons is well approximated by their iPRCs and how that approximation becomes less accurate under strong fluctuating input currents. It also connected iPRC structure to aspects of cellular physiology that could be estimated in simple current-clamp experiments. This allowed us to directly compare the iPRC predicted by our theory with the iPRC estimated from the response to EPSPs or current pulses in individual cells. We found that theoretically predicted iPRCs agreed well with estimates derived from synaptic stimuli, but not with those estimated from the response to somatic current injection. The difference between synaptic currents and those applied experimentally at the soma may arise from differences in the dynamics of charge redistribution on the dendrites and axon. Ultimately, our approach allowed us to identify novel ways in which voltage-dependent conductances interact with AHP conductances to influence synaptic integration that will apply to a wide range of cell types.

Integration of cortical and pallidal inputs in the basal ganglia-recipient thalamus of singing birds.

The basal ganglia-recipient thalamus receives inhibitory inputs from the pallidum and excitatory inputs from cortex, but it is unclear how these inputs interact during behavior. We recorded simultaneously from thalamic neurons and their putative synaptically connected pallidal inputs in singing zebra finches. We find, first, that each pallidal spike produces an extremely brief (∼5 ms) pulse of inhibition that completely suppresses thalamic spiking. As a result, thalamic spikes are entrained to pallidal spikes with submillisecond precision. Second, we find that the number of thalamic spikes that discharge within a single pallidal interspike interval (ISI) depends linearly on the duration of that interval but does not depend on pallidal activity prior to the interval. In a detailed biophysical model, our results were not easily explained by the postinhibitory "rebound" mechanism previously observed in anesthetized birds and in brain slices, nor could most of our data be characterized as "gating" of excitatory transmission by inhibitory pallidal input. Instead, we propose a novel "entrainment" mechanism of pallidothalamic transmission that highlights the importance of an excitatory conductance that drives spiking, interacting with brief pulses of pallidal inhibition. Building on our recent finding that cortical inputs can drive syllable-locked rate modulations in thalamic neurons during singing, we report here that excitatory inputs affect thalamic spiking in two ways: by shortening the latency of a thalamic spike after a pallidal spike and by increasing thalamic firing rates within individual pallidal ISIs. We present a unifying biophysical model that can reproduce all known modes of pallidothalamic transmission--rebound, gating, and entrainment--depending on the amount of excitation the thalamic neuron receives.

Dynamic spike threshold and zero membrane slope conductance shape the response of subthalamic neurons to cortical input.

The subthalamic nucleus (STN) provides a second entry point for cortical input to the basal ganglia, supplementing the corticostriatal pathway. We examined the way intrinsic properties shape the response of the STN to cortical excitation, recording from rat STN in vivo and in brain slices. STN cells exhibited a near-zero slope conductance-and hence an effectively infinite membrane time constant-at subthreshold potentials. This makes STN cells exceptional temporal integrators, consistent with the common view that basal ganglia nuclei use rate coding. However, STN cells also exhibited a drop in spike threshold triggered by larger EPSPs, allowing them to fire time-locked spikes in response to coincident input. In addition to promoting coincidence detection, the threshold dynamics associated with larger EPSPs reduced the probability of firing spikes outside of a narrow window immediately after the stimulus, even on trials in which the EPSP did not directly trigger a spike. This shift in stimulus-evoked firing pattern would allow downstream structures to distinguish coincidence-triggered spikes from other spikes and thereby permit coincidence detection and rate coding to operate in parallel in the same cell. Thus, STN cells can combine two functions-integration and coincidence detection-that are normally considered mutually exclusive. This could support rapid communication between cortex and basal ganglia targets that bypasses the striatum without disrupting slower rate coding pathways.

Organization of the songbird basal ganglia, including area X.

Area X is a songbird basal ganglia nucleus that is required for vocal learning. Both Area X and its immediate surround, the medial striatum (MSt), contain cells displaying either striatal or pallidal characteristics. We used pathway-tracing techniques to compare directly the targets of Area X and MSt with those of the lateral striatum (LSt) and globus pallidus (GP). We found that the zebra finch LSt projects to the GP, substantia nigra pars reticulata (SNr) and pars compacta (SNc), but not the thalamus. The GP is reciprocally connected with the subthalamic nucleus (STN) and projects to the SNr and motor thalamus analog, the ventral intermediate area (VIA). In contrast to the LSt, Area X and surrounding MSt project to the ventral pallidum (VP) and dorsal thalamus via pallidal-like neurons. A dorsal strip of the MSt contains spiny neurons that project to the VP. The MSt, but not Area X, projects to the ventral tegmental area (VTA) and SNc, but neither MSt nor Area X projects to the SNr. Largely distinct populations of SNc and VTA dopaminergic neurons innervate Area X and surrounding the MSt. Finally, we provide evidence consistent with an indirect pathway from the cerebellum to the basal ganglia, including Area X. Area X projections thus differ from those of the GP and LSt, but are similar to those of the MSt. These data clarify the relationships among different portions of the oscine basal ganglia as well as among the basal ganglia of birds and mammals.

Reinforcement learning with modulated spike timing dependent synaptic plasticity.

Spike timing-dependent synaptic plasticity (STDP) has emerged as the preferred framework linking patterns of pre- and postsynaptic activity to changes in synaptic strength. Although synaptic plasticity is widely believed to be a major component of learning, it is unclear how STDP itself could serve as a mechanism for general purpose learning. On the other hand, algorithms for reinforcement learning work on a wide variety of problems, but lack an experimentally established neural implementation. Here, we combine these paradigms in a novel model in which a modified version of STDP achieves reinforcement learning. We build this model in stages, identifying a minimal set of conditions needed to make it work. Using a performance-modulated modification of STDP in a two-layer feedforward network, we can train output neurons to generate arbitrarily selected spike trains or population responses. Furthermore, a given network can learn distinct responses to several different input patterns. We also describe in detail how this model might be implemented biologically. Thus our model offers a novel and biologically plausible implementation of reinforcement learning that is capable of training a neural population to produce a very wide range of possible mappings between synaptic input and spiking output.

Electrophysiological properties of neurons in the basal ganglia of the domestic chick: conservation and divergence in the evolution of the avian basal ganglia.

Although the basal ganglia of birds and mammals share an enormous number of anatomical, histochemical, and electrophysiological characteristics, studies in songbirds have revealed some important differences. Specifically, a specialized region of songbird striatum (the input structure of the basal ganglia) has an anatomical projection and a physiologically defined cell type that are characteristic of the globus pallidus. At present, it is not clear if these differences result from adaptations specific to songbirds and perhaps a few other avian taxa or are common to all birds. We shed some light on this issue by characterizing the morphology and electrophysiological properties of basal ganglia neurons in an avian species that is only distantly related to songbirds: the domestic chick. We recorded neurons in chick basal ganglia in a brain slice preparation, using the whole cell technique. We found that chick striatum, like songbird striatum, contains a pallidum-like cell type never reported in mammalian striatum, supporting the hypothesis that this feature is common to all birds. We also discovered that spiny neurons, the most common cell type in the striatum of all amniotes, possess a diverse set of physiological properties in chicks that distinguish them from both mammals and songbirds. This study revealed an unexpectedly complex pattern of conservation and divergence in the properties of neurons recorded in avian striatum.

Evidence for "direct" and "indirect" pathways through the song system basal ganglia.

Song learning in oscine birds relies on a circuit known as the "anterior forebrain pathway," which includes a specialized region of the avian basal ganglia. This region, area X, is embedded within a telencephalic structure considered homologous to the striatum, the input structure of the mammalian basal ganglia. Area X has many features in common with the mammalian striatum, yet has distinctive traits, including largely aspiny projection neurons that directly innervate the thalamus and a cell type that physiologically resembles neurons recorded in the mammalian globus pallidus. We have proposed that area X is a mixture of striatum and globus pallidus and has the same functional organization as circuits in the mammalian basal ganglia. Using electrophysiological and anatomical approaches, we found that area X contains a functional analog of the "direct" striatopallidothalamic pathway of mammals: axons of the striatal spiny neurons make close contacts on the somata and dendrites of pallidal cells. A subset of pallidal neurons project directly to the thalamus. Surprisingly, we found evidence that many pallidal cells may not project to the thalamus, but rather participate in a functional analog of the mammalian "indirect" pathway, which may oppose the effects of the direct pathway. Our results deepen our understanding of how information flows through area X and provide more support for the notion that song learning in oscines employs physiological mechanisms similar to basal ganglia-dependent forms of motor learning in mammals.

The avian song system in comparative perspective.

The song system of oscine birds has become a versatile model system that is used to study diverse problems in neurobiology. Because the song system is often studied with the intention of applying the results to mammalian systems, it is important to place song system brain nuclei in a broader context and to understand the relationships between these avian structures and regions of the mammalian brain. This task has been impeded by the distinctiveness of the song system and the vast apparent differences between the forebrains of birds and mammals. Fortunately, accumulating data on the development, histochemistry, and anatomical organization of avian and mammalian brains has begun to shed light on this issue. We now know that the forebrains of birds and mammals are more alike than they first appeared, even though many questions remain unanswered. Furthermore, the song system is not as singular as it seemed-it has much in common with other neural systems in birds and mammals. These data provide a firmer foundation for extrapolating knowledge of the song system to mammalian systems and suggest how the song system might have evolved.

Presynaptic depression of glutamatergic synaptic transmission by D1-like dopamine receptor activation in the avian basal ganglia.

Vocal behavior in songbirds exemplifies a rich integration of motor, cognitive, and social functions that are shared among vertebrates. As a part of the underlying neural substrate, the song system, the anterior forebrain pathway (AFP) is required for song learning and maintenance. The AFP resembles the mammalian basal ganglia-thalamocortical loop in its macroscopic organization, neuronal intrinsic properties, and microcircuitry. Area X, the first station in the AFP, is a part of the basal ganglia essential for vocal learning. It receives glutamatergic inputs from pallial structures and sends GABAergic outputs to thalamic structures. It also receives dense dopaminergic innervation from the midbrain. The role of this innervation is essentially unknown. Here we provide evidence that dopamine (DA) can modulate the glutamatergic inputs to spiny neurons in area X. In whole-cell voltage-clamp recordings from neurons in brain slices of adult zebra finches, we found that activation of D1-like DA receptors depresses ionotropic glutamatergic synaptic current in area X spiny neurons. This effect is mediated by a presynaptic site of action, mimicked by activation of adenylyl cyclase, and blocked by protein kinase A inhibitor and an adenosine A1 receptor antagonist. These results suggest that, in addition to altering the input-output function of spiny neurons by modulating their excitability, as we have shown previously, DA can directly influence the excitatory inputs to these neurons as well. Thus, DA can exert fine control over information processing through spiny neurons in area X, the dynamics of the AFP output, and ultimately song learning and maintenance.

A telencephalic nucleus essential for song learning contains neurons with physiological characteristics of both striatum and globus pallidus.

The song system of oscine birds has frequently been presented as a model system for motor learning in vertebrates. This practice has been bolstered by the growing recognition that one part of the song system that is essential for song learning, area X, is a component of the avian striatum. The mammalian striatum, the input structure of the basal ganglia, has been implicated in a number of motor-related functions, including motor learning, suggesting that song learning in birds and motor learning in mammals may use similar physiological mechanisms. We studied the intrinsic physiological properties of area X neurons in brain slices to see how closely they match properties identified in mammalian striatal neurons and to collect data that are necessary to understand how area X processes information. We found that area X contains all four physiological cell types present in the mammalian striatum and that each is very similar to its mammalian counterpart. We also found a fifth cell type in area X that has not been reported in mammalian striatum; instead, this cell type resembles neurons that have been recorded in the mammalian globus pallidus. This pallidum-like cell type morphologically resembles the projection neurons of area X. We suggest that area X contains a pathway equivalent to the "direct" striatopallidothalamic pathway through the mammalian basal ganglia, with the striatal and pallidal components intermingled in one nucleus.

Electrophysiological analysis of a songbird basal ganglia circuit essential for vocal plasticity.

The discrete, interconnected nuclei of the songbird brain, collectively termed the song system, underlie the learning and production of song. Two main forebrain pathways have been identified that contribute to song production, learning, and adult plasticity. A posterior "motor pathway" including nucleus HVc (used as the proper name), the robust nucleus of the archistriatum (RA) and descending projections to the brainstem, is essential for song production. An "anterior forebrain pathway," arising from HVc, passing through area X of the lobus parolfactorius, the medial portion of the dorsolateral nucleus of the anterior thalamus and the lateral magnocellular nucleus of the anterior neostriatum, and finally terminating in RA, is essential for song learning and adult plasticity. The fact that the lobus parolfactorius is thought to form a part of the avian striatum implies several predictions for the connections of area X and for the properties of its neurons. Here, we review the existing anatomical and electrophysiological data bearing on the nature of area X as a striatal structure. In general, the data strongly favor the notion that area X is striatal. One set of observations, however, is at odds with that idea, and we provide and partially test the hypothesis that area X also contains a pallidal component. We discuss further tests of this idea and implications for thinking of the song system as a basal ganglia loop similar to that described in mammals.