A hypothesis for basal ganglia-dependent reinforcement learning in the songbird.

Most of our motor skills are not innately programmed, but are learned by a combination of motor exploration and performance evaluation, suggesting that they proceed through a reinforcement learning (RL) mechanism. Songbirds have emerged as a model system to study how a complex behavioral sequence can be learned through an RL-like strategy. Interestingly, like motor sequence learning in mammals, song learning in birds requires a basal ganglia (BG)-thalamocortical loop, suggesting common neural mechanisms. Here, we outline a specific working hypothesis for how BG-forebrain circuits could utilize an internally computed reinforcement signal to direct song learning. Our model includes a number of general concepts borrowed from the mammalian BG literature, including a dopaminergic reward prediction error and dopamine-mediated plasticity at corticostriatal synapses. We also invoke a number of conceptual advances arising from recent observations in the songbird. Specifically, there is evidence for a specialized cortical circuit that adds trial-to-trial variability to stereotyped cortical motor programs, and a role for the BG in "biasing" this variability to improve behavioral performance. This BG-dependent "premotor bias" may in turn guide plasticity in downstream cortical synapses to consolidate recently learned song changes. Given the similarity between mammalian and songbird BG-thalamocortical circuits, our model for the role of the BG in this process may have broader relevance to mammalian BG function.

Miniature motorized microdrive and commutator system for chronic neural recording in small animals.

The use of chronically implanted electrodes for neural recordings in small, freely behaving animals poses several unique technical challenges. Because of the need for an extremely lightweight apparatus, chronic recording technology has been limited to manually operated microdrives, despite the advantage of motorized manipulators for positioning electrodes. Here we describe a motorized, miniature chronically implantable microdrive for independently positioning three electrodes in the brain. The electrodes are controlled remotely, avoiding the need to disturb the animal during electrode positioning. The microdrive is approximately 6 mm in diameter, 17 mm high and weighs only 1.5 g, including the headstage preamplifier. Use of the motorized microdrive has produced a ten-fold increase in our data yield compared to those experiments done using a manually operated drive. In addition, we are able to record from multiple single neurons in the behaving animal with signal quality comparable to that seen in a head-fixed anesthetized animal. We also describe a motorized commutator that actively tracks animal rotation based on a measurement of torque in the tether.

A miniature head-mounted two-photon microscope. high-resolution brain imaging in freely moving animals.

Two-photon microscopy has enabled anatomical and functional fluorescence imaging in the intact brain of rats. Here, we extend two-photon imaging from anesthetized, head-stabilized to awake, freely moving animals by using a miniaturized head-mounted microscope. Excitation light is conducted to the microscope in a single-mode optical fiber, and images are scanned using vibrations of the fiber tip. Microscope performance was first characterized in the neocortex of anesthetized rats. We readily obtained images of vasculature filled with fluorescently labeled blood and of layer 2/3 pyramidal neurons filled with a calcium indicator. Capillary blood flow and dendritic calcium transients were measured with high time resolution using line scans. In awake, freely moving rats, stable imaging was possible except during sudden head movements.

Active stabilization of electrodes for intracellular recording in awake behaving animals.

Intracellular recording is a powerful electrophysiology technique that has revealed much of what is known about the biophysical properties of neurons. However, neuronal properties are strongly affected by activity dependent and modulatory influences, making it essential, ultimately, to study these properties in behaving animals. Unfortunately, intracellular recording has only been widely applied in vitro, since cardiac and respiratory pulsations make intracellular recording difficult in vivo. In awake behaving animals, spontaneous movements make intracellular recording nearly impossible. Here I present a novel technique to dynamically stabilize the position of a recording electrode relative to the brain. Physiological signals that are predictive of brain motion at the recording site, such as the electrocardiogram (EKG), respiratory pressure, or cranial motion, are used to control a piezoelectric manipulator, making possible stable intracellular recordings in awake active animals.

Ultra-miniature headstage with 6-channel drive and vacuum-assisted micro-wire implantation for chronic recording from the neocortex.

We describe a head-stage, with precision microtranslators for the chronic placement of micro-wire electrodes in the neocortex, that minimizes compressive damage to the brain. The head-stage has a diameter of 5.8 mm and allows six electrodes, separated by 450 microm on a hexagonal grid, to be individually and continuously positioned throughout a depth of approximately 3 mm. Suction is used to transiently support the dura against a curved array of tubes that guide and stabilize the electrodes as a means to prevent compression of the neocortex as the electrodes breach the dura. With this headstage we recorded extracellular signals in a rat immediately after surgery. Single-unit waveforms at a given electrode position were stable for at least several hours in the freely behaving animal and were obtained throughout the depth of the neocortex for at least 2 months. Electrophysiological records and histological examination showed that the upper layers of the neocortex were intact and minimally damaged after the implantation.

The role of nonlinear dynamics of the syrinx in the vocalizations of a songbird.

Birdsong is characterized by the modulation of sound properties over a wide image of timescales. Understanding the mechanisms by which the brain organizes this complex temporal behaviour is a central motivation in the study of the song control and learning system. Here we present evidence that, in addition to central neural control, a further level of temporal organization is provided by nonlinear oscillatory dynamics that are intrinsic to the avian vocal organ. A detailed temporal and spectral examination of song of the zebra finch (Taeniopygia guttata) reveals a class of rapid song modulations that are consistent with transitions in the dynamical state of the syrinx. Furthermore, in vitro experiments show that the syrinx can produce a sequence of oscillatory states that are both spectrally and temporally complex in response to the slow variation of respiratory or syringeal parameters. As a consequence, simple variations in a small number of neural signals can result in a complex acoustic sequence.

Central versus peripheral determinants of patterned spike activity in rat vibrissa cortex during whisking.

We report on the relationship between single-unit activity in primary somatosensory vibrissa cortex of rat and the rhythmic movement of vibrissae. Animals were trained to whisk freely in air in search of food. Electromyographic (EMG) recordings from the mystatial pads served as a reference for the position of the vibrissae. A fast, oscillatory component in single-unit spike trains is correlated with vibrissa position within the whisk cycle. The phase of the correlation for different units is broadly distributed. A second, slowly varying component of spike activity correlates with the amplitude of the whisk cycle. For some units, the phase and amplitude correlations were of sufficient strength to allow the position of the whiskers to be accurately predicted from a single spike train. To determine whether the observed patterned spike activity was driven by motion of the vibrissae, as opposed to central pathways, we reversibly blocked the contralateral facial motor nerve during the behavioral task so that the rat whisked only on the ipsilateral side. The ipsilateral EMG served as a reliable reference signal. The fast, oscillatory component of the spike-EMG correlation disappears when the facial motor nerve is blocked. This implies that the position of vibrissae within a cycle is encoded through direct sensory activation. The slowly varying component of the spike-EMG correlation is unaffected by the block. This implies that the amplitude of whisking is likely to be mediated by corollary discharge. Our results suggest that motor cortex does not relay a reference signal to sensory cortex for positional information of the vibrissae during whisking.

Variability of extracellular spike waveforms of cortical neurons.

1. Here we study the variability in extracellular records of action potentials. Our work is motivated, in part, by the need to construct effective algorithms to classify single-unit waveforms from multiunit recordings. 2. We used microwire electrode pairs (stereotrodes) to record from primary somatosensory cortex of awake, behaving rat. Our data consist of continuous records of extracellular activity and segmented records of extracellular spikes. Spectral and principal component techniques are used to analyze mean single-unit wave-forms, the variability between different instances of a single-unit waveform, and the underlying background activity. 3. The spectrum of the variability between different instances of a single-unit waveforms is not white, and falls off above 1 kHz with a frequency dependence of roughly f-2. This spectrum is different from that of the mean spike waveforms, which falls off roughly as f-4, but is essentially identical with the spectrum of background activity. The spatial coherence of the variability on the 10-micron scale also falls off at high frequencies. 4. The variability between different instances of a single-unit waveform is dominated by a relatively small number of principal components. As a consequence, there is a large anisotropy in the cluster of the spike waveforms. 5. The background noise cannot be represented as a stationary Gaussian random process. In particular, we observed that the spectrum changes significantly between successive 20-ms intervals. Furthermore, the total power in the background activity exhibits larger fluctuations than is consistent with a stationary Gaussian random process. 6. Roughly half of the single-unit spike waveforms exhibit systematic changes as a function of the interspike interval. Although this results in a non-Gaussian distribution in the space of waveforms, the distribution can be modeled by a scalar function of the interspike interval. 7. We use a set of 44 mean single-unit waveforms to define the space of differences between spike waveforms. This characterization, together with that of the background activity, is used to construct a filter that optimizes the detection of differences between single-unit waveforms. Further, an information theoretic measure is defined that characterizes the detectability.

Automatic sorting of multiple unit neuronal signals in the presence of anisotropic and non-Gaussian variability.

Neuronal noise sources and systematic variability in the shape of a spike limit the ability to sort multiple unit waveforms recorded from nervous tissue into their single neuron constituents. Here we present a procedure to efficiently sort spikes in the presence of noise that is anisotropic, i.e., dominated by particular frequencies, and whose amplitude distribution may be non-Gaussian, such as occurs when spike waveforms are a function of interspike interval. Our algorithm uses a hierarchical clustering scheme. First, multiple unit records are sorted into an overly large number of clusters by recursive bisection. Second, these clusters are progressively aggregated into a minimal set of putative single units based on both similarities of spike shape as well as the statistics of spike arrival times, such as imposed by the refractory period. We apply the algorithm to waveforms recorded with chronically implanted micro-wire stereotrodes from neocortex of behaving rat. Natural extension of the algorithm may be used to cluster spike waveforms from records with many input channels, such as those obtained with tetrodes and multiple site optical techniques.

Dynamics of propagating waves in the olfactory network of a terrestrial mollusk: an electrical and optical study.

1. The procerebral (PC) lobe of the terrestrial mollusk Limax maximus contains a highly interconnected network of local olfactory interneurons that receives ipsilateral axonal projections from superior and inferior noses. This network exhibits an approximately 0.7-Hz intrinsic oscillation in its local field potential (LFP). 2. Intracellular recordings show that the lobe contains at least two classes of neurons with activity phase locked to the oscillation. Neurons in one class produce periodic bursts of spikes, followed by a period of hyperpolarization and subsequently a depolarizing afterpotential. There is a small but significant chance for a second burst to occur during the depolarizing afterpotential; this leads to a double event in the LFP. Bursting neurons constitute approximately 10% of the neurons in the lobe. 3. Neurons in the other class fire infrequently and do not produce periodic bursts of action potentials. However, they receive strong, periodic inhibitory input during every event in the LFP. These nonbursting cells constitute the major fraction of neurons in the lobe. There is a clear correlation between the periodic burst of action potentials in the bursting neurons and the hyperpolarization seen in nonbursting neurons. 4. Optical techniques are used to image the spatially averaged transmembrane potentials in preparations stained with voltage-sensitive dyes. The results of simultaneous optical and electrical measurements show that the major part of the optical signal can be interpreted as a superposition of the intracellular signals arising from the bursting and nonbursting neurons. 5. Successive images of the entire PC lobe show waves of electrical activity that span the width of the lobe and travel its full length along a longitudinal axis. The direction of propagation in the unperturbed lobe is always from the distal to the proximal end. The wavelength varies between preparations but is on the order of the length of the preparation. 6. One-dimensional images along the longitudinal axis of the lobe are used to construct a space-time map of the optical activity, from which we calculate the absolute contribution of bursting and nonbursting neurons to the optical signal. The contribution of the intracellular signals from the two cell types appears to vary systematically across the lobe; bursting cells dominate at middle and proximal locations, and nonbursting cells dominate at distal locations. 7. The direction and form of the waves can be perturbed either by microsurgical manipulation of the preparation or by chemical modulation of its synaptic and neuronal properties.(ABSTRACT TRUNCATED AT 400 WORDS)

Waves and stimulus-modulated dynamics in an oscillating olfactory network.

The temporal dynamics of electrical activity in an olfactory organ, the procerebral lobe of the terrestrial mollusc Limax maximus, is studied. The lobe exhibits intrinsic oscillations in its field potential. Intracellular recordings show that the lobe contains two classes of neurons, both with activity phase-locked to the oscillation. Neurons in one class produce periodic bursts of spikes while those in the other class fire infrequently but receive strong, periodic inhibition whose onset coincides with the burst. The large-scale activity of these neurons is imaged in preparations stained with voltage-sensitive dyes. We observe waves of electrical activity that span the width of the lobe and travel its full length along a longitudinal axis. Simultaneous optical and intracellular recordings show that the form of the wave reflects the electrical activity of both classes of neurons. The application of natural odor stimuli causes the electrical activity along the lobe to transiently switch from the state with propagating waves to one with spatially uniform oscillations. The behavioral and computational relevance of this change in global timing is discussed.