Feasibility of dopamine as a vector-valued feedback signal in the basal ganglia.

It is well established that midbrain dopaminergic neurons support reinforcement learning (RL) in the basal ganglia by transmitting a reward prediction error (RPE) to the striatum. In particular, different computational models and experiments have shown that a striatum-wide RPE signal can support RL over a small discrete set of actions (e.g., no/no-go, choose left/right). However, there is accumulating evidence that the basal ganglia functions not as a selector between predefined actions but rather as a dynamical system with graded, continuous outputs. To reconcile this view with RL, there is a need to explain how dopamine could support learning of continuous outputs, rather than discrete action values. Inspired by the recent observations that besides RPE, the firing rates of midbrain dopaminergic neurons correlate with motor and cognitive variables, we propose a model in which dopamine signal in the striatum carries a vector-valued error feedback signal (a loss gradient) instead of a homogeneous scalar error (a loss). We implement a local, "three-factor" corticostriatal plasticity rule involving the presynaptic firing rate, a postsynaptic factor, and the unique dopamine concentration perceived by each striatal neuron. With this learning rule, we show that such a vector-valued feedback signal results in an increased capacity to learn a multidimensional series of real-valued outputs. Crucially, we demonstrate that this plasticity rule does not require precise nigrostriatal synapses but remains compatible with experimental observations of random placement of varicosities and diffuse volume transmission of dopamine.

Complete representation of action space and value in all dorsal striatal pathways.

The dorsal striatum plays a central role in the selection, execution, and evaluation of actions. An emerging model attributes action selection to the matrix and evaluation to the striosome compartment. Here, we use large-scale cell-type-specific calcium imaging to determine the activity of striatal projection neurons (SPNs) during motor and decision behaviors in the three major outputs of the dorsomedial striatum: Oprm1+ striosome versus D1+ direct and A2A+ indirect pathway SPNs. We find that Oprm1+ SPNs show complex tunings to simple movements and value-guided actions, which are conserved across many sessions in a single task but remap between contexts. During decision making, the SPN tuning profiles form a complete representation in which sequential SPN activity jointly encodes task progress and value. We propose that the three major output pathways in the dorsomedial striatum share a similarly complete representation of the entire action space, including task- and phase-specific signals of action value and choice.

Offset analgesia and onset hyperalgesia with different stimulus ranges.

INTRODUCTION: Offset analgesia (OA), a large reduction in pain after a brief increase in intensity of an otherwise stable painful stimulus, has been established by a large body of research. But the opposite effect, onset hyperalgesia (OH), a disproportional hyperalgesic response after a briefly decreased intensity of a painful stimulus, has only been investigated in one previous study. OBJECTIVES: The aim of this study was to induce OA and OH in healthy participants and explore the effects of different stimulus ranges (increase/decrease of temperature) on OA and OH. METHODS: A total of 62 participants were tested in 2 identical experiments. Offset analgesia and OH conditions included 2 different temperature deviations (±1°C/±2°C) from initial temperature and were compared with a constant temperature (control). RESULTS: Offset analgesia was successfully elicited in OA1°C in experiment 1, and in OA1°C and OA2°C in experiment 2. Results indicate a continuous stimulus-response relationship between the stimulus range and the resulting hypoalgesic response. Onset hyperalgesia was only elicited in OH2°C in experiment 1. Exploratory analysis showed that the lack of OH response in experiment 2 could be explained by sex differences, and that OA and OH responses were only weakly correlated. CONCLUSIONS: The asymmetry between pain responses after a brief temperature increase and decrease suggests that different mechanisms are involved in the pain responses to increasing and decreasing temperature. This asymmetry may also be explained by high temperatures in OA condition (+1°C/+2°C above baseline) that could be seen as salient "learning signals," which augment the response to following changes in temperature.

A Spatiomolecular Map of the Striatum.

The striatum is organized into two major outputs formed by striatal projection neuron (SPN) subtypes with distinct molecular identities. In addition, histochemical division into patch and matrix compartments represents an additional spatial organization, proposed to mirror a motor-motivation regionalization. To map the molecular diversity of patch versus matrix SPNs, we genetically labeled mu opioid receptor (Oprm1) expressing neurons and performed single-nucleus RNA sequencing. This allowed us to establish molecular definitions of patch, matrix, and exopatch SPNs, as well as identification of Col11a1+ striatonigral SPNs. At the tissue level, mapping the expression of candidate markers reveals organization of spatial domains, which are conserved in the non-human primate brain. The spatial markers are cell-type independent and instead represent a spatial code found across all SPNs within a spatial domain. The spatiomolecular map establishes a formal system for targeting and studying striatal subregions and SPNs subtypes, beyond the classical striatonigral and striatopallidal division.

Perturbing low dimensional activity manifolds in spiking neuronal networks.

Several recent studies have shown that neural activity in vivo tends to be constrained to a low-dimensional manifold. Such activity does not arise in simulated neural networks with homogeneous connectivity and it has been suggested that it is indicative of some other connectivity pattern in neuronal networks. In particular, this connectivity pattern appears to be constraining learning so that only neural activity patterns falling within the intrinsic manifold can be learned and elicited. Here, we use three different models of spiking neural networks (echo-state networks, the Neural Engineering Framework and Efficient Coding) to demonstrate how the intrinsic manifold can be made a direct consequence of the circuit connectivity. Using this relationship between the circuit connectivity and the intrinsic manifold, we show that learning of patterns outside the intrinsic manifold corresponds to much larger changes in synaptic weights than learning of patterns within the intrinsic manifold. Assuming larger changes to synaptic weights requires extensive learning, this observation provides an explanation of why learning is easier when it does not require the neural activity to leave its intrinsic manifold.