Dopaminergic and prefrontal dynamics co-determine mouse decisions in a spatial gambling task.

The neural mechanisms by which animals initiate goal-directed actions, choose between options, or explore opportunities remain unknown. Here, we develop a spatial gambling task in which mice, to obtain intracranial self-stimulation rewards, self-determine the initiation, direction, vigor, and pace of their actions based on their knowledge of the outcomes. Using electrophysiological recordings, pharmacology, and optogenetics, we identify a sequence of oscillations and firings in the ventral tegmental area (VTA), orbitofrontal cortex (OFC), and prefrontal cortex (PFC) that co-encodes and co-determines self-initiation and choices. This sequence appeared with learning as an uncued realignment of spontaneous dynamics. Interactions between the structures varied with the reward context, particularly the uncertainty associated with the different options. We suggest that self-generated choices arise from a distributed circuit based on an OFC-VTA core determining whether to wait for or initiate actions, while the PFC is specifically engaged by reward uncertainty in action selection and pace.

A VTA GABAergic computational model of dissociated reward prediction error computation in classical conditioning.

Classical Conditioning is a fundamental learning mechanism where the Ventral Striatum is generally thought to be the source of inhibition to Ventral Tegmental Area (VTA) Dopamine neurons when a reward is expected. However, recent evidences point to a new candidate in VTA GABA encoding expectation for computing the reward prediction error in the VTA. In this system-level computational model, the VTA GABA signal is hypothesised to be a combination of magnitude and timing computed in the Peduncolopontine and Ventral Striatum respectively. This dissociation enables the model to explain recent results wherein Ventral Striatum lesions affected the temporal expectation of the reward but the magnitude of the reward was intact. This model also exhibits other features in classical conditioning namely, progressively decreasing firing for early rewards closer to the actual reward, twin peaks of VTA dopamine during training and cancellation of US dopamine after training.

Lactate is an energy substrate for rodent cortical neurons and enhances their firing activity.

Glucose is the mandatory fuel for the brain, yet the relative contribution of glucose and lactate for neuronal energy metabolism is unclear. We found that increased lactate, but not glucose concentration, enhances the spiking activity of neurons of the cerebral cortex. Enhanced spiking was dependent on ATP-sensitive potassium (KATP) channels formed with KCNJ11 and ABCC8 subunits, which we show are functionally expressed in most neocortical neuronal types. We also demonstrate the ability of cortical neurons to take-up and metabolize lactate. We further reveal that ATP is produced by cortical neurons largely via oxidative phosphorylation and only modestly by glycolysis. Our data demonstrate that in active neurons, lactate is preferred to glucose as an energy substrate, and that lactate metabolism shapes neuronal activity in the neocortex through KATP channels. Our results highlight the importance of metabolic crosstalk between neurons and astrocytes for brain function.

Chronic nicotine increases midbrain dopamine neuron activity and biases individual strategies towards reduced exploration in mice.

Long-term exposure to nicotine alters brain circuits and induces profound changes in decision-making strategies, affecting behaviors both related and unrelated to drug seeking and consumption. Using an intracranial self-stimulation reward-based foraging task, we investigated in mice the impact of chronic nicotine on midbrain dopamine neuron activity and its consequence on the trade-off between exploitation and exploration. Model-based and archetypal analysis revealed substantial inter-individual variability in decision-making strategies, with mice passively exposed to nicotine shifting toward a more exploitative profile compared to non-exposed animals. We then mimicked the effect of chronic nicotine on the tonic activity of dopamine neurons using optogenetics, and found that photo-stimulated mice adopted a behavioral phenotype similar to that of mice exposed to chronic nicotine. Our results reveal a key role of tonic midbrain dopamine in the exploration/exploitation trade-off and highlight a potential mechanism by which nicotine affects the exploration/exploitation balance and decision-making.

Online Learning and Memory of Neural Trajectory Replays for Prefrontal Persistent and Dynamic Representations in the Irregular Asynchronous State.

In the prefrontal cortex (PFC), higher-order cognitive functions and adaptive flexible behaviors rely on continuous dynamical sequences of spiking activity that constitute neural trajectories in the state space of activity. Neural trajectories subserve diverse representations, from explicit mappings in physical spaces to generalized mappings in the task space, and up to complex abstract transformations such as working memory, decision-making and behavioral planning. Computational models have separately assessed learning and replay of neural trajectories, often using unrealistic learning rules or decoupling simulations for learning from replay. Hence, the question remains open of how neural trajectories are learned, memorized and replayed online, with permanently acting biological plasticity rules. The asynchronous irregular regime characterizing cortical dynamics in awake conditions exerts a major source of disorder that may jeopardize plasticity and replay of locally ordered activity. Here, we show that a recurrent model of local PFC circuitry endowed with realistic synaptic spike timing-dependent plasticity and scaling processes can learn, memorize and replay large-size neural trajectories online under asynchronous irregular dynamics, at regular or fast (sped-up) timescale. Presented trajectories are quickly learned (within seconds) as synaptic engrams in the network, and the model is able to chunk overlapping trajectories presented separately. These trajectory engrams last long-term (dozen hours) and trajectory replays can be triggered over an hour. In turn, we show the conditions under which trajectory engrams and replays preserve asynchronous irregular dynamics in the network. Functionally, spiking activity during trajectory replays at regular timescale accounts for both dynamical coding with temporal tuning in individual neurons, persistent activity at the population level, and large levels of variability consistent with observed cognitive-related PFC dynamics. Together, these results offer a consistent theoretical framework accounting for how neural trajectories can be learned, memorized and replayed in PFC networks circuits to subserve flexible dynamic representations and adaptive behaviors.

Author Correction: Mice adaptively generate choice variability in a deterministic task.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Mice adaptively generate choice variability in a deterministic task.

Can decisions be made solely by chance? Can variability be intrinsic to the decision-maker or is it inherited from environmental conditions? To investigate these questions, we designed a deterministic setting in which mice are rewarded for non-repetitive choice sequences, and modeled the experiment using reinforcement learning. We found that mice progressively increased their choice variability. Although an optimal strategy based on sequences learning was theoretically possible and would be more rewarding, animals used a pseudo-random selection which ensures high success rate. This was not the case if the animal is exposed to a uniform probabilistic reward delivery. We also show that mice were blind to changes in the temporal structure of reward delivery once they learned to choose at random. Overall, our results demonstrate that a decision-making process can self-generate variability and randomness, even when the rules governing reward delivery are neither stochastic nor volatile.

Target Interneuron Preference in Thalamocortical Pathways Determines the Temporal Structure of Cortical Responses.

Sensory processing relies on fast detection of changes in environment, as well as integration of contextual cues over time. The mechanisms by which local circuits of the cerebral cortex simultaneously perform these opposite processes remain obscure. Thalamic "specific" nuclei relay sensory information, whereas "nonspecific" nuclei convey information on the environmental and behavioral contexts. We expressed channelrhodopsin in the ventrobasal specific (sensory) or the rhomboid nonspecific (contextual) thalamic nuclei. By selectively activating each thalamic pathway, we found that nonspecific inputs powerfully activate adapting (slow-responding) interneurons but weakly connect fast-spiking interneurons, whereas specific inputs exhibit opposite interneuron preference. Specific inputs thereby induce rapid feedforward inhibition that limits response duration, whereas, in the same cortical area, nonspecific inputs elicit delayed feedforward inhibition that enables lasting recurrent excitation. Using a mean field model, we confirm that cortical response dynamics depends on the type of interneuron targeted by thalamocortical inputs and show that efficient recruitment of adapting interneurons prolongs the cortical response and allows the summation of sensory and contextual inputs. Hence, target choice between slow- and fast-responding inhibitory neurons endows cortical networks with a simple computational solution to perform both sensory detection and integration.

Glutamatergic and dopaminergic modulation of cortico-striatal circuits probed by dynamic calcium imaging of networks reconstructed in microfluidic chips.

Although the prefrontal cortex and basal ganglia are functionally interconnected by parallel loops, cellular substrates underlying their interaction remain poorly understood. One novel approach for addressing this issue is microfluidics, a methodology which recapitulates several intrinsic and synaptic properties of cortico-subcortical networks. We developed a microfluidic device where cortical neurons projected onto striatal neurons in a separate compartment. We exploited real-time (low-resolution/high-output) calcium imaging to register network dynamics and characterize the response to glutamatergic and dopaminergic agents. Reconstructed cortico-striatal networks revealed the progressive appearance of cortical VGLUT1 clusters on striatal dendrites, correlating with the emergence of spontaneous and synchronous glutamatergic responses of striatal neurons to concurrent cortical stimulation. Striatal exposure to the NMDA receptor GluN2A subunit antagonist TCN201 did not affect network rhythm, whereas the GluN2B subunit antagonist RO256981 significantly decreased striatal activity. Dopamine application or the D2/D3 receptor agonist, quinpirole, decreased cortico-striatal synchrony whereas the D1 receptor agonist, SKF38393, was ineffective. These data show that cortico-striatal networks reconstructed in a microfluidic environment are synchronized and present characteristics close to those of their in situ counterparts. They should prove instructive for deciphering the molecular substrates of CNS disorders and evaluating the actions of novel therapeutic agents.

Manipulating midbrain dopamine neurons and reward-related behaviors with light-controllable nicotinic acetylcholine receptors.

Dopamine (DA) neurons of the ventral tegmental area (VTA) integrate cholinergic inputs to regulate key functions such as motivation and goal-directed behaviors. Yet the temporal dynamic range and mechanism of action of acetylcholine (ACh) on the modulation of VTA circuits and reward-related behaviors are not known. Here, we used a chemical-genetic approach for rapid and precise optical manipulation of nicotinic neurotransmission in VTA neurons in living mice. We provide direct evidence that the ACh tone fine-tunes the firing properties of VTA DA neurons through ß2-containing (ß2*) nicotinic ACh receptors (nAChRs). Furthermore, locally photo-antagonizing these receptors in the VTA was sufficient to reversibly switch nicotine reinforcement on and off. By enabling control of nicotinic transmission in targeted brain circuits, this technology will help unravel the various physiological functions of nAChRs and may assist in the design of novel therapies relevant to neuropsychiatric disorders.

Acute Stress Affects the Expression of Hippocampal Mu Oscillations in an Age-Dependent Manner.

Anxiolytic drugs are widely used in the elderly, a population particularly sensitive to stress. Stress, aging and anxiolytics all affect low-frequency oscillations in the hippocampus and prefrontal cortex (PFC) independently, but the interactions between these factors remain unclear. Here, we compared the effects of stress (elevated platform, EP) and anxiolytics (diazepam, DZP) on extracellular field potentials (EFP) in the PFC, parietal cortex and hippocampus (dorsal and ventral parts) of adult (8 months) and aged (18 months) Wistar rats. A potential source of confusion in the experimental studies in rodents comes from locomotion-related theta (6-12 Hz) oscillations, which may overshadow the direct effects of anxiety on low-frequency and especially on the high-amplitude oscillations in the Mu range (7-12 Hz), related to arousal. Animals were restrained to avoid any confound and isolate the direct effects of stress from theta oscillations related to stress-induced locomotion. We identified transient, high-amplitude oscillations in the 7-12 Hz range ("Mu-bursts") in the PFC, parietal cortex and only in the dorsal part of hippocampus. At rest, aged rats displayed more Mu-bursts than adults. Stress acted differently on Mu-bursts depending on age: it increases vs. decreases burst, in adult and aged animals, respectively. In contrast DZP (1 mg/kg) acted the same way in stressed adult and age animal: it decreased the occurrence of Mu-bursts, as well as their co-occurrence. This is consistent with DZP acting as a positive allosteric modulator of GABAA receptors, which globally potentiates inhibition and has anxiolytic effects. Overall, the effect of benzodiazepines on stressed animals was to restore Mu burst activity in adults but to strongly diminish them in aged rats. This work suggests Mu-bursts as a neural marker to study the impact of stress and DZP on age.

Nicotinic receptors in the ventral tegmental area promote uncertainty-seeking.

Cholinergic neurotransmission affects decision-making, notably through the modulation of perceptual processing in the cortex. In addition, acetylcholine acts on value-based decisions through as yet unknown mechanisms. We found that nicotinic acetylcholine receptors (nAChRs) expressed in the ventral tegmental area (VTA) are involved in the translation of expected uncertainty into motivational value. We developed a multi-armed bandit task for mice with three locations, each associated with a different reward probability. We found that mice lacking the nAChR ß2 subunit showed less uncertainty-seeking than their wild-type counterparts. Using model-based analysis, we found that reward uncertainty motivated wild-type mice, but not mice lacking the nAChR ß2 subunit. Selective re-expression of the ß2 subunit in the VTA was sufficient to restore spontaneous bursting activity in dopamine neurons and uncertainty-seeking. Our results reveal an unanticipated role for subcortical nAChRs in motivation induced by expected uncertainty and provide a parsimonious account for a wealth of behaviors related to nAChRs in the VTA expressing the ß2 subunit.

Nicotinic alteration of decision-making.

Addiction to nicotine is characterized by impulses, urges and lack of self-control towards cigarettes. A key element in the process of addiction is the development of habits oriented towards nicotine consumption that surpass flexible systems as a consequence of a gradual adaptation to chronic drug exposure. However, the long-term effects of nicotine on brain circuits also induce wide changes in decision-making processes, affecting behaviors unrelated to cigarettes. This review aims at providing an update on the implications of nicotine on general decision-making processes, with an emphasis on impulsivity and risk-taking. As impulsivity is a rather ambiguous behavioral trait, we build on economic and normative theories to better characterize these nicotine-induced alterations in decision-making. Nonetheless, experimental data are sparse and often contradictory. We will discuss how the latest findings on the neurobiological basis of choice behavior may help disentangling these issues. We focus on the role of nicotine acetylcholine receptors and their different subunits, and on the spatio-temporal dynamics (i.e. diversity of the neural circuits, short- and long-term effects) of both endogenous acetylcholine and nicotine action. Finally, we try to link these neurobiological results with neuro-computational models of attention, valuation and action, and of the role of acetylcholine in these decision processes. This article is part of the Special Issue entitled 'The Nicotinic Acetylcholine Receptor: From Molecular Biology to Cognition'.

Effects of cellular homeostatic intrinsic plasticity on dynamical and computational properties of biological recurrent neural networks.

Homeostatic intrinsic plasticity (HIP) is a ubiquitous cellular mechanism regulating neuronal activity, cardinal for the proper functioning of nervous systems. In invertebrates, HIP is critical for orchestrating stereotyped activity patterns. The functional impact of HIP remains more obscure in vertebrate networks, where higher order cognitive processes rely on complex neural dynamics. The hypothesis has emerged that HIP might control the complexity of activity dynamics in recurrent networks, with important computational consequences. However, conflicting results about the causal relationships between cellular HIP, network dynamics, and computational performance have arisen from machine-learning studies. Here, we assess how cellular HIP effects translate into collective dynamics and computational properties in biological recurrent networks. We develop a realistic multiscale model including a generic HIP rule regulating the neuronal threshold with actual molecular signaling pathways kinetics, Dale's principle, sparse connectivity, synaptic balance, and Hebbian synaptic plasticity (SP). Dynamic mean-field analysis and simulations unravel that HIP sets a working point at which inputs are transduced by large derivative ranges of the transfer function. This cellular mechanism ensures increased network dynamics complexity, robust balance with SP at the edge of chaos, and improved input separability. Although critically dependent upon balanced excitatory and inhibitory drives, these effects display striking robustness to changes in network architecture, learning rates, and input features. Thus, the mechanism we unveil might represent a ubiquitous cellular basis for complex dynamics in neural networks. Understanding this robustness is an important challenge to unraveling principles underlying self-organization around criticality in biological recurrent neural networks.