Distributional reinforcement learning in prefrontal cortex.

The prefrontal cortex is crucial for learning and decision-making. Classic reinforcement learning (RL) theories center on learning the expectation of potential rewarding outcomes and explain a wealth of neural data in the prefrontal cortex. Distributional RL, on the other hand, learns the full distribution of rewarding outcomes and better explains dopamine responses. In the present study, we show that distributional RL also better explains macaque anterior cingulate cortex neuronal responses, suggesting that it is a common mechanism for reward-guided learning.

A cognitive map for value-guided choice in ventromedial prefrontal cortex.

The prefrontal cortex is crucial for economic decision-making and representing the value of options. However, how such representations facilitate flexible decisions remains unknown. We reframe economic decision-making in prefrontal cortex in line with representations of structure within the medial temporal lobe because such cognitive map representations are known to facilitate flexible behaviour. Specifically, we framed choice between different options as a navigation process in value space. Here we show that choices in a 2D value space defined by reward magnitude and probability were represented with a grid-like code, analogous to that found in spatial navigation. The grid-like code was present in ventromedial prefrontal cortex (vmPFC) local field potential theta frequency and the result replicated in an independent dataset. Neurons in vmPFC similarly contained a grid-like code, in addition to encoding the linear value of the chosen option. Importantly, both signals were modulated by theta frequency - occurring at theta troughs but on separate theta cycles. Furthermore, we found sharp-wave ripples - a key neural signature of planning and flexible behaviour - in vmPFC, which were modulated by accuracy and reward. These results demonstrate that multiple cognitive map-like computations are deployed in vmPFC during economic decision-making, suggesting a new framework for the implementation of choice in prefrontal cortex.

Complementary task representations in hippocampus and prefrontal cortex for generalizing the structure of problems.

Humans and other animals effortlessly generalize prior knowledge to solve novel problems, by abstracting common structure and mapping it onto new sensorimotor specifics. To investigate how the brain achieves this, in this study, we trained mice on a series of reversal learning problems that shared the same structure but had different physical implementations. Performance improved across problems, indicating transfer of knowledge. Neurons in medial prefrontal cortex (mPFC) maintained similar representations across problems despite their different sensorimotor correlates, whereas hippocampal (dCA1) representations were more strongly influenced by the specifics of each problem. This was true for both representations of the events that comprised each trial and those that integrated choices and outcomes over multiple trials to guide an animal's decisions. These data suggest that prefrontal cortex and hippocampus play complementary roles in generalization of knowledge: PFC abstracts the common structure among related problems, and hippocampus maps this structure onto the specifics of the current situation.

Mymou: A low-cost, wireless touchscreen system for automated training of nonhuman primates.

Training nonhuman primates (NHPs) to perform cognitive tasks is essential for many neuroscientific investigations, yet laboratory training is a time-consuming process with inherent limitations. Habituating NHPs to the laboratory staff and experimental equipment can take months before NHPs are ready to proceed to the primary tasks. Laboratory training also necessarily separates NHPs from their home-room social group and typically involves some form of restraint or limited mobility, and data collection is often limited to a few hours per day so that multiple NHPs can be trained on the same equipment. Consequently, it can often take a year to train NHPs on complex cognitive tasks. To overcome these issues, we developed a low-cost, open-source, wireless touchscreen training system that can be installed in the home-room environment. The automated device can run continuously all day, including over weekends, without experimenter intervention. The system utilizes real-time facial recognition to initiate subject-specific tasks and provide accurate data logging, without the need for implanted microchips or separation of the NHPs. The system allows NHPs to select their preferred reward on each trial and to work when and for as long as they desire, and it can analyze task performance in real time and adapt the task parameters in order to expedite training. We demonstrate that NHPs consistently use this system on a daily basis to quickly learn complex behavioral tasks. The system therefore addresses many of the welfare and experimental limitations of laboratory-based training of NHPs and provides a platform for wireless electrophysiological investigations in more naturalistic, freely moving environments.

Comparison of three gamma oscillations in the mouse entorhinal-hippocampal system.

The entorhinal-hippocampal system is an important circuit in the brain, essential for certain cognitive tasks such as memory and navigation. Different gamma oscillations occur in this circuit, with the medial entorhinal cortex (mEC), CA3 and CA1 all generating gamma oscillations with different properties. These three gamma oscillations converge within CA1, where much work has gone into trying to isolate them from each other. Here, we compared the gamma generators in the mEC, CA3 and CA1 using optogenetically induced theta-gamma oscillations. Expressing channelrhodopsin-2 in principal neurons in each of the three regions allowed for the induction of gamma oscillations via sinusoidal blue light stimulation at theta frequency. Recording the oscillations in CA1 in vivo, we found that CA3 stimulation induced slower gamma oscillations than CA1 stimulation, matching in vivo reports of spontaneous CA3 and CA1 gamma oscillations. In brain slices ex vivo, optogenetic stimulation of CA3 induced slower gamma oscillations than stimulation of either mEC or CA1, whose gamma oscillations were of similar frequency. All three gamma oscillations had a current sink-source pair between the perisomatic and dendritic layers of the same region. Taking advantage of this model to analyse gamma frequency mechanisms in slice, we showed using pharmacology that all three gamma oscillations were dependent on the same types of synaptic receptor, being abolished by blockade of either type A γ-aminobutyric acid receptors or α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid/kainate receptors, and insensitive to blockade of N-methyl-d-aspartate receptors. These results indicate that a fast excitatory-inhibitory feedback loop underlies the generation of gamma oscillations in all three regions.

Intrinsic Cornu Ammonis Area 1 Theta-Nested Gamma Oscillations Induced by Optogenetic Theta Frequency Stimulation.

Gamma oscillations (30-120 Hz) are thought to be important for various cognitive functions, including perception and working memory, and disruption of these oscillations has been implicated in brain disorders, such as schizophrenia and Alzheimer's disease. The cornu ammonis area 1 (CA1) of the hippocampus receives gamma frequency inputs from upstream regions (cornu ammonis area 3 and medial entorhinal cortex) and generates itself a faster gamma oscillation. The exact nature and origin of the intrinsic CA1 gamma oscillation is still under debate. Here, we expressed channel rhodopsin-2 under the CaMKIIα promoter in mice and prepared hippocampal slices to produce a model of intrinsic CA1 gamma oscillations. Sinusoidal optical stimulation of CA1 at theta frequency was found to induce robust theta-nested gamma oscillations with a temporal and spatial profile similar to CA1 gamma in vivo The results suggest the presence of a single gamma rhythm generator with a frequency range of 65-75 Hz at 32 °C. Pharmacological analysis found that the oscillations depended on both AMPA and GABAA receptors. Cell-attached and whole-cell recordings revealed that excitatory neuron firing slightly preceded interneuron firing within each gamma cycle, suggesting that this intrinsic CA1 gamma oscillation is generated with a pyramidal-interneuron circuit mechanism. SIGNIFICANCE STATEMENT: This study demonstrates that the cornu ammonis area 1 (CA1) is capable of generating intrinsic gamma oscillations in response to theta input. This gamma generator is independent of activity in the upstream regions, highlighting that CA1 can produce its own gamma oscillation in addition to inheriting activity from the upstream regions. This supports the theory that gamma oscillations predominantly function to achieve local synchrony, and that a local gamma generated in each area conducts the signal to the downstream region.

Hippocampal network oscillations - recent insights from in vitro experiments.

Network oscillations are present throughout the mammalian brain. They are important for certain cognitive functions, such as learning and memory. The hippocampus exhibits prominent oscillations similar to those seen in other parts of the cortex. Due to its highly organised lamellar structure, ex vivo and in vitro preparations from the hippocampus have provided experimental models within which to study network oscillations. As such, experiments in hippocampal slices continue to progress our understanding about both the mechanisms and functions of cortical network oscillations. Here, advances from the past two years are summarised, and the current state of the field discussed.

The hippocampal cacophony: multiple layers of communication.

Locally generated gamma oscillations synchronize spikes, but the nature of coupling between regions remains unclear. In this issue of Neuron, Schomburg et al. (2014) show that afferent gamma input fails to entrain hippocampal output, suggesting limited propagation of gamma waves.