Author Correction: A hippocampo-cerebellar centred network for the learning and execution of sequence-based navigation.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Belief state representation in the dopamine system.

Learning to predict future outcomes is critical for driving appropriate behaviors. Reinforcement learning (RL) models have successfully accounted for such learning, relying on reward prediction errors (RPEs) signaled by midbrain dopamine neurons. It has been proposed that when sensory data provide only ambiguous information about which state an animal is in, it can predict reward based on a set of probabilities assigned to hypothetical states (called the belief state). Here we examine how dopamine RPEs and subsequent learning are regulated under state uncertainty. Mice are first trained in a task with two potential states defined by different reward amounts. During testing, intermediate-sized rewards are given in rare trials. Dopamine activity is a non-monotonic function of reward size, consistent with RL models operating on belief states. Furthermore, the magnitude of dopamine responses quantitatively predicts changes in behavior. These results establish the critical role of state inference in RL.

Functional circuit architecture underlying parental behaviour.

Parenting is essential for the survival and wellbeing of mammalian offspring. However, we lack a circuit-level understanding of how distinct components of this behaviour are coordinated. Here we investigate how galanin-expressing neurons in the medial preoptic area (MPOAGal) of the hypothalamus coordinate motor, motivational, hormonal and social aspects of parenting in mice. These neurons integrate inputs from a large number of brain areas and the activation of these inputs depends on the animal's sex and reproductive state. Subsets of MPOAGal neurons form discrete pools that are defined by their projection sites. While the MPOAGal population is active during all episodes of parental behaviour, individual pools are tuned to characteristic aspects of parenting. Optogenetic manipulation of MPOAGal projections mirrors this specificity, affecting discrete parenting components. This functional organization, reminiscent of the control of motor sequences by pools of spinal cord neurons, provides a new model for how discrete elements of a social behaviour are generated at the circuit level.

A hippocampo-cerebellar centred network for the learning and execution of sequence-based navigation.

How do we translate self-motion into goal-directed actions? Here we investigate the cognitive architecture underlying self-motion processing during exploration and goal-directed behaviour. The task, performed in an environment with limited and ambiguous external landmarks, constrained mice to use self-motion based information for sequence-based navigation. The post-behavioural analysis combined brain network characterization based on c-Fos imaging and graph theory analysis as well as computational modelling of the learning process. The study revealed a widespread network centred around the cerebral cortex and basal ganglia during the exploration phase, while a network dominated by hippocampal and cerebellar activity appeared to sustain sequence-based navigation. The learning process could be modelled by an algorithm combining memory of past actions and model-free reinforcement learning, which parameters pointed toward a central role of hippocampal and cerebellar structures for learning to translate self-motion into a sequence of goal-directed actions.

Dopamine reward prediction errors reflect hidden-state inference across time.

Midbrain dopamine neurons signal reward prediction error (RPE), or actual minus expected reward. The temporal difference (TD) learning model has been a cornerstone in understanding how dopamine RPEs could drive associative learning. Classically, TD learning imparts value to features that serially track elapsed time relative to observable stimuli. In the real world, however, sensory stimuli provide ambiguous information about the hidden state of the environment, leading to the proposal that TD learning might instead compute a value signal based on an inferred distribution of hidden states (a 'belief state'). Here we asked whether dopaminergic signaling supports a TD learning framework that operates over hidden states. We found that dopamine signaling showed a notable difference between two tasks that differed only with respect to whether reward was delivered in a deterministic manner. Our results favor an associative learning rule that combines cached values with hidden-state inference.

Opposite initialization to novel cues in dopamine signaling in ventral and posterior striatum in mice.

Dopamine neurons are thought to encode novelty in addition to reward prediction error (the discrepancy between actual and predicted values). In this study, we compared dopamine activity across the striatum using fiber fluorometry in mice. During classical conditioning, we observed opposite dynamics in dopamine axon signals in the ventral striatum ('VS dopamine') and the posterior tail of the striatum ('TS dopamine'). TS dopamine showed strong excitation to novel cues, whereas VS dopamine showed no responses to novel cues until they had been paired with a reward. TS dopamine cue responses decreased over time, depending on what the cue predicted. Additionally, TS dopamine showed excitation to several types of stimuli including rewarding, aversive, and neutral stimuli whereas VS dopamine showed excitation only to reward or reward-predicting cues. Together, these results demonstrate that dopamine novelty signals are localized in TS along with general salience signals, while VS dopamine reliably encodes reward prediction error.

How the cerebellum may monitor sensory information for spatial representation.

The cerebellum has already been shown to participate in the navigation function. We propose here that this structure is involved in maintaining a sense of direction and location during self-motion by monitoring sensory information and interacting with navigation circuits to update the mental representation of space. To better understand the processing performed by the cerebellum in the navigation function, we have reviewed: the anatomical pathways that convey self-motion information to the cerebellum; the computational algorithm(s) thought to be performed by the cerebellum from these multi-source inputs; the cerebellar outputs directed toward navigation circuits and the influence of self-motion information on space-modulated cells receiving cerebellar outputs. This review highlights that the cerebellum is adequately wired to combine the diversity of sensory signals to be monitored during self-motion and fuel the navigation circuits. The direct anatomical projections of the cerebellum toward the head-direction cell system and the parietal cortex make those structures possible relays of the cerebellum influence on the hippocampal spatial map. We describe computational models of the cerebellar function showing that the cerebellum can filter out the components of the sensory signals that are predictable, and provides a novelty output. We finally speculate that this novelty output is taken into account by the navigation structures, which implement an update over time of position and stabilize perception during navigation.

Complementary Roles of the Hippocampus and the Dorsomedial Striatum during Spatial and Sequence-Based Navigation Behavior.

We investigated the neural bases of navigation based on spatial or sequential egocentric representation during the completion of the starmaze, a complex goal-directed navigation task. In this maze, mice had to swim along a path composed of three choice points to find a hidden platform. As reported previously, this task can be solved by using two hippocampal-dependent strategies encoded in parallel i) the allocentric strategy requiring encoding of the contextual information, and ii) the sequential egocentric strategy requiring temporal encoding of a sequence of successive body movements associated to specific choice points. Mice were trained during one day and tested the following day in a single probe trial to reveal which of the two strategies was spontaneously preferred by each animal. Imaging of the activity-dependent gene c-fos revealed that both strategies are supported by an overlapping network involving the dorsal hippocampus, the dorsomedial striatum (DMS) and the medial prefrontal cortex. A significant higher activation of the ventral CA1 subregion was observed when mice used the sequential egocentric strategy. To investigate the potential different roles of the dorsal hippocampus and the DMS in both types of navigation, we performed region-specific excitotoxic lesions of each of these two structures. Dorsal hippocampus lesioned mice were unable to optimally learn the sequence but improved their performances by developing a serial strategy instead. DMS lesioned mice were severely impaired, failing to learn the task. Our data support the view that the hippocampus organizes information into a spatio-temporal representation, which can then be used by the DMS to perform goal-directed navigation.

From ear to hand: the role of the auditory-motor loop in pointing to an auditory source.

Studies of the nature of the neural mechanisms involved in goal-directed movements tend to concentrate on the role of vision. We present here an attempt to address the mechanisms whereby an auditory input is transformed into a motor command. The spatial and temporal organization of hand movements were studied in normal human subjects as they pointed toward unseen auditory targets located in a horizontal plane in front of them. Positions and movements of the hand were measured by a six infrared camera tracking system. In one condition, we assessed the role of auditory information about target position in correcting the trajectory of the hand. To accomplish this, the duration of the target presentation was varied. In another condition, subjects received continuous auditory feedback of their hand movement while pointing to the auditory targets. Online auditory control of the direction of pointing movements was assessed by evaluating how subjects reacted to shifts in heard hand position. Localization errors were exacerbated by short duration of target presentation but not modified by auditory feedback of hand position. Long duration of target presentation gave rise to a higher level of accuracy and was accompanied by early automatic head orienting movements consistently related to target direction. These results highlight the efficiency of auditory feedback processing in online motor control and suggest that the auditory system takes advantages of dynamic changes of the acoustic cues due to changes in head orientation in order to process online motor control. How to design an informative acoustic feedback needs to be carefully studied to demonstrate that auditory feedback of the hand could assist the monitoring of movements directed at objects in auditory space.