Neural responses in macaque prefrontal cortex are linked to strategic exploration.

Humans have been shown to strategically explore. They can identify situations in which gathering information about distant and uncertain options is beneficial for the future. Because primates rely on scarce resources when they forage, they are also thought to strategically explore, but whether they use the same strategies as humans and the neural bases of strategic exploration in monkeys are largely unknown. We designed a sequential choice task to investigate whether monkeys mobilize strategic exploration based on whether information can improve subsequent choice, but also to ask the novel question about whether monkeys adjust their exploratory choices based on the contingency between choice and information, by sometimes providing the counterfactual feedback about the unchosen option. We show that monkeys decreased their reliance on expected value when exploration could be beneficial, but this was not mediated by changes in the effect of uncertainty on choices. We found strategic exploratory signals in anterior and mid-cingulate cortex (ACC/MCC) and dorsolateral prefrontal cortex (dlPFC). This network was most active when a low value option was chosen, which suggests a role in counteracting expected value signals, when exploration away from value should to be considered. Such strategic exploration was abolished when the counterfactual feedback was available. Learning from counterfactual outcome was associated with the recruitment of a different circuit centered on the medial orbitofrontal cortex (OFC), where we showed that monkeys represent chosen and unchosen reward prediction errors. Overall, our study shows how ACC/MCC-dlPFC and OFC circuits together could support exploitation of available information to the fullest and drive behavior towards finding more information through exploration when it is beneficial.

Behavioral flexibility is associated with changes in structure and function distributed across a frontal cortical network in macaques.

One of the most influential accounts of central orbitofrontal cortex-that it mediates behavioral flexibility-has been challenged by the finding that discrimination reversal in macaques, the classic test of behavioral flexibility, is unaffected when lesions are made by excitotoxin injection rather than aspiration. This suggests that the critical brain circuit mediating behavioral flexibility in reversal tasks lies beyond the central orbitofrontal cortex. To determine its identity, a group of nine macaques were taught discrimination reversal learning tasks, and its impact on gray matter was measured. Magnetic resonance imaging scans were taken before and after learning and compared with scans from two control groups, each comprising 10 animals. One control group learned discrimination tasks that were similar but lacked any reversal component, and the other control group engaged in no learning. Gray matter changes were prominent in posterior orbitofrontal cortex/anterior insula but were also found in three other frontal cortical regions: lateral orbitofrontal cortex (orbital part of area 12 [12o]), cingulate cortex, and lateral prefrontal cortex. In a second analysis, neural activity in posterior orbitofrontal cortex/anterior insula was measured at rest, and its pattern of coupling with the other frontal cortical regions was assessed. Activity coupling increased significantly in the reversal learning group in comparison with controls. In a final set of experiments, we used similar structural imaging procedures and analyses to demonstrate that aspiration lesion of central orbitofrontal cortex, of the type known to affect discrimination learning, affected structure and activity in the same frontal cortical circuit. The results identify a distributed frontal cortical circuit associated with behavioral flexibility.

The effect of US signalling and the US-CS interval on backward conditioning in mice.

The effect of US signalling and the US-CS interval in backward conditioning was assessed in mice. For one group of mice the presentation of food was signalled by a tone and for another group, food was unsignalled. For half of the mice, within each group, the presentation of food preceded a visual cue by 10 s. For the other half, food was presented at the start of the visual cue (0-s US-CS interval), resulting in simultaneous pairings of these events. A summation test and a subsequent retardation test were used to assess the inhibitory effects of backward conditioning in comparison to training with a non-reinforced visual cue that controlled for the possible effects of latent inhibition and conditioned inhibition caused as a consequence of differential conditioning. In the summation test unsignalled presentations of the US resulted in inhibition when the US-CS interval was 10 s, but not 0 s. Signalled presentations of the US resulted in inhibition, independent of the US-CS interval. In the retardation test, independent of US signalling, a US-CS interval of 10 s failed to result in inhibition, but an interval of 0 s resulted in greater conditioned responding to the backward CS than the control CS. A generalisation decrement account of the effect of signalling the US with a 0-s US-CS interval, which resulted in reduced responding in the summation test and faster acquisition in the retardation test, is discussed.

Dissociable effects of surprise and model update in parietal and anterior cingulate cortex.

Brains use predictive models to facilitate the processing of expected stimuli or planned actions. Under a predictive model, surprising (low probability) stimuli or actions necessitate the immediate reallocation of processing resources, but they can also signal the need to update the underlying predictive model to reflect changes in the environment. Surprise and updating are often correlated in experimental paradigms but are, in fact, distinct constructs that can be formally defined as the Shannon information (IS) and Kullback-Leibler divergence (DKL) associated with an observation. In a saccadic planning task, we observed that distinct behaviors and brain regions are associated with surprise/IS and updating/DKL. Although surprise/IS was associated with behavioral reprogramming as indexed by slower reaction times, as well as with activity in the posterior parietal cortex [human lateral intraparietal area (LIP)], the anterior cingulate cortex (ACC) was specifically activated during updating of the predictive model (DKL). A second saccade-sensitive region in the inferior posterior parietal cortex (human 7a), which has connections to both LIP and ACC, was activated by surprise and modulated by updating. Pupillometry revealed a further dissociation between surprise and updating with an early positive effect of surprise and late negative effect of updating on pupil area. These results give a computational account of the roles of the ACC and two parietal saccade regions, LIP and 7a, by which their involvement in diverse tasks can be understood mechanistically. The dissociation of functional roles between regions within the reorienting/reprogramming network may also inform models of neurological phenomena, such as extinction and Balint syndrome, and neglect.

Changes in attention to relevant and irrelevant stimuli during spatial learning.

Rats were trained in 2 experiments to find a submerged platform that was situated in 1 of 2 of the 4 corners of a rectangular pool with a curved long wall. Different landmarks occupied 2 of the corners on every trial, and the platform was always situated near a landmark. For the place group in each experiment, the location of the platform was indicated by the shape of the pool and stimuli outside the pool (place cues), but not the landmarks within the pool. For the landmark groups, the landmarks, not the place cues, indicated where the platform could be found. During Stage 2, 2 of the place cues were relevant, and 2 of the landmarks were irrelevant, for a new discrimination. The place cues better controlled searching for the platform in the place group than in the landmark group when the place cues had initially been relevant by signaling the presence (Experiment 1) or the absence (Experiment 2) of the platform. The results show that animals pay more attention to relevant than irrelevant cues.

Latent spatial learning in an environment with a distinctive shape.

Four experiments were conducted with rats in order to determine whether being placed on a platform in one corner of a rectangular swimming pool results in latent spatial learning. Rats in Experiments 1-3 received four trials a day of being placed on the platform. During a subsequent test trial, in which they were released into the pool without the platform, the rats exhibited a preference for swimming in the correct corners of the pool (those with the same geometric properties as the corner containing the platform during training), than the two remaining, incorrect corners. This effect was seen when the interval between the final placement trial and the test trial was as much as 24 hr (Experiment 2) and after varying numbers of sessions of placement training (Experiment 3). Experiment 4 revealed that when the test took place in a kite-shaped arena, after placement training in a rectangle, a stronger preference was shown for the corner that was geometrically equivalent to the correct rather than the incorrect corners in the rectangle. The placement treatment is said to result in latent spatial learning based on the development of S-S associations.