Correction: Intrinsic motivation for choice varies with individual risk attitudes and the controllability of the environment.

[This corrects the article DOI: 10.1371/journal.pcbi.1010551.].

Intrinsic motivation for choice varies with individual risk attitudes and the controllability of the environment.

When deciding between options that do or do not lead to future choices, humans often choose to choose. We studied choice seeking by asking subjects to first decide between a choice opportunity or performing a computer-selected action, after which they either chose freely or performed the forced action. Subjects preferred choice when these options were equally rewarded, even deterministically, and traded extrinsic rewards for opportunities to choose. We explained individual variability in choice seeking using reinforcement learning models incorporating risk sensitivity and overvaluation of rewards obtained through choice. Model fits revealed that 28% of subjects were sensitive to the worst possible outcome associated with free choice, and this pessimism reduced their choice preference with increasing risk. Moreover, outcome overvaluation was necessary to explain patterns of individual choice preference across levels of risk. We also manipulated the degree to which subjects controlled stimulus outcomes. We found that degrading coherence between their actions and stimulus outcomes diminished choice preference following forced actions, although willingness to repeat selection of choice opportunities remained high. When subjects chose freely during these repeats, they were sensitive to rewards when actions were controllable but ignored outcomes-even positive ones-associated with reduced controllability. Our results show that preference for choice can be modulated by extrinsic reward properties including reward probability and risk as well as by controllability of the environment.

Can we tackle climate change by behavioral hacking of the dopaminergic system?

Climate change is an undeniable fact that will certainly affect millions of people in the following decades. Despite this danger threatening our economies, wellbeing and our lives in general, there is a lack of immediate response at both the institutional and individual level. How can it be that the human brain cannot interpret this threat and act against it to avoid the immense negative consequences that may ensue? Here we argue that this paradox could be explained by the fact that some key brain mechanisms are potentially poorly tuned to take action against a threat that would take full effect only in the long-term. We present neuro-behavioral evidence in favor of this proposal and discuss the role of the dopaminergic (DA) system in learning accurate prediction of the value of an outcome, and its consequences regarding the climate issue. We discuss how this system discounts the value of delayed outcomes and, consequently, does not favor action against the climate crisis. Finally, according to this framework, we suggest that this view may be reconsidered and, on the contrary, that the DA reinforcement learning system could be a powerful ally if adapted to short-term incentives which promote climate-friendly behaviors. Additionally, the DA system interacts with multiple brain systems, in particular those related to higher cognitive functions, which can adjust its functions depending on psychological, social, or other complex contextual information. Thus, we propose several generic action plans that could help to hack these neuro-behavioral processes to promote climate-friendly actions.

The Geometry of Abstraction in the Hippocampus and Prefrontal Cortex.

The curse of dimensionality plagues models of reinforcement learning and decision making. The process of abstraction solves this by constructing variables describing features shared by different instances, reducing dimensionality and enabling generalization in novel situations. Here, we characterized neural representations in monkeys performing a task described by different hidden and explicit variables. Abstraction was defined operationally using the generalization performance of neural decoders across task conditions not used for training, which requires a particular geometry of neural representations. Neural ensembles in prefrontal cortex, hippocampus, and simulated neural networks simultaneously represented multiple variables in a geometry reflecting abstraction but that still allowed a linear classifier to decode a large number of other variables (high shattering dimensionality). Furthermore, this geometry changed in relation to task events and performance. These findings elucidate how the brain and artificial systems represent variables in an abstract format while preserving the advantages conferred by high shattering dimensionality.

The role of the posterior parietal cortex in saccadic error processing.

Ocular saccades rapidly displace the fovea from one point of interest to another, thus minimizing the loss of visual information and ensuring the seamless continuity of visual perception. However, because of intrinsic variability in sensory-motor processing, saccades often miss their intended target, necessitating a secondary corrective saccade. Behavioral evidence suggests that the oculomotor system estimates saccadic error by relying on two sources of information: the retinal feedback obtained post-saccadically and an internal extra-retinal signal obtained from efference copy or proprioception. However, the neurophysiological mechanisms underlying this process remain elusive. We trained two rhesus monkeys to perform visually guided saccades towards a target that was imperceptibly displaced at saccade onset on some trials. We recorded activity from neurons in the lateral intraparietal area (LIP), an area implicated in visual, attentional and saccadic processing. We found that a subpopulation of neurons detect saccadic motor error by firing more strongly after an inaccurate saccade. This signal did not depend on retinal feedback or on the execution of a secondary corrective saccade. Moreover, inactivating LIP led to a large and selective increase in the latency of small (i.e., natural) corrective saccade initiation. Our results indicate a key role for LIP in saccadic error processing.

Social Hierarchy Representation in the Primate Amygdala Reflects the Emotional Ambiguity of Our Social Interactions.

Group living can help individuals defend against predators and acquire nutrition. However, conflicts between group members can arise (food sharing, mating, etc), requiring individuals to know the social status of each member to promote survival. In our recent paper, we sought to understand how the brain represents the social status of monkeys living in the same colony. Primates learn the social status of their peers through experience, including observation and direct interactions, just like they learn the rewarding or aversive nature of stimuli that predict different types of reinforcement. Group members may thereby be viewed as differing in value. We found in the amygdala, a brain area specialized for emotion, a neural representation of social hierarchy embedded in the same neuronal ensemble engaged in the assignment of motivational significance to previously neutral stimuli. Interestingly, we found 2 subpopulations of amygdala neurons encoding the social status of individuals in an opposite manner. In response to a stimulus, one population encodes similarly appetitive nonsocial images and dominant monkeys as well as aversive nonsocial stimuli and submissive monkeys. The other population encodes the opposite pattern later in time. This mechanism could reflect the emotional ambiguity we face in social situations as each interaction is potentially positive (eg, food access, protection, promotion) or negative (eg, aggression, bullying).

Shared neural coding for social hierarchy and reward value in primate amygdala.

The social brain hypothesis posits that dedicated neural systems process social information. In support of this, neurophysiological data have shown that some brain regions are specialized for representing faces. It remains unknown, however, whether distinct anatomical substrates also represent more complex social variables, such as the hierarchical rank of individuals within a social group. Here we show that the primate amygdala encodes the hierarchical rank of individuals in the same neuronal ensembles that encode the rewards associated with nonsocial stimuli. By contrast, orbitofrontal and anterior cingulate cortices lack strong representations of hierarchical rank while still representing reward values. These results challenge the conventional view that dedicated neural systems process social information. Instead, information about hierarchical rank-which contributes to the assessment of the social value of individuals within a group-is linked in the amygdala to representations of rewards associated with nonsocial stimuli.

Optimal sensorimotor control in eye movement sequences.

Fast and accurate motor behavior requires combining noisy and delayed sensory information with knowledge of self-generated body motion; much evidence indicates that humans do this in a near-optimal manner during arm movements. However, it is unclear whether this principle applies to eye movements. We measured the relative contributions of visual sensory feedback and the motor efference copy (and/or proprioceptive feedback) when humans perform two saccades in rapid succession, the first saccade to a visual target and the second to a memorized target. Unbeknownst to the subject, we introduced an artificial motor error by randomly "jumping" the visual target during the first saccade. The correction of the memory-guided saccade allowed us to measure the relative contributions of visual feedback and efferent copy (and/or proprioceptive feedback) to motor-plan updating. In a control experiment, we extinguished the target during the saccade rather than changing its location to measure the relative contribution of motor noise and target localization error to saccade variability without any visual feedback. The motor noise contribution increased with saccade amplitude, but remained <30% of the total variability. Subjects adjusted the gain of their visual feedback for different saccade amplitudes as a function of its reliability. Even during trials where subjects performed a corrective saccade to compensate for the target-jump, the correction by the visual feedback, while stronger, remained far below 100%. In all conditions, an optimal controller predicted the visual feedback gain well, suggesting that humans combine optimally their efferent copy and sensory feedback when performing eye movements.