Not so smart? "Smart" drugs increase the level but decrease the quality of cognitive effort.

The efficacy of pharmaceutical cognitive enhancers in everyday complex tasks remains to be established. Using the knapsack optimization problem as a stylized representation of difficulty in tasks encountered in daily life, we discover that methylphenidate, dextroamphetamine, and modafinil cause knapsack value attained in the task to diminish significantly compared to placebo, even if the chance of finding the optimal solution (~50%) is not reduced significantly. Effort (decision time and number of steps taken to find a solution) increases significantly, but productivity (quality of effort) decreases significantly. At the same time, productivity differences across participants decrease, even reverse, to the extent that above-average performers end up below average and vice versa. The latter can be attributed to increased randomness of solution strategies. Our findings suggest that "smart drugs" increase motivation, but a reduction in quality of effort, crucial to solve complex problems, annuls this effect.

Task-independent metrics of computational hardness predict human cognitive performance.

The survival of human organisms depends on our ability to solve complex tasks in the face of limited cognitive resources. However, little is known about the factors that drive the complexity of those tasks. Here, building on insights from computational complexity theory, we quantify the computational hardness of cognitive tasks using a set of task-independent metrics related to the computational resource requirements of individual instances of a task. We then examine the relation between those metrics and human behavior and find that they predict both time spent on a task as well as accuracy in three canonical cognitive tasks. Our findings demonstrate that performance in cognitive tasks can be predicted based on generic metrics of their inherent computational hardness.

Epilepsy and Ecstatic Experiences: The Role of the Insula.

Ecstatic epilepsy is a rare form of focal epilepsy in which the aura (beginning of the seizures) consists of a blissful state of mental clarity/feeling of certainty. Such a state has also been described as a "religious" or mystical experience. While this form of epilepsy has long been recognized as a temporal lobe epilepsy, we have accumulated evidence converging toward the location of the symptomatogenic zone in the dorsal anterior insula during the 10 last years. The neurocognitive hypothesis for the genesis of a mental clarity is the suppression of the interoceptive prediction errors and of the unexpected surprise associated with any incoming internal or external signal, usually processed by the dorsal anterior insula. This mimics a perfect prediction of the world and induces a feeling of certainty. The ecstatic epilepsy is thus an amazing model for the role of anterior insula in uncertainty and surprise.

Transferring cognitive talent across domains to reduce the disposition effect in investment.

We consider Theory of Mind (ToM), the ability to correctly predict the intentions of others. To an important degree, good ToM function requires abstraction from one's own particular circumstances. Here, we posit that such abstraction can be transferred successfully to other, non-social contexts. We consider the disposition effect, which is a pervasive cognitive bias whereby investors, including professionals, improperly take their personal trading history into account when deciding on investments. We design an intervention policy whereby we attempt to transfer good ToM function, subconsciously, to personal investment decisions. In a within-subject repeated-intervention laboratory experiment, we record how the disposition effect is reduced by a very significant 85%, but only for those with high scores on the social-cognitive dimension of ToM function. No such transfer is observed in subjects who score well only on the social-perceptual dimension of ToM function. Our findings open up a promising way to exploit cognitive talent in one domain in order to alleviate cognitive deficiencies elsewhere.

How Neurobiology Elucidates the Role of Emotions in Financial Decision-Making.

Over the last 15 years, a revolution has been taking place in neuroscience, whereby models and methods of economics have led to deeper insights into the neurobiological foundations of human decision-making. These have revealed a number of widespread mis-conceptions, among others, about the role of emotions. Furthermore, the findings suggest that a purely behavior-based approach to studying decisions may miss crucial features of human choice long appreciated in biology, such as Pavlovian approach. The findings could help economists formalize elusive concepts such as intuition, as I show here for financial "trading intuition."

Uncertainty and computational complexity.

Modern theories of decision-making typically model uncertainty about decision options using the tools of probability theory. This is exemplified by the Savage framework, the most popular framework in decision-making research. There, decision-makers are assumed to choose from among available decision options as if they maximized subjective expected utility, which is given by the utilities of outcomes in different states weighted with subjective beliefs about the occurrence of those states. Beliefs are captured by probabilities and new information is incorporated using Bayes' Law. The primary concern of the Savage framework is to ensure that decision-makers' choices are rational. Here, we use concepts from computational complexity theory to expose two major weaknesses of the framework. Firstly, we argue that in most situations, subjective utility maximization is computationally intractable, which means that the Savage axioms are implausible. We discuss empirical evidence supporting this claim. Secondly, we argue that there exist many decision situations in which the nature of uncertainty is such that (random) sampling in combination with Bayes' Law is an ineffective strategy to reduce uncertainty. We discuss several implications of these weaknesses from both an empirical and a normative perspective. This article is part of the theme issue 'Risk taking and impulsive behaviour: fundamental discoveries, theoretical perspectives and clinical implications'.

Separating Probability and Reversal Learning in a Novel Probabilistic Reversal Learning Task for Mice.

The exploration/exploitation tradeoff - pursuing a known reward vs. sampling from lesser known options in the hope of finding a better payoff - is a fundamental aspect of learning and decision making. In humans, this has been studied using multi-armed bandit tasks. The same processes have also been studied using simplified probabilistic reversal learning (PRL) tasks with binary choices. Our investigations suggest that protocols previously used to explore PRL in mice may prove beyond their cognitive capacities, with animals performing at a no-better-than-chance level. We sought a novel probabilistic learning task to improve behavioral responding in mice, whilst allowing the investigation of the exploration/exploitation tradeoff in decision making. To achieve this, we developed a two-lever operant chamber task with levers corresponding to different probabilities (high/low) of receiving a saccharin reward, reversing the reward contingencies associated with levers once animals reached a threshold of 80% responding at the high rewarding lever. We found that, unlike in existing PRL tasks, mice are able to learn and behave near optimally with 80% high/20% low reward probabilities. Altering the reward contingencies towards equality showed that some mice displayed preference for the high rewarding lever with probabilities as close as 60% high/40% low. Additionally, we show that animal choice behavior can be effectively modelled using reinforcement learning (RL) models incorporating learning rates for positive and negative prediction error, a perseveration parameter, and a noise parameter. This new decision task, coupled with RL analyses, advances access to investigate the neuroscience of the exploration/exploitation tradeoff in decision making.

Formalizing the Function of Anterior Insula in Rapid Adaptation.

Anterior insula (aIns) is thought to play a crucial role in rapid adaptation in an ever-changing environment. Mathematically, it is known to track risk and surprise. Modern theories of learning, however, assign a dominant role to signed prediction errors (PEs), not to risk and surprise. Risk and surprise only enter to the extent that they modulate the learning rate, in an attempt to approximate Bayesian learning. Even without such modulation, adaptation is still possible, albeit slow. Here, I propose a new theory of learning, reference-model based learning (RMBL), where risk and surprise are central, and PEs play a secondary, though still crucial, role. The primary goal is to bring outcomes in line with expectations in the reference model (RM). Learning is modulated by how large the PEs are relative to model anticipation, i.e., to surprise as defined by the RM. In a target location prediction task where participants were continuously required to adapt, choices appeared to be closer with to RMBL predictions than to Bayesian learning. aIns reaction to surprise was more acute in the more difficult treatment, consistent with its hypothesized role in metacognition. I discuss links with related theories, such as Active Inference, Actor-Critic Models and Reference-Model Based Adaptive Control.

Computational Complexity and Human Decision-Making.

The rationality principle postulates that decision-makers always choose the best action available to them. It underlies most modern theories of decision-making. The principle does not take into account the difficulty of finding the best option. Here, we propose that computational complexity theory (CCT) provides a framework for defining and quantifying the difficulty of decisions. We review evidence showing that human decision-making is affected by computational complexity. Building on this evidence, we argue that most models of decision-making, and metacognition, are intractable from a computational perspective. To be plausible, future theories of decision-making will need to take into account both the resources required for implementing the computations implied by the theory, and the resource constraints imposed on the decision-maker by biology.

Neural computations underlying inverse reinforcement learning in the human brain.

In inverse reinforcement learning an observer infers the reward distribution available for actions in the environment solely through observing the actions implemented by another agent. To address whether this computational process is implemented in the human brain, participants underwent fMRI while learning about slot machines yielding hidden preferred and non-preferred food outcomes with varying probabilities, through observing the repeated slot choices of agents with similar and dissimilar food preferences. Using formal model comparison, we found that participants implemented inverse RL as opposed to a simple imitation strategy, in which the actions of the other agent are copied instead of inferring the underlying reward structure of the decision problem. Our computational fMRI analysis revealed that anterior dorsomedial prefrontal cortex encoded inferences about action-values within the value space of the agent as opposed to that of the observer, demonstrating that inverse RL is an abstract cognitive process divorceable from the values and concerns of the observer him/herself.

How Humans Solve Complex Problems: The Case of the Knapsack Problem.

Life presents us with problems of varying complexity. Yet, complexity is not accounted for in theories of human decision-making. Here we study instances of the knapsack problem, a discrete optimisation problem commonly encountered at all levels of cognition, from attention gating to intellectual discovery. Complexity of this problem is well understood from the perspective of a mechanical device like a computer. We show experimentally that human performance too decreased with complexity as defined in computer science. Defying traditional economic principles, participants spent effort way beyond the point where marginal gain was positive, and economic performance increased with instance difficulty. Human attempts at solving the instances exhibited commonalities with algorithms developed for computers, although biological resource constraints-limited working and episodic memories-had noticeable impact. Consistent with the very nature of the knapsack problem, only a minority of participants found the solution-often quickly-but the ones who did appeared not to realise. Substantial heterogeneity emerged, suggesting why prizes and patents, schemes that incentivise intellectual discovery but discourage information sharing, have been found to be less effective than mechanisms that reveal private information, such as markets.

Decision Neuroscience: Why We Become More Cautious with Age.

A recent study suggests that risk-taking decreases with age and that this may be related to dopamine-modulated changes in Pavlovian approach behavior, and not a reduction in the subjective value of incremental rewards as traditional models from economics and psychology would have claimed.

Behavioral contagion during learning about another agent's risk-preferences acts on the neural representation of decision-risk.

Our attitude toward risk plays a crucial role in influencing our everyday decision-making. Despite its importance, little is known about how human risk-preference can be modulated by observing risky behavior in other agents at either the behavioral or the neural level. Using fMRI combined with computational modeling of behavioral data, we show that human risk-preference can be systematically altered by the act of observing and learning from others' risk-related decisions. The contagion is driven specifically by brain regions involved in the assessment of risk: the behavioral shift is implemented via a neural representation of risk in the caudate nucleus, whereas the representations of other decision-related variables such as expected value are not affected. Furthermore, we uncover neural computations underlying learning about others' risk-preferences and describe how these signals interact with the neural representation of risk in the caudate. Updating of the belief about others' preferences is associated with neural activity in the dorsolateral prefrontal cortex (dlPFC). Functional coupling between the dlPFC and the caudate correlates with the degree of susceptibility to the contagion effect, suggesting that a frontal-subcortical loop, the so-called dorsolateral prefrontal-striatal circuit, underlies the modulation of risk-preference. Taken together, these findings provide a mechanistic account for how observation of others' risky behavior can modulate an individual's own risk-preference.

Neural Mechanisms Behind Identification of Leptokurtic Noise and Adaptive Behavioral Response.

Large-scale human interaction through, for example, financial markets causes ceaseless random changes in outcome variability, producing frequent and salient outliers that render the outcome distribution more peaked than the Gaussian distribution, and with longer tails. Here, we study how humans cope with this evolutionary novel leptokurtic noise, focusing on the neurobiological mechanisms that allow the brain, 1) to recognize the outliers as noise and 2) to regulate the control necessary for adaptive response. We used functional magnetic resonance imaging, while participants tracked a target whose movements were affected by leptokurtic noise. After initial overreaction and insufficient subsequent correction, participants improved performance significantly. Yet, persistently long reaction times pointed to continued need for vigilance and control. We ran a contrasting treatment where outliers reflected permanent moves of the target, as in traditional mean-shift paradigms. Importantly, outliers were equally frequent and salient. There, control was superior and reaction time was faster. We present a novel reinforcement learning model that fits observed choices better than the Bayes-optimal model. Only anterior insula discriminated between the 2 types of outliers. In both treatments, outliers initially activated an extensive bottom-up attention and belief network, followed by sustained engagement of the fronto-parietal control network.

Modeling the Evolution of Beliefs Using an Attentional Focus Mechanism.

For making decisions in everyday life we often have first to infer the set of environmental features that are relevant for the current task. Here we investigated the computational mechanisms underlying the evolution of beliefs about the relevance of environmental features in a dynamical and noisy environment. For this purpose we designed a probabilistic Wisconsin card sorting task (WCST) with belief solicitation, in which subjects were presented with stimuli composed of multiple visual features. At each moment in time a particular feature was relevant for obtaining reward, and participants had to infer which feature was relevant and report their beliefs accordingly. To test the hypothesis that attentional focus modulates the belief update process, we derived and fitted several probabilistic and non-probabilistic behavioral models, which either incorporate a dynamical model of attentional focus, in the form of a hierarchical winner-take-all neuronal network, or a diffusive model, without attention-like features. We used Bayesian model selection to identify the most likely generative model of subjects' behavior and found that attention-like features in the behavioral model are essential for explaining subjects' responses. Furthermore, we demonstrate a method for integrating both connectionist and Bayesian models of decision making within a single framework that allowed us to infer hidden belief processes of human subjects.

Neural mechanisms underlying human consensus decision-making.

Consensus building in a group is a hallmark of animal societies, yet little is known about its underlying computational and neural mechanisms. Here, we applied a computational framework to behavioral and fMRI data from human participants performing a consensus decision-making task with up to five other participants. We found that participants reached consensus decisions through integrating their own preferences with information about the majority group members' prior choices, as well as inferences about how much each option was stuck to by the other people. These distinct decision variables were separately encoded in distinct brain areas-the ventromedial prefrontal cortex, posterior superior temporal sulcus/temporoparietal junction, and intraparietal sulcus-and were integrated in the dorsal anterior cingulate cortex. Our findings provide support for a theoretical account in which collective decisions are made through integrating multiple types of inference about oneself, others, and environments, processed in distinct brain modules.

Economic choices reveal probability distortion in macaque monkeys.

Economic choices are largely determined by two principal elements, reward value (utility) and probability. Although nonlinear utility functions have been acknowledged for centuries, nonlinear probability weighting (probability distortion) was only recently recognized as a ubiquitous aspect of real-world choice behavior. Even when outcome probabilities are known and acknowledged, human decision makers often overweight low probability outcomes and underweight high probability outcomes. Whereas recent studies measured utility functions and their corresponding neural correlates in monkeys, it is not known whether monkeys distort probability in a manner similar to humans. Therefore, we investigated economic choices in macaque monkeys for evidence of probability distortion. We trained two monkeys to predict reward from probabilistic gambles with constant outcome values (0.5 ml or nothing). The probability of winning was conveyed using explicit visual cues (sector stimuli). Choices between the gambles revealed that the monkeys used the explicit probability information to make meaningful decisions. Using these cues, we measured probability distortion from choices between the gambles and safe rewards. Parametric modeling of the choices revealed classic probability weighting functions with inverted-S shape. Therefore, the animals overweighted low probability rewards and underweighted high probability rewards. Empirical investigation of the behavior verified that the choices were best explained by a combination of nonlinear value and nonlinear probability distortion. Together, these results suggest that probability distortion may reflect evolutionarily preserved neuronal processing.

Chimpanzee choice rates in competitive games match equilibrium game theory predictions.

The capacity for strategic thinking about the payoff-relevant actions of conspecifics is not well understood across species. We use game theory to make predictions about choices and temporal dynamics in three abstract competitive situations with chimpanzee participants. Frequencies of chimpanzee choices are extremely close to equilibrium (accurate-guessing) predictions, and shift as payoffs change, just as equilibrium theory predicts. The chimpanzee choices are also closer to the equilibrium prediction, and more responsive to past history and payoff changes, than two samples of human choices from experiments in which humans were also initially uninformed about opponent payoffs and could not communicate verbally. The results are consistent with a tentative interpretation of game theory as explaining evolved behavior, with the additional hypothesis that chimpanzees may retain or practice a specialized capacity to adjust strategy choice during competition to perform at least as well as, or better than, humans have.

Using Neural Data to Test A Theory of Investor Behavior: An Application to Realization Utility.

We use measures of neural activity provided by functional magnetic resonance imaging (fMRI) to test the "realization utility" theory of investor behavior, which posits that people derive utility directly from the act of realizing gains and losses. Subjects traded stocks in an experimental market while we measured their brain activity. We find that all subjects exhibit a strong disposition effect in their trading, even though it is suboptimal. Consistent with the realization utility explanation for this behavior, we find that activity in the ventromedial prefrontal cortex, an area known to encode the value of options during choices, correlates with the capital gains of potential trades; that the neural measures of realization utility correlate across subjects with their individual tendency to exhibit a disposition effect; and that activity in the ventral striatum, an area known to encode information about changes in the present value of experienced utility, exhibits a positive response when subjects realize capital gains. These results provide support for the realization utility model and, more generally, demonstrate how neural data can be helpful in testing models of investor behavior.

In the mind of the market: theory of mind biases value computation during financial bubbles.

The ability to infer intentions of other agents, called theory of mind (ToM), confers strong advantages for individuals in social situations. Here, we show that ToM can also be maladaptive when people interact with complex modern institutions like financial markets. We tested participants who were investing in an experimental bubble market, a situation in which the price of an asset is much higher than its underlying fundamental value. We describe a mechanism by which social signals computed in the dorsomedial prefrontal cortex affect value computations in ventromedial prefrontal cortex, thereby increasing an individual's propensity to 'ride' financial bubbles and lose money. These regions compute a financial metric that signals variations in order flow intensity, prompting inference about other traders' intentions. Our results suggest that incorporating inferences about the intentions of others when making value judgments in a complex financial market could lead to the formation of market bubbles.