Using games to understand the mind.

Board, card or video games have been played by virtually every individual in the world. Games are popular because they are intuitive and fun. These distinctive qualities of games also make them ideal for studying the mind. By being intuitive, games provide a unique vantage point for understanding the inductive biases that support behaviour in more complex, ecological settings than traditional laboratory experiments. By being fun, games allow researchers to study new questions in cognition such as the meaning of 'play' and intrinsic motivation, while also supporting more extensive and diverse data collection by attracting many more participants. We describe the advantages and drawbacks of using games relative to standard laboratory-based experiments and lay out a set of recommendations on how to gain the most from using games to study cognition. We hope this Perspective will lead to a wider use of games as experimental paradigms, elevating the ecological validity, scale and robustness of research on the mind.

Opportunity cost determines free-operant action initiation latency and predicts apathy.

BACKGROUND: Apathy, a disabling and poorly understood neuropsychiatric symptom, is characterised by impaired self-initiated behaviour. It has been hypothesised that the opportunity cost of time (OCT) may be a key computational variable linking self-initiated behaviour with motivational status. OCT represents the amount of reward which is foregone per second if no action is taken. Using a novel behavioural task and computational modelling, we investigated the relationship between OCT, self-initiation and apathy. We predicted that higher OCT would engender shorter action latencies, and that individuals with greater sensitivity to OCT would have higher behavioural apathy. METHODS: We modulated the OCT in a novel task called the 'Fisherman Game', Participants freely chose when to self-initiate actions to either collect rewards, or on occasion, to complete non-rewarding actions. We measured the relationship between action latencies, OCT and apathy for each participant across two independent non-clinical studies, one under laboratory conditions (n = 21) and one online (n = 90). 'Average-reward' reinforcement learning was used to model our data. We replicated our findings across both studies. RESULTS: We show that the latency of self-initiation is driven by changes in the OCT. Furthermore, we demonstrate, for the first time, that participants with higher apathy showed greater sensitivity to changes in OCT in younger adults. Our model shows that apathetic individuals experienced greatest change in subjective OCT during our task as a consequence of being more sensitive to rewards. CONCLUSIONS: Our results suggest that OCT is an important variable for determining free-operant action initiation and understanding apathy.

Risk taking for potential losses but not gains increases with time of day.

Humans exhibit distinct risk preferences when facing choices involving potential gains and losses. These preferences are believed to be subject to neuromodulatory influence, particularly from dopamine and serotonin. As neuromodulators manifest circadian rhythms, this suggests decision making under risk might be affected by time of day. Here, in a large subject sample collected using a smartphone application, we found that risky options with potential losses were increasingly chosen over the course of the day. We observed this result in both a within-subjects design (N = 2599) comparing risky options chosen earlier and later in the day in the same individuals, and in a between-subjects design (N = 26,720) showing our effect generalizes across ages and genders. Using computational modelling, we show this diurnal change in risk preference reflects a decrease in sensitivity to increasing losses, but no change was observed in the relative impacts of gains and losses on choice (i.e., loss aversion). Thus, our findings reveal a striking diurnal modulation in human decision making, a pattern with potential importance for real-life decisions that include voting, medical decisions, and financial investments.

A highly replicable decline in mood during rest and simple tasks.

Does our mood change as time passes? This question is central to behavioural and affective science, yet it remains largely unexamined. To investigate, we intermixed subjective momentary mood ratings into repetitive psychology paradigms. Here we demonstrate that task and rest periods lowered participants' mood, an effect we call 'Mood Drift Over Time'. This finding was replicated in 19 cohorts totalling 28,482 adult and adolescent participants. The drift was relatively large (-13.8% after 7.3 min of rest, Cohen's d = 0.574) and was consistent across cohorts. Behaviour was also impacted: participants were less likely to gamble in a task that followed a rest period. Importantly, the drift slope was inversely related to reward sensitivity. We show that accounting for time using a linear term significantly improves the fit of a computational model of mood. Our work provides conceptual and methodological reasons for researchers to account for time's effects when studying mood and behaviour.

Measuring self-regulation in everyday life: Reliability and validity of smartphone-based experiments in alcohol use disorder.

Self-regulation, the ability to guide behavior according to one's goals, plays an integral role in understanding loss of control over unwanted behaviors, for example in alcohol use disorder (AUD). Yet, experimental tasks that measure processes underlying self-regulation are not easy to deploy in contexts where such behaviors usually occur, namely outside the laboratory, and in clinical populations such as people with AUD. Moreover, lab-based tasks have been criticized for poor test-retest reliability and lack of construct validity. Smartphones can be used to deploy tasks in the field, but often require shorter versions of tasks, which may further decrease reliability. Here, we show that combining smartphone-based tasks with joint hierarchical modeling of longitudinal data can overcome at least some of these shortcomings. We test four short smartphone-based tasks outside the laboratory in a large sample (N = 488) of participants with AUD. Although task measures indeed have low reliability when data are analyzed traditionally by modeling each session separately, joint modeling of longitudinal data increases reliability to good and oftentimes excellent levels. We next test the measures' construct validity and show that extracted latent factors are indeed in line with theoretical accounts of cognitive control and decision-making. Finally, we demonstrate that a resulting cognitive control factor relates to a real-life measure of drinking behavior and yields stronger correlations than single measures based on traditional analyses. Our findings demonstrate how short, smartphone-based task measures, when analyzed with joint hierarchical modeling and latent factor analysis, can overcome frequently reported shortcomings of experimental tasks.

Computational models of subjective feelings in psychiatry.

Research in computational psychiatry is dominated by models of behavior. Subjective experience during behavioral tasks is not well understood, even though it should be relevant to understanding the symptoms of psychiatric disorders. Here, we bridge this gap and review recent progress in computational models for subjective feelings. For example, happiness reflects not how well people are doing, but whether they are doing better than expected. This dependence on recent reward prediction errors is intact in major depression, although depressive symptoms lower happiness during tasks. Uncertainty predicts subjective feelings of stress in volatile environments. Social prediction errors influence feelings of self-worth more in individuals with low self-esteem despite a reduced willingness to change beliefs due to social feedback. Measuring affective state during behavioral tasks provides a tool for understanding psychiatric symptoms that can be dissociable from behavior. When smartphone tasks are collected longitudinally, subjective feelings provide a potential means to bridge the gap between lab-based behavioral tasks and real-life behavior, emotion, and psychiatric symptoms.

Imbalanced basal ganglia connectivity is associated with motor deficits and apathy in Huntington's disease.

The gating of movement depends on activity within the cortico-striato-thalamic loops. Within these loops, emerging from the cells of the striatum, run two opponent pathways-the direct and indirect basal ganglia pathways. Both are complex and polysynaptic, but the overall effect of activity within these pathways is thought to encourage and inhibit movement, respectively. In Huntington's disease, the preferential early loss of striatal neurons forming the indirect pathway is thought to lead to disinhibition, giving rise to the characteristic motor features of the condition. But early Huntington's disease is also associated with apathy, a loss of motivation and failure to engage in goal-directed movement. We hypothesized that in Huntington's disease, motor signs and apathy may be selectively correlated with indirect and direct pathway dysfunction, respectively. We used spectral dynamic casual modelling of resting-state functional MRI data to model effective connectivity in a model of these cortico-striatal pathways. We tested both of these hypotheses in vivo for the first time in a large cohort of patients with prodromal Huntington's disease. Using an advanced approach at the group level we combined parametric empirical Bayes and Bayesian model reduction procedures to generate a large number of competing models and compare them using Bayesian model comparison. With this automated Bayesian approach, associations between clinical measures and connectivity parameters emerge de novo from the data. We found very strong evidence (posterior probability > 0.99) to support both of our hypotheses. First, more severe motor signs in Huntington's disease were associated with altered connectivity in the indirect pathway components of our model and, by comparison, loss of goal-direct behaviour or apathy, was associated with changes in the direct pathway component. The empirical evidence we provide here demonstrates that imbalanced basal ganglia connectivity may play an important role in the pathogenesis of some of commonest and disabling features of Huntington's disease and may have important implications for therapeutics.

A Neurocomputational Model for Intrinsic Reward.

Standard economic indicators provide an incomplete picture of what we value both as individuals and as a society. Furthermore, canonical macroeconomic measures, such as GDP, do not account for non-market activities (e.g., cooking, childcare) that nevertheless impact well-being. Here, we introduce a computational tool that measures the affective value of experiences (e.g., playing a musical instrument without errors). We go on to validate this tool with neural data, using fMRI to measure neural activity in male and female human subjects performing a reinforcement learning task that incorporated periodic ratings of subjective affective state. Learning performance determined level of payment (i.e., extrinsic reward). Crucially, the task also incorporated a skilled performance component (i.e., intrinsic reward) which did not influence payment. Both extrinsic and intrinsic rewards influenced affective dynamics, and their relative influence could be captured in our computational model. Individuals for whom intrinsic rewards had a greater influence on affective state than extrinsic rewards had greater ventromedial prefrontal cortex (vmPFC) activity for intrinsic than extrinsic rewards. Thus, we show that computational modeling of affective dynamics can index the subjective value of intrinsic relative to extrinsic rewards, a "computational hedonometer" that reflects both behavior and neural activity that quantifies the affective value of experience.SIGNIFICANCE STATEMENT Traditional economic indicators are increasingly recognized to provide an incomplete picture of what we value as a society. Standard economic approaches struggle to accurately assign values to non-market activities that nevertheless may be intrinsically rewarding, prompting a need for new tools to measure what really matters to individuals. Using a combination of neuroimaging and computational modeling, we show that despite their lack of instrumental value, intrinsic rewards influence subjective affective state and ventromedial prefrontal cortex (vmPFC) activity. The relative degree to which extrinsic and intrinsic rewards influence affective state is predictive of their relative impacts on neural activity, confirming the utility of our approach for measuring the affective value of experiences and other non-market activities in individuals.

The temporal representation of experience in subjective mood.

Humans refer to their mood state regularly in day-to-day as well as clinical interactions. Theoretical accounts suggest that when reporting on our mood we integrate over the history of our experiences; yet, the temporal structure of this integration remains unexamined. Here, we use a computational approach to quantitatively answer this question and show that early events exert a stronger influence on reported mood (a primacy weighting) compared to recent events. We show that a Primacy model accounts better for mood reports compared to a range of alternative temporal representations across random, consistent, or dynamic reward environments, different age groups, and in both healthy and depressed participants. Moreover, we find evidence for neural encoding of the Primacy, but not the Recency, model in frontal brain regions related to mood regulation. These findings hold implications for the timing of events in experimental or clinical settings and suggest new directions for individualized mood interventions.

Aberrant Striatal Value Representation in Huntington's Disease Gene Carriers 25 Years Before Onset.

BACKGROUND: In this study, we asked whether differences in striatal activity during a reinforcement learning (RL) task with gain and loss domains could be one of the earliest functional imaging features associated with carrying the Huntington's disease (HD) gene. Based on previous work, we hypothesized that HD gene carriers would show either neural or behavioral asymmetry between gain and loss learning. METHODS: We recruited 35 HD gene carriers, expected to demonstrate onset of motor symptoms in an average of 26 years, and 35 well-matched gene-negative control subjects. Participants were placed in a functional magnetic resonance imaging scanner, where they completed an RL task in which they were required to learn to choose between abstract stimuli with the aim of gaining rewards and avoiding losses. Task behavior was modeled using an RL model, and variables from this model were used to probe functional magnetic resonance imaging data. RESULTS: In comparison with well-matched control subjects, gene carriers more than 25 years from motor onset showed exaggerated striatal responses to gain-predicting stimuli compared with loss-predicting stimuli (p = .002) in our RL task. Using computational analysis, we also found group differences in striatal representation of stimulus value (p = .0004). We found no group differences in behavior, cognitive scores, or caudate volumes. CONCLUSIONS: Behaviorally, gene carriers 9 years from predicted onset have been shown to learn better from gains than from losses. Our data suggest that a window exists in which HD-related functional neural changes are detectable long before associated behavioral change and 25 years before predicted motor onset. These represent the earliest functional imaging differences between HD gene carriers and control subjects.

Smartphones and the Neuroscience of Mental Health.

Improvements in understanding the neurobiological basis of mental illness have unfortunately not translated into major advances in treatment. At this point, it is clear that psychiatric disorders are exceedingly complex and that, in order to account for and leverage this complexity, we need to collect longitudinal data sets from much larger and more diverse samples than is practical using traditional methods. We discuss how smartphone-based research methods have the potential to dramatically advance our understanding of the neuroscience of mental health. This, we expect, will take the form of complementing lab-based hard neuroscience research with dense sampling of cognitive tests, clinical questionnaires, passive data from smartphone sensors, and experience-sampling data as people go about their daily lives. Theory- and data-driven approaches can help make sense of these rich data sets, and the combination of computational tools and the big data that smartphones make possible has great potential value for researchers wishing to understand how aspects of brain function give rise to, or emerge from, states of mental health and illness.

Momentary subjective well-being depends on learning and not reward.

Subjective well-being or happiness is often associated with wealth. Recent studies suggest that momentary happiness is associated with reward prediction error, the difference between experienced and predicted reward, a key component of adaptive behaviour. We tested subjects in a reinforcement learning task in which reward size and probability were uncorrelated, allowing us to dissociate between the contributions of reward and learning to happiness. Using computational modelling, we found convergent evidence across stable and volatile learning tasks that happiness, like behaviour, is sensitive to learning-relevant variables (i.e. probability prediction error). Unlike behaviour, happiness is not sensitive to learning-irrelevant variables (i.e. reward prediction error). Increasing volatility reduces how many past trials influence behaviour but not happiness. Finally, depressive symptoms reduce happiness more in volatile than stable environments. Our results suggest that how we learn about our world may be more important for how we feel than the rewards we actually receive.

Many people believe they would be happier if only they had more money. And events such as winning the lottery or receiving a large pay rise do make people happy, at least temporarily. But recent studies suggest that the main factor driving happiness on such occasions is not the size of the reward received. Instead, it is how well that reward matches up with expectations. Receiving a 10% pay rise when you were expecting 1% will make you feel happier than receiving 10% when you had been expecting 20%. This difference between an expected and an actual reward is referred to as a reward prediction error. Reward prediction errors have a key role in learning. They motivate people to repeat behaviours that led to unexpectedly large rewards. But they also enable people to update their beliefs about the world, which is rewarding in itself. Could it be that reward prediction errors are associated with happiness mainly because they help us understand the world a little better than before? To test this idea, Blain and Rutledge designed a task in which the likelihood of receiving a reward was unrelated to the size of the reward. This study design makes it possible to separate out the contributions of learning versus reward to moment-by-moment happiness. In the task, volunteers had to decide which of two cars would win a race. In the 'stable' condition, one of the cars always had an 80% chance of winning. In the 'volatile' condition, one car had an 80% chance of winning for the first 20 trials. The other car then had an 80% chance of winning for the next 20 trials. The volunteers were not told these probabilities in advance, but had to work them out by playing the game. However, on every trial, the volunteers were shown the reward they would receive if they chose either of the cars and that car went on to win. The size of the rewards varied at random and was unrelated to the likelihood of a car winning. Every few trials, the volunteers were asked to indicate their current level of happiness on a scale. The results showed that volunteers were happier after winning than after losing. On average they were also happier in the stable condition than in the volatile condition. This was especially true for volunteers with pre-existing symptoms of depression. Moreover, happiness after wins did not depend on how large the reward they got was, but instead simply on how surprised they were to win. These results suggest that how we learn about the world around us can be more important for how we feel than rewards we receive directly. Measuring happiness in various types of environment could help us understand factors affecting mental health. The current results suggest, for example, that uncertain environments may be especially unpleasant for people with depression. Further research is needed to understand why this might be the case. In the real world, rewards are often uncertain and infrequent, but learning may nevertheless have the potential to boost happiness.

Under the Hood: Using Computational Psychiatry to Make Psychological Therapies More Mechanism-Focused.

Psychological therapies, such as CBT, are an important part of the treatment of a range of psychiatric disorders such as depression and anxiety. There is a growing desire to understand the mechanisms by which such therapies effect change so as to improve treatment outcomes. Here we argue that adopting a computational framework may be one such approach. Computational psychiatry aims to provide a theoretical framework for moving between higher-level psychological states (like emotions, decisions and beliefs) to neural circuits, by modeling these constructs mathematically. These models are explicit hypotheses that contain quantifiable variables and parameters derived from each individual's behavior. This approach has two advantages. Firstly, some of the variables described by these models appears to reflect the neural activity of specific brain regions. Secondly, the parameters estimated by these models may offer a unique description of a patient's symptoms which can be used to both tailor therapy and track its effect. In doing so this approach may offer some additional granularity in understanding how psychological therapies, such as CBT, are working. Although this field shows significant promise, we also highlight several of the key hurdles that must first be overcome before clinical translation of computational insights can be realized.

Neurocomputational mechanisms underpinning aberrant social learning in young adults with low self-esteem.

Low self-esteem is a risk factor for a range of psychiatric disorders. From a cognitive perspective a negative self-image can be maintained through aberrant learning about self-worth derived from social feedback. We previously showed that neural teaching signals that represent the difference between expected and actual social feedback (i.e., social prediction errors) drive fluctuations in self-worth. Here, we used model-based functional magnetic resonance imaging (fMRI) to characterize learning from social prediction errors in 61 participants drawn from a population-based sample (n = 2402) who were recruited on the basis of being in the bottom or top 10% of self-esteem scores. Participants performed a social evaluation task during fMRI scanning, which entailed predicting whether other people liked them as well as the repeated provision of reported feelings of self-worth. Computational modeling results showed that low self-esteem participants had persistent expectations that others would dislike them, and a reduced propensity to update these expectations in response to social prediction errors. Low self-esteem subjects also displayed an enhanced volatility in reported feelings of self-worth, and this was linked to an increased tendency for social prediction errors to determine momentary self-worth. Canonical correlation analysis revealed that individual differences in self-esteem related to several interconnected psychiatric symptoms organized around a single dimension of interpersonal vulnerability. Such interpersonal vulnerability was associated with an attenuated social value signal in ventromedial prefrontal cortex when making predictions about being liked, and enhanced dorsal prefrontal cortex activity upon receipt of social feedback. We suggest these computational signatures of low self-esteem and their associated neural underpinnings might represent vulnerability for development of psychiatric disorder.

Distinct Processing of Aversive Experience in Amygdala Subregions.

BACKGROUND: The amygdala is an anatomically complex medial temporal brain structure whose subregions are considered to serve distinct functions. However, their precise role in mediating human aversive experience remains ill understood. METHODS: We used functional magnetic resonance imaging in 39 healthy volunteers with varying levels of trait anxiety to assess distinct contributions of the basolateral amygdala (BLA) and centromedial amygdala to anticipation and experience of aversive events. Additionally, we examined the relationship between any identified functional subspecialization and measures of subjective reported aversion and trait anxiety. RESULTS: Our results show that the centromedial amygdala is responsive to aversive outcomes but insensitive to predictive aversive cues. In contrast, the BLA encodes an aversive prediction error that quantifies whether cues and outcomes are worse than expected. A neural representation within the BLA for distinct threat levels was mirrored in self-reported subjective anxiety across individuals. Furthermore, high trait-anxious individuals were characterized by indiscriminately heightened BLA activity in response to aversive cues, regardless of actual threat level. CONCLUSIONS: Our results demonstrate that amygdala subregions are distinctly engaged in processing of aversive experience, with elevated and undifferentiated BLA responses to threat emerging as a potential neurobiological mediator of vulnerability to anxiety disorders.

Endogenous fluctuations in the dopaminergic midbrain drive behavioral choice variability.

Human behavior is surprisingly variable, even when facing the same problem under identical circumstances. A prominent example is risky decision making. Economic theories struggle to explain why humans are so inconsistent. Resting-state studies suggest that ongoing endogenous fluctuations in brain activity can influence low-level perceptual and motor processes, but it remains unknown whether endogenous fluctuations also influence high-level cognitive processes including decision making. Here, using real-time functional magnetic resonance imaging, we tested whether risky decision making is influenced by endogenous fluctuations in blood oxygenation level-dependent (BOLD) activity in the dopaminergic midbrain, encompassing ventral tegmental area and substantia nigra. We show that low prestimulus brain activity leads to increased risky choice in humans. Using computational modeling, we show that increased risk taking is explained by enhanced phasic responses to offers in a decision network. Our findings demonstrate that endogenous brain activity provides a physiological basis for variability in complex human behavior.

Machine learning and big data in psychiatry: toward clinical applications.

Psychiatry is a medical field concerned with the treatment of mental illness. Psychiatric disorders broadly relate to higher functions of the brain, and as such are richly intertwined with social, cultural, and experiential factors. This makes them exquisitely complex phenomena that depend on and interact with a large number of variables. Computational psychiatry provides two ways of approaching this complexity. Theory-driven computational approaches employ mechanistic models to make explicit hypotheses at multiple levels of analysis. Data-driven machine-learning approaches can make predictions from high-dimensional data and are generally agnostic as to the underlying mechanisms. Here, we review recent advances in the use of big data and machine-learning approaches toward the aim of alleviating the suffering that arises from psychiatric disorders.

Oxytocin modulates social value representations in the amygdala.

Humans exhibit considerable variation in how they value their own interest relative to the interests of others. Deciphering the neural codes representing potential rewards for self and others is crucial for understanding social decision-making. Here we integrate computational modeling with functional magnetic resonance imaging to investigate the neural representation of social value and the modulation by oxytocin, a nine-amino acid neuropeptide, in participants evaluating monetary allocations to self and other (self-other allocations). We found that an individual's preferred self-other allocation serves as a reference point for computing the value of potential self-other allocations. In more prosocial participants, amygdala activity encoded a social-value-distance signal; that is, the value dissimilarity between potential and preferred allocations. Intranasal oxytocin administration amplified this amygdala representation and increased prosocial behavior in more individualistic participants but not in more prosocial ones. Our results reveal a neurocomputational mechanism underlying social-value representations and suggest that oxytocin may promote prosociality by modulating social-value representations in the amygdala.

Computing Value from Quality and Quantity in Human Decision-Making.

How organisms learn the value of single stimuli through experience is well described. In many decisions, however, value estimates are computed "on the fly" by combining multiple stimulus attributes. The neural basis of this computation is poorly understood. Here we explore a common scenario in which decision-makers must combine information about quality and quantity to determine the best option. Using fMRI, we examined the neural representation of quality, quantity, and their integration into an integrated subjective value signal in humans of both genders. We found that activity within inferior frontal gyrus (IFG) correlated with offer quality, while activity in the intraparietal sulcus (IPS) specifically correlated with offer quantity. Several brain regions, including the anterior cingulate cortex (ACC), were sensitive to an interaction of quality and quantity. However, the ACC was uniquely activated by quality, quantity, and their interaction, suggesting that this region provides a substrate for flexible computation of value from both quality and quantity. Furthermore, ACC signals across subjects correlated with the strength of quality and quantity signals in IFG and IPS, respectively. ACC tracking of subjective value also correlated with choice predictability. Finally, activity in the ACC was elevated for choice trials, suggesting that ACC provides a nexus for the computation of subjective value in multiattribute decision-making.SIGNIFICANCE STATEMENT Would you prefer three apples or two oranges? Many choices we make each day require us to weigh up the quality and quantity of different outcomes. Using fMRI, we show that option quality is selectively represented in the inferior frontal gyrus, while option quantity correlates with areas of the intraparietal sulcus that have previously been associated with numerical processing. We show that information about the two is integrated into a value signal in the anterior cingulate cortex, and the fidelity of this integration predicts choice predictability. Our results demonstrate how on-the-fly value estimates are computed from multiple attributes in human value-based decision-making.