Negative symptoms, striatal dopamine and model-free reward decision-making in schizophrenia.

Negative symptoms, such as lack of motivation or social withdrawal, are highly prevalent and debilitating in patients with schizophrenia. Underlying mechanisms of negative symptoms are incompletely understood, thereby preventing the development of targeted treatments. We hypothesized that in patients with schizophrenia during psychotic remission, impaired influences of both model-based and model-free reward predictions on decision-making ('reward prediction influence', RPI) underlie negative symptoms. We focused on psychotic remission, because psychotic symptoms might confound reward-based decision-making. Moreover, we hypothesized that impaired model-based/model-free RPIs depend on alterations of both associative striatum dopamine synthesis and storage (DSS) and executive functioning. Both factors influence RPI in healthy subjects and are typically impaired in schizophrenia. Twenty-five patients with schizophrenia with pronounced negative symptoms during psychotic remission and 24 healthy controls were included in the study. Negative symptom severity was measured by the Positive and Negative Syndrome Scale negative subscale, model-based/model-free RPI by the two-stage decision task, associative striatum DSS by 18F-DOPA positron emission tomography and executive functioning by the symbol coding task. Model-free RPI was selectively reduced in patients and associated with negative symptom severity as well as with reduced associative striatum DSS (in patients only) and executive functions (both in patients and controls). In contrast, model-based RPI was not altered in patients. Results provide evidence for impaired model-free reward prediction influence as a mechanism for negative symptoms in schizophrenia as well as for reduced associative striatum dopamine and executive dysfunction as relevant factors. Data suggest potential treatment targets for patients with schizophrenia and pronounced negative symptoms.

Neural signatures of reinforcement learning correlate with strategy adoption during spatial navigation.

Human navigation is generally believed to rely on two types of strategy adoption, route-based and map-based strategies. Both types of navigation require making spatial decisions along the traversed way although formal computational and neural links between navigational strategies and mechanisms of value-based decision making have so far been underexplored in humans. Here we employed functional magnetic resonance imaging (fMRI) while subjects located different objects in a virtual environment. We then modelled their paths using reinforcement learning (RL) algorithms, which successfully explained decision behavior and its neural correlates. Our results show that subjects used a mixture of route and map-based navigation and their paths could be well explained by the model-free and model-based RL algorithms. Furthermore, the value signals of model-free choices during route-based navigation modulated the BOLD signals in the ventro-medial prefrontal cortex (vmPFC), whereas the BOLD signals in parahippocampal and hippocampal regions pertained to model-based value signals during map-based navigation. Our findings suggest that the brain might share computational mechanisms and neural substrates for navigation and value-based decisions such that model-free choice guides route-based navigation and model-based choice directs map-based navigation. These findings open new avenues for computational modelling of wayfinding by directing attention to value-based decision, differing from common direction and distances approaches.

Dopamine enhances model-based over model-free choice behavior.

Decision making is often considered to arise out of contributions from a model-free habitual system and a model-based goal-directed system. Here, we investigated the effect of a dopamine manipulation on the degree to which either system contributes to instrumental behavior in a two-stage Markov decision task, which has been shown to discriminate model-free from model-based control. We found increased dopamine levels promote model-based over model-free choice.

Dopamine and performance in a reinforcement learning task: evidence from Parkinson's disease.

The role dopamine plays in decision-making has important theoretical, empirical and clinical implications. Here, we examined its precise contribution by exploiting the lesion deficit model afforded by Parkinson's disease. We studied patients in a two-stage reinforcement learning task, while they were ON and OFF dopamine replacement medication. Contrary to expectation, we found that dopaminergic drug state (ON or OFF) did not impact learning. Instead, the critical factor was drug state during the performance phase, with patients ON medication choosing correctly significantly more frequently than those OFF medication. This effect was independent of drug state during initial learning and appears to reflect a facilitation of generalization for learnt information. This inference is bolstered by our observation that neural activity in nucleus accumbens and ventromedial prefrontal cortex, measured during simultaneously acquired functional magnetic resonance imaging, represented learnt stimulus values during performance. This effect was expressed solely during the ON state with activity in these regions correlating with better performance. Our data indicate that dopamine modulation of nucleus accumbens and ventromedial prefrontal cortex exerts a specific effect on choice behaviour distinct from pure learning. The findings are in keeping with the substantial other evidence that certain aspects of learning are unaffected by dopamine lesions or depletion, and that dopamine plays a key role in performance that may be distinct from its role in learning.

Mapping value based planning and extensively trained choice in the human brain.

Investigations of the underlying mechanisms of choice in humans have focused on learning from prediction errors, leaving the computational structure of value based planning comparatively underexplored. Using behavioral and neuroimaging analyses of a minimax decision task, we found that the computational processes underlying forward planning are expressed in the anterior caudate nucleus as values of individual branching steps in a decision tree. In contrast, values represented in the putamen pertain solely to values learned during extensive training. During actual choice, both striatal areas showed a functional coupling to ventromedial prefrontal cortex, consistent with this region acting as a value comparator. Our findings point toward an architecture of choice in which segregated value systems operate in parallel in the striatum for planning and extensively trained choices, with medial prefrontal cortex integrating their outputs.

The influence of social status on pre-school children's eating habits, caries experience and caries prevention behavior.

OBJECTIVES: To assess the prevalence of Early Childhood Caries (ECC) in a county in Northern Hesse and to correlate this parameter to various independent variables. Additionally to investigate the relationship between preventive measures and the socioeconomic status (SES). METHODS: In spring 2006, 1,082 preschool children were examined. According to WHO-criteria d(3+4)mft scores were recorded. Information about eating habits and preventive measures were collected by structured questionnaires. To compare the mean caries scores and preventive measures of various subgroups, non-parametric tests were performed. Variables associated with caries were included in a binary stepwise backward logistic regression analysis. RESULTS: The mean d(3+4)mft score amounted to 1.88. Children with high SES had significantly less caries than children with low SES. Significant positive and negative associations of feeding practices and preventive measures to d(3+4)mft scores were observed. Differences between feeding practices and preventive measures were dependent on SES. CONCLUSIONS: Long-term use of baby bottles at night is the most important factor of ECC. Differences in feeding practices and preventive measures in the various SES groups are evident but not that significant as supposed.

Hedging your bets by learning reward correlations in the human brain.

Human subjects are proficient at tracking the mean and variance of rewards and updating these via prediction errors. Here, we addressed whether humans can also learn about higher-order relationships between distinct environmental outcomes, a defining ecological feature of contexts where multiple sources of rewards are available. By manipulating the degree to which distinct outcomes are correlated, we show that subjects implemented an explicit model-based strategy to learn the associated outcome correlations and were adept in using that information to dynamically adjust their choices in a task that required a minimization of outcome variance. Importantly, the experimentally generated outcome correlations were explicitly represented neuronally in right midinsula with a learning prediction error signal expressed in rostral anterior cingulate cortex. Thus, our data show that the human brain represents higher-order correlation structures between rewards, a core adaptive ability whose immediate benefit is optimized sampling.

The human prefrontal cortex mediates integration of potential causes behind observed outcomes.

Prefrontal cortex has long been implicated in tasks involving higher order inference in which decisions must be rendered, not only about which stimulus is currently rewarded, but also which stimulus dimensions are currently relevant. However, the precise computational mechanisms used to solve such tasks have remained unclear. We scanned human participants with functional MRI, while they performed a hierarchical intradimensional/extradimensional shift task to investigate what strategy subjects use while solving higher order decision problems. By using a computational model-based analysis, we found behavioral and neural evidence that humans solve such problems not by occasionally shifting focus from one to the other dimension, but by considering multiple explanations simultaneously. Activity in human prefrontal cortex was better accounted for by a model that integrates over all available evidences than by a model in which attention is selectively gated. Importantly, our model provides an explanation for how the brain determines integration weights, according to which it could distribute its attention. Our results demonstrate that, at the point of choice, the human brain and the prefrontal cortex in particular are capable of a weighted integration of information across multiple evidences.

Economic choices can be made using only stimulus values.

Decision-making often involves choices between different stimuli, each of which is associated with a different physical action. A growing consensus suggests that the brain makes such decisions by assigning a value to each available option and then comparing them to make a choice. An open question in decision neuroscience is whether the brain computes these choices by comparing the values of stimuli directly in goods space or instead by first assigning values to the associated actions and then making a choice over actions. We used a functional MRI paradigm in which human subjects made choices between different stimuli with and without knowledge of the actions required to obtain the different stimuli. We found neural correlates of the value of the chosen stimulus (a postdecision signal) in ventromedial prefrontal cortex before the actual stimulus-action pairing was revealed. These findings provide support for the hypothesis that the brain is capable of making choices in the space of goods without first transferring values into action space.

Neural computations underlying action-based decision making in the human brain.

Action-based decision making involves choices between different physical actions to obtain rewards. To make such decisions the brain needs to assign a value to each action and then compare them to make a choice. Using fMRI in human subjects, we found evidence for action-value signals in supplementary motor cortex. Separate brain regions, most prominently ventromedial prefrontal cortex, were involved in encoding the expected value of the action that was ultimately taken. These findings differentiate two main forms of value signals in the human brain: those relating to the value of each available action, likely reflecting signals that are a precursor of choice, and those corresponding to the expected value of the action that is subsequently chosen, and therefore reflecting the consequence of the decision process. Furthermore, we also found signals in the dorsomedial frontal cortex that resemble the output of a decision comparator, which implicates this region in the computation of the decision itself.

Beyond a relay nucleus: neuroimaging views on the human LGN.

The lateral geniculate nucleus (LGN) is the thalamic station in the retinocortical projection and has traditionally been viewed as the gateway for sensory information to enter the cortex. Here, we review recent studies of the human LGN that have investigated the retinotopic organization, physiologic response properties, and modulation of neural activity by selective attention and by visual awareness in a binocular rivalry paradigm. In the retinotopy studies, we found that the contralateral visual field was represented with the lower field in the medial-superior portion and the upper field in the lateral-inferior portion of each LGN. The fovea was represented in posterior and superior portions, with increasing eccentricities represented more anteriorly. Functional MRI responses increased monotonically with stimulus contrast in the LGN and in visual cortical areas. In the LGN, the dynamic response range of the contrast function was larger and contrast gain was lower than in the cortex. In our attention studies, we found that directed attention to a spatial location modulated neural activity in the LGN in several ways: it enhanced neural responses to attended stimuli, attenuated responses to ignored stimuli, and increased baseline activity in the absence of visual stimulation. Furthermore, we showed in a binocular rivalry paradigm that neural activity in the LGN correlated strongly with the subjects' reported percepts. The overall view that emerges from these studies is that the human LGN plays a role in perception and cognition far beyond that of a relay nucleus and, rather, needs to be considered as an early gatekeeper in the control of visual attention and awareness.

Neural correlates of binocular rivalry in the human lateral geniculate nucleus.

When dissimilar images are presented to the two eyes, they compete for perceptual dominance so that only one image is visible at a time while the other one is suppressed. Neural correlates of such binocular rivalry have been found at multiple stages of visual processing, including striate and extrastriate visual cortex. However, little is known about the role of subcortical processing during binocular rivalry. Here we used fMRI to measure neural activity in the human LGN while subjects viewed contrast-modulated gratings presented dichoptically. Neural activity in the LGN correlated strongly with the subjects' reported percepts, such that activity increased when a high-contrast grating was perceived and decreased when a low-contrast grating was perceived. Our results provide evidence for a functional role of the LGN in binocular rivalry and suggest that the LGN, traditionally viewed as the gateway to the visual cortex, may be an early gatekeeper of visual awareness.