Differentiating between Bayesian parameter learning and structure learning based on behavioural and pupil measures.

Within predictive processing two kinds of learning can be distinguished: parameter learning and structure learning. In Bayesian parameter learning, parameters under a specific generative model are continuously being updated in light of new evidence. However, this learning mechanism cannot explain how new parameters are added to a model. Structure learning, unlike parameter learning, makes structural changes to a generative model by altering its causal connections or adding or removing parameters. Whilst these two types of learning have recently been formally differentiated, they have not been empirically distinguished. The aim of this research was to empirically differentiate between parameter learning and structure learning on the basis of how they affect pupil dilation. Participants took part in a within-subject computer-based learning experiment with two phases. In the first phase, participants had to learn the relationship between cues and target stimuli. In the second phase, they had to learn a conditional change in this relationship. Our results show that the learning dynamics were indeed qualitatively different between the two experimental phases, but in the opposite direction as we originally expected. Participants were learning more gradually in the second phase compared to the first phase. This might imply that participants built multiple models from scratch in the first phase (structure learning) before settling on one of these models. In the second phase, participants possibly just needed to update the probability distribution over the model parameters (parameter learning).

Functional magnetic resonance imaging responses during perceptual decision-making at 3 and 7 T in human cortex, striatum, and brainstem.

While functional magnetic resonance imaging (fMRI) at ultra-high field (7 T) promises a general increase in sensitivity compared to lower field strengths, the benefits may be most pronounced for specific applications. The current study aimed to evaluate the relative benefit of 7 over 3 T fMRI for the assessment of responses evoked in different brain regions by a well-controlled cognitive task. At 3 and 7 T, the same participants made challenging perceptual decisions about visual motion combined with monetary rewards for correct choices. Previous work on this task has extensively characterized the underlying cognitive computations and single-cell responses in cortical and subcortical structures. We quantified the evoked fMRI responses in extrastriate visual cortical areas, the striatum, and the brainstem during the decision interval and the post-feedback interval of the task. The dependence of response amplitudes on field strength during the decision interval differed between cortical, striatal, and brainstem regions, with a generally bigger 7 versus 3 T benefit in subcortical structures. We also found stronger responses during relatively easier than harder decisions at 7 T for dopaminergic midbrain nuclei, in line with reward expectation. Our results demonstrate the potential of 7 T fMRI for illuminating the contribution of small brainstem nuclei to the orchestration of cognitive computations in the human brain.

Processing of Prediction Errors in Mentalizing Areas.

When seeing people perform actions, we are able to quickly predict the action's outcomes. These predictions are not solely based on the observed actions themselves but utilize our prior knowledge of others. It has been suggested that observed outcomes that are not in line with these predictions result in prediction errors, which require additional processing to be integrated or updated. However, there is no consensus on whether this is indeed the case for the kind of high-level social-cognitive processes involved in action observation. In this fMRI study, we investigated whether observation of unexpected outcomes causes additional activation in line with the processing of prediction errors and, if so, whether this activation overlaps with activation in brain areas typically associated with social-cognitive processes. In the first part of the experiment, participants watched animated movies of two people playing a bowling game, one experienced and one novice player. In cases where the player's score was higher or lower than expected based on their skill level, there was increased BOLD activity in areas that were also activated during a theory of mind task that participants performed in the second part of the experiment. These findings are discussed in the light of different theoretical accounts of human social-cognitive processing.

Author Correction: Task-evoked pupil responses reflect internal belief states.

A correction to this article has been published and is linked from the HTML and PDF versions of this paper. The error has not been fixed in the paper.

Task-evoked pupil responses reflect internal belief states.

Perceptual decisions about the state of the environment are often made in the face of uncertain evidence. Internal uncertainty signals are considered important regulators of learning and decision-making. A growing body of work has implicated the brain's arousal systems in uncertainty signaling. Here, we found that two specific computational variables, postulated by recent theoretical work, evoke boosts of arousal at different times during a perceptual decision: decision confidence (the observer's internally estimated probability that a choice was correct given the evidence) before feedback, and prediction errors (deviations from expected reward) after feedback. We monitored pupil diameter, a peripheral marker of central arousal state, while subjects performed a challenging perceptual choice task with a delayed monetary reward. We quantified evoked pupil responses during decision formation and after reward-linked feedback. During both intervals, decision difficulty and accuracy had interacting effects on pupil responses. Pupil responses negatively scaled with decision confidence prior to feedback and scaled with uncertainty-dependent prediction errors after feedback. This pattern of pupil responses during both intervals was in line with a model using the observer's graded belief about choice accuracy to anticipate rewards and compute prediction errors. We conclude that pupil-linked arousal systems are modulated by internal belief states.

Dynamic modulation of decision biases by brainstem arousal systems.

Decision-makers often arrive at different choices when faced with repeated presentations of the same evidence. Variability of behavior is commonly attributed to noise in the brain's decision-making machinery. We hypothesized that phasic responses of brainstem arousal systems are a significant source of this variability. We tracked pupil responses (a proxy of phasic arousal) during sensory-motor decisions in humans, across different sensory modalities and task protocols. Large pupil responses generally predicted a reduction in decision bias. Using fMRI, we showed that the pupil-linked bias reduction was (i) accompanied by a modulation of choice-encoding pattern signals in parietal and prefrontal cortex and (ii) predicted by phasic, pupil-linked responses of a number of neuromodulatory brainstem centers involved in the control of cortical arousal state, including the noradrenergic locus coeruleus. We conclude that phasic arousal suppresses decision bias on a trial-by-trial basis, thus accounting for a significant component of the variability of choice behavior.

Creating Colored Letters: Familial Markers of Grapheme-Color Synesthesia in Parietal Lobe Activation and Structure.

Perception is inherently subjective, and individual differences in phenomenology are well illustrated by the phenomenon of synesthesia (highly specific, consistent, and automatic cross-modal experiences, in which the external stimulus corresponding to the additional sensation is absent). It is unknown why some people develop synesthesia and others do not. In the current study, we tested whether neural markers related to having synesthesia in the family were evident in brain function and structure. Relatives of synesthetes (who did not have any type of synesthesia themselves) and matched controls read specially prepared books with colored letters for several weeks and were scanned before and after reading using magnetic resonance imaging. Effects of acquired letter-color associations were evident in brain activation. Training-related activation (while viewing black letters) in the right angular gyrus of the parietal lobe was directly related to the strength of the learned letter-color associations (behavioral Stroop effect). Within this obtained angular gyrus ROI, the familial trait of synesthesia related to brain activation differences while participants viewed both black and colored letters. Finally, we compared brain structure using voxel-based morphometry and diffusion tensor imaging to test for group differences and training effects. One cluster in the left superior parietal lobe had significantly more coherent white matter in the relatives compared with controls. No evidence for experience-dependent plasticity was obtained. For the first time, we present evidence suggesting that the (nonsynesthete) relatives of grapheme-color synesthetes show atypical grapheme processing as well as increased brain connectivity.

Visual cortex activity predicts subjective experience after reading books with colored letters.

One of the most astonishing properties of synesthesia is that the evoked concurrent experiences are perceptual. Is it possible to acquire similar effects after learning cross-modal associations that resemble synesthetic mappings? In this study, we examine whether brain activation in early visual areas can be directly related to letter-color associations acquired by training. Non-synesthetes read specially prepared books with colored letters for several weeks and were scanned using functional magnetic resonance imaging. If the acquired letter-color associations were visual in nature, then brain activation in visual cortex while viewing the trained black letters (compared to untrained black letters) should predict the strength of the associations, the quality of the color experience, or the vividness of visual mental imagery. Results showed that training-related activation of area V4 was correlated with differences in reported subjective color experience. Trainees who were classified as having stronger 'associator' types of color experiences also had more negative activation for trained compared to untrained achromatic letters in area V4. In contrast, the strength of the acquired associations (measured as the Stroop effect) was not reliably reflected in visual cortex activity. The reported vividness of visual mental imagery was related to veridical color activation in early visual cortex, but not to the acquired color associations. We show for the first time that subjective experience related to a synesthesia-training paradigm was reflected in visual brain activation.

Training synesthetic letter-color associations by reading in color.

Synesthesia is a rare condition in which a stimulus from one modality automatically and consistently triggers unusual sensations in the same and/or other modalities. A relatively common and well-studied type is grapheme-color synesthesia, defined as the consistent experience of color when viewing, hearing and thinking about letters, words and numbers. We describe our method for investigating to what extent synesthetic associations between letters and colors can be learned by reading in color in nonsynesthetes. Reading in color is a special method for training associations in the sense that the associations are learned implicitly while the reader reads text as he or she normally would and it does not require explicit computer-directed training methods. In this protocol, participants are given specially prepared books to read in which four high-frequency letters are paired with four high-frequency colors. Participants receive unique sets of letter-color pairs based on their pre-existing preferences for colored letters. A modified Stroop task is administered before and after reading in order to test for learned letter-color associations and changes in brain activation. In addition to objective testing, a reading experience questionnaire is administered that is designed to probe for differences in subjective experience. A subset of questions may predict how well an individual learned the associations from reading in color. Importantly, we are not claiming that this method will cause each individual to develop grapheme-color synesthesia, only that it is possible for certain individuals to form letter-color associations by reading in color and these associations are similar in some aspects to those seen in developmental grapheme-color synesthetes. The method is quite flexible and can be used to investigate different aspects and outcomes of training synesthetic associations, including learning-induced changes in brain function and structure.

A taste for words and sounds: a case of lexical-gustatory and sound-gustatory synesthesia.

Gustatory forms of synesthesia involve the automatic and consistent experience of tastes that are triggered by non-taste related inducers. We present a case of lexical-gustatory and sound-gustatory synesthesia within one individual, SC. Most words and a subset of non-linguistic sounds induce the experience of taste, smell and physical sensations for SC. SC's lexical-gustatory associations were significantly more consistent than those of a group of controls. We tested for effects of presentation modality (visual vs. auditory), taste-related congruency, and synesthetic inducer-concurrent direction using a priming task. SC's performance did not differ significantly from a trained control group. We used functional magnetic resonance imaging to investigate the neural correlates of SC's synesthetic experiences by comparing her brain activation to the literature on brain networks related to language, music, and sound processing, in addition to synesthesia. Words that induced a strong taste were contrasted to words that induced weak-to-no tastes ("tasty" vs. "tasteless" words). Brain activation was also measured during passive listening to music and environmental sounds. Brain activation patterns showed evidence that two regions are implicated in SC's synesthetic experience of taste and smell: the left anterior insula and left superior parietal lobe. Anterior insula activation may reflect the synesthetic taste experience. The superior parietal lobe is proposed to be involved in binding sensory information across sub-types of synesthetes. We conclude that SC's synesthesia is genuine and reflected in her brain activation. The type of inducer (visual-lexical, auditory-lexical, and non-lexical auditory stimuli) could be differentiated based on patterns of brain activity.

Pseudo-synesthesia through reading books with colored letters.

BACKGROUND: Synesthesia is a phenomenon where a stimulus produces consistent extraordinary subjective experiences. A relatively common type of synesthesia involves perception of color when viewing letters (e.g. the letter 'a' always appears as light blue). In this study, we examine whether traits typically regarded as markers of synesthesia can be acquired by simply reading in color. METHODOLOGY/PRINCIPAL FINDINGS: Non-synesthetes were given specially prepared colored books to read. A modified Stroop task was administered before and after reading. A perceptual crowding task was administered after reading. Reading one book (>49,000 words) was sufficient to induce effects regarded as behavioral markers for synesthesia. The results of the Stroop tasks indicate that it is possible to learn letter-color associations through reading in color (F(1, 14) = 5.85, p = .030). Furthermore, Stroop effects correlated with subjective reports about experiencing letters in color (r(13) = 0.51, p = .05). The frequency of viewing letters is related to the level of association as seen by the difference in the Stroop effect size between upper- and lower-case letters (t(14) = 2.79, p = .014) and in a subgroup of participants whose Stroop effects increased as they continued to read in color. Readers did not show significant performance advantages on the crowding task compared to controls. Acknowledging the many differences between trainees and synesthetes, results suggest that it may be possible to acquire a subset of synesthetic behavioral traits in adulthood through training. CONCLUSION/SIGNIFICANCE: To our knowledge, this is the first evidence of acquiring letter-color associations through reading in color. Reading in color appears to be a promising avenue in which we may explore the differences and similarities between synesthetes and non-synesthetes. Additionally, reading in color is a plausible method for a long-term 'synesthetic' training program.

Brain areas involved in synaesthesia: a review.

Despite a recent upsurge of research, much remains unknown about the neurobiological mechanisms underlying synaesthesia. By integrating results obtained so far in Magnetic Resonance Imaging (MRI) studies, this contribution sheds light on the role of particular brain regions in synaesthetic experiences. First, in accordance with its sensory nature, it seems that the sensory brain areas corresponding to the type of synaesthetic experience are activated. Synaesthetic colour experiences can activate colour regions in occipito-temporal cortex, but this is not necessarily restricted to V4. Furthermore, sensory and motor brain regions have been obtained that extend beyond the particular type of synaesthesia studied. Second, differences in experimental setup, number and type of synaesthetes tested, and method to delineate regions of interest may help explain inconsistent results obtained in the BOLD-MRI (Blood Oxygen Level Dependent functional MRI) studies. Third, an overview of obtained results shows that a network of brain areas rather than a single brain region underlies synaesthesia. Six brain regions of overlapping results emerge, these regions are in sensory and motor regions as well as 'higher level' regions in parietal and frontal lobe. We propose that these regions are related to three different cognitive processes inherently part of synaesthesia; the sensory processes, the (attentional) 'binding' processes, and cognitive control processes. Finally, we discuss how these functional and structural brain properties might relate to the development of synaesthesia. In particular, we believe this relationship is better understood by separating the question what underlies the presence of synaesthesia ('trait') from what determines particular synaesthetic associations ('type').