Dissecting abstract, modality-specific and experience-dependent coding of affect in the human brain.

Emotion and perception are tightly intertwined, as affective experiences often arise from the appraisal of sensory information. Nonetheless, whether the brain encodes emotional instances using a sensory-specific code or in a more abstract manner is unclear. Here, we answer this question by measuring the association between emotion ratings collected during a unisensory or multisensory presentation of a full-length movie and brain activity recorded in typically developed, congenitally blind and congenitally deaf participants. Emotional instances are encoded in a vast network encompassing sensory, prefrontal, and temporal cortices. Within this network, the ventromedial prefrontal cortex stores a categorical representation of emotion independent of modality and previous sensory experience, and the posterior superior temporal cortex maps the valence dimension using an abstract code. Sensory experience more than modality affects how the brain organizes emotional information outside supramodal regions, suggesting the existence of a scaffold for the representation of emotional states where sensory inputs during development shape its functioning.

Convolutional neural networks for vision neuroscience: significance, developments, and outstanding issues.

Convolutional Neural Networks (CNN) are a class of machine learning models predominately used in computer vision tasks and can achieve human-like performance through learning from experience. Their striking similarities to the structural and functional principles of the primate visual system allow for comparisons between these artificial networks and their biological counterparts, enabling exploration of how visual functions and neural representations may emerge in the real brain from a limited set of computational principles. After considering the basic features of CNNs, we discuss the opportunities and challenges of endorsing CNNs as in silico models of the primate visual system. Specifically, we highlight several emerging notions about the anatomical and physiological properties of the visual system that still need to be systematically integrated into current CNN models. These tenets include the implementation of parallel processing pathways from the early stages of retinal input and the reconsideration of several assumptions concerning the serial progression of information flow. We suggest design choices and architectural constraints that could facilitate a closer alignment with biology provide causal evidence of the predictive link between the artificial and biological visual systems. Adopting this principled perspective could potentially lead to new research questions and applications of CNNs beyond modeling object recognition.

Hedonic and autonomic responses in promoting affective touch.

Interpersonal touch is intrinsically reciprocal since it entails a person promoting and another receiving the touch. While several studies have investigated the beneficial effects of receiving affective touch, the affective experience of caressing another individual remains largely unknown. Here, we investigated the hedonic and autonomic responses (skin conductance and heart rate) in the person promoting affective touch. We also examined whether interpersonal relationship, gender, and eye contact modulate these responses. As expected, caressing the partner was perceived as more pleasant than caressing a stranger, especially if the affective touch occurred together with mutual eye contact. Promoting affective touch to the partner also resulted in a decrease of both autonomic responses and anxiety levels, suggesting the occurrence of a calming effect. Additionally, these effects were more pronounced in females compared to males, indicating that hedonic and autonomic aspects of affective touch are modulated by both social relationship and gender. These findings show for the first time that caressing a beloved one is not only pleasant but also reduces autonomic responses and anxiety in the person promoting the touch. This might suggest that affective touch has an instrumental role for romantic partners in promoting and reinforcing their affective bonding.

A modality-independent proto-organization of human multisensory areas.

The processing of multisensory information is based upon the capacity of brain regions, such as the superior temporal cortex, to combine information across modalities. However, it is still unclear whether the representation of coherent auditory and visual events requires any prior audiovisual experience to develop and function. Here we measured brain synchronization during the presentation of an audiovisual, audio-only or video-only version of the same narrative in distinct groups of sensory-deprived (congenitally blind and deaf) and typically developed individuals. Intersubject correlation analysis revealed that the superior temporal cortex was synchronized across auditory and visual conditions, even in sensory-deprived individuals who lack any audiovisual experience. This synchronization was primarily mediated by low-level perceptual features, and relied on a similar modality-independent topographical organization of slow temporal dynamics. The human superior temporal cortex is naturally endowed with a functional scaffolding to yield a common representation across multisensory events.

A deep neural network model of the primate superior colliculus for emotion recognition.

Although sensory processing is pivotal to nearly every theory of emotion, the evaluation of the visual input as 'emotional' (e.g. a smile as signalling happiness) has been traditionally assumed to take place in supramodal 'limbic' brain regions. Accordingly, subcortical structures of ancient evolutionary origin that receive direct input from the retina, such as the superior colliculus (SC), are traditionally conceptualized as passive relay centres. However, mounting evidence suggests that the SC is endowed with the necessary infrastructure and computational capabilities for the innate recognition and initial categorization of emotionally salient features from retinal information. Here, we built a neurobiologically inspired convolutional deep neural network (DNN) model that approximates physiological, anatomical and connectional properties of the retino-collicular circuit. This enabled us to characterize and isolate the initial computations and discriminations that the DNN model of the SC can perform on facial expressions, based uniquely on the information it directly receives from the virtual retina. Trained to discriminate facial expressions of basic emotions, our model matches human error patterns and above chance, yet suboptimal, classification accuracy analogous to that reported in patients with V1 damage, who rely on retino-collicular pathways for non-conscious vision of emotional attributes. When presented with gratings of different spatial frequencies and orientations never 'seen' before, the SC model exhibits spontaneous tuning to low spatial frequencies and reduced orientation discrimination, as can be expected from the prevalence of the magnocellular (M) over parvocellular (P) projections. Likewise, face manipulation that biases processing towards the M or P pathway affects expression recognition in the SC model accordingly, an effect that dovetails with variations of activity in the human SC purposely measured with ultra-high field functional magnetic resonance imaging. Lastly, the DNN generates saliency maps and extracts visual features, demonstrating that certain face parts, like the mouth or the eyes, provide higher discriminative information than other parts as a function of emotional expressions like happiness and sadness. The present findings support the contention that the SC possesses the necessary infrastructure to analyse the visual features that define facial emotional stimuli also without additional processing stages in the visual cortex or in 'limbic' areas. This article is part of the theme issue 'Cracking the laugh code: laughter through the lens of biology, psychology and neuroscience'.

Investigation of Brain Activation Patterns Related to the Feminization or Masculinization of Body and Face Images across Genders.

Previous studies demonstrated sex-related differences in several areas of the human brain, including patterns of brain activation in males and females when observing their own bodies and faces (versus other bodies/faces or morphed versions of themselves), but a complex paradigm touching multiple aspects of embodied self-identity is still lacking. We enrolled 24 healthy individuals (12 M, 12 F) in 3 different fMRI experiments: the vision of prototypical body silhouettes, the vision of static images of the face of the participants morphed with prototypical male and female faces, the vision of short videos showing the dynamic transformation of the morphing. We found differential sexual activations in areas linked to self-identity and to the ability to attribute mental states: In Experiment 1, the male group activated more the bilateral thalamus when looking at sex congruent body images, while the female group activated more the middle and inferior temporal gyrus. In Experiment 2, the male group activated more the supplementary motor area when looking at their faces; the female group activated more the dorsomedial prefrontal cortex (dmPFC). In Experiment 3, the female group activated more the dmPFC when observing either the feminization or the masculinization of their face. The defeminization produced more activations in females in the left superior parietal lobule and middle occipital gyrus. The performance of all classifiers built using single ROIs exceeded chance level, reaching an area under the ROC curves > 0.85 in some cases (notably, for Experiment 2 using the V1 ROI). The results of the fMRI tasks showed good agreement with previously published studies, even if our sample size was small. Therefore, our functional MRI protocol showed significantly different patterns of activation in males and females, but further research is needed both to investigate the gender-related differences in activation when observing a morphing of their face/body, and to validate our paradigm using a larger sample.

Markers of emotion regulation processes: A neuroimaging and behavioral study of reappraising abilities.

Emotion regulation (ER) is a core element for individual well-being, and dysregulated emotional states are prominent in several mental disorders. Moreover, dispositional use of adaptive ER strategies, such as cognitive reappraisal, is usually associated to better psychological outcomes and less emotional problems. Thus, identifying markers of emotion dysregulation could serve as a key point for developing treatments against risks of psychopathological outcomes. Neuroimaging techniques could represent a useful tool within these aims, focusing on neurobiological markers of psychopathological illness. Given the well known gender differences in using ER strategies, we examined behavioral and neuroimaging patterns associated with dispositional use of reappraisal among a non-clinical female sample. We found that the individual predisposition to use cognitive reappraisal as an emotion regulation strategy was associated with decreased levels of dysregulation. From a neurobiological perspective, difficulties in using reappraisal were associated with decreased resting-state functional connectivity (rs-FC) between the Middle Temporal Gyrus and occipito-parietal regions. Moreover, rs-FC between prefrontal and occipito-parietal brain regions was negatively associated with emotion dysregulation levels. Microstructural anomalies across white matter tracts connecting temporal, parietal, and occipital brain regions were associated to difficulties in using reappraisal. Our findings suggest that specific behavioral and neurobiological substrates are linked to reappraising abilities. Furthermore, the ability to implement adaptive ER strategies could serve as protective factor against developing emotion dysregulation.

Default and control network connectivity dynamics track the stream of affect at multiple timescales.

In everyday life, the stream of affect results from the interaction between past experiences, expectations and the unfolding of events. How the brain represents the relationship between time and affect has been hardly explored, as it requires modeling the complexity of everyday life in the laboratory setting. Movies condense into hours a multitude of emotional responses, synchronized across subjects and characterized by temporal dynamics alike real-world experiences. Here, we use time-varying intersubject brain synchronization and real-time behavioral reports to test whether connectivity dynamics track changes in affect during movie watching. The results show that polarity and intensity of experiences relate to the connectivity of the default mode and control networks and converge in the right temporoparietal cortex. We validate these results in two experiments including four independent samples, two movies and alternative analysis workflows. Finally, we reveal chronotopic connectivity maps within the temporoparietal and prefrontal cortex, where adjacent areas preferentially encode affect at specific timescales.

Functional neuroanatomy of racial categorization from visual perception: A meta-analytic study.

We effortlessly sort people into different racial groups from their visual appearance and implicitly generate racial bias affecting cognition and behavior. As these mental activities provide the proximate mechanisms for social behaviours, it becomes essential to understand the neural activity underlying differences between own-race and other-race visual categorization. Yet intrinsic limitations of individual neuroimaging studies, owing to reduced sample size, inclusion of multiple races, and interactions between races in the participants and in the displayed visual stimuli, dampens generalizability of results. In the present meta-analytic study, we applied multimodal techniques to partly overcome these hurdles, and we investigated the entire functional neuroimaging literature on race categorization, therefore including more than 2000 Black, White and Asian participants. Our data-driven approach shows that own- and other-race visual categorization involves partly segregated neural networks, with distinct connectivity and functional profiles, and defined hierarchical organization. Categorization of own-race mainly engages areas related to cognitive components of empathy and mentalizing, such as the medial prefrontal cortex and the inferior frontal gyrus. These areas are functionally co-activated with cortical structures involved in auto-biographical memories and social knowledge. Conversely, other-race categorization recruits areas implicated in, and functionally connected with, visuo-attentive processing, like the fusiform gyrus and the inferior parietal lobule, and areas engaged in affective functions, like the amygdala. These results contribute to a better definition of the neural networks involved in the visual parcelling of social categories based on race, and help to situate these processes within a common neural space.

Enhanced dynamic functional connectivity (whole-brain chronnectome) in chess experts.

Multidisciplinary approaches have demonstrated that the brain is potentially modulated by the long-term acquisition and practice of specific skills. Chess playing can be considered a paradigm for shaping brain function, with complex interactions among brain networks possibly enhancing cognitive processing. Dynamic network analysis based on resting-state magnetic resonance imaging (rs-fMRI) can be useful to explore the effect of chess playing on whole-brain fluidity/dynamism (the chronnectome). Dynamic connectivity parameters of 18 professional chess players and 20 beginner chess players were evaluated applying spatial independent component analysis (sICA), sliding-time window correlation, and meta-state approaches to rs-fMRI data. Four indexes of meta-state dynamic fluidity were studied: i) the number of distinct meta-states a subject pass through, ii) the number of switches from one meta-state to another, iii) the span of the realized meta-states (the largest distance between two meta-states that subjects occupied), and iv) the total distance travelled in the state space. Professional chess players exhibited an increased dynamic fluidity, expressed as a higher number of occupied meta-states (meta-state numbers, 75.8 ± 7.9 vs 68.8 ± 12.0, p = 0.043 FDR-corrected) and changes from one meta-state to another (meta-state changes, 77.1 ± 7.3 vs 71.2 ± 11.0, p = 0.043 FDR-corrected) than beginner chess players. Furthermore, professional chess players exhibited an increased dynamic range, with increased traveling between successive meta-states (meta-state total distance, 131.7 ± 17.8 vs 108.7 ± 19.7, p = 0.0004 FDR-corrected). Chess playing may induce changes in brain activity through the modulation of the chronnectome. Future studies are warranted to evaluate if these potential effects lead to enhanced cognitive processing and if "gaming" might be used as a treatment in clinical practice.

Changes in peri-calcarine cortical thickness in blindsight.

Blindsight is the ability of patients with primary visual cortex (V1) damage to process information in their clinically blind visual field in the absence of conscious awareness. In addition to those with localized V1 lesions, some patients exhibiting this phenomenon have had a cerebral hemisphere removed or disconnected from the rest of the brain for the treatment of drug-resistant epilepsy (hemispherectomy). Research into the underlying neural substrates of blindsight has long implicated the intact visual cortex in maintaining residual vision and supporting visuo-guided responses to stimuli presented ipsilaterally within the blind visual field while operating outside the geniculo-striate pathway. A recent study demonstrated functional reorganization in the dorsal visual areas of the intact hemisphere, thereby supporting its compensatory role in non-conscious vision. In this study, we used cortical thickness analysis to examine anatomical differences in the visual cortex of the intact hemisphere of three subjects with varying degrees of cortical damage and well documented blindsight: two with a right hemispherectomy (complete and partial), and one with a left V1 lesion. T1-weighted MRI data were obtained for the subjects while control data were chosen from publicly available NKI-dataset to match closely the acquisition parameters of our blindsight cases. Our results show significant increases in cortical thickness in the visual cortex of all blindsight subjects compared to healthy controls, irrespective of age-onset, etiology, and extent of the damage. Our findings add to accumulating evidence from behavioral, functional imaging, and tractography studies of cerebral compensation and reorganization.

Imageability effect on the functional brain activity during a naming to definition task.

Lexical competence includes both the ability to relate words to the external world as accessed through (mainly) visual perception (referential competence) and the ability to relate words to other words (inferential competence). We investigated the role of visual imagery in lexical inferential competence by using an auditory version of an inferential naming-to-definition task, in which visual imageability of both definitions and target words was manipulated. A visual imageability-related brain activity (bilateral posterior-parietal lobe and ventrotemporal cortex, including fusiform gyrus) was found during a "pure" inferential performance. The definition effect in high vs. low imageability contrast suggests that a visual-imagery strategy is spontaneously activated during the retrieval of a word from a high imageable definition; such an effect appears to be independent of whether the target word is high or low imageable. This contributes to the understanding of the neural correlates of semantic processing and the differential role of spontaneous visual imagery, depending on the semantic properties of the processed stimuli.

The neural correlates of hedonic and eudaimonic happiness: An fMRI study.

The current study investigated the neural correlates of both hedonic and eudaimonic happiness. Seventeen students underwent fMRI while cued with written sentences describing three classes of life events: Hedonic events (HE), eudaimonic events (EE) and neutral events (NE). We asked participants to imagine and empathize with all these event types. Results showed that, compared to NE, both HE and EE activated a network involving frontal, temporal and parietal regions, as well as subcortical structures. However, in the HE/EE comparison, HE showed enhanced activity in frontal medial/middle regions and anterior cingulate cortex; by contrast, EE showed increased activity in the right precentral gyrus. Findings suggest that hedonic and eudaimonic happiness activate similar neural correlates. However, both kinds of happiness are also associated with distinctive brain areas serving distinctive functions.

The inner fluctuations of the brain in presymptomatic Frontotemporal Dementia: The chronnectome fingerprint.

Frontotemporal Dementia (FTD) is preceded by a long period of subtle brain changes, occurring in the absence of overt cognitive symptoms, that need to be still fully characterized. Dynamic network analysis based on resting-state magnetic resonance imaging (rs-fMRI) is a potentially powerful tool for the study of preclinical FTD. In the present study, we employed a "chronnectome" approach (recurring, time-varying patterns of connectivity) to evaluate measures of dynamic connectivity in 472 at-risk FTD subjects from the Genetic Frontotemporal dementia research Initiative (GENFI) cohort. We considered 249 subjects with FTD-related pathogenetic mutations and 223 mutation non-carriers (HC). Dynamic connectivity was evaluated using independent component analysis and sliding-time window correlation to rs-fMRI data, and meta-state measures of global brain flexibility were extracted. Results show that presymptomatic FTD exhibits diminished dynamic fluidity, visiting less meta-states, shifting less often across them, and travelling through a narrowed meta-state distance, as compared to HC. Dynamic connectivity changes characterize preclinical FTD, arguing for the desynchronization of the inner fluctuations of the brain. These changes antedate clinical symptoms, and might represent an early signature of FTD to be used as a biomarker in clinical trials.

Functional neuroanatomy of blindsight revealed by activation likelihood estimation meta-analysis.

Blindsight, the residual abilities of patients with cortical blindness to respond proficiently to stimuli they do not consciously acknowledge, offers a unique opportunity to study the functional and anatomical mechanisms sustaining visual awareness. Over decades, the phenomenon has been documented in a wide number of different patients, across independent laboratories, and for a variety of tasks and stimulus properties. Nevertheless, the functional neuroanatomy of blindsight remains elusive and alternative proposals have been put forth. To tackle this issue from a novel perspective, we performed a quantitative Activation Likelihood Estimation (ALE) meta-analysis on the neuroimaging literature available on blindsight. Significant activity was reported in subcortical structures, such as the superior colliculus, pulvinar and amygdala, as well as in cortical extrastriate areas along the dorsal and ventral visual stream. This data-driven functional network collectively defines the extant neural fingerprint of blindsight. To further characterize the unique combination of segregation and integration in brain networks engaged in blindsight, we measured the relationship between active areas and experimental features in the original studies, their clustering and hierarchical organization. Results support a network-based organization in the functional neuroanatomy of blindsight, which likely reflects the intersection of different stimulus properties and behavioural tasks examined. We suggest that the conceptualization of blindsight as a constellation of multiple nonconscious visual abilities is better apt as a summary of present-day wisdom, thereby mirroring the variety of existing V1-independent pathway and their different functional roles.

Classical paintings may trigger pain and pleasure in the gendered brain.

The human body is the most common object of pictorial representation in western art and its representations trigger a vast range of experiences from pain to pleasure. The goal of this study was to investigate brain activity triggered by paintings of male and female body images exemplifying conditions associated with pleasure or pain. Our findings show participant-general as well as gender specific brain activity for either the pain or the pleasure conditions. Although our participants were fully aware that they were viewing artworks, the inferior parietal lobule - known for its role in the perception of emotional body images - and the somatosensory cortex related to touch were selectively active for female body paintings in all participants in the pleasure conditions. As regards gender we observed that the sight of female bodies activated the subgenual anterior cingulate cortex in males, an area known to subserve autonomic arousal. In contrast, in females the sight of the male body activated reward and control related parts of the dorsal anterior cingulate cortex. This study supports the notion that some basic evolutionary processes operate when we view body images, also when they are cultural heritage paintings far removed from daily experience.

How do morphological alterations caused by chronic pain distribute across the brain? A meta-analytic co-alteration study.

•In chronic pain, gray matter (GM) alterations are not distributed randomly across the brain.•The pattern of co-alterations resembles that of brain connectivity.•The alterations' distribution partly rely on the pathways of functional connectivity.•This method allows us to identify tendencies in the distribution of GM co-alteration related to chronic pain.

Action Observation Areas Represent Intentions From Subtle Kinematic Features.

Mirror neurons have been proposed to underlie humans' ability to understand others' actions and intentions. Despite 2 decades of research, however, the exact computational and neuronal mechanisms implied in this ability remain unclear. In the current study, we investigated whether, in the absence of contextual cues, regions considered to be part of the human mirror neuron system represent intention from movement kinematics. A total of 21 participants observed reach-to-grasp movements, performed with either the intention to drink or to pour while undergoing functional magnetic resonance imaging. Multivoxel pattern analysis revealed successful decoding of intentions from distributed patterns of activity in a network of structures comprising the inferior parietal lobule, the superior parietal lobule, the inferior frontal gyrus, and the middle frontal gyrus. Consistent with the proposal that parietal regions play a key role in intention understanding, classifier weights were higher in the inferior parietal region. These results provide the first demonstration that putative mirror neuron regions represent subtle differences in movement kinematics to read the intention of an observed motor act.

Intact hemisphere and corpus callosum compensate for visuomotor functions after early visual cortex damage.

Unilateral damage to the primary visual cortex (V1) leads to clinical blindness in the opposite visual hemifield, yet nonconscious ability to transform unseen visual input into motor output can be retained, a condition known as "blindsight." Here we combined psychophysics, functional magnetic resonance imaging, and tractography to investigate the functional and structural properties that enable the developing brain to partly overcome the effects of early V1 lesion in one blindsight patient. Visual stimuli appeared in either the intact or blind hemifield and simple responses were given with either the left or right hand, thereby creating conditions where visual input and motor output involve the same or opposite hemisphere. When the V1-damaged hemisphere was challenged by incoming visual stimuli, or controlled manual responses to these unseen stimuli, the corpus callosum (CC) dynamically recruited areas in the visual dorsal stream and premotor cortex of the intact hemisphere to compensate for altered visuomotor functions. These compensatory changes in functional brain activity were paralleled by increased connections in posterior regions of the CC, where fibers connecting homologous areas of the parietal cortex course.