Characterizing the discriminability of visual categorical information in strongly connected voxels.

Functional brain responses are strongly influenced by connectivity. Recently, we demonstrated a major example of this: category discriminability within occipitotemporal cortex (OTC) is enhanced for voxel sets that share strong functional connectivity to distal brain areas, relative to those that share lesser connectivity. That is, within OTC regions, sets of 'most-connected' voxels show improved multivoxel pattern discriminability for tool-, face-, and place stimuli relative to voxels with weaker connectivity to the wider brain. However, understanding whether these effects generalize to other domains (e.g. body perception network), and across different levels of the visual processing streams (e.g. dorsal as well as ventral stream areas) is an important extension of this work. Here, we show that this so-called connectivity-guided decoding (CGD) effect broadly generalizes across a wide range of categories (tools, faces, bodies, hands, places). This effect is robust across dorsal stream areas, but less consistent in earlier ventral stream areas. In the latter regions, category discriminability is generally very high, suggesting that extraction of category-relevant visual properties is less reliant on connectivity to downstream areas. Further, CGD effects are primarily expressed in a category-specific manner: For example, within the network of tool regions, discriminability of tool information is greater than non-tool information. The connectivity-guided decoding approach shown here provides a novel demonstration of the crucial relationship between wider brain connectivity and complex local-level functional responses at different levels of the visual processing streams. Further, this approach generates testable new hypotheses about the relationships between connectivity and local selectivity.

Converging evidence that left extrastriate body area supports visual sensitivity to social interactions.

Navigating our complex social world requires processing the interactions we observe. Recent psychophysical and neuroimaging studies provide parallel evidence that the human visual system may be attuned to efficiently perceive dyadic interactions. This work implies, but has not yet demonstrated, that activity in body-selective cortical regions causally supports efficient visual perception of interactions. We adopt a multi-method approach to close this important gap. First, using a large fMRI dataset (n = 92), we found that the left hemisphere extrastriate body area (EBA) responds more to face-to-face than non-facing dyads. Second, we replicated a behavioral marker of visual sensitivity to interactions: categorization of facing dyads is more impaired by inversion than non-facing dyads. Third, in a pre-registered experiment, we used fMRI-guided transcranial magnetic stimulation to show that online stimulation of the left EBA, but not a nearby control region, abolishes this selective inversion effect. Activity in left EBA, thus, causally supports the efficient perception of social interactions.

Testing cognitive theories with multivariate pattern analysis of neuroimaging data.

Multivariate pattern analysis (MVPA) has emerged as a powerful method for the analysis of functional magnetic resonance imaging, electroencephalography and magnetoencephalography data. The new approaches to experimental design and hypothesis testing afforded by MVPA have made it possible to address theories that describe cognition at the functional level. Here we review a selection of studies that have used MVPA to test cognitive theories from a range of domains, including perception, attention, memory, navigation, emotion, social cognition and motor control. This broad view reveals properties of MVPA that make it suitable for understanding the 'how' of human cognition, such as the ability to test predictions expressed at the item or event level. It also reveals limitations and points to future directions.

Asymmetric visual representation of sex from facial appearance.

We efficiently infer others' traits from their faces, and these inferences powerfully shape our social behaviour. Here, we investigated how sex is represented in facial appearance. Based on previous findings from sex-judgment tasks, we hypothesized that the perceptual encoding of sex is not balanced but rather polarized: for the processes that generate a sex percept, the default output is "male," and the representation of female faces extends that of the male, engaging activity over unique detectors that are not activated by male faces. We tested this hypothesis with the logic of Treisman's studies of visual search asymmetries, predicting that observers should more readily detect the presence of female faces amongst male distractors than vice versa. Across three experiments (N = 32 each), each using different face stimuli, we confirmed this prediction in response time and sensitivity measures. We apply GIST analyses to the face stimuli to exclude that the search asymmetry is explained by differences in image homogeneity. These findings demonstrate a property of the coding that links facial appearance with a significant social trait: the female face is coded as an extension of a male default. We offer a mechanistic description of perceptual detectors to account for our findings and posit that the origins of this polarized coding scheme are an outcome of biased early developmental experience.

The role of motion in the neural representation of social interactions in the posterior temporal cortex.

Humans are an inherently social species, with multiple focal brain regions sensitive to various visual social cues such as faces, bodies, and biological motion. More recently, research has begun to investigate how the brain responds to more complex, naturalistic social scenes, identifying a region in the posterior superior temporal sulcus (SI-pSTS; i.e., social interaction pSTS), amongst others, as an important region for processing social interaction. This research, however, has presented images or videos, and thus the contribution of motion to social interaction perception in these brain regions is not yet understood. In the current study, 22 participants viewed videos, image sequences, scrambled image sequences and static images of either social interactions or non-social independent actions. Combining univariate and multivariate analyses, we confirm that bilateral SI-pSTS plays a central role in dynamic social interaction perception but is much less involved when 'interactiveness' is conveyed solely with static cues. Regions in the social brain, including SI-pSTS and extrastriate body area (EBA), showed sensitivity to both motion and interactive content. While SI-pSTS is somewhat more tuned to video interactions than is EBA, both bilateral SI-pSTS and EBA showed a greater response to social interactions compared to non-interactions and both regions responded more strongly to videos than static images. Indeed, both regions showed higher responses to interactions than independent actions in videos and intact sequences, but not in other conditions. Exploratory multivariate regression analyses suggest that selectivity for simple visual motion does not in itself drive interactive sensitivity in either SI-pSTS or EBA. Rather, selectivity for interactions expressed in point-light animations, and selectivity for static images of bodies, make positive and independent contributions to this effect across the LOTC region. Our results strongly suggest that EBA and SI-pSTS work together during dynamic interaction perception, at least when interactive information is conveyed primarily via body information. As such, our results are also in line with proposals of a third visual stream supporting dynamic social scene perception.

Asymmetric visual representation of sex from human body shape.

We efficiently infer others' states and traits from their appearance, and these inferences powerfully shape our social behaviour. One key trait is sex, which is strongly cued by the appearance of the body. What are the visual representations that link body shape to sex? Previous studies of visual sex judgment tasks find observers have a bias to report "male", particularly for ambiguous stimuli. This finding implies a representational asymmetry - that for the processes that generate a sex percept, the default output is "male", and "female" is determined by the presence of additional perceptual evidence. That is, female body shapes are positively coded by reference to a male default shape. This perspective makes a novel prediction in line with Treisman's studies of visual search asymmetries: female body targets should be more readily detected amongst male distractors than vice versa. Across 10 experiments (N = 32 each) we confirmed this prediction and ruled out alternative low-level explanations. The asymmetry was found with profile and frontal body silhouettes, frontal photographs, and schematised icons. Low-level confounds were controlled by balancing silhouette images for size and homogeneity, and by matching physical properties of photographs. The female advantage was nulled for inverted icons, but intact for inverted photographs, suggesting reliance on distinct cues to sex for different body depictions. Together, these findings demonstrate a principle of the perceptual coding that links bodily appearance with a significant social trait: the female body shape is coded as an extension of a male default. We conclude by offering a visual experience account of how these asymmetric representations arise in the first place.

Examining the value of body gestures in social reward contexts.

Brain regions associated with the processing of tangible rewards (such as money, food, or sex) are also involved in anticipating social rewards and avoiding social punishment. To date, studies investigating the neural underpinnings of social reward have presented feedback via static or dynamic displays of faces to participants. However, research demonstrates that participants find another type of social stimulus, namely, biological motion, rewarding as well, and exert effort to engage with this type of stimulus. Here we examine whether feedback presented via body gestures in the absence of facial cues also acts as a rewarding stimulus and recruits reward-related brain regions. To achieve this, we investigated the neural underpinnings of anticipating social reward and avoiding social disapproval presented via gestures alone, using a social incentive delay task. As predicted, the anticipation of social reward and avoidance of social disapproval engaged reward-related brain regions, including the nucleus accumbens, in a manner similar to previous studies' reports of feedback presented via faces and money. This study provides the first evidence that human body motion alone engages brain regions associated with reward processing in a similar manner to other social (i.e. faces) and non-social (i.e. money) rewards. The findings advance our understanding of social motivation in human perception and behavior.

Neural representations of haptic object size in the human brain revealed by multivoxel fMRI patterns.

The brain must interpret sensory input from diverse receptor systems to estimate object properties. Much has been learned about the brain mechanisms behind these processes in vision, but our understanding of haptic perception remains less clear. Here we examined haptic judgments of object size, which require integrating multiple cutaneous and proprioceptive afferent signals, as a model problem. To identify candidate human brain regions that support this process, participants (n = 16) in an event-related functional MRI experiment grasped objects to categorize them as one of four sizes. Object sizes were calibrated psychophysically to be equally distinct for each participant. We applied representational similarity logic to whole brain, multivoxel searchlight analyses to identify brain regions that exhibit size-relevant voxelwise activity patterns. Of particular interest was to identify regions for which more similar sizes produce more similar patterns of activity, which constitutes evidence of a metric size code. Regions of the intraparietal sulcus and the lateral prefrontal cortex met this criterion, both within hands and across hands. We suggest that these regions compute representations of haptic size that abstract over the specific peripheral afferent signals generated in a grasp. Results of a matched visual size task, performed by the same participants and analyzed in the same fashion, identified similar regions, indicating that these representations may be partly modality general. We consider these results with respect to perspectives on magnitude estimation in general and to computational views on perceptual signal integration.NEW & NOTEWORTHY Our understanding of the neural basis of haptics (perceiving the world through touch) remains incomplete. We used functional MRI to study human haptic judgments of object size, which require integrating multiple afferent signals. Multivoxel pattern analyses identified intraparietal and prefrontal regions that encode size haptically in a metric and hand-invariant fashion. Effector-independent haptic size estimates are useful on their own and in combination with other sensory estimates for a variety of perceptual and motor tasks.

Perceiving emotion and sex from the body: evidence from the Garner task for independent processes.

The appearance of the body signals socially relevant states and traits, but the how these cues are perceived is not well understood. Here we examined judgments of emotion and sex from the body's appearance. Understanding how we extract these cues is important because they are both salient and socially relevant. Participants viewed body images and either reported the emotion expressed by each body while ignoring its sex, or else reported the sex while ignoring its emotion. Following Garner's logic, two types of blocks were compared. In control blocks, the task-irrelevant dimension was fixed (e.g. all male in an emotion judgment task), whereas in orthogonal blocks it varied orthogonally to the task-relevant dimension (e.g. male-female). Where two dimensions draw on shared processes, interference results in relatively slower responses during orthogonal blocks. In contrast, a finding of no Garner interference - efficient selection of the task-relevant dimension - is taken to reflect independent processes. Bayesian analyses revealed evidence of no Garner interference between sex and emotion judgments, showing that extraction of these distinct signals from the body's appearance proceeds along largely parallel processing streams. These findings are informative about the mental architecture behind our perception of socially relevant characteristics of other people.

Automatic imitation remains unaffected under cognitive load.

Automaticity has been argued to be a core feature of the mental processes that guide social interactions, such as those underpinning imitative behaviors. To date, however, there is little known about the automaticity of imitative tendencies. In the current study, we used a finger movement stimulus-response compatibility task to index processes associated with controlling the urge to copy other people's actions. In addition, we manipulated the level of load placed on a secondary cognitive task to test if there is a capacity limit in the systems that filter distractor finger movement stimuli. Across three experiments, we showed that whether letter strings (Experiment 1), faces (Experiment 2), or hand postures (Experiment 3) are held in working memory, there was no impact on compatibility effects in the main task. These findings show that the cognitive operations that generate imitative tendencies are relatively efficient in that they operate the same whether or not a central resource is taxed heavily with nonsocial (letter strings) or social stimuli (faces and hand postures). Therefore, in the sense of persisting in the presence of a demanding cognitive load, this type of imitation behavior can be considered automatic. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Neural networks supporting social evaluation of bodies based on body shape.

Body shape cues inferences regarding personality and health, but the neural processes underpinning such inferences remain poorly understood. Across two fMRI experiments, we test the extent to which neural networks associated with body perception and theory-of-mind (ToM) support social inferences based on body shape. Participants observed obese, muscular, and slim bodies that cued distinct social inferences as revealed in behavioural pilot experiments. To investigate judgment intentionality, the first fMRI experiment required participants to detect repeat presentations of bodies, whereas in fMRI Experiment 2 participants intentionally formed an impression. Body and ToM networks were localized using independent functional localisers. Experiment 1 revealed no differential network engagement for muscular or obese compared to slim bodies. By contrast, in Experiment 2, compared to slim bodies, forming impressions of muscular bodies engaged the body-network more, whereas the ToM-network was engaged more when forming impressions of obese bodies. These results demonstrate that social judgments based on body shape do not rely on a single neural mechanism, but rather on multiple mechanisms that are separately sensitive to body fat and muscularity. Moreover, dissociable responses are only apparent when intentionally forming an impression. Thus, these experiments show how segregated networks operate to extract socially-relevant information cued by body shape.

fMRI repetition suppression reveals no sensitivity to trait judgments from faces in face perception or theory-of-mind networks.

The human face cues a wealth of social information, but the neural mechanisms that underpin social attributions from faces are not well known. In the current fMRI experiment, we used repetition suppression to test the hypothesis that populations of neurons in face perception and theory-of-mind neural networks would show sensitivity to faces that cue distinct trait judgments. Although faces were accurately discriminated based on associated traits, our results showed no evidence that face or theory-of-mind networks showed repetition suppression for face traits. Thus, we do not provide evidence for population coding models of face perception that include sensitivity to high and low trait features. Due to aspects of the experimental design, which bolstered statistical power and sensitivity, we have reasonable confidence that we could detect effects of a moderate size, should they exist. The null findings reported here, therefore, add value to models of neural organisation in social perception by showing instances where effects are absent or small. To test the generalisability of our findings, future work should test different types of trait judgment and different types of facial stimuli, in order to further probe the neurobiological bases of impression formation based on facial appearance.

Category selectivity in human visual cortex: Beyond visual object recognition.

Human ventral temporal cortex shows a categorical organization, with regions responding selectively to faces, bodies, tools, scenes, words, and other categories. Why is this? Traditional accounts explain category selectivity as arising within a hierarchical system dedicated to visual object recognition. For example, it has been proposed that category selectivity reflects the clustering of category-associated visual feature representations, or that it reflects category-specific computational algorithms needed to achieve view invariance. This visual object recognition framework has gained renewed interest with the success of deep neural network models trained to "recognize" objects: these hierarchical feed-forward networks show similarities to human visual cortex, including categorical separability. We argue that the object recognition framework is unlikely to fully account for category selectivity in visual cortex. Instead, we consider category selectivity in the context of other functions such as navigation, social cognition, tool use, and reading. Category-selective regions are activated during such tasks even in the absence of visual input and even in individuals with no prior visual experience. Further, they are engaged in close connections with broader domain-specific networks. Considering the diverse functions of these networks, category-selective regions likely encode their preferred stimuli in highly idiosyncratic formats; representations that are useful for navigation, social cognition, or reading are unlikely to be meaningfully similar to each other and to varying degrees may not be entirely visual. The demand for specific types of representations to support category-associated tasks may best account for category selectivity in visual cortex. This broader view invites new experimental and computational approaches.

Evidence for Integrated Visual Face and Body Representations in the Anterior Temporal Lobes.

Research on visual face perception has revealed a region in the ventral anterior temporal lobes, often referred to as the anterior temporal face patch (ATFP), which responds strongly to images of faces. To date, the selectivity of the ATFP has been examined by contrasting responses to faces against a small selection of categories. Here, we assess the selectivity of the ATFP in humans with a broad range of visual control stimuli to provide a stronger test of face selectivity in this region. In Experiment 1, participants viewed images from 20 stimulus categories in an event-related fMRI design. Faces evoked more activity than all other 19 categories in the left ATFP. In the right ATFP, equally strong responses were observed for both faces and headless bodies. To pursue this unexpected finding, in Experiment 2, we used multivoxel pattern analysis to examine whether the strong response to face and body stimuli reflects a common coding of both classes or instead overlapping but distinct representations. On a voxel-by-voxel basis, face and whole-body responses were significantly positively correlated in the right ATFP, but face and body-part responses were not. This finding suggests that there is shared neural coding of faces and whole bodies in the right ATFP that does not extend to individual body parts. In contrast, the same approach revealed distinct face and body representations in the right fusiform gyrus. These results are indicative of an increasing convergence of distinct sources of person-related perceptual information proceeding from the posterior to the anterior temporal cortex.

Linking person perception and person knowledge in the human brain.

Neuroscience research has examined separately how we detect human agents on the basis of their face and body (person perception) and how we reason about their thoughts, traits or intentions (person knowledge). Neuroanatomically distinct networks have been associated with person perception and person knowledge, but it remains unknown how multiple features of a person (e.g. thin and kind) are linked to form a holistic identity representation. In this fMRI experiment, we investigated the hypothesis that when encountering another person specialised person perception circuits would be functionally coupled with circuits involved in person knowledge. In a factorial design, we paired bodies or names with trait-based or neutral statements, and independent localiser scans identified body-selective and mentalising networks. When observing a body paired with a trait-implying statement, functional connectivity analyses demonstrated that body-selective patches in bilateral fusiform gyri were functionally coupled with nodes of the mentalising network. We demonstrate that when forming a representation of a person circuits for representing another person's physical appearance are linked to circuits that are engaged when reasoning about trait-based character. These data support the view that a 'who' system for social cognition involves communication between perceptual and inferential mechanisms when forming a representation of another's identity.

Body selectivity in occipitotemporal cortex: Causal evidence.

Perception of others' bodies provides information that is useful for a number of important social-cognitive processes. Evidence from neuroimaging methods has identified focal cortical regions that are highly selective for perceiving bodies and body parts, including the extrastriate body area (EBA) and fusiform body area (FBA). Our understanding of the functional properties of these regions, and their causal contributions to behavior, has benefitted from the study of neuropsychological patients and particularly from investigations using transcranial magnetic stimulation (TMS). We review this evidence, focusing on TMS studies that are revealing of how (and when) activity in EBA contributes to detecting people in natural scenes; to resolving their body shape, movements, actions, individual parts, and identities; and to guiding goal-directed behavior. These findings are considered in reference to a framework for body perception in which the patterns of neural activity in EBA and FBA jointly serve to make explicit the elements of the visual scene that correspond to the body and its parts. These representations are modulated by other sources of information such as prior knowledge, and are shared with wider brain networks involved in many aspects of social cognition.

The lateral occipitotemporal cortex in action.

Understanding and responding to other people's actions is fundamental for social interactions. Whereas many studies emphasize the importance of parietal and frontal regions for these abilities, several lines of recent research show that the human lateral occipitotemporal cortex (LOTC) represents varied aspects of action, ranging from perception of tools and bodies and the way they typically move, to understanding the meaning of actions, to performing overt actions. Here, we highlight common themes across these lines of work, which have informed theories related to high-level vision, concepts, social cognition, and apraxia. We propose that patterns of activity in LOTC form representational spaces, the dimensions of which capture perceptual, semantic, and motor knowledge of how actions change the state of the world.

Brain correlates of experience-dependent changes in stimulus discrimination based on the amount and schedule of exposure.

One product of simple exposure to similar visual stimuli is that they become easier to distinguish. The early visual cortex and other brain areas (such as the prefrontal cortex) have been implicated in such perceptual learning effects, but the anatomical specificity within visual cortex and the relationship between sensory cortex and other brain areas has yet to be examined. Moreover, while variations in the schedule (rather than merely the amount) of exposure influence experience-dependent improvement in discrimination, the neural sequelae of exposure schedule have not been fully investigated. In an event-related fMRI study, participants were exposed to confusable pairs of faces, scenes and dot patterns, using either intermixed or blocked presentation schedules. Participants then performed same/different judgements with exposed and novel pairs of stimuli. Stimulus independent activation, which was correlated with experience-dependent improvement in discrimination, was seen in frontal areas (e.g. frontal and supplementary eye fields and dorsolateral prefrontal cortex) and in early visual cortex (V1-4). In all regions, the difference in activation between exposed and novel stimuli decreased as a function of the degree of discrimination improvement. Overall levels of BOLD activation differed across regions, consistent with the possibility that, as a consequence of experience, processing shifts from initial engagement of early visual regions to higher order visual areas. Similar relationships were observed when contrasting intermixed with blocked exposure, suggesting that the schedule of exposure primarily influences the degree of, rather than the mechanisms for, discrimination performance.

A critical role for the hippocampus and perirhinal cortex in perceptual learning of scenes and faces: complementary findings from amnesia and FMRI.

It is debated whether subregions within the medial temporal lobe (MTL), in particular the hippocampus (HC) and perirhinal cortex (PrC), play domain-sensitive roles in learning. In the present study, two patients with differing degrees of MTL damage were first exposed to pairs of highly similar scenes, faces, and dot patterns and then asked to make repeated same/different decisions to preexposed and nonexposed (novel) pairs from the three categories (Experiment 1). We measured whether patients would show a benefit of prior exposure (preexposed > nonexposed) and whether repetition of nonexposed (and preexposed) pairs at test would benefit discrimination accuracy. Although selective HC damage impaired learning of scenes, but not faces and dot patterns, broader MTL damage involving the HC and PrC compromised discrimination learning of scenes and faces but left dot pattern learning unaffected. In Experiment 2, a similar task was run in healthy young participants in the MRI scanner. Functional region-of-interest analyses revealed that posterior HC and posterior parahippocampal gyrus showed greater activity during scene pattern learning, but not face and dot pattern learning, whereas PrC, anterior HC, and posterior fusiform gyrus were recruited during discrimination learning for faces, but not scenes and dot pattern learning. Critically, activity in posterior HC and PrC, but not the other functional region-of-interest analyses, was modulated by accuracy (correct > incorrect within a preferred category). Therefore, both approaches revealed a key role for the HC and PrC in discrimination learning, which is consistent with representational accounts in which subregions in these MTL structures store complex spatial and object representations, respectively.

Crossmodal and action-specific: neuroimaging the human mirror neuron system.

The notion of a frontoparietal human mirror neuron system (HMNS) has been used to explain a range of social phenomena. However, most human neuroimaging studies of this system do not address critical 'mirror' properties: neural representations should be action specific and should generalise across visual and motor modalities. Studies using repetition suppression (RS) and, particularly, multivariate pattern analysis (MVPA) highlight the contribution to action perception of anterior parietal regions. Further, these studies add to mounting evidence that suggests the lateral occipitotemporal cortex plays a role in the HMNS, but they offer less support for the involvement of the premotor cortex. Neuroimaging, particularly through application of MVPA, has the potential to reveal the properties of the HMNS in further detail, which could challenge prevailing views about its neuroanatomical organisation.