Faces under continuous flash suppression capture attention faster than objects, but without a face-evoked steady-state visual potential: Is curvilinearity responsible for the behavioral effect?

Face perception is a vital part of human social interactions. The social value of faces makes their efficient detection evolutionarily advantageous. It has been suggested that this might occur nonconsciously, but experimental results are equivocal thus far. Here, we probe nonconscious face perception using a novel combination of binocular rivalry with continuous flash suppression and steady-state visually evoked potentials. In the first two experiments, participants viewed either non-face objects, neutral faces (Study 1), or fearful faces (Study 2). Consistent with the hypothesis that faces are processed nonconsciously, we found that faces broke through suppression faster than objects. We did not, however, observe a concomitant face-selective steady-state visually evoked potential. Study 3 was run to reconcile this paradox. We hypothesized that the faster breakthrough time was due to a mid-level visual feature, curvilinearity, rather than high-level category membership, which would explain the behavioral difference without neural evidence of face-selective processing. We tested this hypothesis by presenting participants with four different groups of stimuli outside of conscious awareness: rectilinear objects (e.g., chessboard), curvilinear objects (e.g., dartboard), faces, and objects that were not dominantly curvilinear or rectilinear. We found that faces and curvilinear objects broke through suppression faster than objects and rectilinear objects. Moreover, there was no difference between faces and curvilinear objects. These results support our hypothesis that the observed behavioral advantage for faces is due to their curvilinearity, rather than category membership.

Early identity recognition of familiar faces is not dependent on holistic processing.

It is widely accepted that holistic processing is critical for early face recognition, but recent work has suggested a larger role for feature-based processing. The earliest step in familiar face recognition is thought to be matching a perceptual representation of a familiar face to a stored representation of that face, which is thought to be indexed by the N250r event-related potential (ERP). In the current face-priming studies, we investigated whether this perceptual representation can be effectively activated by feature-based processing. In the first experiment, prime images were familiar whole faces, isolated eyes or isolated mouths. Whole faces and isolated eyes, but not isolated mouths, effectively modulated the N250r. In the second experiment, prime images were familiar whole faces presented either upright or inverted. Inverted face primes were no less effective than upright face primes in modulating the N250r. Together, the results of these studies indicate that activation of the earliest face recognition processes is not dependent on holistic processing of a typically configured face. Rather, feature-based processing can effectively activate the perceptual memory of a familiar face. However, not all features are effective primes as we found eyes, but not mouths, were effective in activating early face recognition.

Sensitivity to Faces with Typical and Atypical Part Configurations within Regions of the Face-processing Network: An fMRI Study.

Perception of faces has been shown to engage a domain-specific set of brain regions, including the occipital face area (OFA) and the fusiform face area (FFA). It is commonly held that the OFA is responsible for the detection of faces in the environment, whereas the FFA is responsible for processing the identity of the face. However, an alternative model posits that the FFA is responsible for face detection and subsequently recruits the OFA to analyze the face parts in the service of identification. An essential prediction of the former model is that the OFA is not sensitive to the arrangement of internal face parts. In the current fMRI study, we test the sensitivity of the OFA and FFA to the configuration of face parts. Participants were shown faces in which the internal parts were presented in a typical configuration (two eyes above a nose above a mouth) or in an atypical configuration (the locations of individual parts were shuffled within the face outline). Perception of the atypical faces evoked a significantly larger response than typical faces in the OFA and in a wide swath of the surrounding posterior occipitotemporal cortices. Surprisingly, typical faces did not evoke a significantly larger response than atypical faces anywhere in the brain, including the FFA (although some subthreshold differences were observed). We propose that face processing in the FFA results in inhibitory sculpting of activation in the OFA, which accounts for this region's weaker response to typical than to atypical configurations.

Stimulus-induced reversal of information flow through a cortical network for animacy perception.

Decades of research have demonstrated that a region of the right fusiform gyrus (FG) and right posterior superior temporal sulcus (pSTS) responds preferentially to static faces and biological motion, respectively. Despite this view, both regions activate in response to both stimulus categories and to a range of other stimuli, such as goal-directed actions, suggesting that these regions respond to characteristics of animate agents more generally. Here we propose a neural model for animacy detection composed of processing streams that are initially differentially sensitive to cues signaling animacy, but that ultimately act in concert to support reasoning about animate agents. We use dynamic causal modeling, a measure of effective connectivity, to demonstrate that the directional flow of information between the FG and pSTS is initially dependent on the characteristics of the animate agent presented, a key prediction of our proposed network for animacy detection.

Face, eye, and body selective responses in fusiform gyrus and adjacent cortex: an intracranial EEG study.

Functional MRI (fMRI) studies have investigated the degree to which processing of whole faces, face-parts, and bodies are differentially localized within the fusiform gyrus and adjacent ventral occipitotemporal cortex. While some studies have emphasized the spatial differentiation of processing into discrete areas, others have emphasized the overlap of processing and the importance of distributed patterns of activity. Intracranial EEG (iEEG) recorded from subdural electrodes provides excellent temporal and spatial resolution of local neural activity, and thus provides an alternative method to fMRI for studying differences and commonalities in face and body processing. In this study we recorded iEEG from 12 patients while they viewed images of novel faces, isolated eyes, headless bodies, and flowers. Event-related potential analysis identified 69 occipitotemporal sites at which there was a face-, eye-, or body-selective response when contrasted to flowers. However, when comparing faces, eyes, and bodies to each other at these sites, we identified only 3 face-specific, 13 eye-specific, and 1 body-specific electrodes. Thus, at the majority of sites, faces, eyes, and bodies evoked similar responses. However, we identified ten locations at which the amplitude of the responses spatially varied across adjacent electrodes, indicating that the configuration of current sources and sinks were different for faces, eyes, and bodies. Our results also demonstrate that eye-sensitive regions are more abundant and more purely selective than face- or body-sensitive regions, particularly in lateral occipitotemporal cortex.

Repetition suppression of face-selective evoked and induced EEG recorded from human cortex.

In functional MRI studies, repetition suppression refers to the reduction of hemodynamic activation to repeated stimulus presentation. For example, the repeated presentation of a face reduces the hemodynamic response evoked by faces in the fusiform gyrus. The neural events that underlie repetition suppression are not well understood. Indeed, in contrast to the hemodynamic response, the face-specific N200 recorded from subdural electrodes on the ventral occipitotemporal cortex, primarily along the fusiform gyrus, has been reported to be insensitive to face-identity repetition. We have previously described a face-specific broadband gamma (30-100 Hz) response at ventral face-specific N200 sites that is functionally dissociable from the N200. In this study, we investigate whether gamma and other components of the electroencephalogram spectrum are affected by face-identity repetition independently of the N200. Participants viewed sequentially presented identical faces. At sites on and around the fusiform gyrus, we found that face repetition modulated alpha (8-12 Hz), low-gamma (30-60 Hz), and high-gamma (60-100 Hz) synchrony, but not the N200. These findings provide evidence of a spatially co-localized progression of face processing. Whereas the N200 reflects an initial obligatory response that is less sensitive to face-identity repetition, the subsequent spectral fluctuations reflect more elaborative face processing and are thus sensitive to face novelty. It is notable that the observed modulations were different for different frequency bands. We observed repetition suppression of broadband gamma, but repetition enhancement of alpha synchrony. This difference is discussed with regard to an existing model of repetition suppression and behavioral repetition priming.

Probabilistic atlases for face and biological motion perception: an analysis of their reliability and overlap.

Neuroimaging research has identified several category-selective regions in visual cortex that respond most strongly when viewing an exemplar image from a preferred category, such as faces. Recent studies, however, have suggested a more complex pattern of activation that has been heretofore unrecognized, e.g., the presence of additional patches of activation to faces beyond the well-studied fusiform face area, and the activation of ostensible face selective regions by animate motion of non-biological forms. Here, we characterize the spatial pattern of brain activity evoked by viewing faces or biological motion in large fMRI samples (N>120). We create probabilistic atlases for both face and biological motion activation, and directly compare their spatial patterns of activation. Our findings support the suggestion that the fusiform face area is composed of at least two separable foci of activation. The face-evoked response in the fusiform and nearby ventral temporal cortex has good reliability across runs; however, we found surprisingly high variability in lateral brain regions by faces, and for all brain regions by biological motion, which had an overall much lower effect size. We found that faces and biological motion evoke substantially overlapping activation distributions in both ventral and lateral occipitotemporal cortices. The peaks of activation for these different categories within these overlapping regions were close but distinct.

The fMRI BOLD signal tracks electrophysiological spectral perturbations, not event-related potentials.

Functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) are primary tools of the psychological neurosciences. It is therefore important to understand the relationship between hemodynamic and electrophysiological responses. An early study by Huettel and colleagues found that the coupling of fMRI blood-oxygen-level-dependent signal (BOLD) and subdurally-recorded signal-averaged event-related potentials (ERPs) was not consistent across brain regions. Instead, a growing body of evidence now indicates that hemodynamic changes measured by fMRI reflect non-phase-locked changes in high frequency power rather than the phase-locked ERP. Here, we revisit the data from Huettel and colleagues and measure event-related spectral perturbations (ERSPs) to examine the time course of frequency changes. We found that, unlike the ERP, γ-ERSP power was consistently coupled with the hemodynamic response across three visual cortical regions. Stimulus duration modulated the BOLD signal and the γ-ERSP in the peri-calcarine and fusiform cortices, whereas there was no such modulation of either physiological signal in the lateral temporal-occipital cortex. This finding reconciles the original report with the more recent literature and demonstrates that the ERP and ERSP reflect dissociable aspects of neural activity.

Autism spectrum traits predict the neural response to eye gaze in typical individuals.

Autism Spectrum Disorders (ASD) are neurodevelopmental disorders characterised by impaired social interaction and communication, restricted interests and repetitive behaviours. The severity of these characteristics are posited to lie on a continuum extending into the typical population, and typical adults' performance on behavioural tasks that are impaired in ASD is correlated with the extent to which they display autistic traits (as measured by Autism Spectrum Quotient, AQ). Individuals with ASD also show structural and functional differences in brain regions involved in social perception. Here we show that variation in AQ in typically developing individuals is associated with altered brain activity in the neural circuit for social attention perception while viewing others' eye gaze. In an fMRI experiment, participants viewed faces looking at variable or constant directions. In control conditions, only the eye region was presented or the heads were shown with eyes closed but oriented at variable or constant directions. The response to faces with variable vs. constant eye gaze direction was associated with AQ scores in a number of regions (posterior superior temporal sulcus, intraparietal sulcus, temporoparietal junction, amygdala, and MT/V5) of the brain network for social attention perception. No such effect was observed for heads with eyes closed or when only the eyes were presented. The results demonstrate a relationship between neurophysiology and autism spectrum traits in the typical (non-ASD) population and suggest that changes in the functioning of the neural circuit for social attention perception is associated with an extended autism spectrum in the typical population.

Task-invariant brain responses to the social value of faces.

In two fMRI experiments (n = 44) using tasks with different demands-approach-avoidance versus one-back recognition decisions-we measured the responses to the social value of faces. The face stimuli were produced by a parametric model of face evaluation that reduces multiple social evaluations to two orthogonal dimensions of valence and power [Oosterhof, N. N., & Todorov, A. The functional basis of face evaluation. Proceedings of the National Academy of Sciences, U.S.A., 105, 11087-11092, 2008]. Independent of the task, the response within regions of the occipital, fusiform, and lateral prefrontal cortices was sensitive to the valence dimension, with larger responses to low-valence faces. Additionally, there were extensive quadratic responses in the fusiform gyri and dorsal amygdala, with larger responses to faces at the extremes of the face valence continuum than faces in the middle. In all these regions, participants' avoidance decisions correlated with brain responses, with faces more likely to be avoided evoking stronger responses. The findings suggest that both explicit and implicit face evaluation engage multiple brain regions involved in attention, affect, and decision making.

Amygdala and dorsomedial prefrontal cortex responses to appearance-based and behavior-based person impressions.

We explored the neural correlates of learning about people when the affective value of both facial appearance and behavioral information is manipulated. Participants were presented with faces that were either rated as high or low on trustworthiness. Subsequently, we paired these faces with positive, negative, or no behavioral information. Prior to forming face-behavior associations, a cluster in the right amygdala responded more strongly to untrustworthy than to trustworthy faces. During learning, a cluster in the dorsomedial prefrontal cortex (dmPFC) responded more strongly to faces paired with behaviors than faces not paired with behaviors. We also observed that the activity in the dmPFC was correlated with behavioral learning performance assessed after scanning. Interestingly, individual differences in the initial amygdala response to face trustworthiness prior to learning modulated the relationship between dmPFC activity and learning. This finding suggests that the activity of the amygdala can affect the interaction between dmPFC activity and learning.

The relationship of γ oscillations and face-specific ERPs recorded subdurally from occipitotemporal cortex.

The perception of faces evokes characteristic electrophysiological responses at discrete loci in human fusiform gyrus and adjacent ventral occipitotemporal cortical sites. Prominent among these responses are a surface-negative potential at ∼200-ms postonset (face-N200) and face-induced spectral perturbations in the gamma band (face-γERSP). The degree to which these responses represent activity in the same cortical loci and the degree to which they are influenced by the same perceptual and task variables are unknown. We evaluated this anatomical colocalization and functional correlation in 2 experiments in which the electrocorticogram was recorded from subdural electrodes in 51 participants. Experiment 1 investigated the category specificity of the γERSP and its colocalization with the face-N200. Experiment 2 examined differences in face-N200 and face-γERSP to face stimuli that varied in featural complexity. We found that γERSP is a category-specific phenomenon with separate, though overlapping, category sensitivities as the N200. Further, the presence of face-γERSP at an electrode site significantly predicted the presence and amplitude of face-N200 at that site. However, the converse was not true in that face-N200 was evoked by impoverished face stimuli that did not induce face-γERSP. These results demonstrate that these electrophysiological responses reflect separate components of the brain's face processing system.

Autism spectrum traits in the typical population predict structure and function in the posterior superior temporal sulcus.

Autism spectrum disorders (ASDs) are typically characterized by impaired social interaction and communication, narrow interests, and repetitive behaviors. The heterogeneity in the severity of these characteristics across individuals with ASD has led some researchers to suggest that these disorders form a continuum which extends into the general, or "typical," population, and there is growing evidence that the extent to which typical adults display autistic traits, as measured using the autism-spectrum quotient (AQ), predicts performance on behavioral tasks that are impaired in ASD. Here, we show that variation in autism spectrum traits is related to cortical structure and function within the typical population. Voxel-based morphometry showed that increased AQ scores were associated with decreased white matter volume in the posterior superior temporal sulcus (pSTS), a region important in processing socially relevant stimuli and associated with structural and functional impairments in ASD. In addition, AQ was correlated with the extent of cortical deactivation of an adjacent area of pSTS during a Stroop task relative to rest, reflecting variation in resting state function. The results provide evidence that autism spectrum characteristics are reflected in neural structure and function across the typical (non-ASD) population.

Common neural mechanisms for the evaluation of facial trustworthiness and emotional expressions as revealed by behavioral adaptation.

People rapidly and automatically evaluate faces along many social dimensions. Here, we focus on judgments of trustworthiness, which approximate basic valence evaluation of faces, and test whether these judgments are an overgeneralization of the perception of emotional expressions. We used a behavioral adaptation paradigm to investigate whether the previously noted perceptual similarities between trustworthiness and emotional expressions of anger and happiness extend to their underlying neural representations. We found that adapting to angry or happy facial expressions causes trustworthiness evaluations of subsequently rated neutral faces to increase or decrease, respectively. Further, we found no such modulation of trustworthiness evaluations after participants were adapted to fearful expressions, suggesting that this effect is specific to angry and happy expressions. We conclude that, in line with the overgeneralization hypothesis, a common neural system is engaged during the evaluation of facial trustworthiness and expressions of anger and happiness.

Selective attention modulates face-specific induced gamma oscillations recorded from ventral occipitotemporal cortex.

EEG studies from subdural electrodes have demonstrated a face-specific event-related potential (face-N200) recorded from human ventral occipitotemporal cortex. The insensitivity of face-N200 to task manipulations has supported the proposal that face-N200 reflects an initial obligatory response to faces. This result stands in striking contrast to results of neuroimaging studies that have demonstrated strong task sensitivity of the fusiform hemodynamic response evoked by faces, and thus has created a paradox in the face perception literature. We recorded field potentials directly from the cortical surface of 16 patients while they selectively attended to faces or houses. Here we report that face-specific gamma activity recorded at face-N200 sites is strongly modulated by selective attention, while face-N200 is not. Our results reconcile prior electrophysiological and hemodynamic studies of face perception, and suggest that attentional modulation of the face response follows an initial phase that is largely insensitive to attention.

Distributed representations of dynamic facial expressions in the superior temporal sulcus.

Previous research on the superior temporal sulcus (STS) has shown that it responds more to facial expressions than to neutral faces. Here, we extend our understanding of the STS in two ways. First, using targeted high-resolution fMRI measurements of the lateral cortex and multivoxel pattern analysis, we show that the response to seven categories of dynamic facial expressions can be decoded in both the posterior STS (pSTS) and anterior STS (aSTS). We were also able to decode patterns corresponding to these expressions in the frontal operculum (FO), a structure that has also been shown to respond to facial expressions. Second, we measured the similarity structure of these representations and found that the similarity structure in the pSTS significantly correlated with the perceptual similarity structure of the expressions. This was the case regardless of whether we used pattern classification or more traditional correlation techniques to extract the neural similarity structure. These results suggest that distributed representations in the pSTS could underlie the perception of facial expressions.

Differential activation of frontoparietal attention networks by social and symbolic spatial cues.

Perception of both gaze-direction and symbolic directional cues (e.g. arrows) orient an observer's attention toward the indicated location. It is unclear, however, whether these similar behavioral effects are examples of the same attentional phenomenon and, therefore, subserved by the same neural substrate. It has been proposed that gaze, given its evolutionary significance, constitutes a 'special' category of spatial cue. As such, it is predicted that the neural systems supporting spatial reorienting will be different for gaze than for non-biological symbols. We tested this prediction using functional magnetic resonance imaging to measure the brain's response during target localization in which laterally presented targets were preceded by uninformative gaze or arrow cues. Reaction times were faster during valid than invalid trials for both arrow and gaze cues. However, differential patterns of activity were evoked in the brain. Trials including invalid rather than valid arrow cues resulted in a stronger hemodynamic response in the ventral attention network. No such difference was seen during trials including valid and invalid gaze cues. This differential engagement of the ventral reorienting network is consistent with the notion that the facilitation of target detection by gaze cues and arrow cues is subserved by different neural substrates.

Connectivity analysis reveals a cortical network for eye gaze perception.

Haxby et al. (Haxby JV, Hoffman EA, Gobbini MI. 2000. The distributed human neural system for face perception. Trends Cogn Sci. 4:223-233.) proposed that eye gaze processing results from an interaction between a "core" face-specific system involved in visual analysis and an "extended" system involved in spatial attention, more generally. However, the full gaze perception network has remained poorly specified. In the context of a functional magnetic resonance imaging study, we used psychophysiological interactions (PPIs) to identify brain regions that showed differential connectivity (correlation) with core face perception structures (posterior superior temporal sulcus [pSTS] and fusiform gyrus [FG]) when viewing gaze shifts relative to control eye movements (opening/closing the eyes). The PPIs identified altered connectivity between the pSTS and MT/V5, intraparietal sulcus, frontal eye fields, superior temporal gyrus (STG), supramarginal gyrus, and middle frontal gyrus (MFG). The FG showed altered connectivity with the same areas of the STG and MFG, demonstrating the contribution of both dorsal and ventral core face areas to gaze perception. We propose that this network provides an interactive system that alerts us to seen changes in other agents' gaze direction, makes us aware of their altered focus of spatial attention, and prepares a corresponding shift in our own attention.

Implicit working memory.

Working Memory (WM) plays a crucial role in many high-level cognitive processes (e.g., reasoning, decision making, goal pursuit and cognitive control). The prevalent view holds that active components of WM are predominantly intentional and conscious. This conception is oftentimes expressed explicitly, but it is best reflected in the nature of major WM tasks: All of them are blatantly explicit. We developed two new WM paradigms that allow for an examination of the role of conscious awareness in WM. Results from five studies show that WM can operate unintentionally and outside of conscious awareness, thus suggesting that the current view should be expanded to include implicit WM.

The role of the amygdala in implicit evaluation of emotionally neutral faces.

The amygdala is involved in the evaluation of novel stimuli, including faces. We examined whether the amygdala is engaged during the evaluation of emotionally neutral faces along trait-specific dimensions such as trustworthiness and attractiveness or along a general valence dimension. Using behavioral data from evaluation of faces on 14 trait dimensions and fMRI data from an implicit evaluation paradigm, we show that the extent to which the amygdala responds to variations of faces on specific dimensions is a function of the valence content of these dimensions. Variations on dimensions with clear valence connotations (e.g. trustworthiness) engaged the amygdala more strongly than variations on dimensions with less clear valence connotations (e.g. dominance). In addition to the amygdala, several other regions--right superior occipital gyrus, right middle temporal/occipital gyrus and bilateral fusiform gyri--were involved in valence evaluation of faces. However, the relation between these regions and face valence was accounted for by the amygdala's response to faces. The findings suggest that the amygdala (i) automatically evaluates novel faces along a general valence dimension; and (ii) modulates a face responsive network of regions in occipital and temporal cortices.