Your place or mine? The neural dynamics of personally familiar scene recognition suggests category independent familiarity encoding.

Recognizing a stimulus as familiar is an important capacity in our everyday life. Recent investigation of visual processes has led to important insights into the nature of the neural representations of familiarity for human faces. Still, little is known about how familiarity affects the neural dynamics of non-face stimulus processing. Here we report the results of an EEG study, examining the representational dynamics of personally familiar scenes. Participants viewed highly variable images of their own apartments and unfamiliar ones, as well as personally familiar and unfamiliar faces. Multivariate pattern analyses were used to examine the time course of differential processing of familiar and unfamiliar stimuli. Time-resolved classification revealed that familiarity is decodable from the EEG data similarly for scenes and faces. The temporal dynamics showed delayed onsets and peaks for scenes as compared to faces. Familiarity information, starting at 200 ms, generalized across stimulus categories and led to a robust familiarity effect. In addition, familiarity enhanced category representations in early (250-300 ms) and later (>400 ms) processing stages. Our results extend previous face familiarity results to another stimulus category and suggest that familiarity as a construct can be understood as a general, stimulus-independent processing step during recognition.

The neural dynamics of familiarity-dependent face identity representation.

Recognizing a face as belonging to a given identity is essential in our everyday life. Clearly, the correct identification of a face is only possible for familiar people, but 'familiarity' covers a wide range-from people we see every day to those we barely know. Although several studies have shown that the processing of familiar and unfamiliar faces is substantially different, little is known about how the degree of familiarity affects the neural dynamics of face identity processing. Here, we report the results of a multivariate EEG analysis, examining the representational dynamics of face identity across several familiarity levels. Participants viewed highly variable face images of 20 identities, including the participants' own face, personally familiar (PF), celebrity and unfamiliar faces. Linear discriminant classifiers were trained and tested on EEG patterns to discriminate pairs of identities of the same familiarity level. Time-resolved classification revealed that the neural representations of identity discrimination emerge around 100 ms post-stimulus onset, relatively independently of familiarity level. In contrast, identity decoding between 200 and 400 ms is determined to a large extent by familiarity: it can be recovered with higher accuracy and for a longer duration in the case of more familiar faces. In addition, we found no increased discriminability for faces of PF persons compared to those of highly familiar celebrities. One's own face benefits from processing advantages only in a relatively late time-window. Our findings provide new insights into how the brain represents face identity with various degrees of familiarity and show that the degree of familiarity modulates the available identity-specific information at a relatively early time window.

No semantic information is necessary to evoke general neural signatures of face familiarity: evidence from cross-experiment classification.

Recent theories on the neural correlates of face identification stressed the importance of the available identity-specific semantic and affective information. However, whether such information is essential for the emergence of neural signal of familiarity has not yet been studied in detail. Here, we explored the shared representation of face familiarity between perceptually and personally familiarized identities. We applied a cross-experiment multivariate pattern classification analysis (MVPA), to test if EEG patterns for passive viewing of personally familiar and unfamiliar faces are useful in decoding familiarity in a matching task where familiarity was attained thorough a short perceptual task. Importantly, no additional semantic, contextual, or affective information was provided for the familiarized identities during perceptual familiarization. Although the two datasets originate from different sets of participants who were engaged in two different tasks, familiarity was still decodable in the sorted, same-identity matching trials. This finding indicates that the visual processing of the faces of personally familiar and purely perceptually familiarized identities involve similar mechanisms, leading to cross-classifiable neural patterns.

Characterizing the shared signals of face familiarity: Long-term acquaintance, voluntary control, and concealed knowledge.

In a recent study using cross-experiment multivariate classification of EEG patterns, we found evidence for a shared familiarity signal for faces, patterns of neural activity that successfully separate trials for familiar and unfamiliar faces across participants and modes of familiarization. Here, our aim was to expand upon this research to further characterize the spatio-temporal properties of this signal. By utilizing the information content present for incidental exposure to personally familiar and unfamiliar faces, we tested how the information content in the neural signal unfolds over time under different task demands - giving truthful or deceptive responses to photographs of genuinely familiar and unfamiliar individuals. For this goal, we re-analyzed data from two previously published experiments using within-experiment leave-one-subject-out and cross-experiment classification of face familiarity. We observed that the general face familiarity signal, consistent with its previously described spatio-temporal properties, is present for long-term personally familiar faces under passive viewing, as well as for acknowledged and concealed familiarity responses. Also, central-posterior regions contain information related to deception. We propose that signals in the 200-400 ms window are modulated by top-down task-related anticipation, while the patterns in the 400-600 ms window are influenced by conscious effort to deceive. To our knowledge, this is the first report describing the representational dynamics of concealed knowledge for faces, using time-resolved multivariate classification.

A neural measure of the degree of face familiarity.

Recognizing a face as familiar is essential in our everyday life. However, 'familiarity' covers a wide range - from people we see every day to those we barely know. Although face recognition is studied extensively, little is known about how the degree of familiarity affects neural face processing, despite the critical social importance of this dimension. Here we report the results of a multivariate cross-classification EEG experiment, where we study the temporal representational dynamics of the degree of familiarity. Participants viewed highly variable face images of 20 identities. Importantly, we measured face familiarity using subjective familiarity ratings in addition to testing explicit knowledge and reaction times in a face matching task. A machine learning algorithm, trained to discriminate familiar and unfamiliar faces from a separate study, was used to predict the degree of face familiarity from the pattern of the EEG data. We found that the neural representations of the degree of familiarity emerge between 400 - 600 msec post-stimulus onset for famous persons. The correlation between decoding performance and behavioral familiarity was more reliable, occurred earlier and lasted longer when personally familiar and viewers' own faces were included in the analysis. Our findings provide new insights into how the brain represents faces with various degrees of familiarity and show that the degree of familiarity can be decoded reliably from the EEG at a relatively late time window. These results support the idea that representations of familiar faces form part of a general neural signature of the familiarity component of recognition memory processes.

Person identity-specific adaptation effects in the ventral occipito-temporal cortex.

Identifying the faces of familiar persons requires the ability to assign several different images of a face to a common identity. Previous research showed that the occipito-temporal cortex, including the fusiform and the occipital face areas, is sensitive to personal identity. Still, the viewpoint, facial expression and image-independence of this information are currently under heavy debate. Here we adapted a rapid serial visual stimulation paradigm Johnston et al. (2016, https://doi.org/10.1016/j.cortex.2016.10.002) and presented highly variable ambient-face images of famous persons to measure functional magnetic resonance imaging (fMRI) adaptation. fMRI adaptation is considered as the neuroimaging manifestation of repetition suppression, a neural phenomenon currently explained as a correlate of reduced predictive error responses for expected stimuli. We revisited the question of image-invariant identity-specific encoding mechanisms of the occipito-temporal cortex, using fMRI adaptation with a particular interest in predictive mechanisms. Participants were presented with trials containing eight different images of a famous person, images of eight different famous persons or seven different images of a particular famous person followed by an identity change to violate potential expectation effects about person identity. We found an image-independent adaptation effect of identity for famous faces in the fusiform face area. However, in contrast to previous electrophysiological studies, using similar paradigms, no release of the adaptation effect was observed when identity-specific expectations were violated. Our results support recent multivariate pattern analysis studies, showing image-independent identity encoding in the core face-processing areas of the occipito-temporal cortex. These results are discussed in the frame of recent identity-processing models and predictive mechanisms.

Evidence for a General Neural Signature of Face Familiarity.

We explored the neural signatures of face familiarity using cross-participant and cross-experiment decoding of event-related potentials, evoked by unknown and experimentally familiarized faces from a set of experiments with different participants, stimuli, and familiarization-types. Human participants of both sexes were either familiarized perceptually, via media exposure, or by personal interaction. We observed significant cross-experiment familiarity decoding involving all three experiments, predominantly over posterior and central regions of the right hemisphere in the 270-630 ms time window. This shared face familiarity effect was most prominent across the Media and the Personal, as well as between the Perceptual and Personal experiments. Cross-experiment decodability makes this signal a strong candidate for a general neural indicator of face familiarity, independent of familiarization methods, participants, and stimuli. Furthermore, the sustained pattern of temporal generalization suggests that it reflects a single automatic processing cascade that is maintained over time.

Cautious or causal? Key implicit sequence learning paradigms should not be overlooked when assessing the role of DLPFC (Commentary on Prutean et al.).

The role of the dorsolateral prefrontal cortex (DLPFC) in implicit sequence/statistical learning has received considerable attention in recent cognitive neuroscience research. Studies have used non-invasive brain stimulation methods to test whether the DLPFC plays a role in the incidental acquisition and expression of implicit sequence learning. In a recent study, Prutean et al. has concluded that stimulating the left or the right DLPFC might not affect the expression of implicit sequence learning measured by the Serial Reaction Time (SRT) task. The authors speculated that the previous results revealing improved implicit sequence learning following DLPFC stimulation might have been found because explicit awareness accumulated with the use of Alternating Serial Reaction Time (ASRT) task. Our response presents solid evidence that the ASRT task measures implicit sequence learning that remains unconscious both at the judgment and structural level. Therefore, contrary to the conclusion of Prutean et al., we argue that the DLPFC could have a crucial effect on implicit sequence learning that may be task-dependent. We suggest that future research should focus on the specific cognitive processes that may be differentially involved in the SRT versus ASRT tasks, and test what the role of the DLPFC is in those specific cognitive processes.

Inhibition of the occipital face area modulates the electrophysiological signals of face familiarity: A combined cTBS-EEG study.

The occipital face area (OFA) is hierarchically one of the first stages of the face processing network. It has originally been thought to be involved in early, structural processing steps, but currently more and more studies challenge this view and propose that it also takes part in higher level face processing, such as identification and recognition. Here we tested whether the OFA is involved in the initial steps of recognition memory and plays a causal role in the differential processing of familiar and unfamiliar faces. We used an offline, inhibitory continuous theta-burst stimulation (cTBS) protocol over the right OFA and the vertex as control site. Electroencephalographic (EEG) recording of event-related potentials (ERPs), elicited by visually presented familiar (famous) and unfamiliar faces was performed before and after stimulation. We observed a difference in ERPs for famous and unfamiliar faces in a time-window corresponding to the N250 component. Importantly, this difference was significantly increased by cTBS of the right OFA, suggesting its causal role in the differential processing of familiar and unfamiliar faces. The enhancement occurred focally, at electrodes close to the right hemispheric cTBS site, as well as over similar occipito-temporal sites of the contralateral hemisphere. To the best of our knowledge, this is the first study showing the causal role of the rOFA in the differential processing of familiar and unfamiliar faces, using combined cTBS and EEG recording methods. These results are discussed with respect to the nature of familiar face representations, supported by an extensive, bilateral network.

Getting to Know You: Emerging Neural Representations during Face Familiarization.

The successful recognition of familiar persons is critical for social interactions. Despite extensive research on the neural representations of familiar faces, we know little about how such representations unfold as someone becomes familiar. In three EEG experiments on human participants of both sexes, we elucidated how representations of face familiarity and identity emerge from different qualities of familiarization: brief perceptual exposure (Experiment 1), extensive media familiarization (Experiment 2), and real-life personal familiarization (Experiment 3). Time-resolved representational similarity analysis revealed that familiarization quality has a profound impact on representations of face familiarity: they were strongly visible after personal familiarization, weaker after media familiarization, and absent after perceptual familiarization. Across all experiments, we found no enhancement of face identity representation, suggesting that familiarity and identity representations emerge independently during face familiarization. Our results emphasize the importance of extensive, real-life familiarization for the emergence of robust face familiarity representations, constraining models of face perception and recognition memory.SIGNIFICANCE STATEMENT Despite extensive research on the neural representations of familiar faces, we know little about how such representations unfold as someone becomes familiar. To elucidate how face representations change as we get familiar with someone, we conducted three EEG experiments where we used brief perceptual exposure, extensive media familiarization, or real-life personal familiarization. Using multivariate representational similarity analysis, we demonstrate that the method of familiarization has a profound impact on face representations, and emphasize the importance of real-life familiarization. Additionally, familiarization shapes representations of face familiarity and identity differently: as we get to know someone, familiarity signals seem to appear before the formation of identity representations.

The occipital face area is causally involved in identity-related visual-semantic associations.

Faces are processed in a network of areas within regions of the ventral visual stream. However, familiar faces typically are characterized by additional associated information, such as episodic memories or semantic biographical information as well. The acquisition of such non-sensory, identity-specific knowledge plays a crucial role in our ability to recognize and identify someone we know. The occipital face area (OFA), an early part of the core face-processing network, is recently found to be involved in the formation of identity-specific memory traces but it is currently unclear if this role is limited to unimodal visual information. The current experiments used transcranial magnetic stimulation (TMS) to test whether the OFA is involved in the association of a face with identity-specific semantic information, such as the name or job title of a person. We applied an identity-learning task where unfamiliar faces were presented together with a name and a job title in the first encoding phase. Simultaneously, TMS pulses were applied either to the left or right OFA or to Cz, as a control. In the subsequent retrieval phase, the previously seen faces were presented either with two names or with two job titles and the task of the participants was to select the semantic information previously learned. We found that the stimulation of the right or left OFA reduced subsequent retrieval performance for the face-associated job titles. This suggests a causal role of the OFA in the association of faces and related semantic information. Furthermore, in contrast to prior findings, we did not observe hemispherical differences of the TMS intervention, suggesting a similar role of the left and right OFAs in the formation of the visual-semantic associations. Our results suggest the necessity to reconsider the hierarchical face-perception models and support the distributed and recurrent models.

The Neural Dynamics of Familiar Face Recognition.

In real-life situations, the appearance of a person's face can vary substantially across different encounters, making face recognition a challenging task for the visual system. Recent fMRI decoding studies have suggested that face recognition is supported by identity representations located in regions of the occipitotemporal cortex. Here, we used EEG to elucidate the temporal emergence of these representations. Human participants viewed a set of highly variable face images of 4 highly familiar celebrities (2 males and 2 females), while performing an orthogonal task. Univariate analyses of event-related EEG responses revealed a pronounced differentiation between male and female faces, but not between identities of the same sex. Using multivariate representational similarity analysis, we observed a gradual emergence of face identity representations, with an increasing degree of invariance. Face identity information emerged rapidly, starting shortly after 100 ms from stimulus onset, but was modulated by sex differences and image similarities. From 400 ms after onset and predominantly in the right hemisphere, identity representations showed 2 invariance properties: 1) they equally discriminated identities of opposite sexes and of the same sex, and 2) they were tolerant to image-based variations. These invariant representations may be a crucial prerequisite for successful face recognition in everyday situations, where the appearance of a familiar person can vary drastically.

TMS of the occipital face area modulates cross-domain identity priming.

Accumulating evidence suggests that besides its function in early facial feature processing, the role of the right occipital face area (rOFA) extends to higher level, image-independent processing. Recent studies hint at the possibility that the activity of this region can be modulated by semantic information as well. To test whether the OFA is sensitive to semantic information in a functionally relevant way, we implemented a cross-domain, name-face priming paradigm combined with state-dependent transcranial magnetic stimulation, whereby stimulation preferentially facilitates the processing of attributes encoded by less active neural populations. Our volunteers performed a familiarity decision task for target face images preceded by primes that were either the name of the same identity (congruent), a name of a different person (incongruent), or the character string 'XXXXX' (no prime). Stimulating the rOFA at target stimulus onset, we observed the disappearance of the behavioral disadvantage of incongruent primes, compared to the vertex control condition. Performance in the congruent and no prime conditions remained intact. This result suggests the existence of neural populations in the rOFA that take part in the semantic processing of identity, probably in interplay with other nodes in the extended face network.

Neuroimaging results suggest the role of prediction in cross-domain priming.

The repetition of a stimulus leads to shorter reaction times as well as to the reduction of neural activity. Previous encounters with closely related stimuli (primes) also lead to faster and often to more accurate processing of subsequent stimuli (targets). For instance, if the prime is a name, and the target is a face, the recognition of a persons' face is facilitated by prior presentation of his/her name. A possible explanation for this phenomenon is that the prime allows predicting the occurrence of the target. To the best of our knowledge, so far, no study tested the neural correlates of such cross-domain priming with fMRI. To fill this gap, here we used names of famous persons as primes, and congruent or incongruent faces as targets. We found that congruent primes not only reduced RT, but also lowered the BOLD signal in bilateral fusiform (FFA) and occipital (OFA) face areas. This suggests that semantic information affects not only behavioral performance, but also neural responses in relatively early processing stages of the occipito-temporal cortex. We interpret our results in the framework of predictive coding theories.

The occipital face area is causally involved in the formation of identity-specific face representations.

Transcranial magnetic stimulation (TMS) and neuroimaging studies suggest a role of the right occipital face area (rOFA) in early facial feature processing. However, the degree to which rOFA is necessary for the encoding of facial identity has been less clear. Here we used a state-dependent TMS paradigm, where stimulation preferentially facilitates attributes encoded by less active neural populations, to investigate the role of the rOFA in face perception and specifically in image-independent identity processing. Participants performed a familiarity decision task for famous and unknown target faces, preceded by brief (200 ms) or longer (3500 ms) exposures to primes which were either an image of a different identity (DiffID), another image of the same identity (SameID), the same image (SameIMG), or a Fourier-randomized noise pattern (NOISE) while either the rOFA or the vertex as control was stimulated by single-pulse TMS. Strikingly, TMS to the rOFA eliminated the advantage of SameID over DiffID condition, thereby disrupting identity-specific priming, while leaving image-specific priming (better performance for SameIMG vs. SameID) unaffected. Our results suggest that the role of rOFA is not limited to low-level feature processing, and emphasize its role in image-independent facial identity processing and the formation of identity-specific memory traces.

Causal evidence of the involvement of the right occipital face area in face-identity acquisition.

There is growing evidence that the occipital face area (OFA), originally thought to be involved in the construction of a low-level representation of the physical features of a face, is also taking part in higher-level face processing. To test whether the OFA is causally involved in the learning of novel face identities, we have used transcranial magnetic stimulation (TMS) together with a sequential sorting - face matching paradigm (Andrews et al. 2015). First, participants sorted images of two unknown persons during the initial learning phase while either their right OFA or the Vertex was stimulated using TMS. In the subsequent test phase, we measured the participants' face matching performance for novel images of the previously trained identities and for two novel identities. We found that face-matching performance accuracy was higher for the trained as compared to the novel identities in the vertex control group, suggesting that the sorting task led to incidental learning of the identities involved. However, no such difference was observed between trained and novel identities in the rOFA stimulation group. Our results support the hypothesis that the role of the rOFA is not limited to the processing of low-level physical features, but it has a significant causal role in face identity encoding and in the formation of identity-specific memory-traces.

Alternating Current Stimulation for Vision Restoration after Optic Nerve Damage: A Randomized Clinical Trial.

BACKGROUND: Vision loss after optic neuropathy is considered irreversible. Here, repetitive transorbital alternating current stimulation (rtACS) was applied in partially blind patients with the goal of activating their residual vision. METHODS: We conducted a multicenter, prospective, randomized, double-blind, sham-controlled trial in an ambulatory setting with daily application of rtACS (n = 45) or sham-stimulation (n = 37) for 50 min for a duration of 10 week days. A volunteer sample of patients with optic nerve damage (mean age 59.1 yrs) was recruited. The primary outcome measure for efficacy was super-threshold visual fields with 48 hrs after the last treatment day and at 2-months follow-up. Secondary outcome measures were near-threshold visual fields, reaction time, visual acuity, and resting-state EEGs to assess changes in brain physiology. RESULTS: The rtACS-treated group had a mean improvement in visual field of 24.0% which was significantly greater than after sham-stimulation (2.5%). This improvement persisted for at least 2 months in terms of both within- and between-group comparisons. Secondary analyses revealed improvements of near-threshold visual fields in the central 5° and increased thresholds in static perimetry after rtACS and improved reaction times, but visual acuity did not change compared to shams. Visual field improvement induced by rtACS was associated with EEG power-spectra and coherence alterations in visual cortical networks which are interpreted as signs of neuromodulation. Current flow simulation indicates current in the frontal cortex, eye, and optic nerve and in the subcortical but not in the cortical regions. CONCLUSION: rtACS treatment is a safe and effective means to partially restore vision after optic nerve damage probably by modulating brain plasticity. This class 1 evidence suggests that visual fields can be improved in a clinically meaningful way. TRIAL REGISTRATION: ClinicalTrials.gov NCT01280877.

Causal evidence of the involvement of the number form area in the visual detection of numbers and letters.

Recent research suggests the existence of a visual area selectively processing numbers in the human inferior temporal cortex (number form area (NFA); Abboud et al., 2015; Grotheer et al., 2016; Shum et al., 2013). The NFA is thought to be involved in the preferential encoding of numbers over false characters, letters and non-number words (Grotheer et al., 2016; Shum et al., 2013), independently of the sensory modality (Abboud et al., 2015). However, it is not yet clear if this area is mandatory for normal number processing. The present study exploited the fact that high-resolution fMRI can be applied to identify the NFA individually (Grotheer et al., 2016) and tested if transcranial magnetic stimulation (TMS) of this area interferes with stimulus processing in a selective manner. Double-pulse TMS targeted at the right NFA significantly impaired the detection of briefly presented and masked Arabic numbers in comparison to vertex stimulation. This suggests the NFA to be necessary for fluent number processing. Surprisingly, TMS of the NFA also impaired the detection of Roman letters. On the other hand, stimulation of the lateral occipital complex (LO) had neither an effect on the detection of numbers nor on letters. Our results show, for the first time, that the NFA is causally involved in the early visual processing of numbers as well as of letters.

Monitoring transcranial direct current stimulation induced changes in cortical excitability during the serial reaction time task.

The measurement of the motor evoked potential (MEP) amplitudes using single pulse transcranial magnetic stimulation (TMS) is a common method to observe changes in motor cortical excitability. The level of cortical excitability has been shown to change during motor learning. Conversely, motor learning can be improved by using anodal transcranial direct current stimulation (tDCS). In the present study, we aimed to monitor cortical excitability changes during an implicit motor learning paradigm, a version of the serial reaction time task (SRTT). Responses from the first dorsal interosseous (FDI) and forearm flexor (FLEX) muscles were recorded before, during and after the performance of the SRTT. Online measurements were combined with anodal, cathodal or sham tDCS for the duration of the SRTT. Negative correlations between the amplitude of online FDI MEPs and SRTT reaction times (RTs) were observed across the learning blocks in the cathodal condition (higher average MEP amplitudes associated with lower RTs) but no significant differences in the anodal and sham conditions. tDCS did not have an impact on SRTT performance, as would be predicted based on previous studies. The offline before-after SRTT MEP amplitudes showed an increase after anodal and a tendency to decrease after cathodal stimulation, but these changes were not significant. The combination of different interventions during tDCS might result in reduced efficacy of the stimulation that in future studies need further attention.

Bi-frontal transcranial alternating current stimulation in the ripple range reduced overnight forgetting.

High frequency oscillations in the hippocampal structures recorded during sleep have been proved to be essential for long-term episodic memory consolidation in both animals and in humans. The aim of this study was to test if transcranial Alternating Current Stimulation (tACS) of the dorsolateral prefrontal cortex (DLPFC) in the hippocampal ripple range, applied bi-frontally during encoding, could modulate declarative memory performance, measured immediately after encoding, and after a night's sleep. An associative word-pair learning test was used. During an evening encoding phase, participants received 1 mA 140 Hz tACS or sham stimulation over both DLPFCs for 10 min while being presented twice with a list of word-pairs. Cued recall performance was investigated 10 min after training and the morning following the training session. Forgetting from evening to morning was observed in the sham condition, but not in the 140 Hz stimulation condition. 140 Hz tACS during encoding may have an effect on the consolidation of declarative material.