Identifying conceptual neural responses to symbolic numerals.

The goal of measuring conceptual processing in numerical cognition is distanced by the possibility that neural responses to symbolic numerals are influenced by physical stimulus confounds. Here, we targeted conceptual responses to parity (even versus odd), using electroencephalogram (EEG) frequency-tagging with a symmetry/asymmetry design. Arabic numerals (2-9) were presented at 7.5 Hz in 50 s sequences; odd and even numbers were alternated to target differential, 'asymmetry' responses to parity at 3.75 Hz (7.5 Hz/2). Parity responses were probed with four different stimulus sets, increasing in intra-numeral stimulus variability, and with two control conditions composed of non-conceptual numeral alternations. Significant asymmetry responses were found over the occipitotemporal cortex to all conditions, even for the arbitrary controls. The large physical-differences control condition elicited the largest response in the stimulus set with the lowest variability (one font). Only in the stimulus set with the highest variability (20 drawn, coloured exemplars/numeral) did the response to parity surpass both control conditions. These findings show that physical differences across small sets of Arabic numerals can strongly influence, and even account for, automatic brain responses. However, carefully designed control conditions and highly variable stimulus sets may be used towards identifying truly conceptual neural responses.

Categorical consistency facilitates implicit learning of color-number associations.

In making sense of the environment, we implicitly learn to associate stimulus attributes that frequently occur together. Is such learning favored for categories over individual items? Here, we introduce a novel paradigm for directly comparing category- to item-level learning. In a category-level experiment, even numbers (2,4,6,8) had a high-probability of appearing in blue, and odd numbers (3,5,7,9) in yellow. Associative learning was measured by the relative performance on trials with low-probability (p = .09) to high-probability (p = .91) number colors. There was strong evidence for associative learning: low-probability performance was impaired (40ms RT increase and 8.3% accuracy decrease relative to high-probability). This was not the case in an item-level experiment with a different group of participants, in which high-probability colors were non-categorically assigned (blue: 2,3,6,7; yellow: 4,5,8,9; 9ms RT increase and 1.5% accuracy increase). The categorical advantage was upheld in an explicit color association report (83% accuracy vs. 43% at the item-level). These results support a conceptual view of perception and suggest empirical bases of categorical, not item-level, color labeling of learning materials.

Automatic, Early Color-Specific Neural Responses to Object Color Knowledge.

Some familiar objects are associated with specific colors, e.g., rubber ducks with yellow. Whether and at what stage neural responses occur to these color associations remain open questions. We recorded frequency-tagged electroencephalogram (EEG) responses to periodic presentations of yellow-associated objects, shown among sequences of non-periodic blue-, red-, and green-associated objects. Both color and grayscale versions of the objects elicited yellow-specific responses, indicating an automatic activation of color knowledge from object shape. Follow-up experiments replicated these effects with green-specific responses, and demonstrated modulated responses for incongruent color/object associations. Importantly, the onset of color-specific responses was as early to grayscale as actually colored stimuli (before 100 ms), the latter additionally eliciting a conventional later response (approximately 140-230 ms) to actual stimulus color. This suggests that the neural representation of familiar objects includes both diagnostic shape and color properties, such that shape can elicit associated color-specific responses before actual color-specific responses occur.

Contrast Adaptation in Face Perception Revealed Through EEG and Behavior.

Exposure to a face can produce biases in the perception of subsequent faces. Typically, these face aftereffects are studied by adapting to an individual face or category (e.g., faces of a given gender) and can result in renormalization of perceptions such that the adapting face appears more neutral. These shifts are analogous to chromatic adaptation, where a renormalization for the average adapting color occurs. However, in color vision, adaptation can also adjust to the variance or range of colors in the distribution. We examined whether this variance or contrast adaptation also occurs for faces, using an objective EEG measure to assess response changes following adaptation. An average female face was contracted or expanded along the horizontal or vertical axis to form four images. Observers viewed a 20 s sequence of the four images presented in a fixed order at a rate of 6 Hz, while responses to the faces were recorded with EEG. A 6 Hz signal was observed over right occipito-temporal channels, indicating symmetric responses to the four images. This test sequence was repeated after 20 s adaptation to alternations between two of the faces (e.g., horizontal contracted and expanded). This adaptation resulted in an additional signal at 3 Hz, consistent with asymmetric responses to adapted and non-adapted test faces. Adapting pairs have the same mean (undistorted) as the test sequence and thus should not bias responses driven only by the mean. Instead, the results are consistent with selective adaptation to the distortion axis. A 3 Hz signal was also observed after adapting to face pairs selected to induce a mean bias (e.g., expanded vertical and expanded horizontal), and this signal was not significantly different from that observed following adaption to a single image that did not form part of the test sequence (e.g., a single image expanded both vertically and horizontally). In a further experiment, we found that this variance adaptation can also be observed behaviorally. Our results suggest that adaptation calibrates face perception not only for the average characteristics of the faces we experience but also for the gamut of faces to which we are exposed.

Varying Stimulus Duration Reveals Consistent Neural Activity and Behavior for Human Face Individuation.

Establishing consistent relationships between neural activity and behavior is a challenge in human cognitive neuroscience research. We addressed this issue using variable time constraints in an oddball frequency-sweep design for visual discrimination of complex images (face exemplars). Sixteen participants viewed sequences of ascending presentation durations, from 25 to 333 ms (40-3 Hz stimulation rate) while their electroencephalogram (EEG) was recorded. Throughout each sequence, the same unfamiliar face picture was repeated with variable size and luminance changes while different unfamiliar facial identities appeared every 1 s (1 Hz). A neural face individuation response, tagged at 1 Hz and its unique harmonics, emerged over the occipito-temporal cortex at 50 ms stimulus duration (25-100 ms across individuals), with an optimal response reached at 170 ms stimulus duration. In a subsequent experiment, identity changes appeared non-periodically within fixed-frequency sequences while the same participants performed an explicit face individuation task. The behavioral face individuation response also emerged at 50 ms presentation time, and behavioral accuracy correlated with individual participants' neural response amplitude in a weighted middle stimulus duration range (50-125 ms). Moreover, the latency of the neural response peaking between 180 and 200 ms correlated strongly with individuals' behavioral accuracy in this middle duration range, as measured independently. These observations point to the minimal (50 ms) and optimal (170 ms) stimulus durations for human face individuation and provide novel evidence that inter-individual differences in the magnitude and latency of early, high-level neural responses are predictive of behavioral differences in performance at this function.

Harmonic Amplitude Summation for Frequency-tagging Analysis.

In the approach of frequency tagging, stimuli that are presented periodically generate periodic responses of the brain. Following a transformation into the frequency domain, the brain's response is often evident at the frequency of stimulation, F, and its higher harmonics (2F, 3F, etc.). This approach is increasingly used in neuroscience, as it affords objective measures to characterize brain function. However, whether these specific harmonic frequency responses should be combined for analysis-and if so, how-remains an outstanding issue. In most studies, higher harmonic responses have not been described or were described only individually; in other studies, harmonics have been combined with various approaches, for example, averaging and root-mean-square summation. A rationale for these approaches in the context of frequency-based analysis principles and an understanding of how they relate to the brain's response amplitudes in the time domain have been missing. Here, with these elements addressed, the summation of (baseline-corrected) harmonic amplitude is recommended.

Does automatic human face categorization depend on head orientation?

Whether human categorization of visual stimuli as faces is optimal for full-front views, best revealing diagnostic features but lacking depth cues, remains largely unknown. To address this question, we presented 16 human observers with unsegmented natural images of different living and non-living objects at a fast rate (f = 12 Hz), with natural face images appearing at f/9 = 1.33 Hz. Faces posing all full-front or at ¾ side view angles appeared in separate sequences. Robust frequency-tagged 1.33 Hz (and harmonic) occipito-temporal electroencephalographic (EEG) responses reflecting face-selective neural activity did not differ in overall amplitude between full-front and ¾ side views. Despite this, alternating between full-front and ¾ side views within a sequence led to significant responses at specific harmonics of .67 Hz (f/18), objectively isolating view-dependent face-selective responses over occipito-temporal regions. Critically, a time-domain analysis showed that these view-dependent face-selective responses reflected only an earlier response to full-front than ¾ side views by 8-13 ms. Overall, these findings indicate that the face-selective neural representation is as robust for ¾ side faces as for full-front faces in the human brain, but full-front views provide a slightly earlier processing-time advantage as compared to rotated face views.

Color Vision: Decoding Color Space.

A new study has used magnetoencephalography to track cortical responses to color as they emerge in time. Similarities and differences within these neural responses parallel characteristics of the perceptual experience of color.

Understanding human individuation of unfamiliar faces with oddball fast periodic visual stimulation and electroencephalography.

To investigate face individuation (FI), a critical brain function in the human species, an oddball fast periodic visual stimulation (FPVS) approach was recently introduced (Liu-Shuang et al., Neuropsychologia, 2014, 52, 57). In this paradigm, an image of an unfamiliar "base" facial identity is repeated at a rapid rate F (e.g., 6 Hz) and different unfamiliar "oddball" facial identities are inserted every nth item, at a F/n rate (e.g., every 5th item, 1.2 Hz). This stimulation elicits FI responses at F/n and its harmonics (2F/n, 3F/n, etc.), reflecting neural discrimination between oddball versus base facial identities, which is quantified in the frequency domain of the electroencephalogram (EEG). This paradigm, used in 20 published studies, demonstrates substantial advantages for measuring FI in terms of validity, objectivity, reliability, and sensitivity. Human intracerebral recordings suggest that this FI response originates from neural populations in the lateral inferior occipital and fusiform gyri, with a right hemispheric dominance consistent with the localization of brain lesions specifically affecting facial identity recognition (prosopagnosia). Here, we summarize the contributions of the oddball FPVS framework toward understanding FI, including its (a)typical development, with early studies supporting the application of this technique to clinical testing (e.g., autism spectrum disorder). This review also includes an in-depth analysis of the paradigm's methodology, with guidelines for designing future studies. A large-scale group analysis compiling data across 130 observers provides insights into the oddball FPVS FI response properties. Overall, we recommend the oddball FPVS paradigm as an alternative approach to behavioral or traditional event-related potential EEG measures of face individuation.

Neural correlates of perceptual color inferences as revealed by #thedress.

Color constancy involves disambiguating the spectral characteristics of lights and surfaces, for example to distinguish red in white light from white in red light. Solving this problem appears especially challenging for bluish tints, which may be attributed more often to shading, and this bias may underlie the individual differences in whether people described the widely publicized image of #thedress as blue-black or white-gold. To probe these higher-level color inferences, we examined neural correlates of the blue-bias, using frequency-tagging and high-density electroencephalography to monitor responses to 3-Hz alternations between different color versions of #thedress. Specifically, we compared relative neural responses to the original "blue" dress image alternated with the complementary "yellow" image (formed by inverting the chromatic contrast of each pixel). This image pair produced a large modulation of the electroencephalography amplitude at the alternation frequency, consistent with a perceived contrast difference between the blue and yellow images. Furthermore, decoding topographical differences in the blue-yellow asymmetries over occipitoparietal channels predicted blue-black and white-gold observers with over 80% accuracy. The blue-yellow asymmetry was stronger than for a "red" versus "green" pair matched for the same component differences in L versus M or S versus LM chromatic contrast as the blue-yellow pair and thus cannot be accounted for by asymmetries within either precortical cardinal mechanism. Instead, the results may point to neural correlates of a higher-level perceptual representation of surface colors.

All-or-none face categorization in the human brain.

Visual categorization is integral for our interaction with the natural environment. In this process, similar selective responses are produced to a class of variable visual inputs. Whether categorization is supported by partial (graded) or absolute (all-or-none) neural responses in high-level human brain regions is largely unknown. We address this issue with a novel frequency-sweep paradigm probing the evolution of face categorization responses between the minimal and optimal stimulus presentation times. In a first experiment, natural images of variable non-face objects were progressively swept from 120 to 3 â€‹Hz (8.33-333 â€‹ms duration) in rapid serial visual presentation sequences. Widely variable face exemplars appeared every 1 â€‹s, enabling an implicit frequency-tagged face-categorization electroencephalographic (EEG) response at 1 â€‹Hz. Face-categorization activity emerged with stimulus durations as brief as 17 â€‹ms (17-83 â€‹ms across individual participants) but was significant with 33 â€‹ms durations at the group level. The face categorization response amplitude increased until 83 â€‹ms stimulus duration (12 â€‹Hz), implying graded categorization responses. In a second EEG experiment, faces appeared non-periodically throughout such sequences at fixed presentation rates, while participants explicitly categorized faces. A strong correlation between response amplitude and behavioral accuracy across frequency rates suggested that dilution from missed categorizations, rather than a decreased response to each face stimulus, accounted for the graded categorization responses as found in Experiment 1. This was supported by (1) the absence of neural responses to faces that participants failed to categorize explicitly in Experiment 2 and (2) equivalent amplitudes and spatio-temporal signatures of neural responses to behaviorally categorized faces across presentation rates. Overall, these observations provide original evidence that high-level visual categorization of faces, starting at about 100 â€‹ms following stimulus onset in the human brain, is variable across observers tested under tight temporal constraints, but occurs in an all-or-none fashion.

The contribution of color information to rapid face categorization in natural scenes.

Color's contribution to rapid categorization of natural images is debated. We examine its effect on high-level face categorization responses using fast periodic visual stimulation (Rossion et al., 2015). A high-density electroencephalogram (EEG) was recorded during presentation of sequences of natural object images every 83 ms (i.e., at F = 12.0 Hz). Natural face images were embedded in the sequence at a fixed interval of F/9 (1.33 Hz). There were four conditions: (a) full-color images; (b) grayscale images; and (c) and (d) phase-scrambled images from Conditions 1 and 2, respectively, making faces and objects unrecognizable. Observers' task was to respond to color changes of the fixation cross (Experiment 1). We found face-categorization responses at 1.33 Hz and its harmonics (2.67 Hz, etc.) over occipitotemporal areas, with right-hemisphere dominance; responses to color images were not significantly different from those to grayscale images. Behavioral analysis revealed longer response times when images contained color, despite nearly-all-correct performance in all conditions, suggesting that color change in the task might detract from color's contribution to face categorization. We subsequently changed the task to responding to fixation shape changes so that such response-time differences were eliminated (Experiment 2). The aggregate face-categorization response became 21.6% stronger to color than to grayscale images. This color advantage occurred late, at 290-415 ms after stimulus onset. Our results suggest that the color advantage for face categorization interacts with behavior, and that color only has a moderate and relatively late contribution to rapid face categorization in natural images.

Directional Visual Motion Is Represented in the Auditory and Association Cortices of Early Deaf Individuals.

Individuals who are deaf since early life may show enhanced performance at some visual tasks, including discrimination of directional motion. The neural substrates of such behavioral enhancements remain difficult to identify in humans, although neural plasticity has been shown for early deaf people in the auditory and association cortices, including the primary auditory cortex (PAC) and STS region, respectively. Here, we investigated whether neural responses in auditory and association cortices of early deaf individuals are reorganized to be sensitive to directional visual motion. To capture direction-selective responses, we recorded fMRI responses frequency-tagged to the 0.1-Hz presentation of central directional (100% coherent random dot) motion persisting for 2 sec contrasted with nondirectional (0% coherent) motion for 8 sec. We found direction-selective responses in the STS region in both deaf and hearing participants, but the extent of activation in the right STS region was 5.5 times larger for deaf participants. Minimal but significant direction-selective responses were also found in the PAC of deaf participants, both at the group level and in five of six individuals. In response to stimuli presented separately in the right and left visual fields, the relative activation across the right and left hemispheres was similar in both the PAC and STS region of deaf participants. Notably, the enhanced right-hemisphere activation could support the right visual field advantage reported previously in behavioral studies. Taken together, these results show that the reorganized auditory cortices of early deaf individuals are sensitive to directional motion. Speculatively, these results suggest that auditory and association regions can be remapped to support enhanced visual performance.

Dissociable effects of inter-stimulus interval and presentation duration on rapid face categorization.

Fast periodic visual stimulation combined with electroencephalography (FPVS-EEG) has unique sensitivity and objectivity in measuring rapid visual categorization processes. It constrains image processing time by presenting stimuli rapidly through brief stimulus presentation durations and short inter-stimulus intervals. However, the selective impact of these temporal parameters on visual categorization is largely unknown. Here, we presented natural images of objects at a rate of 10 or 20 per second (10 or 20 Hz), with faces appearing once per second (1 Hz), leading to two distinct frequency-tagged EEG responses. Twelve observers were tested with three squarewave image presentation conditions: 1) with an ISI, a traditional 50% duty cycle at 10 Hz (50-ms stimulus duration separated by a 50-ms ISI); 2) removing the ISI and matching the rate, a 100% duty cycle at 10 Hz (100-ms duration with 0-ms ISI); 3) removing the ISI and matching the stimulus presentation duration, a 100% duty cycle at 20 Hz (50-ms duration with 0-ms ISI). The face categorization response was significantly decreased in the 20 Hz 100% condition. The conditions at 10 Hz showed similar face-categorization responses, peaking maximally over the right occipito-temporal (ROT) cortex. However, the onset of the 10 Hz 100% response was delayed by about 20 ms over the ROT region relative to the 10 Hz 50% condition, likely due to immediate forward-masking by preceding images. Taken together, these results help to interpret how the FPVS-EEG paradigm sets temporal constraints on visual image categorization.

Visual adaptation reveals an objective electrophysiological measure of high-level individual face discrimination.

The ability to individualize faces is a fundamental human brain function. Following visual adaptation to one individual face, the suppressed neural response to this identity becomes discriminable from an unadapted facial identity at a neural population level. Here, we investigate a simple and objective measure of individual face discrimination with electroencephalographic (EEG) frequency tagging following adaptation. In a first condition, (1) two facial identities are presented in alternation at a rate of six images per second (6 Hz; 3 Hz identity repetition rate) for a 20 s testing sequence, following 10-s adaptation to one of the facial identities; this results in a significant identity discrimination response at 3 Hz in the frequency domain of the EEG over right occipito-temporal channels, replicating our previous findings. Such a 3 Hz response is absent for two novel conditions, in which (2) the faces are inverted and (3) an identity physically equidistant from the two faces is adapted. These results indicate that low-level visual features present in inverted or unspecific facial identities are not sufficient to produce the adaptation effect found for upright facial stimuli, which appears to truly reflect identity-specific perceptual representations in the human brain.

Uncovering the neural magnitude and spatio-temporal dynamics of natural image categorization in a fast visual stream.

Perceptual categorization occurs rapidly under natural viewing conditions. Yet, the neural spatio-temporal dynamics of category-selective processes to single-glanced, natural (i.e., unsegmented) images in a rapidly changing presentation stream remain unknown. We presented human observers with natural images of objects at a fast periodic rate of 12.5Hz, i.e., every 80ms. Images of faces were inserted every 3, 5, 7, 9, or 11 stimuli, defining stimulus-onset-asynchronies (SOAs) between 240-880ms, i.e., presentation frequencies (Fs) between 4.17-1.14Hz. Robust face-selective responses were objectively identified and quantified at F and its harmonics (2F, 3F, etc.) for every condition in the electroencephalogram (EEG). The summed-harmonic face-selective response was significantly reduced by 25% at the lowest face SOA, i.e. 240ms between two faces, but remained stable from 400ms SOA onward. This high-level, right lateralized face-selective response emerged at about 100ms post-stimulus onset and progressed spatially throughout four successive time-windows (i.e., P1-face, N1-face, P2-face, P3-face) from posterior to anterior occipito-temporal electrode sites. The total duration of a category-selective response to a briefly presented face stimulus in a rapid sequence of objects was estimated to be 420ms. Uncovering the neural spatio-temporal dynamics of category-selectivity in a rapid stream of natural images goes well beyond previous evidence obtained from spatially and temporally isolated stimuli, opening an avenue for understanding human vision and its relationship to categorization behavior.

A single glance at natural face images generate larger and qualitatively different category-selective spatio-temporal signatures than other ecologically-relevant categories in the human brain.

Although humans discriminate natural images of faces from other categories at a single glance, clarifying the neural specificity and spatio-temporal dynamics of this process without low-level visual confounds remains a challenge. We recorded high-density scalp electroencephalogram while presenting natural images of various objects at a fast periodic rate (5.88images/s). In different stimulation sequences, numerous variable exemplars of three categories associated with cortical specialization in neuroimaging - faces, body parts, or houses - appeared every five images (5.88Hz/5=1.18Hz). In these fast periodic visual stimulation (FPVS) sequences, common low- and high-level visual processes between these categories and other objects are captured at the 5.88Hz frequency, while high-level category-selective responses are objectively quantified at the 1.18Hz frequency and harmonics. Category-selective responses differed quantitatively and qualitatively between faces, body parts and houses. First, they were much larger (2-4 times) for faces over the whole scalp. Second, specific and reliable scalp topographical maps of category-selective responses pointed to distinct principle neural sources for faces (ventral occipito-temporal), body parts (lateral occipito-temporal) and houses (dorso-medial occipital). Category-selective EEG responses were found at multiple time-windows from 110 to 600ms post-stimulus onset. Faces elicited the most complex spatio-temporal profile with up to four selective responses, although body parts and houses also elicited selective responses more complex than previously described. These observations indicate that a single glance at natural face images inserted in a rapid stream of natural objects generates a quantitatively and qualitatively unique category-selective spatio-temporal signature in occipito-temporal cortical areas of the human brain.

Visual adaptation provides objective electrophysiological evidence of facial identity discrimination.

Discrimination of facial identities is a fundamental function of the human brain that is challenging to examine with macroscopic measurements of neural activity, such as those obtained with functional magnetic resonance imaging (fMRI) and electroencephalography (EEG). Although visual adaptation or repetition suppression (RS) stimulation paradigms have been successfully implemented to this end with such recording techniques, objective evidence of an identity-specific discrimination response due to adaptation at the level of the visual representation is lacking. Here, we addressed this issue with fast periodic visual stimulation (FPVS) and EEG recording combined with a symmetry/asymmetry adaptation paradigm. Adaptation to one facial identity is induced through repeated presentation of that identity at a rate of 6 images per second (6 Hz) over 10 sec. Subsequently, this identity is presented in alternation with another facial identity (i.e., its anti-face, both faces being equidistant from an average face), producing an identity repetition rate of 3 Hz over a 20 sec testing sequence. A clear EEG response at 3 Hz is observed over the right occipito-temporal (ROT) cortex, indexing discrimination between the two facial identities in the absence of an explicit behavioral discrimination measure. This face identity discrimination occurs immediately after adaptation and disappears rapidly within 20 sec. Importantly, this 3 Hz response is not observed in a control condition without the single-identity 10 sec adaptation period. These results indicate that visual adaptation to a given facial identity produces an objective (i.e., at a pre-defined stimulation frequency) electrophysiological index of visual discrimination between that identity and another, and provides a unique behavior-free quantification of the effect of visual adaptation.

Global shape information increases but color information decreases the composite face effect.

The separation of visual shape and surface information may be useful for understanding holistic face perception--that is, the perception of a face as a single unit (Jiang, Blanz, & Rossion, 2011, Visual Cognition, 19, 1003-1034). A widely used measure of holistic face perception is the composite face effect (CFE), in which identical top face halves appear different when aligned with bottom face halves from different identities. In the present study the influences of global face shape (ie contour of the face) and color information on the CFE are investigated, with the hypothesis that global face shape supports but color impairs holistic face perception as measured in this paradigm. In experiment 1 the CFE is significantly increased when face stimuli possess natural global shape information than when cropped to a generic (ie oval) global shape; this effect is not found when the stimuli are presented inverted. In experiment 2 the CFE is significantly decreased when face stimuli are presented with color information than when presented in grayscale. These findings indicate that grayscale stimuli maintaining natural global face shape information provide the most adept measure of holistic face perception in the behavioral composite face paradigm. More generally, they show that reducing different types of information diagnostic for individual face perception can have opposite effects on the CFE, illustrating the functional dissociation between shape and surface information in face perception.