Contrast versus identity encoding in the face image follow distinct orientation selectivity profiles.

Orientation selectivity is a fundamental property of primary visual encoding. High-level processing stages also show some form of orientation dependence, with face identification preferentially relying on horizontally-oriented information. How high-level orientation tuning emerges from primary orientation biases is unclear. In the same group of participants, we derived the orientation selectivity profile at primary and high-level visual processing stages using a contrast detection and an identity matching task. To capture the orientation selectivity profile, we calculated the difference in performance between all tested orientations (0, 45, 90, and 135°) for each task and for upright and inverted faces, separately. Primary orientation selectivity was characterized by higher sensitivity to oblique as compared to cardinal orientations. The orientation profile of face identification showed superior horizontal sensitivity to face identity. In each task, performance with upright and inverted faces projected onto qualitatively similar a priori models of orientation selectivity. Yet the fact that the orientation selectivity profiles of contrast detection in upright and inverted faces correlated significantly while such correlation was absent for identification indicates a progressive dissociation of orientation selectivity profiles from primary to high-level stages of orientation encoding. Bayesian analyses further indicate a lack of correlation between the orientation selectivity profiles in the contrast detection and face identification tasks, for upright and inverted faces. From these findings, we conclude that orientation selectivity shows distinct profiles at primary and high-level stages of face processing and that a transformation must occur from general cardinal attenuation when processing basic properties of the face image to horizontal tuning when encoding more complex properties such as identity.

Coarse-to-fine information integration in human vision.

Coarse-to-fine theories of vision propose that the coarse information carried by the low spatial frequencies (LSF) of visual input guides the integration of finer, high spatial frequency (HSF) detail. Whether and how LSF modulates HSF processing in naturalistic broad-band stimuli is still unclear. Here we used multivariate decoding of EEG signals to separate the respective contribution of LSF and HSF to the neural response evoked by broad-band images. Participants viewed images of human faces, monkey faces and phase-scrambled versions that were either broad-band or filtered to contain LSF or HSF. We trained classifiers on EEG scalp-patterns evoked by filtered scrambled stimuli and evaluated the derived models on broad-band scrambled and intact trials. We found reduced HSF contribution when LSF was informative towards image content, indicating that coarse information does guide the processing of fine detail, in line with coarse-to-fine theories. We discuss the potential cortical mechanisms underlying such coarse-to-fine feedback.

Visual working memory performance in aphantasia.

Aphantasia, i.e., the congenital inability to experience voluntary mental imagery, offers a new model for studying the functional role of mental imagery in (visual) cognition. However, until now, there have been no studies investigating whether aphantasia can be linked to specific impairments in cognitive functioning. Here, we assess visual working memory performance in an aphantasic individual. We find that she performs significantly worse than controls on the most difficult (i.e., requiring the highest degree of precision) visual working memory trials. Surprisingly, her performance on a task designed to involve mental imagery did not differ from controls', although she lacked metacognitive insight into her performance. Together, these results indicate that although a lack of mental imagery can be compensated for under some conditions, mental imagery has a functional role in other areas of visual cognition, one of which is high-precision working memory.

State-Dependent TMS Reveals Representation of Affective Body Movements in the Anterior Intraparietal Cortex.

In humans, recognition of others' actions involves a cortical network that comprises, among other cortical regions, the posterior superior temporal sulcus (pSTS), where biological motion is coded and the anterior intraparietal sulcus (aIPS), where movement information is elaborated in terms of meaningful goal-directed actions. This action observation system (AOS) is thought to encode neutral voluntary actions, and possibly some aspects of affective motor repertoire, but the role of the AOS' areas in processing affective kinematic information has never been examined. Here we investigated whether the AOS plays a role in representing dynamic emotional bodily expressions. In the first experiment, we assessed behavioral adaptation effects of observed affective movements. Participants watched series of happy or fearful whole-body point-light displays (PLDs) as adapters and were then asked to perform an explicit categorization of the emotion expressed in test PLDs. Participants were slower when categorizing any of the two emotions as long as it was congruent with the emotion in the adapter sequence. We interpreted this effect as adaptation to the emotional content of PLDs. In the second experiment, we combined this paradigm with TMS applied over either the right aIPS, pSTS, and the right half of the occipital pole (corresponding to Brodmann's area 17 and serving as control) to examine the neural locus of the adaptation effect. TMS over the aIPS (but not over the other sites) reversed the behavioral cost of adaptation, specifically for fearful contents. This demonstrates that aIPS contains an explicit representation of affective body movements.SIGNIFICANCE STATEMENT In humans, a network of areas, the action observation system, encodes voluntary actions. However, the role of these brain regions in processing affective kinematic information has not been investigated. Here we demonstrate that the aIPS contains a representation of affective body movements. First, in a behavioral experiment, we found an adaptation after-effect for emotional PLDs, indicating the existence of a neural representation selective for affective information in biological motion. To examine the neural locus of this effect, we then combined the adaptation paradigm with TMS. Stimulation of the aIPS (but not over pSTS and control site) reversed the behavioral cost of adaptation, specifically for fearful contents, demonstrating that aIPS contains a representation of affective body movements.

Moving Beyond Attentional Biases: Shifting the Interhemispheric Balance between Left and Right Posterior Parietal Cortex Modulates Attentional Control Processes.

The concept of interhemispheric competition has been very influential in attention research, and the occurrence of biased attention due to an imbalance in posterior parietal cortex (PPC) is well documented. In this context, the vast majority of studies have assessed attentional performance with tasks that did not include an explicit experimental manipulation of attention, and, as a consequence, it remains largely unknown how these findings relate to core attentional constructs such as endogenous and exogenous control and spatial orienting and reorienting. We here addressed this open question by creating an imbalance between left and right PPC with transcranial direct current stimulation, resulting in right-hemispheric dominance, and assessed performance on three experimental paradigms that isolate distinct attentional processes. The comparison between active and sham transcranial direct current stimulations revealed a highly informative pattern of results with differential effects across tasks. Our results demonstrate the functional necessity of PPC for endogenous and exogenous attentional control and, importantly, link the concept of interhemispheric competition to core attentional processes, thus moving beyond the notion of biased attention after noninvasive brain stimulation over PPC.

Attention, working memory, and phenomenal experience of WM content: memory levels determined by different types of top-down modulation.

What is the role of top-down attentional modulation in consciously accessing working memory (WM) content? In influential WM models, information can exist in different states, determined by allocation of attention; placing the original memory representation in the center of focused attention gives rise to conscious access. Here we discuss various lines of evidence indicating that such attentional modulation is not sufficient for memory content to be phenomenally experienced. We propose that, in addition to attentional modulation of the memory representation, another type of top-down modulation is required: suppression of all incoming visual information, via inhibition of early visual cortex. In this view, there are three distinct memory levels, as a function of the top-down control associated with them: (1) Nonattended, nonconscious associated with no attentional modulation; (2) attended, phenomenally nonconscious memory, associated with attentional enhancement of the actual memory trace; (3) attended, phenomenally conscious memory content, associated with enhancement of the memory trace and top-down suppression of all incoming visual input.

How is working memory content consciously experienced? The 'conscious copy' model of WM introspection.

We address the issue of how visual information stored in working memory (WM) is introspected. In other words, how do we become aware of WM content in order to consciously examine or manipulate it? Influential models of WM have suggested that WM representations are either conscious by definition, or directly accessible for conscious inspection. We propose that WM introspection does not operate on the actual memory trace but rather requires a new representation to be created for the conscious domain. This conscious representation exists in addition and in parallel to the actual memory representation. The existence of such a separate representation is revealed by and reflected in the qualitatively different functional characteristics between the actual memory trace and its conscious experience, and their distinct interactions within external visual input. Our model differs from state-based models in that WM introspection does not involve a change in the state of WM content, but rather involves the creation of a new, second representation existing in parallel to the original memory trace.

Two distinct neural mechanisms in early visual cortex determine subsequent visual processing.

Neuroscience research has conventionally focused on how the brain processes sensory information, after the information has been received. Recently, increased interest focuses on how the state of the brain upon receiving inputs determines and biases their subsequent processing and interpretation. Here, we investigated such 'pre-stimulus' brain mechanisms and their relevance for objective and subjective visual processing. Using non-invasive focal brain stimulation [transcranial magnetic stimulation (TMS)] we disrupted spontaneous brain state activity within early visual cortex (EVC) before onset of visual stimulation, at two different pre-stimulus-onset-asynchronies (pSOAs). We found that TMS pulses applied to EVC at either 20 msec or 50 msec before onset of a simple orientation stimulus both prevented this stimulus from reaching visual awareness. Interestingly, only the TMS-induced visual suppression following TMS at a pSOA of ?20 msec was retinotopically specific, while TMS at a pSOA of ?50 msec was not. In a second experiment, we used more complex symbolic arrow stimuli, and found TMS-induced suppression only when disrupting EVC at a pSOA of ? ?60 msec, which, in line with Experiment 1, was not retinotopically specific. Despite this topographic unspecificity of the ?50 msec effect, the additional control measurements as well as tracking and removal of eye blinks, suggested that also this effect was not the result of an unspecific artifact, and thus neural in origin. We therefore obtained evidence of two distinct neural mechanisms taking place in EVC, both determining whether or not subsequent visual inputs are successfully processed by the human visual system.

The chronometry of visual perception: review of occipital TMS masking studies.

Transcranial magnetic stimulation (TMS) continues to deliver on its promise as a research tool. In this review article we focus on the application of TMS to early visual cortex (V1, V2, V3) in studies of visual perception and visual awareness. Depending on the asynchrony between visual stimulus onset and TMS pulse (SOA), TMS can suppress visual perception, allowing one to track the time course of functional relevance (chronometry) of early visual cortex for vision. This procedure has revealed multiple masking effects ('dips'), some consistently (∼+100ms SOA) but others less so (∼-50ms, ∼-20ms, ∼+30ms, ∼+200ms SOA). We review the state of TMS masking research, focusing on the evidence for these multiple dips, the relevance of several experimental parameters to the obtained 'masking curve', and the use of multiple measures of visual processing (subjective measures of awareness, objective discrimination tasks, priming effects). Lastly, we consider possible future directions for this field. We conclude that while TMS masking has yielded many fundamental insights into the chronometry of visual perception already, much remains unknown. Not only are there several temporal windows when TMS pulses can induce visual suppression, even the well-established 'classical' masking effect (∼+100ms) may reflect more than one functional visual process.

The cortex-based alignment approach to TMS coil positioning.

TMS allows noninvasive manipulation of brain activity in healthy participants and patients. The effectiveness of TMS experiments critically depends on precise TMS coil positioning, which is best for most brain areas when a frameless stereotactic system is used to target activation foci based on individual fMRI data. From a purely scientific perspective, individual fMRI-guided TMS is thus the method of choice to ensure optimal TMS efficiency. Yet, from a more practical perspective, such individual functional data are not always available, and therefore alternative TMS coil positioning approaches are often applied, for example, based on functional group data reported in Talairach coordinates. We here propose a novel method for TMS coil positioning that is based on functional group data, yet only requires individual anatomical data. We used cortex-based alignment (CBA) to transform individual anatomical data to an atlas brain that includes probabilistic group maps of two functional regions (FEF and hMT+/V5). Then, these functional group maps were back-transformed to the individual brain anatomy, preserving functional-anatomical correspondence. As a proof of principle, the resulting CBA-based functional targets in individual brain space were compared with individual FEF and hMT+/V5 hotspots as conventionally localized with individual fMRI data and with targets based on Talairach coordinates as commonly done in TMS research in case only individual anatomical data are available. The CBA-based approach significantly improved localization of functional brain areas compared with traditional Talairach-based targeting. Given the widespread availability of CBA schemes and preexisting functional group data, the proposed procedure is easy to implement and at no additional measurement costs. However, the accuracy of individual fMRI-guided TMS remains unparalleled, and the CBA-based approach should only be the method of choice when individual functional data cannot be obtained or experimental factors argue against it.

Time- and task-dependent non-neural effects of real and sham TMS.

Transcranial magnetic stimulation (TMS) is widely used in experimental brain research to manipulate brain activity in humans. Next to the intended neural effects, every TMS pulse produces a distinct clicking sound and sensation on the head which can also influence task performance. This necessitates careful consideration of control conditions in order to ensure that behavioral effects of interest can be attributed to the neural consequences of TMS and not to non-neural effects of a TMS pulse. Surprisingly, even though these non-neural effects of TMS are largely unknown, they are often assumed to be unspecific, i.e. not dependent on TMS parameters. This assumption is inherent to many control strategies in TMS research but has recently been challenged on empirical grounds. Here, we further develop the empirical basis of control strategies in TMS research. We investigated the time-dependence and task-dependence of the non-neural effects of TMS and compared real and sham TMS over vertex. Critically, we show that non-neural TMS effects depend on a complex interplay of these factors. Although TMS had no direct neural effects, both pre- and post-stimulus TMS time windows modulated task performance on both a sensory detection task and a cognitive angle judgment task. For the most part, these effects were quantitatively similar across tasks but effect sizes were clearly different. Moreover, the effects of real and sham TMS were almost identical with interesting exceptions that shed light on the relative contribution of auditory and somato-sensory aspects of a TMS pulse. Knowledge of such effects is of critical importance for the interpretation of TMS experiments and helps deciding what constitutes an appropriate control condition. Our results broaden the empirical basis of control strategies in TMS research and point at potential pitfalls that should be avoided.

The temporal dynamics of early visual cortex involvement in behavioral priming.

Transcranial magnetic stimulation (TMS) allows for non-invasive interference with ongoing neural processing. Applied in a chronometric design over early visual cortex (EVC), TMS has proved valuable in indicating at which particular time point EVC must remain unperturbed for (conscious) vision to be established. In the current study, we set out to examine the effect of EVC TMS across a broad range of time points, both before (pre-stimulus) and after (post-stimulus) the onset of symbolic visual stimuli. Behavioral priming studies have shown that the behavioral impact of a visual stimulus can be independent from its conscious perception, suggesting two independent neural signatures. To assess whether TMS-induced suppression of visual awareness can be dissociated from behavioral priming in the temporal domain, we thus implemented three different measures of visual processing, namely performance on a standard visual discrimination task, a subjective rating of stimulus visibility, and a visual priming task. To control for non-neural TMS effects, we performed electrooculographical recordings, placebo TMS (sham), and control site TMS (vertex). Our results suggest that, when considering the appropriate control data, the temporal pattern of EVC TMS disruption on visual discrimination, subjective awareness and behavioral priming are not dissociable. Instead, TMS to EVC disrupts visual perception holistically, both when applied before and after the onset of a visual stimulus. The current findings are discussed in light of their implications on models of visual awareness and (subliminal) priming.

Posttraining transcranial magnetic stimulation of striate cortex disrupts consolidation early in visual skill learning.

Practice-induced improvements in skilled performance reflect "offline " consolidation processes extending beyond daily training sessions. According to visual learning theories, an early, fast learning phase driven by high-level areas is followed by a late, asymptotic learning phase driven by low-level, retinotopic areas when higher resolution is required. Thus, low-level areas would not contribute to learning and offline consolidation until late learning. Recent studies have challenged this notion, demonstrating modified responses to trained stimuli in primary visual cortex (V1) and offline activity after very limited training. However, the behavioral relevance of modified V1 activity for offline consolidation of visual skill memory in V1 after early training sessions remains unclear. Here, we used neuronavigated transcranial magnetic stimulation (TMS) directed to a trained retinotopic V1 location to test for behaviorally relevant consolidation in human low-level visual cortex. Applying TMS to the trained V1 location within 45 min of the first or second training session strongly interfered with learning, as measured by impaired performance the next day. The interference was conditional on task context and occurred only when training in the location targeted by TMS was followed by training in a second location before TMS. In this condition, high-level areas may become coupled to the second location and uncoupled from the previously trained low-level representation, thereby rendering consolidation vulnerable to interference. Our data show that, during the earliest phases of skill learning in the lowest-level visual areas, a behaviorally relevant form of consolidation exists of which the robustness is controlled by high-level, contextual factors.

Topographic contribution of early visual cortex to short-term memory consolidation: a transcranial magnetic stimulation study.

The neural correlates for retention of visual information in visual short-term memory are considered separate from those of sensory encoding. However, recent findings suggest that sensory areas may play a role also in short-term memory. We investigated the functional relevance, spatial specificity, and temporal characteristics of human early visual cortex in the consolidation of capacity-limited topographic visual memory using transcranial magnetic stimulation (TMS). Topographically specific TMS pulses were delivered over lateralized occipital cortex at 100, 200, or 400 ms into the retention phase of a modified change detection task with low or high memory loads. For the high but not the low memory load, we found decreased memory performance for memory trials in the visual field contralateral, but not ipsilateral to the side of TMS, when pulses were delivered at 200 ms into the retention interval. A behavioral version of the TMS experiment, in which a distractor stimulus (memory mask) replaced the TMS pulses, further corroborated these findings. Our findings suggest that retinotopic visual cortex contributes to the short-term consolidation of topographic visual memory during early stages of the retention of visual information. Further, TMS-induced interference decreased the strength (amplitude) of the memory representation, which most strongly affected the high memory load trials.

Behavior in oblivion: the neurobiology of subliminal priming.

Subliminal priming refers to behavioral modulation by an unconscious stimulus, and can thus be regarded as a form of unconscious visual processing. Theories on recurrent processing have suggested that the neural correlate of consciousness (NCC) comprises of the non-hierarchical transfer of stimulus-related information. According to these models, the neural correlate of subliminal priming (NCSP) corresponds to the visual processing within the feedforward sweep. Research from cognitive neuroscience on these two concepts and the relationship between them is discussed here. Evidence favoring the necessity of recurrent connectivity for visual awareness is accumulating, although some questions, such as the need for global versus local recurrent processing, are not clarified yet. However, this is not to say that recurrent processing is sufficient for consciousness, as a neural definition of consciousness in terms of recurrent connectivity would imply. We argue that the limited interest cognitive neuroscience currently has for the NCSP is undeserved, because the discovery of the NCSP can give insight into why people do (and do not) express certain behavior.

Visual awareness suppression by pre-stimulus brain stimulation; a neural effect.

Transcranial magnetic stimulation (TMS) has established the functional relevance of early visual cortex (EVC) for visual awareness with great temporal specificity non-invasively in conscious human volunteers. Many studies have found a suppressive effect when TMS was applied over EVC 80-100 ms after the onset of the visual stimulus (post-stimulus TMS time window). Yet, few studies found task performance to also suffer when TMS was applied even before visual stimulus presentation (pre-stimulus TMS time window). This pre-stimulus TMS effect, however, remains controversially debated and its origin had mainly been ascribed to TMS-induced eye-blinking artifacts. Here, we applied chronometric TMS over EVC during the execution of a visual discrimination task, covering an exhaustive range of visual stimulus-locked TMS time windows ranging from -80 pre-stimulus to 300 ms post-stimulus onset. Electrooculographical (EoG) recordings, sham TMS stimulation, and vertex TMS stimulation controlled for different types of non-neural TMS effects. Our findings clearly reveal TMS-induced masking effects for both pre- and post-stimulus time windows, and for both objective visual discrimination performance and subjective visibility. Importantly, all effects proved to be still present after post hoc removal of eye blink trials, suggesting a neural origin for the pre-stimulus TMS suppression effect on visual awareness. We speculate based on our data that TMS exerts its pre-stimulus effect via generation of a neural state which interacts with subsequent visual input.

TMS effects on subjective and objective measures of vision: stimulation intensity and pre- versus post-stimulus masking.

Transcranial magnetic stimulation (TMS) can be used to mask visual stimuli, disrupting visual task performance or preventing visual awareness. While TMS masking studies generally fix stimulation intensity, we hypothesized that varying the intensity of TMS pulses in a masking paradigm might inform several ongoing debates concerning TMS disruption of vision as measured subjectively versus objectively, and pre-stimulus (forward) versus post-stimulus (backward) TMS masking. We here show that both pre-stimulus TMS pulses and post-stimulus TMS pulses could strongly mask visual stimuli. We found no dissociations between TMS effects on the subjective and objective measures of vision for any masking window or intensity, ruling out the option that TMS intensity levels determine whether dissociations between subjective and objective vision are obtained. For the post-stimulus time window particularly, we suggest that these data provide new constraints for (e.g. recurrent) models of vision and visual awareness. Finally, our data are in line with the idea that pre-stimulus masking operates differently from conventional post-stimulus masking.

FMRI effective connectivity and TMS chronometry: complementary accounts of causality in the visuospatial judgment network.

BACKGROUND: While traditionally quite distinct, functional neuroimaging (e.g. functional magnetic resonance imaging: fMRI) and functional interference techniques (e.g. transcranial magnetic stimulation: TMS) increasingly address similar questions of functional brain organization, including connectivity, interactions, and causality in the brain. Time-resolved TMS over multiple brain network nodes can elucidate the relative timings of functional relevance for behavior ("TMS chronometry"), while fMRI functional or effective connectivity (fMRI EC) can map task-specific interactions between brain regions based on the interrelation of measured signals. The current study empirically assessed the relation between these different methods. METHODOLOGY/PRINCIPAL FINDINGS: One group of 15 participants took part in two experiments: one fMRI EC study, and one TMS chronometry study, both of which used an established cognitive paradigm involving one visuospatial judgment task and one color judgment control task. Granger causality mapping (GCM), a data-driven variant of fMRI EC analysis, revealed a frontal-to-parietal flow of information, from inferior/middle frontal gyrus (MFG) to posterior parietal cortex (PPC). FMRI EC-guided Neuronavigated TMS had behavioral effects when applied to both PPC and to MFG, but the temporal pattern of these effects was similar for both stimulation sites. At first glance, this would seem in contradiction to the fMRI EC results. However, we discuss how TMS chronometry and fMRI EC are conceptually different and show how they can be complementary and mutually constraining, rather than contradictory, on the basis of our data. CONCLUSIONS/SIGNIFICANCE: The findings that fMRI EC could successfully localize functionally relevant TMS target regions on the single subject level, and conversely, that TMS confirmed an fMRI EC identified functional network to be behaviorally relevant, have important methodological and theoretical implications. Our results, in combination with data from earlier studies by our group (Sack et al., 2007, Cerebral Cortex), lead to informed speculations on complex brain mechanisms, and TMS disruption thereof, underlying visuospatial judgment. This first in-depth empirical and conceptual comparison of fMRI EC and TMS chronometry thereby shows the complementary insights offered by the two methods.

Dynamic premotor-to-parietal interactions during spatial imagery.

The neurobiological processes underlying mental imagery are a matter of debate and controversy among neuroscientists, cognitive psychologists, philosophers, and biologists. Recent neuroimaging studies demonstrated that the execution of mental imagery activates large frontoparietal and occipitotemporal networks in the human brain. These previous imaging studies, however, neglected the crucial interplay within and across the widely distributed cortical networks of activated brain regions. Here, we combined time-resolved event-related functional magnetic resonance imaging with analyses of interactions between brain regions (functional and effective brain connectivity) to unravel the premotor-parietal dynamics underlying spatial imagery. Participants had to sequentially construct and spatially transform a mental visual object based on either verbal or visual instructions. By concurrently accounting for the full spatiotemporal pattern of brain activity and network connectivity, we functionally segregated an early from a late premotor-parietal imagery network. Moreover, we revealed that the modality-specific information upcoming from sensory brain regions is first sent to the premotor cortex and then to the medial-dorsal parietal cortex, i.e., top-down from the motor to the perceptual pole during spatial imagery. Importantly, we demonstrate that the premotor cortex serves as the central relay station, projecting to parietal cortex at two functionally distinct stages during spatial imagery. Our approach enabled us to disentangle the multicomponential cognitive construct of mental imagery into its different cognitive subelements. We discuss and explicitly assign these mental subprocesses to each of the revealed effective brain connectivity networks and present an integrative neurobiological model of spatial imagery.