Frontal eye field neurons selectively signal the reward value of prior actions.

The consequences of individual actions are typically unknown until well after they are executed. This fact necessitates a mechanism that bridges delays between specific actions and reward outcomes. We looked for the presence of such a mechanism in the post-movement activity of neurons in the frontal eye field (FEF), a visuomotor area in prefrontal cortex. Monkeys performed an oculomotor gamble task in which they made eye movements to different locations associated with dynamically varying reward outcomes. Behavioral data showed that monkeys tracked reward history and made choices according to their own risk preferences. Consistent with previous studies, we observed that the activity of FEF neurons is correlated with the expected reward value of different eye movements before a target appears. Moreover, we observed that the activity of FEF neurons continued to signal the direction of eye movements, the expected reward value, and their interaction well after the movements were completed and when targets were no longer within the neuronal response field. In addition, this post-movement information was also observed in local field potentials, particularly in low-frequency bands. These results show that neural signals of prior actions and expected reward value persist across delays between those actions and their experienced outcomes. These memory traces may serve a role in reward-based learning in which subjects need to learn actions predicting delayed reward.

Parietal Cortex Regulates Visual Salience and Salience-Driven Behavior.

Unique stimuli stand out. Despite an abundance of competing sensory stimuli, the detection of the most salient ones occurs without effort, and that detection contributes to the guidance of adaptive behavior. Neurons sensitive to the salience of visual stimuli are widespread throughout the primate visual system and are thought to shape the selection of visual targets. However, a neural source of salience remains elusive. In an attempt to identify a source of visual salience, we reversibly inactivated parietal cortex and simultaneously recorded salience signals in prefrontal cortex. Inactivation of parietal cortex not only caused pronounced and selective reductions of salience signals in prefrontal cortex but also diminished the influence of salience on visually guided behavior. These observations demonstrate a causal role of parietal cortex in regulating salience signals within the brain and in controlling salience-driven behavior.

Dissonant Representations of Visual Space in Prefrontal Cortex during Eye Movements.

We used local field potentials (LFPs) and spikes to investigate representations of visual space in prefrontal cortex and the dynamics of those representations during eye movements. Spatial information contained in LFPs of the frontal eye field (FEF) was differentially distributed across frequencies, with a majority of that information being carried in alpha and high-gamma bands and minimal signal in the low-gamma band. During fixation, spatial information from alpha and high-gamma bands and spiking activity was robust across cortical layers. Receptive fields (RFs) derived from alpha and high-gamma bands were retinocentrically organized, and they were spatially correlated both with each other and with spiking RFs. However, alpha and high-gamma RFs probed before eye movements were dissociated. Whereas high-gamma and spiking RFs immediately converged toward the movement goal, alpha RFs remained largely unchanged during the initial probe response, but they converged later. These observations reveal possible mechanisms of dynamic spatial representations that underlie visual perception during eye movements.

Spatial working memory alters the efficacy of input to visual cortex.

Prefrontal cortex modulates sensory signals in extrastriate visual cortex, in part via its direct projections from the frontal eye field (FEF), an area involved in selective attention. We find that working memory-related activity is a dominant signal within FEF input to visual cortex. Although this signal alone does not evoke spiking responses in areas V4 and MT during memory, the gain of visual responses in these areas increases, and neuronal receptive fields expand and shift towards the remembered location, improving the stimulus representation by neuronal populations. These results provide a basis for enhancing the representation of working memory targets and implicate persistent FEF activity as a basis for the interdependence of working memory and selective attention.

Two Types of Receptive Field Dynamics in Area V4 at the Time of Eye Movements?

How we perceive the world as stable despite the frequent disruptions of the retinal image caused by eye movements is one of the fundamental questions in sensory neuroscience. Seemingly convergent evidence points towards a mechanism which dynamically updates representations of visual space in anticipation of a movement (Wurtz, 2008). In particular, receptive fields (RFs) of neurons, predominantly within oculomotor and attention related brain structures (Duhamel et al., 1992; Walker et al., 1995; Umeno and Goldberg, 1997), are thought to "remap" to their future, post-movement location prior to an impending eye movement. New studies (Neupane et al., 2016a,b) report observations on RF dynamics at the time of eye movements of neurons in area V4. These dynamics are interpreted as being largely dominated by a remapping of RFs. Critically, these observations appear at odds with a previous study reporting a different type of RF dynamics within the same brain structure (Tolias et al., 2001), consisting of a shrinkage and shift of RFs towards the movement target. Importantly, RFs have been measured with different techniques in those studies. Here, we measured V4 RFs comparable to Neupane et al. (2016a,b) and observe a shrinkage and shift of RFs towards the movement target when analyzing the immediate stimulus response (Zirnsak et al., 2014). When analyzing the late stimulus response (Neupane et al., 2016a,b), we observe RF shifts resembling remapping. We discuss possible causes for these shifts and point out important issues which future studies on RF dynamics need to address.

Neural Mechanisms of Selective Visual Attention.

Selective visual attention describes the tendency of visual processing to be confined largely to stimuli that are relevant to behavior. It is among the most fundamental of cognitive functions, particularly in humans and other primates for whom vision is the dominant sense. We review recent progress in identifying the neural mechanisms of selective visual attention. We discuss evidence from studies of different varieties of selective attention and examine how these varieties alter the processing of stimuli by neurons within the visual system, current knowledge of their causal basis, and methods for assessing attentional dysfunctions. In addition, we identify some key questions that remain in identifying the neural mechanisms that give rise to the selective processing of visual information.

Social and attention-to-detail subclusters of autistic traits differentially predict looking at eyes and face identity recognition ability.

This study distinguished between different subclusters of autistic traits in the general population and examined the relationships between these subclusters, looking at the eyes of faces, and the ability to recognize facial identity. Using the Autism Spectrum Quotient (AQ) measure in a university-recruited sample, we separate the social aspects of autistic traits (i.e., those related to communication and social interaction; AQ-Social) from the non-social aspects, particularly attention-to-detail (AQ-Attention). We provide the first evidence that these social and non-social aspects are associated differentially with looking at eyes: While AQ-Social showed the commonly assumed tendency towards reduced looking at eyes, AQ-Attention was associated with increased looking at eyes. We also report that higher attention-to-detail (AQ-Attention) was then indirectly related to improved face recognition, mediated by increased number of fixations to the eyes during face learning. Higher levels of socially relevant autistic traits (AQ-Social) trended in the opposite direction towards being related to poorer face recognition (significantly so in females on the Cambridge Face Memory Test). There was no evidence of any mediated relationship between AQ-Social and face recognition via reduced looking at the eyes. These different effects of AQ-Attention and AQ-Social suggest face-processing studies in Autism Spectrum Disorder might similarly benefit from considering symptom subclusters. Additionally, concerning mechanisms of face recognition, our results support the view that more looking at eyes predicts better face memory.

The What and Where of Visual Attention.

The selective processing of sensory input during attention is known to take many forms, and different forms of attention likely reflect varying underlying neural mechanisms. Bichot and colleagues (2015) identify neurons that appear specialized for the control of feature-based visual attention.

Saccades and shifting receptive fields: anticipating consequences or selecting targets?

Saccadic eye movements cause frequent and substantial displacements of the retinal image, but those displacements go unnoticed. It has been widely assumed that this perceived stability emerges from the shifting of visual receptive fields from their current, presaccadic locations to their future, postsaccadic locations in anticipation of the retinal consequences of saccades. Although evidence consistent with this anticipatory remapping has accumulated over the years, more recent work suggests an alternative view. In this opinion article, we examine the evidence of presaccadic receptive field shifts and their relationship to the perceptual changes that accompany saccades. We argue that both reflect the selection of targets for saccades rather than the anticipation of a displaced retinal image.

Global selection of saccadic target features by neurons in area v4.

Psychophysical and neurophysiological studies indicate that during the preparation of saccades, visual processing at the target location is facilitated automatically by the deployment of attention. It has been assumed that the neural mechanisms involved in presaccadic shifts of attention are purely spatial in nature. Saccade preparation modulates the visual responses of neurons within extrastriate area V4, where the responses to targets are enhanced and responses to nontargets are suppressed. We tested whether this effect also engages a nonspatial form of modulation. We measured the responses of area V4 neurons to oriented gratings in two monkeys (Macaca mulatta) making delayed saccades to targets distant from the neuronal receptive field (RF). We varied the orientation of both the RF stimulus and the saccadic target. We found that, in addition to the spatial modulation, saccade preparation involves a feature-dependent modulation of V4 neuronal responses. Specifically, we found that the suppression of area V4 responses to nontarget stimuli during the preparation of saccades depends on the features of the saccadic target. Presaccadic suppression was absent when the features of the saccadic target matched the features preferred by individual V4 neurons. This feature-dependent modulation occurred in the absence of any feature-attention task. We show that our observations are consistent with a computational framework in which feature-based effects automatically emerge from saccade-related feedback signals that are spatial in nature.

Visual space is compressed in prefrontal cortex before eye movements.

We experience the visual world through a series of saccadic eye movements, each one shifting our gaze to bring objects of interest to the fovea for further processing. Although such movements lead to frequent and substantial displacements of the retinal image, these displacements go unnoticed. It is widely assumed that a primary mechanism underlying this apparent stability is an anticipatory shifting of visual receptive fields (RFs) from their presaccadic to their postsaccadic locations before movement onset. Evidence of this predictive 'remapping' of RFs has been particularly apparent within brain structures involved in gaze control. However, critically absent among that evidence are detailed measurements of visual RFs before movement onset. Here we show that during saccade preparation, rather than remap, RFs of neurons in a prefrontal gaze control area massively converge towards the saccadic target. We mapped the visual RFs of prefrontal neurons during stable fixation and immediately before the onset of eye movements, using multi-electrode recordings in monkeys. Following movements from an initial fixation point to a target, RFs remained stationary in retinocentric space. However, in the period immediately before movement onset, RFs shifted by as much as 18 degrees of visual angle, and converged towards the target location. This convergence resulted in a threefold increase in the proportion of RFs responding to stimuli near the target region. In addition, like in human observers, the population of prefrontal neurons grossly mislocalized presaccadic stimuli as being closer to the target. Our results show that RF shifts do not predict the retinal displacements due to saccades, but instead reflect the overriding perception of target space during eye movements.

Anticipatory saccade target processing and the presaccadic transfer of visual features.

As we shift our gaze to explore the visual world, information enters cortex in a sequence of successive snapshots, interrupted by phases of blur. Our experience, in contrast, appears like a movie of a continuous stream of objects embedded in a stable world. This perception of stability across eye movements has been linked to changes in spatial sensitivity of visual neurons anticipating the upcoming saccade, often referred to as shifting receptive fields (Duhamel et al., 1992; Walker et al., 1995; Umeno and Goldberg, 1997; Nakamura and Colby, 2002). How exactly these receptive field dynamics contribute to perceptual stability is currently not clear. Anticipatory receptive field shifts toward the future, postsaccadic position may bridge the transient perisaccadic epoch (Sommer and Wurtz, 2006; Wurtz, 2008; Melcher and Colby, 2008). Alternatively, a presaccadic shift of receptive fields toward the saccade target area (Tolias et al., 2001) may serve to focus visual resources onto the most relevant objects in the postsaccadic scene (Hamker et al., 2008). In this view, shifts of feature detectors serve to facilitate the processing of the peripheral visual content before it is foveated. While this conception is consistent with previous observations on receptive field dynamics and on perisaccadic compression (Ross et al., 1997; Morrone et al., 1997; Kaiser and Lappe, 2004), it predicts that receptive fields beyond the saccade target shift toward the saccade target rather than in the direction of the saccade. We have tested this prediction in human observers via the presaccadic transfer of the tilt-aftereffect (Melcher, 2007).

Split of spatial attention as predicted by a systems-level model of visual attention.

Can we attend to multiple distinct spatial locations at the same time? According to a recent psychophysical study [J. Dubois et al. (2009)Journal of Vision, 9, 3.1-11] such a split of spatial attention might be limited to short periods of time. Following N. P. Bichot et al. [(1999)Perception & Psychophysics, 61, 403-423] subjects had to report the identity of multiple letters that were briefly presented at different locations, while two of these locations (targets) were relevant for a concurrent shape comparison task. In addition to the design used by Bichot et al. stimulus onset asynchrony between shape onset and letters was systematically varied. In general, the performance of subjects was superior at target locations. Furthermore, for short stimulus onset asynchronies, performance was simultaneously increasing at both target locations. For longer stimulus onset asynchronies, however, performance deteriorated at one of the target locations while increasing at the other target location. It was hypothesized that this dynamic deployment of attention might be caused by competitive processes in saccade-related structures such as the frontal eye field. Here we simulated the task of Dubois et al. using a systems-level model of attention. Our results are consistent with recent findings in the frontal eye field obtained during covert visual search, and they support the view of a transient deployment of spatial attention to multiple stimuli in the early epoch of target selection.

Computational models of spatial updating in peri-saccadic perception.

Perceptual phenomena that occur around the time of a saccade, such as peri-saccadic mislocalization or saccadic suppression of displacement, have often been linked to mechanisms of spatial stability. These phenomena are usually regarded as errors in processes of trans-saccadic spatial transformations and they provide important tools to study these processes. However, a true understanding of the underlying brain processes that participate in the preparation for a saccade and in the transfer of information across it requires a closer, more quantitative approach that links different perceptual phenomena with each other and with the functional requirements of ensuring spatial stability. We review a number of computational models of peri-saccadic spatial perception that provide steps in that direction. Although most models are concerned with only specific phenomena, some generalization and interconnection between them can be obtained from a comparison. Our analysis shows how different perceptual effects can coherently be brought together and linked back to neuronal mechanisms on the way to explaining vision across saccades.

Attention alters feature space in motion processing.

Attending to a particular feature modulates the activity of neurons throughout the visual field with the result that relevant features are enhanced while irrelevant features are suppressed. Do these modulatory influences merely lead to a gating of relevant features, or does attention have a direct impact on the representation of feature space, leading to a different percept depending on the content of attention? We observed that direction estimates of the static motion aftereffect drastically change when human observers attend to a stimulus whose motion direction differs from the one of the adaptor. This observation suggests that feature-based attention might operate by local magnifications of feature space between relevant and irrelevant features.

The spatial distribution of receptive field changes in a model of peri-saccadic perception: predictive remapping and shifts towards the saccade target.

At the time of an impending saccade receptive fields (RFs) undergo dynamic changes, that is, their spatial profile is altered. This phenomenon has been observed in several monkey visual areas. Although their link to eye movements is obvious, neither the exact pattern nor their function is fully clear. Several RF shifts have been interpreted in terms of predictive remapping mediating visual stability. In particular, even prior to saccade onset some cells become responsive to stimuli presented in their future, post-saccadic RF. In visual area V4, however, the overall effect of RF dynamics consists of a shrinkage and shift of RFs towards the saccade target. These observations have been linked to a pre-saccadically enhanced processing of the future fixation. In order to better understand these seemingly different outcomes, we analyzed the RF shifts predicted by a recently proposed computational model of peri-saccadic perception (Hamker, Zirnsak, Calow, & Lappe, 2008). This model unifies peri-saccadic compression, pre-saccadic attention shifts, and peri-saccadic receptive field dynamics in a common framework of oculomotor reentry signals in extrastriate visual cortical maps. According to the simulations that we present in the current paper, a spatially selective oculomotor feedback signal leads to RF dynamics which are both consistent with the observations made in studies aiming to investigate predictive remapping and saccade target shifts. Thus, the seemingly distinct experimental observations could be grounded in the same neural mechanism leading to different RF dynamics dependent on the location of the RF in visual space.

The peri-saccadic perception of objects and space.

Eye movements affect object localization and object recognition. Around saccade onset, briefly flashed stimuli appear compressed towards the saccade target, receptive fields dynamically change position, and the recognition of objects near the saccade target is improved. These effects have been attributed to different mechanisms. We provide a unifying account of peri-saccadic perception explaining all three phenomena by a quantitative computational approach simulating cortical cell responses on the population level. Contrary to the common view of spatial attention as a spotlight, our model suggests that oculomotor feedback alters the receptive field structure in multiple visual areas at an intermediate level of the cortical hierarchy to dynamically recruit cells for processing a relevant part of the visual field. The compression of visual space occurs at the expense of this locally enhanced processing capacity.

About the influence of post-saccadic mechanisms for visual stability on peri-saccadic compression of object location.

Peri-saccadic perception experiments have revealed a multitude of mislocalization phenomena. For instance, a briefly flashed stimulus is perceived closer to the saccade target, whereas a displacement of the saccade target goes usually unnoticeable. This latter saccadic suppression of displacement has been explained by a built-in characteristic of the perceptual system: the assumption that during a saccade, the environment remains stable. We explored whether the mislocalization of a briefly flashed stimulus toward the saccade target also grounds in the built-in assumption of a stable environment. If the mislocalization of a peri-saccadically flashed stimulus originates from a post-saccadic alignment process, an additional location marker at the position of the upcoming flash should counteract compression. Alternatively, compression might be the result of peri-saccadic attentional phenomena. In this case, mislocalization should occur even if the position of the flashed stimulus is marked. When subjects were asked about their perceived location, they mislocalized the stimulus toward the saccade target, even though they were fully aware of the correct stimulus location. Thus, our results suggest that the uncertainty about the location of a flashed stimulus is not inherently relevant for compression.

V4 receptive field dynamics as predicted by a systems-level model of visual attention using feedback from the frontal eye field.

Visual attention is generally considered to facilitate the processing of the attended stimulus. Its mechanisms, however, are still under debate. We have developed a systems-level model of visual attention which predicts that attentive effects emerge by the interactions between different brain areas. Recent physiological studies have provided evidence that attention also alters the receptive field structure. For example, V4 receptive fields typically shrink and shift towards the saccade target around saccade onset. We show that receptive field dynamics are inherently predicted by the mechanism of feedback in our model. According to the model an oculomotor feedback signal from an area involved in the competition for the saccade target location, e.g. the frontal eye field, enhances the gain of V4 cells. V4 receptive field dynamics can be observed after pooling the gain modulated responses to obtain a certain degree of spatial invariance. The time course of the receptive field dynamics in the model resemble those obtained from macaque V4.