Perceptual learning across saccades: Feature but not location specific.

Perceptual learning is the ability to enhance perception through practice. The hallmark of perceptual learning is its specificity for the trained location and stimulus features, such as orientation. For example, training in discriminating a grating's orientation improves performance only at the trained location but not in other untrained locations. Perceptual learning has mostly been studied using stimuli presented briefly while observers maintained gaze at one location. However, in everyday life, stimuli are actively explored through eye movements, which results in successive projections of the same stimulus at different retinal locations. Here, we studied perceptual learning of orientation discrimination across saccades. Observers were trained to saccade to a peripheral grating and to discriminate its orientation change that occurred during the saccade. The results showed that training led to transsaccadic perceptual learning (TPL) and performance improvements which did not generalize to an untrained orientation. Remarkably, however, for the trained orientation, we found a complete transfer of TPL to the untrained location in the opposite hemifield suggesting high flexibility of reference frame encoding in TPL. Three control experiments in which participants were trained without saccades did not show such transfer, confirming that the location transfer was contingent upon eye movements. Moreover, performance at the trained location, but not at the untrained location, was also improved in an untrained fixation task. Our results suggest that TPL has both, a location-specific component that occurs before the eye movement and a saccade-related component that involves location generalization.

A dynamic 1/f noise protocol to assess visual attention without biasing perceptual processing.

Psychophysical paradigms measure visual attention via localized test items to which observers must react or whose features have to be discriminated. These items, however, potentially interfere with the intended measurement, as they bias observers' spatial and temporal attention to their location and presentation time. Furthermore, visual sensitivity for conventional test items naturally decreases with retinal eccentricity, which prevents direct comparison of central and peripheral attention assessments. We developed a stimulus that overcomes these limitations. A brief oriented discrimination signal is seamlessly embedded into a continuously changing 1/f noise field, such that observers cannot anticipate potential test locations or times. Using our new protocol, we demonstrate that local orientation discrimination accuracy for 1/f filtered signals is largely independent of retinal eccentricity. Moreover, we show that items present in the visual field indeed shape the distribution of visual attention, suggesting that classical studies investigating the spatiotemporal dynamics of visual attention via localized test items may have obtained a biased measure. We recommend our protocol as an efficient method to evaluate the behavioral and neurophysiological correlates of attentional orienting across space and time.

The effect of spatial structure on presaccadic attention costs and benefits assessed with dynamic 1/f noise.

Already before the onset of a saccadic eye movement, we preferentially process visual information at the upcoming eye fixation. This 'presaccadic shift of attention' is typically assessed via localized test items, which potentially bias the attention measurement. Here we show how presaccadic attention shapes perception from saccade origin to target when no scene-structuring items are presented. Participants made saccades into a 1/f ('pink') noise field, in which we embedded a brief orientation signal at various locations shortly before saccade onset. Local orientation discrimination performance served as a proxy for the allocation of attention. Results demonstrate that (1) the presaccadic attention shift is accompanied by considerable attentional costs at the presaccadic eye fixation; (2) saccades are preceded by shifts of attention to their goal location even if they are directed into an unstructured visual field, but the spread of attention, compared to target-directed saccades, is broad; We conclude that the absence or presence of saccade target objects markedly shapes the distribution of presaccadic attention, and likely the underlying (space-based or object-based) cortical control mechanism. Our findings demonstrate the relevance of an item-free approach for measuring attentional dynamics across the visual field.

Eye and hand movements disrupt attentional control.

Voluntary attentional control is the ability to selectively focus on a subset of visual information in the presence of other competing stimuli-a marker of cognitive control enabling flexible, goal-driven behavior. To test its robustness, we contrasted attentional control with the most common source of attentional orienting in daily life: attention shifts prior to goal-directed eye and hand movements. In a multi-tasking paradigm, human participants attended at a location while planning eye or hand movements elsewhere. Voluntary attentional control suffered with every simultaneous action plan, even under reduced task difficulty and memory load-factors known to interfere with attentional control. Furthermore, the performance cost was limited to voluntary attention: We observed simultaneous attention benefits at two movement targets without attentional competition between them. This demonstrates that the visual system allows for the concurrent representation of multiple attentional foci. Since attentional control is extremely fragile and dominated by premotor attention shifts, we propose that action-driven selection plays the superordinate role for visual selection.

Sounds are remapped across saccades.

To achieve visual space constancy, our brain remaps eye-centered projections of visual objects across saccades. Here, we measured saccade trajectory curvature following the presentation of visual, auditory, and audiovisual distractors in a double-step saccade task to investigate if this stability mechanism also accounts for localized sounds. We found that saccade trajectories systematically curved away from the position at which either a light or a sound was presented, suggesting that both modalities are represented in eye-centered oculomotor centers. Importantly, the same effect was observed when the distractor preceded the execution of the first saccade. These results suggest that oculomotor centers keep track of visual, auditory and audiovisual objects by remapping their eye-centered representations across saccades. Furthermore, they argue for the existence of a supra-modal map which keeps track of multi-sensory object locations across our movements to create an impression of space constancy.

Attention capture outside the oculomotor range.

Neurophysiological studies have demonstrated that attentional orienting is associated with activity in fronto-parietal brain areas that play a pivotal role in oculomotor control, such as the lateral intraparietal cortex (LIP), the frontal eye fields (FEF), and the superior colliculus (SC) (e.g., [1]). Accordingly, based on the influential premotor theory of attention, which posits that even covert shifts of spatial attention in the absence of eye movements are elicited by preceding activation in the oculomotor system [2], it has been claimed that attention can only be allocated to where we can potentially make an eye movement [3]. There are two forms of covert spatial attention: exogenous attention is automatic, stimulus-driven, and transiently deployed in ∼100 ms. Conversely, endogenous attention is voluntary, goal-driven, and deployed in a slower (∼300 ms) and sustained manner [4]. Notably, it has been postulated that only exogenous attention, but not endogenous attention, would be restricted to locations within the so-called oculomotor range that is accessible by saccadic eye movements [5,6]. To test this claim, we used a dissociation approach that allowed us to evaluate exogenous attention shifts to locations within and beyond observers' oculomotor range via their disruptive, attention capturing costs for endogenous attention. We found that salient events equally grab exogenous attention both inside and outside the oculomotor range, demonstrating that exogenous attention can shift to locations not reachable by the eyes.

Stimulus blanking reveals contrast-dependent transsaccadic feature transfer.

Across saccadic eye movements, the visual system receives two successive static images corresponding to the pre- and the postsaccadic projections of the visual field on the retina. The existence of a mechanism integrating the content of these images is today still a matter of debate. Here, we studied the transfer of a visual feature across saccades using a blanking paradigm. Participants moved their eyes to a peripheral grating and discriminated a change in its orientation occurring during the eye movement. The grating was either constantly on the screen or briefly blanked during and after the saccade. Moreover, it either was of the same luminance as the background (i.e., isoluminant) or anisoluminant with respect to it. We found that for anisoluminant gratings, the orientation discrimination across saccades was improved when a blank followed the onset of the eye movement. Such effect was however abolished with isoluminant gratings. Additionally, performance was also improved when an anisoluminant grating presented before the saccade was followed by an isoluminant one. These results demonstrate that a detailed representation of the presaccadic image was transferred across saccades allowing participants to perform better on the transsaccadic orientation task. While such a transfer of visual orientation across saccade is masked in real-life anisoluminant conditions, the use of a blank and of an isoluminant postsaccadic grating allowed to reveal its existence.

Theory of visual attention (TVA) in action: Assessing premotor attention in simultaneous eye-hand movements.

Attention shifts that precede goal-directed eye and hand movements are regarded as markers of motor target selection. Whether effectors compete for a single, shared attentional resource during simultaneous eye-hand movements or whether attentional resources can be allocated independently towards multiple target locations is controversially debated. Independent, effector-specific target selection mechanisms underlying parallel allocation of visuospatial attention to saccade and reach targets would predict an increase of the overall attention capacity with the number of active effectors. We test this hypothesis in a modified Theory of Visual Attention (TVA; Bundesen, 1990) paradigm. Participants reported briefly presented letters during eye, hand, or combined eye-hand movement preparation to centrally cued locations. Modeling the data according to TVA allowed us to assess both the overall attention capacity and the deployment of visual attention to individual locations in the visual work space. In two experiments, we show that attention is predominantly allocated to the motor targets-without pronounced competition between effectors. The parallel benefits at eye and hand targets, however, have concomitant costs at non-motor locations, and the overall attention capacity does not increase by the simultaneous recruitment of both effector systems. Moreover, premotor shifts of attention dominate over voluntary deployment of processing resources, yielding severe impairments of voluntary attention allocation. We conclude that attention shifts to multiple effector targets without mutual competition given that sufficient processing resources can be withdrawn from movement-irrelevant locations.

Visual attention and eye movement control during oculomotor competition.

Saccadic eye movements are typically preceded by selective shifts of visual attention. Recent evidence, however, suggests that oculomotor selection can occur in the absence of attentional selection when saccades erroneously land in between nearby competing objects (saccade averaging). This study combined a saccade task with a visual discrimination task to investigate saccade target selection during episodes of competition between a saccade target and a nearby distractor. We manipulated the spatial predictability of target and distractor locations and asked participants to execute saccades upon variably delayed go-signals. This allowed us to systematically investigate the capacity to exert top-down eye movement control (as reflected in saccade endpoints) based on the spatiotemporal dynamics of visual attention during movement preparation (measured as visual sensitivity). Our data demonstrate that the predictability of target and distractor locations, despite not affecting the deployment of visual attention prior to movement preparation, largely improved the accuracy of short-latency saccades. Under spatial uncertainty, a short go-signal delay likewise enhanced saccade accuracy substantially, which was associated with a more selective deployment of attentional resources to the saccade target. Moreover, we observed a systematic relationship between the deployment of visual attention and saccade accuracy, with visual discrimination performance being significantly enhanced at the saccade target relative to the distractor only before the execution of saccades accurately landing at the saccade target. Our results provide novel insights linking top-down eye movement control to the operation of selective visual attention during movement preparation.

Displacement detection is suppressed by the post-saccadic stimulus.

To establish a perceptually stable world despite the large retinal shifts caused by saccadic eye movements, the visual system reduces its sensitivity to the displacement of visual stimuli during saccades (e.g. saccadic suppression of displacement, SSD). Previous studies have demonstrated that inserting a temporal blank right after a saccade improves displacement detection performance. This 'blanking effect' suggests that visual information right after the saccade may play an important role in SSD. To understand the mechanisms underlying SSD, we here compare the effect of pre- and post-saccadic stimulus contrast on displacement detection during a saccade with and without inserting a blank. Our results show that observers' sensitivity to detect visual displacement was reduced by increasing post-saccadic stimulus contrast, but a blank relieves the impairment. We successfully explain the results with a model proposing that parvo-pathway signals suppress the magno-pathway processes responsible for detecting displacements across saccades. Our results suggest that the suppression of the magno-pathway by parvo-pathway signals immediately after a saccade causes SSD, which helps to achieve the perceptual stability of the visual world across saccades.

Spatiotopic and saccade-specific transsaccadic memory for object detail.

The content and nature of transsaccadic memory are still a matter of debate. Brief postsaccadic target blanking was demonstrated to recover transsaccadic memory and defeat saccadic suppression of displacement. We examined whether blanking would also support transsaccadic transfer of detailed form information. Observers saccaded to a peripheral, checkerboard-like stimulus and reported whether an intrasaccadic change had occurred in its upper or lower half. On half of the trials, the stimulus was blanked for 200 ms with saccade onset. In a fixation condition, observers kept fixation but the stimulus was displaced from periphery to fixation, mimicking the retinal events of the saccade condition. Results show that stimulus blanking improves transsaccadic change detection, with performance being far superior to the retinally equivalent fixation condition. Our findings argue in favor of a remapped memory trace that can be accessed only in the blanking condition, when not being overwritten by the salient postsaccadic stimulus.

Sensitivity measures of visuospatial attention.

Measuring visual sensitivity has become popular to determine the spatial deployment of visual attention. Critically, the accuracy of the measurement depends on the quality of the stimulus used. We evaluated the strengths and weaknesses of six commonly used stimuli for assessing visual attention. While preparing an eye movement to a cued item, participants discriminated a stimulus-specific visual feature, either at the cued location or at other equidistant uncued locations. Stimuli differed in their visual features (digital letters, Gabors, crosses, pink noise, random dot kinematograms, and Gabor streams) and their presentation mode (static or dynamic stimuli). We evaluated these stimuli regarding their temporal and spatial specificity and their impact on saccade preparation. We assessed presaccadic visual sensitivity as a correlate of visual spatial attention and discuss the stimulus-specific time course, spatial specificity, and magnitude of the measured attention modulation. Irrespective of the stimulus type, we observed a clear increase of visual sensitivity at the cued location. Time course, spatial specificity, and magnitude of this improvement, however, were specific to each stimulus. Based on our findings, we present guidelines to select the stimulus best suited to measure visuospatial attention depending on the respective research question.

The spread of presaccadic attention depends on the spatial configuration of the visual scene.

When preparing a saccade, attentional resources are focused at the saccade target and its immediate vicinity. Here we show that this does not hold true when saccades are prepared toward a recently extinguished target. We obtained detailed maps of orientation sensitivity when participants prepared a saccade toward a target that either remained on the screen or disappeared before the eyes moved. We found that attention was mainly focused on the immediate surround of the visible target and spread to more peripheral locations as a function of the distance from the cue and the delay between the target's disappearance and the saccade. Interestingly, this spread was not accompanied with a spread of the saccade endpoint. These results suggest that presaccadic attention and saccade programming are two distinct processes that can be dissociated as a function of their interaction with the spatial configuration of the visual scene.

Investigating the Deployment of Visual Attention Before Accurate and Averaging Saccades via Eye Tracking and Assessment of Visual Sensitivity.

This experimental protocol was designed to investigate whether visual attention is obligatorily deployed at the endpoint of saccades. To this end, we recorded the eye position of human participants engaged in a saccade task via eye tracking and assessed visual orientation discrimination performance at various locations during saccade preparation. Importantly, instead of using a single saccade target paradigm for which the saccade endpoint typically coincides roughly with the target, this protocol comprised the presentation of two nearby saccade targets, leading to a distinct spatial dissociation between target locations and saccade endpoint on a substantial number of trials. The paradigm allowed us to compare presaccadic visual discrimination performance at the endpoint of accurate saccades (landing at one of the saccade targets) and of averaging saccades (landing at an intermediate location in between the two targets). We observed a selective enhancement of visual sensitivity at the endpoint of accurate saccades but not at the endpoint of averaging saccades. Rather, before the execution of averaging saccades, visual sensitivity was equally enhanced at both targets, suggesting that saccade averaging follows from unresolved attentional selection among the saccade targets. These results argue against a mandatory coupling between visual attention and saccade programming based on a direct measure of presaccadic visual sensitivity rather than saccadic reaction times, which have been used in other protocols to draw similar conclusions. While our protocol provides a useful framework to investigate the relationship between visual attention and saccadic eye movements at the behavioral level, it can also be combined with electrophysiological measures to extend insights at the neuronal level.

Visual attention is not limited to the oculomotor range.

Both patients with eye movement disorders and healthy participants whose oculomotor range had been experimentally reduced have been reported to show attentional deficits at locations unreachable by their eyes. Whereas previous studies were mainly based on the evaluation of reaction times, we measured visual sensitivity before saccadic eye movements and during fixation at locations either within or beyond participants' oculomotor range. Participants rotated their heads to prevent them from performing large rightward saccades. In this posture, an attentional cue was presented inside or outside their oculomotor range. Participants either made a saccade to the cue or maintained fixation while they discriminated the orientation of a visual noise patch. In contrast to previous reports, we found that the cue attracted visual attention regardless of whether it was presented within or beyond participants' oculomotor range during both fixation and saccade preparation. Moreover, when participants aimed to look to a cue that they could not reach with their eyes, we observed no benefit at their actual saccade endpoint. This demonstrates that spatial attention is not coupled to the executed oculomotor program but instead can be deployed unrestrictedly also toward locations to which no saccade can be executed. Our results are compatible with the view that covert and overt attentional orienting are guided by feedback projections of visual and visuomotor neurons of the gaze control system, irrespective of oculomotor limitations.

Saccade selection and inhibition: motor and attentional components.

Motor responses are fundamentally spatial in their function and neural organization. However, studies of inhibitory motor control, focused on global stopping of all actions, have ignored whether inhibitory control can be exercised selectively for specific actions. We used a new approach to elicit and measure motor inhibition by asking human participants to either look at (select) or avoid looking at (inhibit) a location in space. We found that instructing a location to be avoided resulted in an inhibitory bias specific to that location. When compared with the facilitatory bias observed in the Look task, it differed significantly in both its spatiotemporal dynamics and its modulation of attentional processing. While action selection was evident in oculomotor system and interacted with attentional processing, action inhibition was evident mainly in the oculomotor system. Our findings suggest that action inhibition is implemented by spatially specific mechanisms that are separate from action selection. NEW & NOTEWORTHY We show that cognitive control of saccadic responses evokes separable action selection and inhibition processes. Both action selection and inhibition are represented in the saccadic system, but only action selection interacts with the attentional system.

Independent Effects of Eye and Hand Movements on Visual Working Memory.

Both eye and hand movements have been shown to selectively interfere with visual working memory. We investigated working memory in the context of simultaneous eye-hand movements to approach the question whether the eye and the hand movement systems independently interact with visual working memory. Participants memorized several locations and performed eye, hand, or simultaneous eye-hand movements during the maintenance interval. Subsequently, we tested spatial working memory at the eye or the hand motor goal, and at action-irrelevant locations. We found that for single eye and single hand movements, memory at the eye or hand target was significantly improved compared to action-irrelevant locations. Remarkably, when an eye and a hand movement were prepared in parallel, but to distinct locations, memory at both motor targets was enhanced-with no tradeoff between the two separate action goals. This suggests that eye and hand movements independently enhance visual working memory at their goal locations, resulting in an overall working memory performance that is higher than that expected when recruiting only one effector.

The influence of obstacles on grasp planning.

When reaching to grasp for an object in the absence of obstacles, the choice of contact points is highly consistent within and between healthy humans, suggesting a preplanning of grasping movements (Gilster et al. in Exp Brain Res 217:137-151, 2012). In real life, objects may obstruct the favored contact points at a target object, requiring adjustments to avoid collision. In the present study, we investigated how an obstacle that directly obstructs the favored contact points for two-digit grasping changes the planning and execution of reach-to-grasp movements. Furthermore, we elucidated to what extent an obstacle placed at various angular positions around the target object (thereby not directly obstructing the favored contact points) still influences trajectories, contact points, and time-related parameters. When obstacles directly obstructed favored contact points participants either chose a completely new contact point or grasped the object only slightly away from the favored contact point. Obstacles located near the favored contact points but not directly obstructing them still resulted in a repulsive effect, meaning that contact points were shifted away from the obstacle to ensure sufficient distance to the obstacle. We found that the position of an obstacle even influences the direction in which the fingers set off. This leads to a deviation of the trajectory very early in the time course, yielding longer movement times if the main contact points are obstructed. Taken together, the early significant influence of obstacles on the grasping movement supports the assumption that grasping movements are preplanned.

Independent selection of eye and hand targets suggests effector-specific attentional mechanisms.

Both eye and hand movements bind visual attention to their target locations during movement preparation. However, it remains contentious whether eye and hand targets are selected jointly by a single selection system, or individually by independent systems. To unravel the controversy, we investigated the deployment of visual attention - a proxy of motor target selection - in coordinated eye-hand movements. Results show that attention builds up in parallel both at the eye and the hand target. Importantly, the allocation of attention to one effector's motor target was not affected by the concurrent preparation of the other effector's movement at any time during movement preparation. This demonstrates that eye and hand targets are represented in separate, effector-specific maps of action-relevant locations. The eye-hand synchronisation that is frequently observed on the behavioral level must emerge from mutual influences of the two effector systems at later, post-attentional processing stages.

Visual attention is not deployed at the endpoint of averaging saccades.

The premotor theory of attention postulates that spatial attention arises from the activation of saccade areas and that the deployment of attention is the consequence of motor programming. Yet attentional and oculomotor processes have been shown to be dissociable at the neuronal level in covert attention tasks. To investigate a potential dissociation at the behavioral level, we instructed human participants to move their eyes (saccade) towards 1 of 2 nearby, competing saccade targets. The spatial distribution of visual attention was determined using oriented visual stimuli presented either at the target locations, between them, or at several other equidistant locations. Results demonstrate that accurate saccades towards one of the targets were associated with presaccadic enhancement of visual sensitivity at the respective saccade endpoint compared to the nonsaccaded target location. In contrast, averaging saccades, landing between the 2 targets, were not associated with attentional facilitation at the saccade endpoint. Rather, attention before averaging saccades was equally deployed at the 2 target locations. Taken together, our results reveal that visual attention is not obligatorily coupled to the endpoint of a subsequent saccade. Rather, our results suggest that the oculomotor program depends on the state of attentional selection before saccade onset and that saccade averaging arises from unresolved attentional selection.