The role of prediction and visual tracking strategies during manual interception: An exploration of individual differences.

The interception (or avoidance) of moving objects is a common component of various daily living tasks; however, it remains unclear whether precise alignment of foveal vision with a target is important for motor performance. Furthermore, there has also been little examination of individual differences in visual tracking strategy and the use of anticipatory gaze adjustments. We examined the importance of in-flight tracking and predictive visual behaviors using a virtual reality environment that required participants (n = 41) to intercept tennis balls projected from one of two possible locations. Here, we explored whether different tracking strategies spontaneously arose during the task, and which were most effective. Although indices of closer in-flight tracking (pursuit gain, tracking coherence, tracking lag, and saccades) were predictive of better interception performance, these relationships were rather weak. Anticipatory gaze shifts toward the correct release location of the ball provided no benefit for subsequent interception. Nonetheless, two interceptive strategies were evident: 1) early anticipation of the ball's onset location followed by attempts to closely track the ball in flight (i.e., predictive strategy); or 2) positioning gaze between possible onset locations and then using peripheral vision to locate the moving ball (i.e., a visual pivot strategy). Despite showing much poorer in-flight foveal tracking of the ball, participants adopting a visual pivot strategy performed slightly better in the task. Overall, these results indicate that precise alignment of the fovea with the target may not be critical for interception tasks, but that observers can adopt quite varied visual guidance approaches.

Classification of short and long term mild traumatic brain injury using computerized eye tracking.

Accurate, and objective diagnosis of brain injury remains challenging. This study evaluated useability and reliability of computerized eye-tracker assessments (CEAs) designed to assess oculomotor function, visual attention/processing, and selective attention in recent mild traumatic brain injury (mTBI), persistent post-concussion syndrome (PPCS), and controls. Tests included egocentric localisation, fixation-stability, smooth-pursuit, saccades, Stroop, and the vestibulo-ocular reflex (VOR). Thirty-five healthy adults performed the CEA battery twice to assess useability and test-retest reliability. In separate experiments, CEA data from 55 healthy, 20 mTBI, and 40 PPCS adults were used to train a machine learning model to categorize participants into control, mTBI, or PPCS classes. Intraclass correlation coefficients demonstrated moderate (ICC > .50) to excellent (ICC > .98) reliability (p < .05) and satisfactory CEA compliance. Machine learning modelling categorizing participants into groups of control, mTBI, and PPCS performed reasonably (balanced accuracy control: 0.83, mTBI: 0.66, and PPCS: 0.76, AUC-ROC: 0.82). Key outcomes were the VOR (gaze stability), fixation (vertical error), and pursuit (total error, vertical gain, and number of saccades). The CEA battery was reliable and able to differentiate healthy, mTBI, and PPCS patients reasonably well. While promising, the diagnostic model accuracy should be improved with a larger training dataset before use in clinical environments.

Column setting and text justification influence return-sweep eye movement behavior during Chinese multi-line reading.

People regularly read multi-line texts in different formats and publishers, internationally, must decide how to present text to make reading most effective and efficient. Relatively few studies have examined multi-line reading, and fewer still Chinese multi-line reading. Here, we examined whether texts presented in single or double columns, and either left-justified or fully-justified affect Chinese reading. Text format had minimal influence on overall reading time; however, it significantly impacted return-sweeps (large saccades moving the eyes from the end of one line of text to the beginning of the next). Return-sweeps were launched and landed further away from margins and involved more corrective saccades in single- than double-column format. For left- compared to fully-justified format, return-sweeps were launched and landed closer to margins. More corrective saccades also occurred. Our results showed more efficient return-sweep behavior for fully- than left-justified text. Moreover, there were clear trade-off effects such that formats requiring increased numbers of shorter return-sweeps produced more accurate targeting and reduced numbers of corrective fixations, whereas formats requiring reduced numbers of longer return-sweeps caused less accurate targeting and an increased rate of corrective fixations. Overall, our results demonstrate that text formats substantially affect return-sweep eye movement behavior during Chinese reading without affecting efficiency and effectiveness, that is, the overall time it takes to read and understand the text.

Dorsal pulvinar inactivation leads to spatial selection bias without perceptual deficit.

The dorsal pulvinar has been implicated in visuospatial attentional and perceptual confidence processing. Pulvinar lesions in humans and monkeys lead to spatial neglect symptoms, including an overt spatial saccade bias during free choices. However, it remains unclear whether disrupting the dorsal pulvinar during target selection that relies on a perceptual decision leads to a perceptual impairment or a more general spatial orienting and choice deficit. To address this question, we reversibly inactivated the unilateral dorsal pulvinar by injecting GABA-A agonist THIP while two macaque monkeys performed a color discrimination saccade task with varying perceptual difficulty. We used Signal Detection Theory and simulations to dissociate perceptual sensitivity (d-prime) and spatial selection bias (response criterion) effects. We expected a decrease in d-prime if dorsal pulvinar affects perceptual discrimination and a shift in response criterion if dorsal pulvinar is mainly involved in spatial orienting. After the inactivation, we observed response criterion shifts away from contralesional stimuli, especially when two competing stimuli in opposite hemifields were present. Notably, the d-prime and overall accuracy remained largely unaffected. Our results underline the critical contribution of the dorsal pulvinar to spatial orienting and action selection while showing it to be less important for visual perceptual discrimination.

Evidence of elevated situational awareness for active duty soldiers during navigation of a virtual environment.

U.S. service members maintain constant situational awareness (SA) due to training and experience operating in dynamic and complex environments. Work examining how military experience impacts SA during visual search of a complex naturalistic environment, is limited. Here, we compare Active Duty service members and Civilians' physiological behavior during a navigational visual search task in an open-world virtual environment (VE) while cognitive load was manipulated. We measured eye-tracking and electroencephalogram (EEG) outcomes from Active Duty (N = 21) and Civilians (N = 15) while they navigated a desktop VE at a self-regulated pace. Participants searched and counted targets (N = 15) presented among distractors, while cognitive load was manipulated with an auditory Math Task. Results showed Active Duty participants reported significantly greater/closer to the correct number of targets compared to Civilians. Overall, Active Duty participants scanned the VE with faster peak saccade velocities and greater average saccade magnitudes compared to Civilians. Convolutional Neural Network (CNN) response (EEG P-300) was significantly weighted more to initial fixations for the Active Duty group, showing reduced attentional resources on object refixations compared to Civilians. There were no group differences in fixation outcomes or overall CNN response when comparing targets versus distractor objects. When cognitive load was manipulated, only Civilians significantly decreased their average dwell time on each object and the Active Duty group had significantly fewer numbers of correct answers on the Math Task. Overall, the Active Duty group explored the VE with increased scanning speed and distance and reduced cognitive re-processing on objects, employing a different, perhaps expert, visual search strategy indicative of increased SA. The Active Duty group maintained SA in the main visual search task and did not appear to shift focus to the secondary Math Task. Future work could compare how a stress inducing environment impacts these groups' physiological or cognitive markers and performance for these groups.

Clinical value of video oculomotor evaluation in the differential diagnosis of multiple system atrophy and Parkinson's disease.

BACKGROUND: Multiple system atrophy (MSA) is a neurodegenerative disease that progresses rapidly and has a poor prognosis. This study aimed to assess the value of video oculomotor evaluation (VOE) in the differential diagnosis of MSA and Parkinson's disease (PD). METHODS: In total, 28 patients with MSA, 31 patients with PD, and 30 age- and sex-matched healthy controls (HC) were screened and included in this study. The evaluation consisted of a gaze-holding test, smooth pursuit eye movement (SPEM), random saccade, and optokinetic nystagmus (OKN). RESULTS: The MSA and PD groups had more abnormalities and decreased SPEM gain than the HC group (64.29%, 35.48%, 10%, p < .001). The SPEM gain in the MSA group was significantly lower than that in the PD group at specific frequencies. Patients with MSA and PD showed prolonged latencies in all saccade directions compared with those with HC. However, the two diseases had no significant differences in the saccade parameters. The OKN gain gradually decreased from the HC to the PD and the MSA groups (p < .05). Compared with the PD group, the gain in the MSA group was further decreased in the OKN test at 30°/s (Left, p = .010; Right p = .016). Receiver operating characteristic curves showed that the combination of oculomotor parameters with age and course of disease could aid in the differential diagnosis of patients with MSA and PD, with a sensitivity of 89.29% and a specificity of 70.97%. CONCLUSIONS: The combination of oculomotor parameters and clinical data may aid in the differential diagnosis of MSA and PD. Furthermore, VOE is vital in the identification of neurodegenerative diseases.

Infants' reorienting efficiency depends on parental autistic traits and predicts future socio-communicative behaviors.

Attentional reorienting is dysfunctional not only in children with autism spectrum disorder (ASD), but also in infants who will develop ASD, thus constituting a potential causal factor of future social interaction and communication abilities. Following the research domain criteria framework, we hypothesized that the presence of subclinical autistic traits in parents should lead to atypical infants' attentional reorienting, which in turn should impact on their future socio-communication behavior in toddlerhood. During an attentional cueing task, we measured the saccadic latencies in a large sample (total enrolled n = 89; final sample n = 71) of 8-month-old infants from the general population as a proxy for their stimulus-driven attention. Infants were grouped in a high parental traits (HPT; n = 23) or in a low parental traits (LPT; n = 48) group, according to the degree of autistic traits self-reported by their parents. Infants (n = 33) were then longitudinally followed to test their socio-communicative behaviors at 21 months. Results show a sluggish reorienting system, which was a longitudinal predictor of future socio-communicative skills at 21 months. Our combined transgenerational and longitudinal findings suggest that the early functionality of the stimulus-driven attentional network-redirecting attention from one event to another-could be directly connected to future social and communication development.

Jointly looking to the past and the future in visual working memory.

Working memory enables us to bridge past sensory information to upcoming future behaviour. Accordingly, by its very nature, working memory is concerned with two components: the past and the future. Yet, in conventional laboratory tasks, these two components are often conflated, such as when sensory information in working memory is encoded and tested at the same location. We developed a task in which we dissociated the past (encoded location) and future (to-be-tested location) attributes of visual contents in working memory. This enabled us to independently track the utilisation of past and future memory attributes through gaze, as observed during mnemonic selection. Our results reveal the joint consideration of past and future locations. This was prevalent even at the single-trial level of individual saccades that were jointly biased to the past and future. This uncovers the rich nature of working memory representations, whereby both past and future memory attributes are retained and can be accessed together when memory contents become relevant for behaviour.

Oculomotor behavior can be adjusted on the basis of artificial feedback signals indicating externally caused errors.

Whether a saccade is accurate and has reached the target cannot be evaluated during its execution, but relies on post-saccadic feedback. If the eye has missed the target object, a secondary corrective saccade has to be made to align the fovea with the target. If a systematic post-saccadic error occurs, adaptive changes to the oculomotor behavior are made, such as shortening or lengthening the saccade amplitude. Systematic post-saccadic errors are typically attributed internally to erroneous motor commands. The corresponding adaptive changes to the motor command reduce the error and the need for secondary corrective saccades, and, in doing so, restore accuracy and efficiency. However, adaptive changes to the oculomotor behavior also occur if a change in saccade amplitude is beneficial for task performance, or if it is rewarded. Oculomotor learning thus is more complex than reducing a post-saccadic position error. In the current study, we used a novel oculomotor learning paradigm and investigated whether human participants are able to adapt their oculomotor behavior to improve task performance even when they attribute the error externally. The task was to indicate the intended target object among several objects to a simulated human-machine interface by making eye movements. The participants were informed that the system itself could make errors. The decoding process depended on a distorted landing point of the saccade, resulting in decoding errors. Two different types of visual feedback were added to the post-saccadic scene and we compared how participants used the different feedback types to adjust their oculomotor behavior to avoid errors. We found that task performance improved over time, regardless of the type of feedback. Thus, error feedback from the simulated human-machine interface was used for post-saccadic error evaluation. This indicates that 1) artificial visual feedback signals and 2) externally caused errors might drive adaptive changes to oculomotor behavior.

Microsaccade-inspired event camera for robotics.

Neuromorphic vision sensors or event cameras have made the visual perception of extremely low reaction time possible, opening new avenues for high-dynamic robotics applications. These event cameras' output is dependent on both motion and texture. However, the event camera fails to capture object edges that are parallel to the camera motion. This is a problem intrinsic to the sensor and therefore challenging to solve algorithmically. Human vision deals with perceptual fading using the active mechanism of small involuntary eye movements, the most prominent ones called microsaccades. By moving the eyes constantly and slightly during fixation, microsaccades can substantially maintain texture stability and persistence. Inspired by microsaccades, we designed an event-based perception system capable of simultaneously maintaining low reaction time and stable texture. In this design, a rotating wedge prism was mounted in front of the aperture of an event camera to redirect light and trigger events. The geometrical optics of the rotating wedge prism allows for algorithmic compensation of the additional rotational motion, resulting in a stable texture appearance and high informational output independent of external motion. The hardware device and software solution are integrated into a system, which we call artificial microsaccade-enhanced event camera (AMI-EV). Benchmark comparisons validated the superior data quality of AMI-EV recordings in scenarios where both standard cameras and event cameras fail to deliver. Various real-world experiments demonstrated the potential of the system to facilitate robotics perception both for low-level and high-level vision tasks.

The impact of presentation modes on mental rotation processing: a comparative analysis of eye movements and performance.

Mental rotation is the ability to rotate mental representations of objects in space. Shepard and Metzler's shape-matching tasks, frequently used to test mental rotation, involve presenting pictorial representations of 3D objects. This stimulus material has raised questions regarding the ecological validity of the test for mental rotation with actual visual 3D objects. To systematically investigate differences in mental rotation with pictorial and visual stimuli, we compared data of N = 54 university students from a virtual reality experiment. Comparing both conditions within subjects, we found higher accuracy and faster reaction times for 3D visual figures. We expected eye tracking to reveal differences in participants' stimulus processing and mental rotation strategies induced by the visual differences. We statistically compared fixations (locations), saccades (directions), pupil changes, and head movements. Supplementary Shapley values of a Gradient Boosting Decision Tree algorithm were analyzed, which correctly classified the two conditions using eye and head movements. The results indicated that with visual 3D figures, the encoding of spatial information was less demanding, and participants may have used egocentric transformations and perspective changes. Moreover, participants showed eye movements associated with more holistic processing for visual 3D figures and more piecemeal processing for pictorial 2D figures.

Effects of post-saccadic oscillations on visual processing times.

Saccadic eye movements enable us to search for the target of interest in a crowded scene or, in the case of goal-directed saccades, to simply bring the image of the peripheral target to the very centre of the fovea. This mechanism extends the use of the superior image processing performance of the fovea over a large visual field. We know that visual information is processed quickly at the end of each saccade but estimates of the times involved remain controversial. This study aims to investigate the processing of visual information during post fixation oscillations of the eyeball. A new psychophysical test measures the combined eye movement response latencies, including fixation duration and visual processing times. When the test is used in conjunction with an eye tracker, each component that makes up the 'integrated saccade latency' time, from the onset of the peripheral stimulus to the correct interpretation of the information carried by the stimulus, can be measured and the discrete components delineated. The results show that the time required to process and encode the stimulus attribute of interest at the end of a saccade is longer than the time needed to carry out the same task in the absence of an eye movement. We propose two principal hypotheses, each of which can account for this finding. 1. The known inhibition of afferent retinal signals during fast eye movements extends beyond the end point of the saccade. 2. The extended visual processing times measured when saccades are involved are caused by the transient loss of spatial resolution due to eyeball instability during post-saccadic oscillations. The latter can best be described as retinal image smear with greater loss of spatial resolution expected for stimuli of low luminance contrast.

Decision-related activity and movement selection in primate visual cortex.

Fluctuations in the activity of sensory neurons often predict perceptual decisions. This connection can be quantified with a metric called choice probability (CP), and there is a longstanding debate about whether CP reflects a causal influence on decisions or an echo of decision-making activity elsewhere in the brain. Here, we show that CP can reflect a third variable, namely, the movement used to indicate the decision. In a standard visual motion discrimination task, neurons in the middle temporal (MT) area of primate cortex responded more strongly during trials that involved a saccade toward their receptive fields. This variability accounted for much of the CP observed across the neuronal population, and it arose through training. Moreover, pharmacological inactivation of MT biased behavioral responses away from the corresponding visual field locations. These results demonstrate that training on a task with fixed sensorimotor contingencies introduces movement-related activity in sensory brain regions and that this plasticity can shape the neural circuitry of perceptual decision-making.

Post-Saccadic Oscillations of the Pupil and Lens Reduce Fixation Stability in Retinitis Pigmentosa and Age-Related Macular Degeneration.

Purpose: Post-saccadic oscillations (PSOs) reflect movements of gaze that result from motion of the pupil and lens relative to the eyeball rather than eyeball rotations. Here, we analyzed the characteristics of PSOs in subjects with age-related macular degeneration (AMD), retinitis pigmentosa (RP), and normal vision (NV). Our aim was to assess the differences in PSOs between people with vision loss and healthy controls because PSOs affect retinal image stability after each saccade. Methods: Participants completed a horizontal saccade task and their gaze was measured using a pupil-based eye tracker. Oscillations occurring in the 80 to 200 ms post-saccadic period were described with a damped oscillation model. We compared the amplitude, decay time constant, and frequency of the PSOs for the three different groups. We also examined the correlation between these PSO parameters and the amplitude, peak velocity, and final deceleration of the preceding saccades. Results: Subjects with vision loss (AMD, n = 6, and RP, n = 5) had larger oscillation amplitudes, longer decay constants, and lower frequencies than subjects with NV (n = 7). The oscillation amplitudes increased with increases in saccade deceleration in all three groups. The other PSO parameters, however, did not show consistent correlations with either saccade amplitude or peak velocity. Conclusions: Post-saccadic fixation stability in AMD and RP is reduced due to abnormal PSOs. The differences with respect to NV are not due to differences in saccade kinematics, suggesting that anatomic and neuronal variations affect the suspension of the iris and the lens in the patients' eyes.

The execution of saccadic eye movements suppresses visual processing of both color and luminance in the early visual cortex of humans.

Our eyes execute rapid, directional movements known as saccades, occurring several times per second, to focus on objects of interest in our environment. During these movements, visual sensitivity is temporarily reduced. Despite numerous studies on this topic, the underlying mechanism remains elusive, including a lingering debate on whether saccadic suppression affects the parvocellular visual pathway. To address this issue, we conducted a study employing steady-state visual evoked potentials (SSVEPs) elicited by chromatic and luminance stimuli while observers performed saccadic eye movements. We also employed an innovative analysis pipeline to enhance the signal-to-noise ratio, yielding superior results compared to the previous method. Our findings revealed a clear suppression effect on SSVEP signals during saccades compared to fixation periods. Notably, this suppression effect was comparable for both chromatic and luminance stimuli. We went further to measure the suppression effect across various contrast levels, which enabled us to model SSVEP responses with contrast response functions. The results suggest that saccades primarily reduce response gain without significantly affecting contrast gain and that this reduction applies uniformly to both chromatic and luminance pathways. In summary, our study provides robust evidence that saccades similarly suppress visual processing in both the parvocellular and magnocellular pathways within the human early visual cortex, as indicated by SSVEP responses. The observation that saccadic eye movements impact response gain rather than contrast gain implies that they influence visual processing through a multiplicative mechanism.NEW & NOTEWORTHY The present study demonstrates that saccadic eye movements reduce the processing of both luminance and chromatic stimuli in the early visual cortex of humans. By modeling the contrast response function, the study further shows that saccades affect visual processing by reducing the response gain rather than altering the contrast gain, suggesting that a multiplicative mechanism of visual attenuation affects both parvocellular and magnocellular pathways.

Oculomotor Dysfunction in Idiopathic and LRRK2-Parkinson's Disease and At-Risk Individuals.

Background: Video-oculography constitutes a highly-sensitive method of characterizing ocular movements, which could detect subtle premotor changes and contribute to the early diagnosis of Parkinson's disease (PD). Objective: To investigate potential oculomotor differences between idiopathic PD (iPD) and PD associated with the G2019S variant of LRRK2 (L2PD), as well as to evaluate oculomotor function in asymptomatic carriers of the G2019S variant of LRRK2. Methods: The study enrolled 129 subjects: 30 PD (16 iPD, 14 L2PD), 23 asymptomatic carriers, 13 non-carrier relatives of L2PD patients, and 63 unrelated HCs. The video-oculographic evaluation included fixation, prosaccade, antisaccade, and memory saccade tests. Results: We did not find significant differences between iPD and L2PD. Compared to controls, PD patients displayed widespread oculomotor deficits including larger microsaccades, hypometric vertical prosaccades, increased latencies in all tests, and lower percentages of successful antisaccades and memory saccades. Non-carrier relatives showed oculomotor changes with parkinsonian features, such as fixation instability and hypometric vertical saccades. Asymptomatic carriers shared multiple similarities with PD, including signs of unstable fixation and hypometric vertical prosaccades; however, they were able to reach percentages of successful antisaccade and memory saccades similar to controls, although at the expense of longer latencies. Classification accuracy of significant oculomotor parameters to differentiate asymptomatic carriers from HCs ranged from 0.68 to 0.74, with BCEA, a marker of global fixation instability, being the parameter with the greatest classification accuracy. Conclusions: iPD and LRRK2-G2019S PD patients do not seem to display a differential oculomotor profile. Several oculomotor changes in asymptomatic carriers of LRRK2 mutations could be considered premotor biomarkers.

Peripheral vision and crowding in mental maze-solving.

Solving a maze effectively relies on both perception and cognition. Studying maze-solving behavior contributes to our knowledge about these important processes. Through psychophysical experiments and modeling simulations, we examine the role of peripheral vision, specifically visual crowding in the periphery, in mental maze-solving. Experiment 1 measured gaze patterns while varying maze complexity, revealing a direct relationship between visual complexity and maze-solving efficiency. Simulations of the maze-solving task using a peripheral vision model confirmed the observed crowding effects while making an intriguing prediction that saccades provide a conservative measure of how far ahead observers can perceive the path. Experiment 2 confirms that observers can judge whether a point lies on the path at considerably greater distances than their average saccade. Taken together, our findings demonstrate that peripheral vision plays a key role in mental maze-solving.

Bidirectional brain-body interactions during natural story listening.

Narratives can synchronize neural and physiological signals between individuals, but the relationship between these signals, and the underlying mechanism, is unclear. We hypothesized a top-down effect of cognition on arousal and predicted that auditory narratives will drive not only brain signals but also peripheral physiological signals. We find that auditory narratives entrained gaze variation, saccade initiation, pupil size, and heart rate. This is consistent with a top-down effect of cognition on autonomic function. We also hypothesized a bottom-up effect, whereby autonomic physiology affects arousal. Controlled breathing affected pupil size, and heart rate was entrained by controlled saccades. Additionally, fluctuations in heart rate preceded fluctuations of pupil size and brain signals. Gaze variation, pupil size, and heart rate were all associated with anterior-central brain signals. Together, these results suggest bidirectional causal effects between peripheral autonomic function and central brain circuits involved in the control of arousal.

Common structure of saccades and microsaccades in visual perception.

We obtain large amounts of external information through our eyes, a process often considered analogous to picture mapping onto a camera lens. However, our eyes are never as still as a camera lens, with saccades occurring between fixations and microsaccades occurring within a fixation. Although saccades are agreed to be functional for information sampling in visual perception, it remains unknown if microsaccades have a similar function when eye movement is restricted. Here, we demonstrated that saccades and microsaccades share common spatiotemporal structures in viewing visual objects. Twenty-seven adults viewed faces and houses in free-viewing and fixation-controlled conditions. Both saccades and microsaccades showed distinctive spatiotemporal patterns between face and house viewing that could be discriminated by pattern classifications. The classifications based on saccades and microsaccades could also be mutually generalized. Importantly, individuals who showed more distinctive saccadic patterns between faces and houses also showed more distinctive microsaccadic patterns. Moreover, saccades and microsaccades showed a higher structure similarity for face viewing than house viewing and a common orienting preference for the eye region over the mouth region. These findings suggested a common oculomotor program that is used to optimize information sampling during visual object perception.