Quantifying task-related gaze.

Competing theories attempt to explain what guides eye movements when exploring natural scenes: bottom-up image salience and top-down semantic salience. In one study, we apply language-based analyses to quantify the well-known observation that task influences gaze in natural scenes. Subjects viewed ten scenes as if they were performing one of two tasks. We found that the semantic similarity between the task and the labels of objects in the scenes captured the task-dependence of gaze (t(39) = 13.083; p < 0.001). In another study, we examined whether image salience or semantic salience better predicts gaze during a search task, and if viewing strategies are affected by searching for targets of high or low semantic relevance to the scene. Subjects searched 100 scenes for a high- or low-relevance object. We found that image salience becomes a worse predictor of gaze across successive fixations, while semantic salience remains a consistent predictor (X2(1, N=40) = 75.148, p < .001). Furthermore, we found that semantic salience decreased as object relevance decreased (t(39) = 2.304; p = .027). These results suggest that semantic salience is a useful predictor of gaze during task-related scene viewing, and that even in target-absent trials, gaze is modulated by the relevance of a search target to the scene in which it might be located.

Visual search patterns during exploration of naturalistic scenes are driven by saliency cues in individuals with cerebral visual impairment.

We investigated the relative influence of image salience and image semantics during the visual search of naturalistic scenes, comparing performance in individuals with cerebral visual impairment (CVI) and controls with neurotypical development. Participants searched for a prompted target presented as either an image or text cue. Success rate and reaction time were collected, and gaze behavior was recorded with an eye tracker. A receiver operating characteristic (ROC) analysis compared the distribution of individual gaze landings based on predictions of image salience (using Graph-Based Visual Saliency) and image semantics (using Global Vectors for Word Representations combined with Linguistic Analysis of Semantic Salience) models. CVI participants were less likely and were slower in finding the target. Their visual search behavior was also associated with a larger visual search area and greater number of fixations. ROC scores were also lower in CVI compared to controls for both model predictions. Furthermore, search strategies in the CVI group were not affected by cue type, although search times and accuracy showed a significant correlation with verbal IQ scores for text-cued searches. These results suggest that visual search patterns in CVI are driven mainly by image salience and provide further characterization of higher-order processing deficits observed in this population.

Object identification in cerebral visual impairment characterized by gaze behavior and image saliency analysis.

Individuals with cerebral visual impairment (CVI) have difficulties identifying common objects, especially when presented as cartoons or abstract images. In this study, participants were shown a series of images of ten common objects, each from five possible categories ranging from abstract black & white line drawings to color photographs. Fifty individuals with CVI and 50 neurotypical controls verbally identified each object and success rates and reaction times were collected. Visual gaze behavior was recorded using an eye tracker to quantify the extent of visual search area explored and number of fixations. A receiver operating characteristic (ROC) analysis was also carried out to compare the degree of alignment between the distribution of individual eye gaze patterns and image saliency features computed by the graph-based visual saliency (GBVS) model. Compared to controls, CVI participants showed significantly lower success rates and longer reaction times when identifying objects. In the CVI group, success rate improved moving from abstract black & white images to color photographs, suggesting that object form (as defined by outlines and contours) and color are important cues for correct identification. Eye tracking data revealed that the CVI group showed significantly greater visual search areas and number of fixations per image, and the distribution of eye gaze patterns in the CVI group was less aligned with the high saliency features of the image compared to controls. These results have important implications in helping to understand the complex profile of visual perceptual difficulties associated with CVI.

Low-level factors increase gaze-guidance under cognitive load: A comparison of image-salience and semantic-salience models.

Growing evidence links eye movements and cognitive functioning, however there is debate concerning what image content is fixated in natural scenes. Competing approaches have argued that low-level/feedforward and high-level/feedback factors contribute to gaze-guidance. We used one low-level model (Graph Based Visual Salience, GBVS) and a novel language-based high-level model (Global Vectors for Word Representation, GloVe) to predict gaze locations in a natural image search task, and we examined how fixated locations during this task vary under increasing levels of cognitive load. Participants (N = 30) freely viewed a series of 100 natural scenes for 10 seconds each. Between scenes, subjects identified a target object from the scene a specified number of trials (N) back among three distracter objects of the same type but from alternate scenes. The N-back was adaptive: N-back increased following two correct trials and decreased following one incorrect trial. Receiver operating characteristic (ROC) analysis of gaze locations showed that as cognitive load increased, there was a significant increase in prediction power for GBVS, but not for GloVe. Similarly, there was no significant difference in the area under the ROC between the minimum and maximum N-back achieved across subjects for GloVe (t(29) = -1.062, p = 0.297), while there was a cohesive upwards trend for GBVS (t(29) = -1.975, p = .058), although not significant. A permutation analysis showed that gaze locations were correlated with GBVS indicating that salient features were more likely to be fixated. However, gaze locations were anti-correlated with GloVe, indicating that objects with low semantic consistency with the scene were more likely to be fixated. These results suggest that fixations are drawn towards salient low-level image features and this bias increases with cognitive load. Additionally, there is a bias towards fixating improbable objects that does not vary under increasing levels of cognitive load.

Cognitive load influences oculomotor behavior in natural scenes.

Cognitive neuroscience researchers have identified relationships between cognitive load and eye movement behavior that are consistent with oculomotor biomarkers for neurological disorders. We develop an adaptive visual search paradigm that manipulates task difficulty and examine the effect of cognitive load on oculomotor behavior in healthy young adults. Participants (N = 30) free-viewed a sequence of 100 natural scenes for 10 s each, while their eye movements were recorded. After each image, participants completed a 4 alternative forced choice task in which they selected a target object from one of the previously viewed scenes, among 3 distracters of the same object type but from alternate scenes. Following two correct responses, the target object was selected from an image increasingly farther back (N-back) in the image stream; following an incorrect response, N decreased by 1. N-back thus quantifies and individualizes cognitive load. The results show that response latencies increased as N-back increased, and pupil diameter increased with N-back, before decreasing at very high N-back. These findings are consistent with previous studies and confirm that this paradigm was successful in actively engaging working memory, and successfully adapts task difficulty to individual subject's skill levels. We hypothesized that oculomotor behavior would covary with cognitive load. We found that as cognitive load increased, there was a significant decrease in the number of fixations and saccades. Furthermore, the total duration of saccades decreased with the number of events, while the total duration of fixations remained constant, suggesting that as cognitive load increased, subjects made fewer, longer fixations. These results suggest that cognitive load can be tracked with an adaptive visual search task, and that oculomotor strategies are affected as a result of greater cognitive demand in healthy adults.

Transfer and retention of oculomotor alignment rehabilitation training.

Ocular alignment defects such as strabismus affect around 5% of people and are associated with binocular vision impairments. Current nonsurgical treatments are controversial and have high levels of recidivism. In this study, we developed a rehabilitation method for ocular alignment training and examined the rate of learning, transfer to untrained alignments, and retention over time. Ocular alignment was controlled with a real-time dichoptic feedback paradigm where a static fixation target and white gaze-contingent ring were presented to the dominant eye and a black gaze-contingent ring with no fixation target was presented to the nondominant eye. Observers were required to move their eyes to center the rings on the target, with real-time feedback provided by the size of the rings. Offsetting the ring of the nondominant temporal or nasal visual field required convergent or divergent ocular deviation, respectively, to center the ring on the fixation target. Learning was quantified as the time taken to achieve target deviation of 2° (easy, E) or 4° (hard, H) for convergence (CE, CH) or divergence (DE, DH) over 40 trials. Thirty-two normally sighted observers completed two training sequences separated by one week. Subjects were randomly assigned to a training sequence: CE-CH-DE, CH-CE-DE, DE-DH-CE, or DH-DE-CE. The results showed that training was retained over the course of approximately one week across all conditions. Training on an easy deviation angle transferred to untrained hard angles within convergence or divergence but not between these directions. We conclude that oculomotor alignment can be rapidly trained, retained, and transferred with a feedback-based dichoptic paradigm. Feedback-based oculomotor training may therefore provide a noninvasive method for the rehabilitation of ocular alignment defects.

Object-based warping in three-dimensional environments.

Object-based warping is a powerful visual illusion wherein space between features within figural regions is regularly overestimated compared with those within ground regions. Originally, the effect was only examined in displays of two-dimensional (2D) stimuli. The present study sought to examine whether object-based warping persists in more naturalistic viewing conditions, where additional contextual cues are present. Stimuli were presented with either three-dimensional (3D) printed objects (Experiment 1) or 3D objects in virtual reality (Experiments 2-4). The testing metric was actual distance of features (dots) compared with estimated distances made by participants. Responses for the 3D printed stimuli were measured with replica dots on a slide ruler device. The virtual reality experiments collected responses either with a computer mouse or motion-tracked controller and included manipulations of object type, spatial separation, viewing distance of stimuli, and head motion. A standard warping effect in 3D was observed in all experiments, although the effect was not present in one condition that elicits warping in 2D (Occluded Rectangle). The final experiment resolves this discrepancy by reducing the multicomponent object (Occluded Rectangle) to a single component figure, while demonstrating the influence of depth cues on the warping effect under occlusion. Collectively, these experiments reveal that object-based warping is a powerful effect, even in naturalistic settings.