Extracting blinks from continuous eye-tracking data in a mind wandering paradigm.

Mind wandering is a universal phenomenon in which a person's attention decouples from stimuli within their current environment. Researchers have sought objective, less disruptive indicators of cognitive disengagement, resulting in a focus eye tracking and blink characteristics. Such research has found positive associations between mind wandering and blink characteristics, typically in reading tasks. However, extracting blinks accurately from continuous eye-tracking data is complex, and the literature contains inconsistently reported data processing methods, some of which may have an elevated risk of identifying noise as signal. Further, the relationship between attentional disengagement and blink durations has not been fully explored in multiple task modalities. We conducted three modality-specific experiments while recording eye movements. Blink durations varied as a function of stimulus/task engagingness; less engaging tasks yielded longer blink durations, suggesting a link between blinking and mind wandering. Recommendations are provided for researchers seeking to accurately derive blink events from continuous, binocular, eye-tracking data.

Eye movements during listening reveal spontaneous grammatical processing.

Recent research using eye-tracking typically relies on constrained visual contexts in particular goal-oriented contexts, viewing a small array of objects on a computer screen and performing some overt decision or identification. Eyetracking paradigms that use pictures as a measure of word or sentence comprehension are sometimes touted as ecologically invalid because pictures and explicit tasks are not always present during language comprehension. This study compared the comprehension of sentences with two different grammatical forms: the past progressive (e.g., was walking), which emphasizes the ongoing nature of actions, and the simple past (e.g., walked), which emphasizes the end-state of an action. The results showed that the distribution and timing of eye movements mirrors the underlying conceptual structure of this linguistic difference in the absence of any visual stimuli or task constraint: Fixations were shorter and saccades were more dispersed across the screen, as if thinking about more dynamic events when listening to the past progressive stories. Thus, eye movement data suggest that visual inputs or an explicit task are unnecessary to solicit analog representations of features such as movement, that could be a key perceptual component to grammatical comprehension.

Degree of certainty modulates anticipatory processes in real time.

In the present study, we investigated how degree of certainty modulates anticipatory processes using a modified spatial cuing task in which participants made an anticipatory hand movement with the computer mouse toward one of two probabilistic targets. A cue provided information of the location of the upcoming target with 100% validity (certain condition), 75% validity (semicertain condition) or gave no information of the location (uncertain condition). We found that the degree of certainty associated with the probabilistic precue on the upcoming target location affected the spatiotemporal characteristics of the anticipatory hand movements in a systematic way. In the case of semicertainty, we found evidence that the anticipatory processes were modulated in a way consistent with a model of graded probability matching biased toward certainty. In the case of uncertainty regarding two equally likely locations, we observed large between- and within-subject variability in the patterns of anticipatory hand movements, suggesting that individual differences in the strategies employed may become relevant when the likelihoods of response options are equal.

Drawing from memory: hand-eye coordination at multiple scales.

Eyes move to gather visual information for the purpose of guiding behavior. This guidance takes the form of perceptual-motor interactions on short timescales for behaviors like locomotion and hand-eye coordination. More complex behaviors require perceptual-motor interactions on longer timescales mediated by memory, such as navigation, or designing and building artifacts. In the present study, the task of sketching images of natural scenes from memory was used to examine and compare perceptual-motor interactions on shorter and longer timescales. Eye and pen trajectories were found to be coordinated in time on shorter timescales during drawing, and also on longer timescales spanning study and drawing periods. The latter type of coordination was found by developing a purely spatial analysis that yielded measures of similarity between images, eye trajectories, and pen trajectories. These results challenge the notion that coordination only unfolds on short timescales. Rather, the task of drawing from memory evokes perceptual-motor encodings of visual images that preserve coarse-grained spatial information over relatively long timescales as well.

Negation without symbols: the importance of recurrence and context in linguistic negation.

A simple recurrent network with a perceptual simulation layer was trained on a corpus of affirmative and negated sentences. Linguistic negation can be encoded by the network via the inclusion (or absence) of features and categories associated with the senses, in one step, without the need for an explicit logical operation or for treating the negating word any differently than any other words. Visualizing negation as a trajectory in perceptual simulation space is explored in detail, and the implications for artificial intelligence, embodied computational models, and more practical implications of everyday use of negations are discussed.

Processing motion implied in language: eye-movement differences during aspect comprehension.

Previous research on language comprehension has used the eyes as a window into processing. However, these methods are entirely reliant upon using visual or orthographic stimuli that map onto the linguistic stimuli being used. The potential danger of this method is that the pictures used may not perfectly match the internal aspects of language processing. Thus, a method was developed in which participants listened to stories while wearing a head-mounted eyetracker. Preliminary results demonstrate that this method is uniquely suited to measure responses to stimuli in the absence of visual stimulation.

On the temporal dynamics of language-mediated vision and vision-mediated language.

Recent converging evidence suggests that language and vision interact immediately in non-trivial ways, although the exact nature of this interaction is still unclear. Not only does linguistic information influence visual perception in real-time, but visual information also influences language comprehension in real-time. For example, in visual search tasks, incremental spoken delivery of the target features (e.g., "Is there a red vertical?") can increase the efficiency of conjunction search because only one feature is heard at a time. Moreover, in spoken word recognition tasks, the visual presence of an object whose name is similar to the word being spoken (e.g., a candle present when instructed to "pick up the candy") can alter the process of comprehension. Dense sampling methods, such as eye-tracking and reach-tracking, richly illustrate the nature of this interaction, providing a semi-continuous measure of the temporal dynamics of individual behavioral responses. We review a variety of studies that demonstrate how these methods are particularly promising in further elucidating the dynamic competition that takes place between underlying linguistic and visual representations in multimodal contexts, and we conclude with a discussion of the consequences that these findings have for theories of embodied cognition.

Continuous dynamics of color categorization.

In order to assess whether color categorization is sensitive to within-category differences in hue, we monitored mouse trajectories in a modified categorization task. Participants saw color swatches from a blue-green continuum and categorized them with a computer mouse by selecting one of two colored regions at the top of a monitor. An analysis of the mouse trajectories showed that the deviation toward the competing category was a function of hue: As hues approached the category boundary, they increasingly deviated to the competitor. This work presents evidence for parallel activation on the level of hue and category processing for color, as well as simultaneous activation of perceptually adjacent categories. Thus, a dynamic process sensitive to fine-grained within-category detail best characterizes color categorization.