Exploring the impact of nonverbal social behavior on learning outcomes in instructional video design.

Online education has become increasingly popular in recent years, and video lectures have emerged as a common instructional format. While the importance of instructors' nonverbal social cues such as gaze, facial expression, and gestures for learning progress in face-to-face teaching is well-established, their impact on instructional videos is not fully understood. Most studies on nonverbal social cues in instructional videos focus on isolated cues rather than considering multimodal nonverbal behavior patterns and their effects on the learning progress. This study examines the role of instructors' nonverbal immediacy (a construct capturing multimodal nonverbal behaviors that reduce psychological distance) in video lectures with respect to learners' cognitive, affective, and motivational outcomes. We carried out an eye-tracking experiment with 87 participants (Mage = 24.11, SD = 4.80). Results of multilevel path analyses indicate that high nonverbal immediacy substantially increases learners' state motivation and enjoyment, but does not affect cognitive learning. Analyses of learners' eye movements show that learners allocate more attention to the instructor than to the learning material with increasing levels of nonverbal immediacy displayed by the instructor. The study highlights the importance of considering the role of multimodal nonverbal behavior patterns in online education and provides insights for effective video lecture design.

How body postures affect gaze control in scene viewing under specific task conditions.

Gaze movements during visual exploration of natural scenes are typically investigated with the static picture viewing paradigm in the laboratory. While this paradigm is attractive for its highly controlled conditions, limitations in the generalizability of the resulting findings to more natural viewing behavior have been raised frequently. Here, we address the combined influences of body posture and viewing task on gaze behavior with the static picture viewing paradigm under free viewing as a baseline condition. We recorded gaze data using mobile eye tracking during postural manipulations in scene viewing. Specifically, in Experiment 1, we compared gaze behavior during head-supported sitting and quiet standing under two task conditions. We found that task affects temporal and spatial gaze parameters, while posture produces no effects on temporal and small effects on spatial parameters. In Experiment 2, we further investigated body posture by introducing four conditions (sitting with chin rest, head-free sitting, quiet standing, standing on an unstable platform). Again, we found no effects on temporal and small effects on spatial gaze parameters. In our experiments, gaze behavior is largely unaffected by body posture, while task conditions readily produce effects. We conclude that results from static picture viewing may allow predictions of gaze statistics under more natural viewing conditions, however, viewing tasks should be chosen carefully because of their potential effects on gaze characteristics.

A dynamical scan-path model for task-dependence during scene viewing.

In real-world scene perception, human observers generate sequences of fixations to move image patches into the high-acuity center of the visual field. Models of visual attention developed over the last 25 years aim to predict two-dimensional probabilities of gaze positions for a given image via saliency maps. Recently, progress has been made on models for the generation of scan paths under the constraints of saliency as well as attentional and oculomotor restrictions. Experimental research demonstrated that task constraints can have a strong impact on viewing behavior. Here, we propose a scan-path model for both fixation positions and fixation durations, which include influences of task instructions and interindividual differences. Based on an eye-movement experiment with four different task conditions, we estimated model parameters for each individual observer and task condition using a fully Bayesian dynamical modeling framework using a joint spatial-temporal likelihood approach with sequential estimation. Resulting parameter values demonstrate that model properties such as the attentional span are adjusted to task requirements. Posterior predictive checks indicate that our dynamical model can reproduce task differences in scan-path statistics across individual observers. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Data assimilation in dynamical cognitive science.

Dynamical models make specific assumptions about cognitive processes that generate human behavior. In data assimilation, these models are tested against time-ordered data. Recent progress on Bayesian data assimilation demonstrates that this approach combines the strengths of statistical modeling of individual differences with the those of dynamical cognitive models.

Does Local Coherence Lead to Targeted Regressions and Illusions of Grammaticality?

Local coherence effects arise when the human sentence processor is temporarily misled by a locally grammatical but globally ungrammatical analysis (The coach smiled at the player tossed a frisbee by the opposing team). It has been suggested that such effects occur either because sentence processing occurs in a bottom-up, self-organized manner rather than under constant grammatical supervision, or because local coherence can disrupt processing due to readers maintaining uncertainty about previous input. We report the results of an eye-tracking study in which subjects read German grammatical and ungrammatical sentences that either contained a locally coherent substring or not and gave binary grammaticality judgments. In our data, local coherence affected on-line processing immediately at the point of the manipulation. There was, however, no indication that local coherence led to illusions of grammaticality (a prediction of self-organization), and only weak, inconclusive support for local coherence leading to targeted regressions to critical context words (a prediction of the uncertain-input approach). We discuss implications for self-organized and noisy-channel models of local coherence.

Predictive modeling of parafoveal information processing during reading.

Skilled reading requires information processing of the fixated and the not-yet-fixated words to generate precise control of gaze. Over the last 30 years, experimental research provided evidence that word processing is distributed across the perceptual span, which permits recognition of the fixated (foveal) word as well as preview of parafoveal words to the right of fixation. However, theoretical models have been unable to differentiate the specific influences of foveal and parafoveal information on saccade control. Here we show how parafoveal word difficulty modulates spatial and temporal control of gaze in a computational model to reproduce experimental results. In a fully Bayesian framework, we estimated model parameters for different models of parafoveal processing and carried out large-scale predictive simulations and model comparisons for a gaze-contingent reading experiment. We conclude that mathematical modeling of data from gaze-contingent experiments permits the precise identification of pathways from parafoveal information processing to gaze control, uncovering potential mechanisms underlying the parafoveal contribution to eye-movement control.

A Bayesian approach to dynamical modeling of eye-movement control in reading of normal, mirrored, and scrambled texts.

In eye-movement control during reading, advanced process-oriented models have been developed to reproduce behavioral data. So far, model complexity and large numbers of model parameters prevented rigorous statistical inference and modeling of interindividual differences. Here we propose a Bayesian approach to both problems for one representative computational model of sentence reading (SWIFT; Engbert et al., Psychological Review, 112, 2005, pp. 777-813). We used experimental data from 36 subjects who read the text in a normal and one of four manipulated text layouts (e.g., mirrored and scrambled letters). The SWIFT model was fitted to subjects and experimental conditions individually to investigate between-subject variability. Based on posterior distributions of model parameters, fixation probabilities and durations are reliably recovered from simulated data and reproduced for withheld empirical data, at both the experimental condition and subject levels. A subsequent statistical analysis of model parameters across reading conditions generates model-driven explanations for observable effects between conditions. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

A mathematical model of local and global attention in natural scene viewing.

Understanding the decision process underlying gaze control is an important question in cognitive neuroscience with applications in diverse fields ranging from psychology to computer vision. The decision for choosing an upcoming saccade target can be framed as a selection process between two states: Should the observer further inspect the information near the current gaze position (local attention) or continue with exploration of other patches of the given scene (global attention)? Here we propose and investigate a mathematical model motivated by switching between these two attentional states during scene viewing. The model is derived from a minimal set of assumptions that generates realistic eye movement behavior. We implemented a Bayesian approach for model parameter inference based on the model's likelihood function. In order to simplify the inference, we applied data augmentation methods that allowed the use of conjugate priors and the construction of an efficient Gibbs sampler. This approach turned out to be numerically efficient and permitted fitting interindividual differences in saccade statistics. Thus, the main contribution of our modeling approach is two-fold; first, we propose a new model for saccade generation in scene viewing. Second, we demonstrate the use of novel methods from Bayesian inference in the field of scan path modeling.

Sequential Data Assimilation of the Stochastic SEIR Epidemic Model for Regional COVID-19 Dynamics.

Newly emerging pandemics like COVID-19 call for predictive models to implement precisely tuned responses to limit their deep impact on society. Standard epidemic models provide a theoretically well-founded dynamical description of disease incidence. For COVID-19 with infectiousness peaking before and at symptom onset, the SEIR model explains the hidden build-up of exposed individuals which creates challenges for containment strategies. However, spatial heterogeneity raises questions about the adequacy of modeling epidemic outbreaks on the level of a whole country. Here, we show that by applying sequential data assimilation to the stochastic SEIR epidemic model, we can capture the dynamic behavior of outbreaks on a regional level. Regional modeling, with relatively low numbers of infected and demographic noise, accounts for both spatial heterogeneity and stochasticity. Based on adapted models, short-term predictions can be achieved. Thus, with the help of these sequential data assimilation methods, more realistic epidemic models are within reach.

Modeling the effects of perisaccadic attention on gaze statistics during scene viewing.

How we perceive a visual scene depends critically on the selection of gaze positions. For this selection process, visual attention is known to play a key role in two ways. First, image-features attract visual attention, a fact that is captured well by time-independent fixation models. Second, millisecond-level attentional dynamics around the time of saccade drives our gaze from one position to the next. These two related research areas on attention are typically perceived as separate, both theoretically and experimentally. Here we link the two research areas by demonstrating that perisaccadic attentional dynamics improve predictions on scan path statistics. In a mathematical model, we integrated perisaccadic covert attention with dynamic scan path generation. Our model reproduces saccade amplitude distributions, angular statistics, intersaccadic turning angles, and their impact on fixation durations as well as inter-individual differences using Bayesian inference. Therefore, our result lend support to the relevance of perisaccadic attention to gaze statistics.

No exception from Bayes' rule: The presence and absence of the range effect for saccades explained.

In an influential theoretical model, human sensorimotor control is achieved by a Bayesian decision process, which combines noisy sensory information and learned prior knowledge. A ubiquitous signature of prior knowledge and Bayesian integration in human perception and motor behavior is the frequently observed bias toward an average stimulus magnitude (i.e., a central-tendency bias, range effect, regression-to-the-mean effect). However, in the domain of eye movements, there is a recent controversy about the fundamental existence of a range effect in the saccadic system. Here we argue that the problem of the existence of a range effect is linked to the availability of prior knowledge for saccade control. We present results from two prosaccade experiments that both employ an informative prior structure (i.e., a nonuniform Gaussian distribution of saccade target distances). Our results demonstrate the validity of Bayesian integration in saccade control, which generates a range effect in saccades. According to Bayesian integration principles, the saccadic range effect depends on the availability of prior knowledge and varies in size as a function of the reliability of the prior and the sensory likelihood.

Task-dependence in scene perception: Head unrestrained viewing using mobile eye-tracking.

Real-world scene perception is typically studied in the laboratory using static picture viewing with restrained head position. Consequently, the transfer of results obtained in this paradigm to real-word scenarios has been questioned. The advancement of mobile eye-trackers and the progress in image processing, however, permit a more natural experimental setup that, at the same time, maintains the high experimental control from the standard laboratory setting. We investigated eye movements while participants were standing in front of a projector screen and explored images under four specific task instructions. Eye movements were recorded with a mobile eye-tracking device and raw gaze data were transformed from head-centered into image-centered coordinates. We observed differences between tasks in temporal and spatial eye-movement parameters and found that the bias to fixate images near the center differed between tasks. Our results demonstrate that current mobile eye-tracking technology and a highly controlled design support the study of fine-scaled task dependencies in an experimental setting that permits more natural viewing behavior than the static picture viewing paradigm.

Modulation of oculomotor control during reading of mirrored and inverted texts.

The interplay between cognitive and oculomotor processes during reading can be explored when the spatial layout of text deviates from the typical display. In this study, we investigate various eye-movement measures during reading of text with experimentally manipulated layout (word-wise and letter-wise mirrored-reversed text as well as inverted and scrambled text). While typical findings (e.g., longer mean fixation times, shorter mean saccades lengths) in reading manipulated texts compared to normal texts were reported in earlier work, little is known about changes of oculomotor targeting observed in within-word landing positions under the above text layouts. Here we carry out precise analyses of landing positions and find substantial changes in the so-called launch-site effect in addition to the expected overall slow-down of reading performance. Specifically, during reading of our manipulated text conditions with reversed letter order (against overall reading direction), we find a reduced launch-site effect, while in all other manipulated text conditions, we observe an increased launch-site effect. Our results clearly indicate that the oculomotor system is highly adaptive when confronted with unusual reading conditions.

Microsaccades: Empirical Research and Methodological Advances - Introduction to Part 1 of the Thematic Special Issue.

Recent technical developments and increased affordability of high-speed eye tracking devices have brought microsaccades to the forefront of research in many areas of sensory, perceptual, and cognitive processes. The present thematic issue on "Microsaccades: Empirical Research and Methodological Advances" invited authors to submit original research and reviews encompassing measurements and data analyses in fundamental, translational, and applied studies. We present the first volume of this special issue, comprising 14 articles by research teams around the world. Contributions include the characterization of fixational eye movements and saccadic intrusions in neurological impairments and in visual disease, methodological developments in microsaccade detection, the measurement of fixational eye movements in applied and ecological scenarios, and advances in the current understanding of the relationship between microsaccades and cognition. When fundamental research on microsaccades experienced a renaissance at the turn of the millennium (c.f. 4), one could hardly have been so bold as to predict the manifold applications of research on fixational eye movements in clinic and practice. Through this great variety of areas of focus, some main topics emerge. One such theme is the applicability of microsaccade measures to neurological and visual disease. Whereas microsaccade quantifications have been largely limited to participants with intact visual and oculomotor systems, recent research has extended this interest into the realm of neural and ophthalmic impairment (see 1 for a review). In this volume, Becker et al analyze " Saccadic intrusions in amyotrophic lateral sclerosis (ALS)" and Kang et al study " Fixational eye movement waveforms in amblyopia", delving into the characteristics of fast and slow eye movements. Two other articles focus on how the degradation of visual information, which is relevant to many ophthalmic pathologies, affects microsaccadic features. Tang et al investigate the " Effects of visual blur on microsaccades on visual exploration" and conclude that the precision of an image on the fovea plays an important role in the calibration of microsaccade amplitudes during visual scanning. Otero-Millan et al use different kinds of visual stimuli and viewing tasks in the presence or absence of simulated scotomas, to determine the contributions of foveal and peripheral visual information to microsaccade production. They conclude that " Microsaccade generation requires a foveal anchor". The link between microsaccadic characteristics and cognitive processes has been a mainstay of microsaccade research for almost two decades, since studies in the early 2000s connected microsaccade directions to the spatial location of covert attentional cues (2 3). In the present volume, Dalmaso et al report that " Anticipation of cognitive conflict is reflected in microsaccades", providing new insights about the top-down modulation of microsaccade dynamics. Ryan et al further examine the relationship between " Microsaccades and covert attention" during the performance of a continuous, divided-attention task, and find preliminary evidence that microsaccades track the ongoing allocation of spatial attention. Krueger et al discover that microsaccade rates modulate with visual attention demands and report that " Microsaccades distinguish looking from seeing". Taking the ecological validity of microsaccade investigations one step further, Barnhart et al evaluate microsaccades during the observation of magic tricks and conclude that " Microsaccades reflect the dynamics of misdirected attention in magic". Two articles examine the role of individual differences and intraindividual variability over time on microsaccadic features. In " Reliability and correlates of intra-individual variability in the oculomotor system" Perquin and Bompas find evidence for intra-individual reliability over different time points, while cautioning that its use to classify self-reported individual differences remains unclear. Stafford et al provide a counterpoint in " Can microsaccade rate predict drug response?" by supporting the use of microsaccade occurrence as both a trait measure of individual differences and as a state measure of response to caffeine administration. Methodological and technical advances are the subjects of three papers in this volume. In " Motion tracking of iris features to detect small eye movements" Chaudhary and Pelz describe a new video-based eye tracking methodology that relies on higher-order iris texture features, rather than on lower-order pupil center and corneal reflection features, to detect microsaccades with high confidence. Munz et al present an open source visual analytics system called " VisME: Visual microsaccades explorer" that allows users to interactively vary microsaccade filter parameters and evaluate the resulting effects on microsaccade behavior, with the goal of promoting reproducibility in data analyses. In " What makes a microsaccade? A review of 70 years research prompts a new detection method" Hauperich et al review the microsaccade properties reported between the 1940s and today, and use the stated range of parameters to develop a novel method of microsaccade detection. Lastly, Alexander et al switch the focus from the past of microsaccade research to its future, by discussing the recent and upcoming applications of fixational eye movements to ecologically-valid and real-world scenarios. Their review " Microsaccades in applied environments: real-world applications of fixational eye movement measurements" covers the possibilities and challenges of taking microsaccade measurements out of the lab and into the field. Microsaccades have engaged the interest of scientists from different backgrounds and disciplines for many decades and will certainly continue to do so. One reason for this fascination might be microsaccades' role as a link between basic sensory processes and high-level cognitive phenomena, making them an attractive focus of interdisciplinary research and transdisciplinary applications. Thus, research on microsaccades will not only endure, but keep evolving as the present knowledge base expands. Part 2 of the special issue on microsaccades is already in progress with articles currently under review and will be published in 2021.

Experimental test of Bayesian saccade targeting under reversed reading direction.

During reading, rapid eye movements (saccades) shift the reader's line of sight from one word to another for high-acuity visual information processing. While experimental data and theoretical models show that readers aim at word centers, the eye-movement (oculomotor) accuracy is low compared to other tasks. As a consequence, distributions of saccadic landing positions indicate large (i) random errors and (ii) systematic over- and undershoot of word centers, which additionally depend on saccade lengths (McConkie et al. Visual Research, 28(10), 1107-1118, 1988). Here we show that both error components can be simultaneously reduced by reading texts from right to left in German language (N = 32). We used our experimental data to test a Bayesian model of saccade planning. First, experimental data are consistent with the model. Second, the model makes specific predictions of the effects of the precision of prior and (sensory) likelihood. Our results suggest that it is a more precise sensory likelihood that can explain the reduction of both random and systematic error components.

How spatial frequencies and color drive object search in real-world scenes: A new eye-movement corpus.

When studying how people search for objects in scenes, the inhomogeneity of the visual field is often ignored. Due to physiological limitations, peripheral vision is blurred and mainly uses coarse-grained information (i.e., low spatial frequencies) for selecting saccade targets, whereas high-acuity central vision uses fine-grained information (i.e., high spatial frequencies) for analysis of details. Here we investigated how spatial frequencies and color affect object search in real-world scenes. Using gaze-contingent filters, we attenuated high or low frequencies in central or peripheral vision while viewers searched color or grayscale scenes. Results showed that peripheral filters and central high-pass filters hardly affected search accuracy, whereas accuracy dropped drastically with central low-pass filters. Peripheral filtering increased the time to localize the target by decreasing saccade amplitudes and increasing number and duration of fixations. The use of coarse-grained information in the periphery was limited to color scenes. Central filtering increased the time to verify target identity instead, especially with low-pass filters. We conclude that peripheral vision is critical for object localization and central vision is critical for object identification. Visual guidance during peripheral object localization is dominated by low-frequency color information, whereas high-frequency information, relatively independent of color, is most important for object identification in central vision.

Spatial statistics for gaze patterns in scene viewing: Effects of repeated viewing.

Scene viewing is used to study attentional selection in complex but still controlled environments. One of the main observations on eye movements during scene viewing is the inhomogeneous distribution of fixation locations: While some parts of an image are fixated by almost all observers and are inspected repeatedly by the same observer, other image parts remain unfixated by observers even after long exploration intervals. Here, we apply spatial point process methods to investigate the relationship between pairs of fixations. More precisely, we use the pair correlation function, a powerful statistical tool, to evaluate dependencies between fixation locations along individual scanpaths. We demonstrate that aggregation of fixation locations within 4° is stronger than expected from chance. Furthermore, the pair correlation function reveals stronger aggregation of fixations when the same image is presented a second time. We use simulations of a dynamical model to show that a narrower spatial attentional span may explain differences in pair correlations between the first and the second inspection of the same image.

Disentangling bottom-up versus top-down and low-level versus high-level influences on eye movements over time.

Bottom-up and top-down as well as low-level and high-level factors influence where we fixate when viewing natural scenes. However, the importance of each of these factors and how they interact remains a matter of debate. Here, we disentangle these factors by analyzing their influence over time. For this purpose, we develop a saliency model that is based on the internal representation of a recent early spatial vision model to measure the low-level, bottom-up factor. To measure the influence of high-level, bottom-up features, we use a recent deep neural network-based saliency model. To account for top-down influences, we evaluate the models on two large data sets with different tasks: first, a memorization task and, second, a search task. Our results lend support to a separation of visual scene exploration into three phases: the first saccade, an initial guided exploration characterized by a gradual broadening of the fixation density, and a steady state that is reached after roughly 10 fixations. Saccade-target selection during the initial exploration and in the steady state is related to similar areas of interest, which are better predicted when including high-level features. In the search data set, fixation locations are determined predominantly by top-down processes. In contrast, the first fixation follows a different fixation density and contains a strong central fixation bias. Nonetheless, first fixations are guided strongly by image properties, and as early as 200 ms after image onset, fixations are better predicted by high-level information. We conclude that any low-level, bottom-up factors are mainly limited to the generation of the first saccade. All saccades are better explained when high-level features are considered, and later, this high-level, bottom-up control can be overruled by top-down influences.