Exploring the impact of nonverbal social behavior on learning outcomes in instructional video design.

Online education has become increasingly popular in recent years, and video lectures have emerged as a common instructional format. While the importance of instructors' nonverbal social cues such as gaze, facial expression, and gestures for learning progress in face-to-face teaching is well-established, their impact on instructional videos is not fully understood. Most studies on nonverbal social cues in instructional videos focus on isolated cues rather than considering multimodal nonverbal behavior patterns and their effects on the learning progress. This study examines the role of instructors' nonverbal immediacy (a construct capturing multimodal nonverbal behaviors that reduce psychological distance) in video lectures with respect to learners' cognitive, affective, and motivational outcomes. We carried out an eye-tracking experiment with 87 participants (Mage = 24.11, SD = 4.80). Results of multilevel path analyses indicate that high nonverbal immediacy substantially increases learners' state motivation and enjoyment, but does not affect cognitive learning. Analyses of learners' eye movements show that learners allocate more attention to the instructor than to the learning material with increasing levels of nonverbal immediacy displayed by the instructor. The study highlights the importance of considering the role of multimodal nonverbal behavior patterns in online education and provides insights for effective video lecture design.

Classification of short and long term mild traumatic brain injury using computerized eye tracking.

Accurate, and objective diagnosis of brain injury remains challenging. This study evaluated useability and reliability of computerized eye-tracker assessments (CEAs) designed to assess oculomotor function, visual attention/processing, and selective attention in recent mild traumatic brain injury (mTBI), persistent post-concussion syndrome (PPCS), and controls. Tests included egocentric localisation, fixation-stability, smooth-pursuit, saccades, Stroop, and the vestibulo-ocular reflex (VOR). Thirty-five healthy adults performed the CEA battery twice to assess useability and test-retest reliability. In separate experiments, CEA data from 55 healthy, 20 mTBI, and 40 PPCS adults were used to train a machine learning model to categorize participants into control, mTBI, or PPCS classes. Intraclass correlation coefficients demonstrated moderate (ICC > .50) to excellent (ICC > .98) reliability (p < .05) and satisfactory CEA compliance. Machine learning modelling categorizing participants into groups of control, mTBI, and PPCS performed reasonably (balanced accuracy control: 0.83, mTBI: 0.66, and PPCS: 0.76, AUC-ROC: 0.82). Key outcomes were the VOR (gaze stability), fixation (vertical error), and pursuit (total error, vertical gain, and number of saccades). The CEA battery was reliable and able to differentiate healthy, mTBI, and PPCS patients reasonably well. While promising, the diagnostic model accuracy should be improved with a larger training dataset before use in clinical environments.

Column setting and text justification influence return-sweep eye movement behavior during Chinese multi-line reading.

People regularly read multi-line texts in different formats and publishers, internationally, must decide how to present text to make reading most effective and efficient. Relatively few studies have examined multi-line reading, and fewer still Chinese multi-line reading. Here, we examined whether texts presented in single or double columns, and either left-justified or fully-justified affect Chinese reading. Text format had minimal influence on overall reading time; however, it significantly impacted return-sweeps (large saccades moving the eyes from the end of one line of text to the beginning of the next). Return-sweeps were launched and landed further away from margins and involved more corrective saccades in single- than double-column format. For left- compared to fully-justified format, return-sweeps were launched and landed closer to margins. More corrective saccades also occurred. Our results showed more efficient return-sweep behavior for fully- than left-justified text. Moreover, there were clear trade-off effects such that formats requiring increased numbers of shorter return-sweeps produced more accurate targeting and reduced numbers of corrective fixations, whereas formats requiring reduced numbers of longer return-sweeps caused less accurate targeting and an increased rate of corrective fixations. Overall, our results demonstrate that text formats substantially affect return-sweep eye movement behavior during Chinese reading without affecting efficiency and effectiveness, that is, the overall time it takes to read and understand the text.

Paying attention.

Intentional eye movements depend on where peripheral attention is voluntarily deployed beforehand, but they can be directed elsewhere shortly afterwards.

Sustained bias of spatial attention in a 3 T MRI scanner.

When lying inside a MRI scanner and even in the absence of any motion, the static magnetic field of MRI scanners induces a magneto-hydrodynamic stimulation of subjects' vestibular organ (MVS). MVS thereby not only causes a horizontal vestibular nystagmus but also induces a horizontal bias in spatial attention. In this study, we aimed to determine the time course of MVS-induced biases in both VOR and spatial attention inside a 3 T MRI-scanner as well as their respective aftereffects after participants left the scanner. Eye movements and overt spatial attention in a visual search task were assessed in healthy volunteers before, during, and after a one-hour MVS period. All participants exhibited a VOR inside the scanner, which declined over time but never vanished completely. Importantly, there was also an MVS-induced horizontal bias in spatial attention and exploration, which persisted throughout the entire hour within the scanner. Upon exiting the scanner, we observed aftereffects in the opposite direction manifested in both the VOR and in spatial attention, which were statistically no longer detectable after 7 min. Sustained MVS effects on spatial attention have important implications for the design and interpretation of fMRI-studies and for the development of therapeutic interventions counteracting spatial neglect.

A Multimodal Assistive-Robotic-Arm Control System to Increase Independence After Tetraplegia.

Following tetraplegia, independence for completing essential daily tasks, such as opening doors and eating, significantly declines. Assistive robotic manipulators (ARMs) could restore independence, but typically input devices for these manipulators require functional use of the hands. We created and validated a hands-free multimodal input system for controlling an ARM in virtual reality using combinations of a gyroscope, eye-tracking, and heterologous surface electromyography (sEMG). These input modalities are mapped to ARM functions based on the user's preferences and to maximize the utility of their residual volitional capabilities following tetraplegia. The two participants in this study with tetraplegia preferred to use the control mapping with sEMG button functions and disliked winking commands. Non-disabled participants were more varied in their preferences and performance, further suggesting that customizability is an advantageous component of the control system. Replacing buttons from a traditional handheld controller with sEMG did not substantively reduce performance. The system provided adequate control to all participants to complete functional tasks in virtual reality such as opening door handles, turning stove dials, eating, and drinking, all of which enable independence and improved quality of life for these individuals.

Visual search for real-world scenes in patients with Alzheimer's disease and amnestic mild cognitive impairment.

BACKGROUND: Visual attention-related processes that underlie visual search behavior are impaired in both the early stages of Alzheimer's disease (AD) and amnestic mild cognitive impairment (aMCI), which is considered a risk factor for AD. Although traditional computer-based array tasks have been used to investigate visual search, information on the visual search patterns of AD and MCI patients in real-world environments is limited. AIM: The objective of this study was to evaluate the differences in visual search behaviors among individuals with AD, aMCI, and healthy controls (HCs) in real-world scenes. MATERIALS AND METHODS: A total of 92 participants were enrolled, including 28 with AD, 32 with aMCI, and 32 HCs. During the visual search task, participants were instructed to look at a single target object amid distractors, and their eye movements were recorded. RESULTS: The results indicate that patients with AD made more fixations on distractors and fewer fixations on the target, compared to patients with aMCI and HC groups. Additionally, AD patients had longer fixation durations on distractors and spent less time looking at the target than both patients with aMCI and HCs. DISCUSSION: These findings suggest that visual search behavior is impaired in patients with AD and can be distinguished from aMCI and healthy individuals. For future studies, it is important to longitudinally monitor visual search behavior in the progression from aMCI to AD. CONCLUSION: Our study holds significance in elucidating the interplay between impairments in attention, visual processes, and other underlying cognitive processes, which contribute to the functional decline observed in individuals with AD and aMCI.

The role of prediction and visual tracking strategies during manual interception: An exploration of individual differences.

The interception (or avoidance) of moving objects is a common component of various daily living tasks; however, it remains unclear whether precise alignment of foveal vision with a target is important for motor performance. Furthermore, there has also been little examination of individual differences in visual tracking strategy and the use of anticipatory gaze adjustments. We examined the importance of in-flight tracking and predictive visual behaviors using a virtual reality environment that required participants (n = 41) to intercept tennis balls projected from one of two possible locations. Here, we explored whether different tracking strategies spontaneously arose during the task, and which were most effective. Although indices of closer in-flight tracking (pursuit gain, tracking coherence, tracking lag, and saccades) were predictive of better interception performance, these relationships were rather weak. Anticipatory gaze shifts toward the correct release location of the ball provided no benefit for subsequent interception. Nonetheless, two interceptive strategies were evident: 1) early anticipation of the ball's onset location followed by attempts to closely track the ball in flight (i.e., predictive strategy); or 2) positioning gaze between possible onset locations and then using peripheral vision to locate the moving ball (i.e., a visual pivot strategy). Despite showing much poorer in-flight foveal tracking of the ball, participants adopting a visual pivot strategy performed slightly better in the task. Overall, these results indicate that precise alignment of the fovea with the target may not be critical for interception tasks, but that observers can adopt quite varied visual guidance approaches.

High-speed eye tracking based on a synchronized imaging mechanism by a dual-ring infrared lighting source.

It is a challenge for conventional monocular-camera single-light source eye-tracking methods to achieve high-speed eye tracking. In this work, a dual-ring infrared lighting source was designed to achieve bright and dark pupils in high speed. The eye-tracking method used a dual-ring infrared lighting source and synchronized triggers for the even and odd camera frames to capture bright and dark pupils. A pupillary corneal reflex was calculated by the center coordinates of the Purkinje spot and the pupil. A map function was established to map the relationship between the pupillary corneal reflex and gaze spots. The gaze coordinate was calculated based on the mapping function. The average detection time of each gaze spot was 3.76 ms.

[Neural Mechanisms of Visual Search and Working Memory].

Visual search is a useful experimental paradigm investigating various aspects of attention. For efficient search, participants must avoid revisiting previously viewed objects. Inhibitory tagging and inhibition of return are phenomena related to this process, but their neural mechanisms are yet to be elucidated. Recent studies have shown that the rate of revisit behavior during visual search correlates with working memory capacity. This suggests that top-down signals from the frontal executive system alter the priority map that guides attention and eye movements. With this in mind, we have developed a novel visual search paradigm with many identical distractors and an evaluation model that assesses multiple parameters of working memory. The model incorporated memory capacity, memory decay, and utility rate, and when applied to data obtained from experimental animals, these parameters could be reliably evaluated. Furthermore, using the behavioral paradigm and model, we found that systemic administration of subanesthetic dose of ketamine decreased utility rate and memory capacity, while nicotine administration increased utility rate. Since our behavioral paradigm does not require complex instruction, it can be applied to a variety of patients in future clinical studies.

Visuomotor coordination with gaze, head and arm movements during table tennis forehand rallies.

The purpose of this study was to clarify the temporal coordination between gaze, head, and arm movements during forehand rallies in table tennis. Collegiate male table tennis players (n = 7) conducted forehand rallies at a constant tempo (100, 120, and 150 bpm) using a metronome. In each tempo condition, participants performed 30 strokes (a total of 90 strokes). Gaze, head, and dominant arm (shoulder, elbow, and wrist) movements were recorded with an eye-tracking device equipped with a Gyro sensor and a 3-D motion capture system. The results showed that the effect of head movements relative to gaze movements was significantly higher than that of eye movements in the three tempo conditions. Our results indicate that head movements are closely associated with gaze movements during rallies. Furthermore, cross-correlation coefficients (CCs) between head and arm movements were more than 0.96 (maximum coefficient: 0.99). In addition, head and arm movements were synchronized during rallies. Finally, CCs between gaze and arm movements were more than 0.74 (maximum coefficient: 0.99), indicating that gaze movements are temporally coordinated with arm movements. Taken together, head movements could play important roles not only in gaze tracking but also in the temporal coordination with arm movements during table tennis forehand rallies.

The effects of question previewing on response accuracy and text processing: An eye-movement study.

The present study investigated the impact of manipulating reading strategies (i.e., reading the questions first [QF] or reading the passage first [PF]) during a reading comprehension test where we explored how reading strategy was related to student characteristics (i.e., reading achievement and working memory capacity). Participants' eye movements were monitored as they read 12 passages and answered multiple-choice questions. We examined differences in (a) response accuracy, (b) average total time on words in the text, (c) total task reading time, and (d) time reading text relevant to questions as a function of PF and QF strategies. Analyses were conducted to examine whether findings varied as a function of student characteristics (i.e., reading achievement and working memory capacity) and grade level (Grades 3, 5, and 8). Several interesting findings emerged from our study, including a limited effect of reading strategy use on response accuracy, with only eighth graders demonstrating better accuracy in the QF condition, and several demonstrations of PF leading to more efficient test-taking processes, including (a) longer average total reading times on words in the passage in the PF condition that could be associated with creating a better mental model of the text, (b) often being associated with less total-task time, and (c) being associated with more successful search strategies. Implications for providing teachers and students with strategies are discussed.

Effects of activity-oriented physiotherapy with and without eye movement training on dynamic balance, functional mobility, and eye movements in patients with Parkinson's disease: An assessor-blinded randomised controlled pilot trial.

OBJECTIVES: To describe changes in balance, walking speed, functional mobility, and eye movements following an activity-oriented physiotherapy (AOPT) or its combination with eye movement training (AOPT-E) in patients with Parkinson's disease (PD). To explore the feasibility of a full-scale randomised controlled trial (RCT). METHODS: Using an assessor-blinded pilot RCT, 25 patients with PD were allocated to either AOPT or AOPT-E. Supervised interventions were performed 30 minutes, 4x/weekly, for 4 weeks, alongside inpatient rehabilitation. Outcomes were assessed at baseline and post-intervention, including dynamic balance, walking speed, functional and dual-task mobility, ability to safely balance, health-related quality of life (HRQoL), depression, and eye movements (number/duration of fixations) using a mobile eye tracker. Freezing of gait (FOG), and falls-related self-efficacy were assessed at baseline, post-intervention, and 4-week follow-up. Effect sizes of 0.10 were considered weak, 0.30 moderate, and ≥0.50 strong. Feasibility was assessed using predefined criteria: recruitment, retention and adherence rates, adverse events, falls, and post-intervention acceptability using qualitative interviews. RESULTS: Improvements were observed in dynamic balance (effect size r = 0.216-0.427), walking speed (r = 0.165), functional and dual-task mobility (r = 0.306-0.413), ability to safely balance (r = 0.247), HRQoL (r = 0.024-0.650), and depression (r = 0.403). Falls-related self-efficacy (r = 0.621) and FOG (r = 0.248) showed varied improvements, partly sustained at follow-up. Eye movement improvements were observed after AOPT-E only. Feasibility analysis revealed that recruitment was below target, with less than two patients recruited per month due to COVID-19 restrictions. Feasibility targets were met, with a retention rate of 96% (95% confidence interval [CI]: 77.68-99.79) and a 98.18% (95% CI: 96.12-99.20) adherence rate, exceeding the targets of 80% and 75%, respectively. One adverse event unrelated to the study intervention confirmed intervention safety, and interview data indicated high intervention acceptability. CONCLUSIONS: AOPT-E and AOPT appeared to be effective in patients with PD. Feasibility of a larger RCT was confirmed and is needed to validate results.

Age effects on predictive eye movements for action.

When interacting with the environment, humans typically shift their gaze to where information is to be found that is useful for the upcoming action. With increasing age, people become slower both in processing sensory information and in performing their movements. One way to compensate for this slowing down could be to rely more on predictive strategies. To examine whether we could find evidence for this, we asked younger (19-29 years) and older (55-72 years) healthy adults to perform a reaching task wherein they hit a visual target that appeared at one of two possible locations. In separate blocks of trials, the target could appear always at the same location (predictable), mainly at one of the locations (biased), or at either location randomly (unpredictable). As one might expect, saccades toward predictable targets had shorter latencies than those toward less predictable targets, irrespective of age. Older adults took longer to initiate saccades toward the target location than younger adults, even when the likely target location could be deduced. Thus we found no evidence of them relying more on predictive gaze. Moreover, both younger and older participants performed more saccades when the target location was less predictable, but again no age-related differences were found. Thus we found no tendency for older adults to rely more on prediction.

Prefrontal cortex activity and functional organisation in dual-task ocular pursuit is affected by concurrent upper limb movement.

Tracking a moving object with the eyes seems like a simple task but involves areas of prefrontal cortex (PFC) associated with attention, working memory and prediction. Increasing the demand on these processes with secondary tasks can affect eye movements and/or perceptual judgments. This is particularly evident in chronic or acute neurological conditions such as Alzheimer's disease or mild traumatic brain injury. Here, we combined near infrared spectroscopy and video-oculography to examine the effects of concurrent upper limb movement, which provides additional afference and efference that facilitates tracking of a moving object, in a novel dual-task pursuit protocol. We confirmed the expected effects on judgement accuracy in the primary and secondary tasks, as well as a reduction in eye velocity when the moving object was occluded. Although there was limited evidence of oculo-manual facilitation on behavioural measures, performing concurrent upper limb movement did result in lower activity in left medial PFC, as well as a change in PFC network organisation, which was shown by Graph analysis to be locally and globally more efficient. These findings extend upon previous work by showing how PFC is functionally organised to support eye-hand coordination when task demands more closely replicate daily activities.

Eye movements track prioritized auditory features in selective attention to natural speech.

Over the last decades, cognitive neuroscience has identified a distributed set of brain regions that are critical for attention. Strong anatomical overlap with brain regions critical for oculomotor processes suggests a joint network for attention and eye movements. However, the role of this shared network in complex, naturalistic environments remains understudied. Here, we investigated eye movements in relation to (un)attended sentences of natural speech. Combining simultaneously recorded eye tracking and magnetoencephalographic data with temporal response functions, we show that gaze tracks attended speech, a phenomenon we termed ocular speech tracking. Ocular speech tracking even differentiates a target from a distractor in a multi-speaker context and is further related to intelligibility. Moreover, we provide evidence for its contribution to neural differences in speech processing, emphasizing the necessity to consider oculomotor activity in future research and in the interpretation of neural differences in auditory cognition.

A behavioral advantage for the face pareidolia illusion in peripheral vision.

Investigation of visual illusions helps us understand how we process visual information. For example, face pareidolia, the misperception of illusory faces in objects, could be used to understand how we process real faces. However, it remains unclear whether this illusion emerges from errors in face detection or from slower, cognitive processes. Here, our logic is straightforward; if examples of face pareidolia activate the mechanisms that rapidly detect faces in visual environments, then participants will look at objects more quickly when the objects also contain illusory faces. To test this hypothesis, we sampled continuous eye movements during a fast saccadic choice task-participants were required to select either faces or food items. During this task, pairs of stimuli were positioned close to the initial fixation point or further away, in the periphery. As expected, the participants were faster to look at face targets than food targets. Importantly, we also discovered an advantage for food items with illusory faces but, this advantage was limited to the peripheral condition. These findings are among the first to demonstrate that the face pareidolia illusion persists in the periphery and, thus, it is likely to be a consequence of erroneous face detection.

Improving Eye-Tracking Data Quality: A Framework for Reproducible Evaluation of Detection Algorithms.

High-quality eye-tracking data are crucial in behavioral sciences and medicine. Even with a solid understanding of the literature, selecting the most suitable algorithm for a specific research project poses a challenge. Empowering applied researchers to choose the best-fitting detector for their research needs is the primary contribution of this paper. We developed a framework to systematically assess and compare the effectiveness of 13 state-of-the-art algorithms through a unified application interface. Hence, we more than double the number of algorithms that are currently usable within a single software package and allow researchers to identify the best-suited algorithm for a given scientific setup. Our framework validation on retrospective data underscores its suitability for algorithm selection. Through a detailed and reproducible step-by-step workflow, we hope to contribute towards significantly improved data quality in scientific experiments.

Spiking activity in the visual thalamus is coupled to pupil dynamics across temporal scales.

The processing of sensory information, even at early stages, is influenced by the internal state of the animal. Internal states, such as arousal, are often characterized by relating neural activity to a single "level" of arousal, defined by a behavioral indicator such as pupil size. In this study, we expand the understanding of arousal-related modulations in sensory systems by uncovering multiple timescales of pupil dynamics and their relationship to neural activity. Specifically, we observed a robust coupling between spiking activity in the mouse dorsolateral geniculate nucleus (dLGN) of the thalamus and pupil dynamics across timescales spanning a few seconds to several minutes. Throughout all these timescales, 2 distinct spiking modes-individual tonic spikes and tightly clustered bursts of spikes-preferred opposite phases of pupil dynamics. This multi-scale coupling reveals modulations distinct from those captured by pupil size per se, locomotion, and eye movements. Furthermore, coupling persisted even during viewing of a naturalistic movie, where it contributed to differences in the encoding of visual information. We conclude that dLGN spiking activity is under the simultaneous influence of multiple arousal-related processes associated with pupil dynamics occurring over a broad range of timescales.

Oculomotor behavior can be adjusted on the basis of artificial feedback signals indicating externally caused errors.

Whether a saccade is accurate and has reached the target cannot be evaluated during its execution, but relies on post-saccadic feedback. If the eye has missed the target object, a secondary corrective saccade has to be made to align the fovea with the target. If a systematic post-saccadic error occurs, adaptive changes to the oculomotor behavior are made, such as shortening or lengthening the saccade amplitude. Systematic post-saccadic errors are typically attributed internally to erroneous motor commands. The corresponding adaptive changes to the motor command reduce the error and the need for secondary corrective saccades, and, in doing so, restore accuracy and efficiency. However, adaptive changes to the oculomotor behavior also occur if a change in saccade amplitude is beneficial for task performance, or if it is rewarded. Oculomotor learning thus is more complex than reducing a post-saccadic position error. In the current study, we used a novel oculomotor learning paradigm and investigated whether human participants are able to adapt their oculomotor behavior to improve task performance even when they attribute the error externally. The task was to indicate the intended target object among several objects to a simulated human-machine interface by making eye movements. The participants were informed that the system itself could make errors. The decoding process depended on a distorted landing point of the saccade, resulting in decoding errors. Two different types of visual feedback were added to the post-saccadic scene and we compared how participants used the different feedback types to adjust their oculomotor behavior to avoid errors. We found that task performance improved over time, regardless of the type of feedback. Thus, error feedback from the simulated human-machine interface was used for post-saccadic error evaluation. This indicates that 1) artificial visual feedback signals and 2) externally caused errors might drive adaptive changes to oculomotor behavior.