Anaphoric distance dependencies in visual narrative structure and processing.

Linguistic syntax has often been claimed as uniquely complex due to features like anaphoric relations and distance dependencies. However, visual narratives of sequential images, like those in comics, have been argued to use sequencing mechanisms analogous to those in language. These narrative structures include "refiner" panels that "zoom in" on the contents of another panel. Similar to anaphora in language, refiners indexically connect inexplicit referential information in one unit (refiner, pronoun) to a more informative "antecedent" elsewhere in the discourse. Also like in language, refiners can follow their antecedents (anaphoric) or precede them (cataphoric), along with having either proximal or distant connections. We here explore the constraints on visual narrative refiners created by modulating these features of order and distance. Experiment 1 examined participants' preferences for where refiners are placed in a sequence using a force-choice test, which revealed that refiners are preferred to follow their antecedents and have proximal distances from them. Experiment 2 then showed that distance dependencies lead to slower self-paced viewing times. Finally, measurements of event-related brain potentials (ERPs) in Experiment 3 revealed that these patterns evoke similar brain responses as referential dependencies in language (i.e., N400, LAN, Nref). Across all three studies, the constraints and (neuro)cognitive responses to refiners parallel those shown to anaphora in language, suggesting domain-general constraints on the sequencing of referential dependencies.

Social prioritisation in scene viewing and the effects of a spatial memory load.

When free-viewing scenes, participants tend to preferentially fixate social elements (e.g., people). In the present study, we tested whether this bias would be disrupted by increasing the demands of a secondary dual-task: holding a set of (one or six) spatial locations in memory, presented either simultaneously or sequentially. Following a retention interval, participants judged whether a test location was present in the to-be-remembered stimuli. During the retention interval participants free-viewed scenes containing a social element (a person) and a non-social element (an object) that served as regions of interest. In order to assess the impact of physical salience, the non-social element was presented in both an unaltered baseline version, and in a version where its salience was artificially increased. The results showed that the preference to look at social elements decreased when the demands of the spatial memory task were increased from one to six locations, regardless of presentation mode (simultaneous or sequential). The high-load condition also resulted in more central fixations and reduced exploration of the scene. The results indicate that the social prioritisation effect, and scene viewing more generally, can be affected by a concurrent memory load.

Eye movements in visual impairment.

This Special Issue describes the impact of visual impairment on visuomotor function. It includes contributions that examine gaze control in conditions associated with abnormal visual development such as amblyopia, dyslexia and neurofibromatosis as well as disorders associated with field loss later in life, such as macular degeneration and stroke. Specifically, the papers address both gaze holding (fixation), and gaze-following behavior (single saccades, sequences of saccades and smooth-pursuit) that characterize active vision in daily life and evaluate the influence of both pathological and simulated field loss. Several papers address the challenges to reading and visual search; describing how the patterns of eye movements in these real-world tasks adapt to visual impairment and highlighting how they could serve as diagnostic markers of visuomotor function.

Parent-reported social-communication changes in children diagnosed with autism spectrum disorder during the COVID-19 pandemic in the UK.

Introduction: The coronavirus pandemic has swept across the United Kingdom (UK). Given the ever-evolving situation, little is known about the repercussions of coronavirus and the subsequent lockdowns for children diagnosed with Autism Spectrum Disorder (ASD). Therefore, this study explores the social-communicative impact of the first lockdown (March 2020 - July 2020) in the UK and the return to school period (September 2020 - October 2020), following prolonged disruption to routine, in children diagnosed with ASD. Methods : Parents of autistic children completed 2 separate online surveys following the first lockdown in the UK (n = 176) and also when children returned to school following the summer break (n = 54). Results : The results suggested that self-regulation skills (p < .05) and co-operation skills (p < .05) were most affected over the course of the lockdown. Children's physical activity levels were perceived to significantly increase during the return to school (p < .0001), which was associated with better social-communication outcomes (p < .05). Conclusion : Future work is needed to confirm and explore the findings. Such work could be implemented to protect and improve the social-communicative outcomes of autistic children.

Reading the room: Autistic traits, gaze behaviour, and the ability to infer social relationships.

Individuals high in autistic traits can have difficulty understanding verbal and non-verbal cues, and may display atypical gaze behaviour during social interactions. The aim of this study was to examine differences among neurotypical individuals with high and low levels of autistic traits with regard to their gaze behaviour and their ability to assess peers' social status accurately. Fifty-four university students who completed the 10-item Autism Quotient (AQ-10) were eye-tracked as they watched six 20-second video clips of people ("targets") involved in a group decision-making task. Simulating natural, everyday social interactions, the video clips included moments of debate, humour, interruptions, and cross talk. Results showed that high-scorers on the AQ-10 (i.e., those with more autistic traits) did not differ from the low-scorers in either gaze behaviour or assessing the targets' relative social status. The results based on this neurotypical group of participants suggest that the ability of individuals high in autistic traits to read social cues may be preserved in certain tasks crucial to navigating day-to-day social relationships. These findings are discussed in terms of their implications for theory of mind, weak central coherence, and social motivation theories of autism.

Dancing out for a voice; a narrative review of the literature exploring autism, physical activity, and dance.

Autism Spectrum Disorder is characterised by profound challenges with social communication and social interaction. Currently, there are few therapeutic interventions that successfully target some of the functionally impairing traits associated with autism. Furthermore, many of these interventions comprise a variety of limitations; including, limited accessibility, extensive durations, or the requirement of a trained professional to deliver the intervention. New research suggests that instead of targeting all traits associated with Autism Spectrum Disorder with a single solution, scientific research should focus on providing therapeutic tools that alleviate functionally impairing facets specific to the individual. Owing to the nature of physical activity, sports, and dance (coordinated movement) these activities could provide opportunities to enhance communication skills and social development in autistic children. Therefore, this paper gives a narrative overview of the literature surrounding communication and coordinated movement; outlining what is meant by communication challenges, exploring the benefits of coordinated movement for traits associated with Autism Spectrum Disorder, and delineating how coordinated movement elicits positive outcomes for autistic children.

Eye tracking: empirical foundations for a minimal reporting guideline.

In this paper, we present a review of how the various aspects of any study using an eye tracker (such as the instrument, methodology, environment, participant, etc.) affect the quality of the recorded eye-tracking data and the obtained eye-movement and gaze measures. We take this review to represent the empirical foundation for reporting guidelines of any study involving an eye tracker. We compare this empirical foundation to five existing reporting guidelines and to a database of 207 published eye-tracking studies. We find that reporting guidelines vary substantially and do not match with actual reporting practices. We end by deriving a minimal, flexible reporting guideline based on empirical research (Section "An empirically based minimal reporting guideline").

Meaning above (and in) the head: Combinatorial visual morphology from comics and emoji.

Compositionality is a primary feature of language, but graphics can also create combinatorial meaning, like with items above faces (e.g., lightbulbs to mean inspiration). We posit that these "upfixes" (i.e., upwards affixes) involve a productive schema enabling both stored and novel face-upfix dyads. In two experiments, participants viewed either conventional (e.g., lightbulb) or unconventional (e.g., clover-leaves) upfixes with faces which either matched (e.g., lightbulb/smile) or mismatched (e.g., lightbulb/frown). In Experiment 1, matching dyads sponsored higher comprehensibility ratings and faster response times, modulated by conventionality. In Experiment 2, event-related brain potentials (ERPs) revealed conventional upfixes, regardless of matching, evoked larger N250s, indicating perceptual expertise, but mismatching and unconventional dyads elicited larger semantic processing costs (N400) than conventional-matching dyads. Yet mismatches evoked a late negativity, suggesting congruent novel dyads remained construable compared with violations. These results support that combinatorial graphics involve a constrained productive schema, similar to the lexicon of language.

Do cognitive load and ADHD traits affect the tendency to prioritise social information in scenes?

We report two experiments investigating the effect of working memory (WM) load on selective attention. Experiment 1 was a modified version of Lavie et al. and confirmed that increasing memory load disrupted performance in the classic flanker task. Experiment 2 used the same manipulation of WM load to probe attention during the viewing of complex scenes while also investigating individual differences in attention deficit hyperactivity disorder (ADHD) traits. In the image-viewing task, we measured the degree to which fixations targeted each of two crucial objects: (1) a social object (a person in the scene) and (2) a non-social object of higher or lower physical salience. We compared the extent to which increasing WM load would change the pattern of viewing of the physically salient and socially salient objects. If attending to the social item requires greater default voluntary top-down resources, then the viewing of social objects should show stronger modulation by WM load compared with viewing of physically salient objects. The results showed that the social object was fixated to a greater degree than the other object (regardless of physical salience). Increased salience drew fixations away from the background leading to slightly increased fixations on the non-social object, without changing fixations on the social object. Increased levels of ADHD-like traits were associated with fewer fixations on the social object, but only in the high-salient, low-load condition. Importantly, WM load did not affect the number of fixations on the social object. Such findings suggest rather surprisingly that attending to a social area in complex stimuli is not dependent on the availability of voluntary top-down resources.

Theory of mind affects the interpretation of another person's focus of attention.

People are drawn to social, animate things more than inanimate objects. Previous research has also shown gaze following in humans, a process that has been linked to theory of mind (ToM). In three experiments, we investigated whether animacy and ToM are involved when making judgements about the location of a cursor in a scene. In Experiment 1, participants were told that this cursor represented the gaze of an observer and were asked to decide whether the observer was looking at a target object. This task is similar to that carried out by researchers manually coding eye-tracking data. The results showed that participants were biased to perceive the gaze cursor as directed towards animate objects (faces) compared to inanimate objects. In Experiments 2 and 3 we tested the role of ToM, by presenting the same scenes to new participants but now with the statement that the cursor was generated by a 'random' computer system or by a computer system designed to seek targets. The bias to report that the cursor was directed toward faces was abolished in Experiment 2, and minimised in Experiment 3. Together, the results indicate that people attach minds to the mere representation of an individual's gaze, and this attribution of mind influences what people believe an individual is looking at.

Zooming in on visual narrative comprehension.

The comprehension of visual narratives requires paying attention to certain elements and integrating them across a sequence of images. To study this process, we developed a new approach that modified comic strips according to where observers looked while viewing each sequence. Across three self-paced experiments, we presented sequences of six panels that were sometimes automatically "zoomed-in" or re-framed in order to highlight parts of the image that had been fixated by another group of observers. Fixation zoom panels were rated as easier to understand and produced viewing times more similar to the original comic than panels modified to contain non-fixated or incongruous regions. When a single panel depicting the start of an action was cropped to show only the most fixated region, viewing times were similar to the original narrative despite the reduced information. Modifying such panels also had an impact on the viewing time on subsequent panels, both when zoomed in and when regions were highlighted through an "inset" panel. These findings demonstrate that fixations in a visual narrative are guided to informative elements, and that these elements influence both the current panel and the processing of the sequence.

Zooming in on the cognitive neuroscience of visual narrative.

Visual narratives like comics and films often shift between showing full scenes and close, zoomed-in viewpoints. These zooms are similar to the "spotlight of attention" cast across a visual scene in perception. We here measured ERPs to visual narratives (comic strips) that used zoomed-in and full-scene panels either throughout the whole sequence context or at specific critical panels. Zoomed-in panels were automatically generated on the basis of fixations from prior participants' eye movements to the crucial content of panels (Foulsham & Cohn, 2020). We found that these fixation panels evoked a smaller N300 than full-scenes, indicative of reduced cost for object identification, but that they also evoked a slightly larger amplitude N400 response, suggesting a greater cost for accessing semantic memory with constrained content. Panels in sequences where fixation panels persisted across all positions of the sequence also evoked larger posterior P600s, implying that constrained views required more updating or revision processes throughout the sequence. Altogether, these findings suggest that constraining a visual scene to its crucial parts triggers various processes related not only to the density of its information but also to its integration into a sequential context.

Athlete-Opponent Interdependency Alters Pacing and Information-Seeking Behavior.

PURPOSE: The influence of interdependency between competitors on pacing decision-making and information-seeking behavior has been explored. This has been done by only altering instructions, and thereby action possibilities, while controlling environment (i.e., competitor behavior) and exercise task. METHODS: Twelve participants performed a 4-km time trial on a Velotron cycle ergometer in a randomized, counterbalanced order alone with no virtual opponent (NO), against a virtual opponent with no restrictions (low athlete-opponent interdependency [OP-IND]), or against a virtual opponent who the participant was permitted to overtake only once during the trial (high athlete-opponent interdependency [OP-DEP]). Information-seeking behavior was evaluated using an SMI eye tracker. Differences in pacing, performance, and information-seeking behavior were examined using repeated-measures ANOVA (P < 0.05). RESULTS: Neither mean power output (NO, 298 ± 35 W; OP-IND, 297 ± 38 W; OP-DEP, 296 ± 37 W) nor finishing time (NO, 377.7 ± 17.4 s; OP-IND, 379.3 ± 19.5 s; OP-DEP, 378.5 ± 17.7 s) differed between experimental conditions. However, power output was lower in the first kilometer of OP-DEP compared with the other experimental conditions (NO, 332 ± 59 W; OP-IND, 325 ± 62 W; OP-DEP, 316 ± 58 W; both P < 0.05), and participants decided to wait longer before they overtook their opponent (OP-IND, 137 ± 130 s; OP-DEP, 255 ± 107 s; P = 0.040). Moreover, total fixation time spent on the avatar of the virtual opponent increased when participants were only allowed to overtake once (OP-IND, 23.3 ± 16.6 s; OP-DEP, 55.8 ± 32.7 s; P = 0.002). CONCLUSION: A higher interdependency between athlete and opponent altered pacing behavior in terms of in-race adaptations based on opponent's behavior, and it induced an increased attentional focus on the virtual opponent. Thus, in the context of exercise regulation, attentional cues are likely to be used in an adaptive way according to their availability and situational relevance, consistent with a decision-making framework based on the interdependence of perception and action.

Turning the (virtual) world around: Patterns in saccade direction vary with picture orientation and shape in virtual reality.

Research investigating gaze in natural scenes has identified a number of spatial biases in where people look, but it is unclear whether these are partly due to constrained testing environments (e.g., a participant with their head restrained and looking at a landscape image framed within a computer monitor). We examined the extent to which image shape (square vs. circle), image rotation, and image content (landscapes vs. fractal images) influence eye and head movements in virtual reality (VR). Both the eyes and head were tracked while observers looked at natural scenes in a virtual environment. In line with previous work, we found a bias for saccade directions parallel to the image horizon, regardless of image shape or content. We found that, when allowed to do so, observers move both their eyes and head to explore images. Head rotation, however, was idiosyncratic; some observers rotated a lot, whereas others did not. Interestingly, the head rotated in line with the rotation of landscape but not fractal images. That head rotation and gaze direction respond differently to image content suggests that they may be under different control systems. We discuss our findings in relation to current theories on head and eye movement control and how insights from VR might inform more traditional eye-tracking studies.

Reading and Misleading: Changes in Head and Eye Movements Reveal Attentional Orienting in a Social Context.

Social attention describes how observers orient to social information and exhibit behaviors such as gaze following. These behaviors are examples of how attentional orienting may differ when in the presence of other people, although they have typically been studied without actual social presence. In the present study we ask whether orienting, as measured by head and eye movements, will change when participants are trying to mislead or hide their attention from a bystander. In two experiments, observers performed a preference task while being video-recorded, and subsequent participants were asked to guess the response of the participant based on a video of the head and upper body. In a second condition, observers were told to try to mislead the "guesser". The results showed that participants' preference responses could be guessed from videos of the head and, critically, that participants spontaneously changed their orienting behavior in order to mislead by reducing the rate at which they made large head movements. Masking the eyes with sunglasses suggested that head movements were most important in our setup. This indicates that head and eye movements can be used flexibly according to the socio-communicative context.

Attention to the face is characterised by a difficult to inhibit first fixation to the eyes.

The eyes are preferentially attended over other facial features and recent evidence suggests this bias is difficult to suppress. To further examine the automatic and volitional nature of this bias for eye information, we used a novel prompting face recognition paradigm in 41 adults and measured the location of their first fixations, overall dwell time and behavioural responses. First, patterns of eye gaze were measured during a free-viewing forced choice face recognition paradigm. Second, the task was repeated but with prompts to look to either the eyes or the mouth. Participants showed significantly more first fixations to the eyes than mouth, both when prompted to look at the eyes and when prompted to look at the mouth. The pattern of looking to the eyes when prompted was indistinguishable from the unprompted condition in which participants were free to look where they chose. Notably, the dwell time data demonstrated that the eye bias did not persist over the entire presentation period. Our results suggest a difficult-to-inhibit bias to initially orient to the eyes, which is superseded by volitional, top-down control of eye gaze. Further, the amount of looking to the eyes is at a maximum level spontaneously and cannot be enhanced by explicit instructions.

Understanding the Effect of Information Presentation Order and Orientation on Information Search and Treatment Evaluation.

BACKGROUND: Past research finds that treatment evaluations are more negative when risks are presented after benefits. This study investigates this order effect: manipulating tabular orientation and order of risk-benefit information, and examining information search order and gaze duration via eye-tracking. DESIGN: 108 (Study 1) and 44 (Study 2) participants viewed information about treatment risks and benefits, in either a horizontal (left-right) or vertical (above-below) orientation, with the benefits or risks presented first (left side or at top). For 4 scenarios, participants answered 6 treatment evaluation questions (1-7 scales) that were combined into overall evaluation scores. In addition, Study 2 collected eye-tracking data during the benefit-risk presentation. RESULTS: Participants tended to read one set of information (i.e., all risks or all benefits) before transitioning to the other. Analysis of order of fixations showed this tendency was stronger in the vertical (standardized mean rank difference further from 0, M = ± .88) than horizontal orientation ( M = ± 0.71). Approximately 50% of the time was spent reading benefits when benefits were shown first, but this was reduced to ~40% when risks were presented first (regression coefficient: B = -4.52, p < .001). Eye-tracking measures did not strongly predict treatment evaluations, although time percentage reading benefits positively predicted evaluation when holding other variables constant ( B = 0.02, p = .023). CONCLUSION: These results highlight the impact of seemingly arbitrary design choices on inspection order. For instance, presenting risks where they will be seen first leads to relatively less time spent considering treatment benefits. Other research suggests these changes to inspection order can influence multi-option and multi-attribute choices, and represent an area for future research.

How task demands influence scanpath similarity in a sequential number-search task.

More and more researchers are considering the omnibus eye movement sequence-the scanpath-in their studies of visual and cognitive processing (e.g. Hayes, Petrov, & Sederberg, 2011; Madsen, Larson, Loschky, & Rebello, 2012; Ni et al., 2011; von der Malsburg & Vasishth, 2011). However, it remains unclear how recent methods for comparing scanpaths perform in experiments producing variable scanpaths, and whether these methods supplement more traditional analyses of individual oculomotor statistics. We address this problem for MultiMatch (Jarodzka et al., 2010; Dewhurst et al., 2012), evaluating its performance with a visual search-like task in which participants must fixate a series of target numbers in a prescribed order. This task should produce predictable sequences of fixations and thus provide a testing ground for scanpath measures. Task difficulty was manipulated by making the targets more or less visible through changes in font and the presence of distractors or visual noise. These changes in task demands led to slower search and more fixations. Importantly, they also resulted in a reduction in the between-subjects scanpath similarity, demonstrating that participants' gaze patterns became more heterogenous in terms of saccade length and angle, and fixation position. This implies a divergent strategy or random component to eye-movement behaviour which increases as the task becomes more difficult. Interestingly, the duration of fixations along aligned vectors showed the opposite pattern, becoming more similar between observers in 2 of the 3 difficulty manipulations. This provides important information for vision scientists who may wish to use scanpath metrics to quantify variations in gaze across a spectrum of perceptual and cognitive tasks.