Competition between emotional faces in visuospatial working memory.

Visuospatial working memory (VSWM) helps track the identity and location of people during social interactions. Previous work showed better VSWM when all faces at encoding displayed a happy compared to an angry expression, reflecting a prosocial preference for monitoring who was where. However, social environments are not typically uniform, and certain expressions may more strongly compete for and bias face monitoring according to valence and/or arousal properties. Here, we used heterogeneous encoding displays in which two faces shared one emotion and two shared another, and asked participants to relocate a central neutral probe face after a blank delay. When considering the emotion of the probed face independently of the co-occurring emotion at encoding, an overall happy benefit was replicated. However, accuracy was modulated by the nonprobed emotion, with a relocation benefit for angry over sad, happy over fearful, and sad over happy faces. These effects did not depend on encoding fixation time, stimulus arousal, perceptual similarity, or response bias. Thus, emotional competition for faces in VSWM is complex and appears to rely on more than simple arousal- or valence-biased mechanisms. We propose a "social value (SV)" account to better explain when and why certain emotions may be prioritized in VSWM. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Top-down and bottom-up sources of eye-movement guidance during realistic scene search in Alzheimer's disease.

OBJECTIVE: Visual search is a crucial task in daily life, but in Alzheimer's disease (AD) it has usually been investigated using simple arrays. Here, we used scenes depicting real environments and studied the time course of attentional guidance. METHOD: We analyzed eye-movement differences between mild AD patients and age-matched healthy controls during search. We examined top-down guidance, manipulating the target template (precise picture vs. word cue) and the target-scene semantic consistency (consistent vs. inconsistent), and bottom-up guidance, manipulating the perceptual salience (high vs. low) of targets and distractors. RESULTS: During scene scanning, AD patients had longer search times, made more fixations before the first target fixation, and showed a greater probability of distractor selection, with longer distractor fixation. AD also led to longer target fixation. In patients and controls, picture cues and highly salient targets improved all search phases, whereas consistent targets only improved search initiation (first saccade). Moreover, top-down and bottom-up guidance interacted in initiation and scanning, and this did not differ between the two groups of participants. However, AD led to a smaller picture cue benefit in shortening distractor fixation and greater bottom-up search facilitation during scanning, where a high-salience target reduced the performance gap between patients and controls. CONCLUSIONS: Our study shows the importance of top-down and bottom-up guidance, and their integration, in improving search in AD patients. It suggests that precise target cues and, even more, highly salient targets may act as environmental supports that enhance attentional processing and search performance. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Age-related differences when searching in a real environment: The use of semantic contextual guidance and incidental object encoding.

Visual search is a crucial, everyday activity that declines with ageing. Here, referring to the environmental support account, we hypothesised that semantic contextual associations between the target and the neighbouring objects (e.g., a teacup near a tea bag and a spoon), acting as external cues, may counteract this decline. Moreover, when searching for a target, viewers may encode information about the co-present distractor objects, by simply looking at them. In everyday life, where viewers often search for several targets within the same environment, such distractor objects may often become targets of future searches. Thus, we examined whether incidentally fixating a target during previous trials, when it was a distractor, may also modulate the impact of ageing on search performance. We used everyday object arrays on tables in a real room, where healthy young and older adults had to search sequentially for multiple objects across different trials within the same array. We showed that search was quicker (1) in young than older adults; (2) for targets surrounded by semantically associated objects than unassociated objects, but only in older adults; and (3) for incidentally fixated targets than for targets that were not fixated when they were distractors, with no differences between young and older adults. These results suggest that older viewers use both environmental support based on object semantic associations and object information incidentally encoded to enhance efficiency of real-world search, even in relatively simple environments. This reduces, but does not eliminate, search decline related to ageing.

Top-down and bottom-up guidance in normal aging during scene search.

Age-related differences in visual search have been extensively studied using simple item arrays, showing an attentional decline. Little is known about how aging affects attentional guidance during search in more complex scenes. To study this issue, we analyzed eye-movement behavior in realistic scene search. We examined age-related differences in top-down guidance, manipulating target template specificity (picture vs. word cue) and target-scene semantic consistency (consistent vs. inconsistent), and in bottom-up guidance, manipulating perceptual salience (high vs. low) of targets and distractors. Compared to young adults (YA), older adults (OA) were overall slower, from the first saccade in the scene. They showed a smaller benefit of a specific target template, suggesting that precision of visual information in working memory may decrease with age. The benefit of semantic consistency did not depend on age, suggesting a preserved ability in OA to use knowledge about object occurrence in scenes. OA showed greater bottom-up search facilitation due to target's high salience, which may depend on reduced selection of low-salience stimuli. Attentional capture by distractors was greater in OA than YA, with respect to engagement (probability of distractor fixation), but only following a picture cue, and disengagement (fixation duration on distractors) in all conditions. Overall, our study shows that age-related differences in visual selection of targets and distractors depend on specific task demands in terms of top-down and bottom-up guidance. It also indicates that scene search difficulties in OA can be limited by cognitive and perceptual forms of environmental support. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Emotional scene processing in children and adolescents with attention deficit/hyperactivity disorder: a systematic review.

"Impairments in emotional information processing are frequently reported in attention deficit hyperactivity disorder (ADHD) at a voluntary, explicit level (e.g., emotion recognition) and at an involuntary, implicit level (e.g., emotional interference). Most of previous studies have used faces with emotional expressions, rarely examining other important sources of information usually co-occurring with faces in our every day experience. Here, we examined how the emotional content of an entire visual scene depicting real-world environments and situations is processed in ADHD. We systematically reviewed in PubMed, SCOPUS and ScienceDirect, using the PRISMA guidelines, empirical studies published in English until March 2019, about processing of visual scenes, with or without emotional content, in children and adolescents with ADHD. We included 17 studies among the 154 initially identified. Fifteen used scenes with emotional content (which was task-relevant in seven and irrelevant in eight studies) and two used scenes without emotional content. Even though the interpretation of the results differed according to the theoretical model of emotions of the study and the presence of comorbidity, differences in scene information processing between ADHD and typically developing children and adolescents were reported in all but one study. ADHD children and adolescents show difficulties in the processing of emotional information conveyed by visual scenes, which may stem from a stronger bottom-up impact of emotional stimuli in ADHD, increasing the emotional experience, and from core deficits of the disorder, decreasing the overall processing of the scene".

Author Correction: Reciprocal semantic predictions drive categorization of scene contexts and objects even when they are separate.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Reciprocal semantic predictions drive categorization of scene contexts and objects even when they are separate.

Visual categorization improves when object-context associations in scenes are semantically consistent, thus predictable from schemas stored in long-term memory. However, it is unclear whether this is due to differences in early perceptual processing, in matching of memory representations or in later stages of response selection. We tested these three concurrent explanations across five experiments. At each trial, participants had to categorize a scene context and an object briefly presented within the same image (Experiment 1), or separately in simultaneous images (Experiments 2-5). We analyzed unilateral (Experiments 1, 3) and bilateral presentations (Experiments 2, 4, 5), and presentations on the screen's horizontal midline (Experiments 1-2) and in the upper and lower visual fields (Experiments 3, 4). In all the experiments, we found a semantic consistency advantage for both context categorization and object categorization. This shows that the memory for object-context semantic associations is activated regardless of whether these two scene components are integrated in the same percept. Our study suggests that the facilitation effect of semantic consistency on categorization occurs at the stage of matching the percept with previous knowledge, supporting the object selection account and extending this framework to an object-context reciprocal influence on matching processes (object-context selection account).

Alzheimer's Disease, Visual Search, and Instrumental Activities of Daily Living: A Review and a New Perspective on Attention and Eye Movements.

Many instrumental activities of daily living (IADLs), like cooking and managing finances and medications, involve finding efficiently and in a timely manner one or several objects within complex environments. They may thus be disrupted by visual search deficits. These deficits, present in Alzheimer's disease (AD) from its early stages, arise from impairments in multiple attentional and memory mechanisms. A growing body of research on visual search in AD has examined several factors underlying search impairments in simple arrays. Little is known about how AD patients search in real-world scenes and in real settings, and about how such impairments affect patients' functional autonomy. Here, we review studies on visuospatial attention and visual search in AD. We then consider why analysis of patients' oculomotor behavior is promising to improve understanding of the specific search deficits in AD, and of their role in impairing IADL performance. We also highlight why paradigms developed in research on real-world scenes and real settings in healthy individuals are valuable to investigate visual search in AD. Finally, we indicate future research directions that may offer new insights to improve visual search abilities and autonomy in AD patients.

Remembering who was where: A happy expression advantage for face identity-location binding in working memory.

It is well established that visual working memory (WM) for face identity is enhanced when faces display threatening versus nonthreatening expressions. During social interaction, it is also important to bind person identity with location information in WM to remember who was where, but we lack a clear understanding of how emotional expression influences this. Here, we conducted two touchscreen experiments to investigate how angry versus happy expressions displayed at encoding influenced the precision with which participants relocated a single neutral test face to its original position. Maintenance interval was manipulated (Experiment 2; 1 s, 3 s, 6 s) to assess durability of binding. In both experiments, relocation accuracy was enhanced when faces were happy versus angry, and this happy benefit endured from 1-s to 6-s maintenance interval. Eye movement measures during encoding showed no convincing effects of oculomotor behavior that could readily explain the happy benefit. However, accuracy in general was improved, and the happy benefit was strongest for the last, most recent face fixated at encoding. Improved, durable binding of who was where in the presence of a happy expression may reflect the importance of prosocial navigation. (PsycINFO Database Record

The elephant in the room: Inconsistency in scene viewing and representation.

We examined the extent to which semantic informativeness, consistency with expectations and perceptual salience contribute to object prioritization in scene viewing and representation. In scene viewing (Experiments 1-2), semantic guidance overshadowed perceptual guidance in determining fixation order, with the greatest prioritization for objects that were diagnostic of the scene's depicted event. Perceptual properties affected selection of consistent objects (regardless of their informativeness) but not of inconsistent objects. Semantic and perceptual properties also interacted in influencing foveal inspection, as inconsistent objects were fixated longer than low but not high salience diagnostic objects. While not studied in direct competition with each other (each studied in competition with diagnostic objects), we found that inconsistent objects were fixated earlier and for longer than consistent but marginally informative objects. In change detection (Experiment 3), perceptual guidance overshadowed semantic guidance, promoting detection of highly salient changes. A residual advantage for diagnosticity over inconsistency emerged only when selection prioritization could not be based on low-level features. Overall these findings show that semantic inconsistency is not prioritized within a scene when competing with other relevant information that is essential to scene understanding and respects observers' expectations. Moreover, they reveal that the relative dominance of semantic or perceptual properties during selection depends on ongoing task requirements. (PsycINFO Database Record

Fixational saccades during grating detection and discrimination.

We investigated the patterns of fixational saccades in human observers performing two classical perceptual tasks: grating detection and discrimination. First, participants were asked to detect a vertical or tilted grating with one of three spatial frequencies and one of four luminance contrast levels. In the second experiment, participants had to discriminate the spatial frequency of two supra-threshold gratings. The gratings were always embedded in additive, high- or low-contrast pink noise. We observed that the patterns of fixational saccades were highly idiosyncratic among participants. Moreover, during the grating detection task, the amplitude and the number of saccades were inversely correlated with stimulus visibility. We did not find a systematic relationship between saccade parameters and grating frequency, apart from a slight decrease of saccade amplitude during grating discrimination with higher spatial frequencies. No consistent changes in the number and amplitude of fixational saccades with performance accuracy were reported. Surprisingly, during grating detection, saccade number and amplitude were similar in grating-with-noise and noise-only displays. Grating orientation did not affect substantially saccade direction in either task. The results challenge the idea that, when analyzing low-level spatial properties of visual stimuli, fixational saccades can be adapted in order to extract task-relevant information optimally. Rather, saccadic patterns seem to be overall modulated by task context, stimulus visibility and individual variability.

Disentangling the effects of spatial inconsistency of targets and distractors when searching in realistic scenes.

Previous research has suggested that correctly placed objects facilitate eye guidance, but also that objects violating spatial associations within scenes may be prioritized for selection and subsequent inspection. We analyzed the respective eye guidance of spatial expectations and target template (precise picture or verbal label) in visual search, while taking into account any impact of object spatial inconsistency on extrafoveal or foveal processing. Moreover, we isolated search disruption due to misleading spatial expectations about the target from the influence of spatial inconsistency within the scene upon search behavior. Reliable spatial expectations and precise target template improved oculomotor efficiency across all search phases. Spatial inconsistency resulted in preferential saccadic selection when guidance by template was insufficient to ensure effective search from the outset and the misplaced object was bigger than the objects consistently placed in the same scene region. This prioritization emerged principally during early inspection of the region, but the inconsistent object also tended to be preferentially fixated overall across region viewing. These results suggest that objects are first selected covertly on the basis of their relative size and that subsequent overt selection is made considering object-context associations processed in extrafoveal vision. Once the object was fixated, inconsistency resulted in longer first fixation duration and longer total dwell time. As a whole, our findings indicate that observed impairment of oculomotor behavior when searching for an implausibly placed target is the combined product of disruption due to unreliable spatial expectations and prioritization of inconsistent objects before and during object fixation.

How context information and target information guide the eyes from the first epoch of search in real-world scenes.

This study investigated how the visual system utilizes context and task information during the different phases of a visual search task. The specificity of the target template (the picture or the name of the target) and the plausibility of target position in real-world scenes were manipulated orthogonally. Our findings showed that both target template information and guidance of spatial context are utilized to guide eye movements from the beginning of scene inspection. In both search initiation and subsequent scene scanning, the availability of a specific visual template was particularly useful when the spatial context of the scene was misleading and the availability of a reliable scene context facilitated search mainly when the template was abstract. Target verification was affected principally by the level of detail of target template, and was quicker in the case of a picture cue. The results indicate that the visual system can utilize target template guidance and context guidance flexibly from the beginning of scene inspection, depending upon the amount and the quality of the available information supplied by either of these high-level sources. This allows for optimization of oculomotor behavior throughout the different phases of search within a real-world scene.

Semantic consistency versus perceptual salience in visual scenes: findings from change detection.

In a one-shot change detection task, we investigated the relationship between semantic properties (high consistency, i.e., diagnosticity, versus inconsistency with regard to gist) and perceptual properties (high versus low salience) of objects in guiding attention in visual scenes and in constructing scene representations. To produce the change an object was added or deleted in either the right or left half of coloured drawings of daily-life events. Diagnostic object deletions were more accurately detected than inconsistent ones, indicating rapid inclusion into early scene representation for the most predictable objects. Detection was faster and more accurate for high salience than for low salience changes. An advantage was found for diagnostic object changes in the high salience condition, although it was limited to additions when considering response speed. For inconsistent objects of high salience, deletions were detected faster than additions. These findings may indicate that objects are primarily selected on a perceptual basis with subsequent and supplementary effect of semantic consistency, in the sense of facilitation due to object diagnosticity or lengthening of processing time due to inconsistency.

The right hemisphere advantage in visual change detection depends on temporal factors.

What accounts for the Right Hemisphere (RH) functional superiority in visual change detection? An original task which combines one-shot and divided visual field paradigms allowed us to direct change information initially to the RH or the Left Hemisphere (LH) by deleting, respectively, an object included in the left or right half of a scene presented centrally. We manipulated the perceptual salience and semantic relevance of the change as well as the duration of the Inter-Stimulus Interval (ISI) between the scenes in order to clarify the role of the RH in memory and attention processes, and to explore whether lengthening the ISI would enhance the contribution of the LH. When analyzing data collapsed over the two levels (high vs. low) of salience and of relevance, changes were better detected in the left visual field (lvf) than in the right visual field (rvf) in the case of a short ISI, while no difference emerged in the case of a long ISI. Moreover, lengthening the ISI resulted in a performance decrement in the lvf, both for accuracy and response speed. The fact that the RH superiority was limited to short intervals indicates that stimulus-driven orienting contributes more than perceptual processing to this hemispheric asymmetry. When considering perceptual and semantic properties of the change, the effect of the ISI duration seemed to specifically emerge in the case of low relevance, with an enhancement of accuracy in the rvf when comparing the long with the short ISI. This suggests that the ISI influence on hemispheric performance operates on different levels.

Change detection in complex scenes: hemispheric contribution and the role of perceptual and semantic factors.

The perceptual salience and semantic relevance of objects for the meaning of a scene were evaluated with multiple criteria and then manipulated in a change-detection experiment that used an original combination of one-shot and tachistoscopic divided-visual-field paradigms to study behavioural hemispheric asymmetry. Coloured drawings that depicted meaningful situations were presented centrally and very briefly (120 ms) and only the changes were lateralised by adding an object in the right or in the left visual hemifield. High salience and high relevance improved both response times (RTs) and accuracy, although the overall contribution of salience was greater than that of relevance. Moreover, only for low-salience changes did relevance affect speed. RTs were shorter when a change occurred in the left visual hemifield, suggesting a right-hemisphere advantage for detection of visual change. Also, men responded faster than women. The theoretical and methodological implications are discussed.