Task-irrelevant semantic relationship between objects and scene influence attentional allocation.

Recent behavioral evidence suggests that the semantic relationships between isolated objects can influence attentional allocation, with highly semantically related objects showing an increase in processing efficiency. This semantic influence is present even when it is task-irrelevant (i.e., when semantic information is not central to the task). However, given that objects exist within larger contexts, i.e., scenes, it is critical to understand whether the semantic relationship between a scene and its objects continuously influence attention. Here, we investigated the influence of task-irrelevant scene semantic properties on attentional allocation and the degree to which semantic relationships between scenes and objects interact. Results suggest that task-irrelevant associations between scenes and objects continuously influence attention and that this influence is directly predicted by the perceived strength of semantic associations.

Predicting attentional allocation in real-world environments: The need to investigate crossmodal semantic guidance.

Real-world environments are multisensory, meaningful, and highly complex. To parse these environments in a highly efficient manner, a subset of this information must be selected both within and across modalities. However, the bulk of attention research has been conducted within sensory modalities, with a particular focus on vision. Visual attention research has made great strides, with over a century of research methodically identifying the underlying mechanisms that allow us to select critical visual information. Spatial attention, attention to features, and object-based attention have all been studied extensively. More recently, research has established semantics (meaning) as a key component to allocating attention in real-world scenes, with the meaning of an item or environment affecting visual attentional selection. However, a full understanding of how semantic information modulates real-world attention requires studying more than vision in isolation. The world provides semantic information across all senses, but with this extra information comes greater complexity. Here, we summarize visual attention (including semantic-based visual attention), crossmodal attention, and argue for the importance of studying crossmodal semantic guidance of attention. This article is categorized under: Psychology > Attention Psychology > Perception and Psychophysics.

Holistic processing is modulated by the probability that parts contain task-congruent information.

Holistic processing of face and non-face stimuli has been framed as a perceptual strategy, with classic hallmarks of holistic processing, such as the composite effect, reflecting a failure of selective attention, which is a consequence of this strategy. Further, evidence that holistic processing is impacted by training different patterns of attentional prioritization suggest that it may be a result of learned attention to the whole, which renders it difficult to attend to only part of a stimulus. If so, holistic processing should be modulated by the same factors that shape attentional selection, such as the probability that distracting or task-relevant information will be present. In contrast, other accounts suggest that it is the match to an internal face template that triggers specialized holistic processing mechanisms. Here we probed these accounts by manipulating the probability, across different testing sessions, that the task-irrelevant face part in the composite face task will contain task-congruent or -incongruent information. Attentional accounts of holistic processing predict that when the probability that the task-irrelevant part contains congruent information is low (25%), holistic processing should be attenuated compared to when this probability is high (75%). In contrast, template-based accounts of holistic face processing predict that it will be unaffected by manipulation given the integrity of the faces remains intact. Experiment 1 found evidence consistent with attentional accounts of holistic face processing and Experiment 2 extends these findings to holistic processing of non-face stimuli. These findings are broadly consistent with learned attention accounts of holistic processing.

Assessing the interaction between working memory and perception through time.

Content maintained in visual working memory changes concurrent visual processing, suggesting that visual working memory may recruit an overlapping neural representation with visual perception. However, it remains unclear whether visual working memory representations persist as a sensory code through time, or are recoded later into an abstract code. Here, we directly contrasted a temporal decay + visual code account and a temporal decay + abstract code account within the temporal dynamics of the interaction between working memory and perception. By manipulating the ISI (inter-stimulus interval) between working memory encoding and a perceptual discrimination task, we found that task-relevant and therefore actively maintained perceptual information parametrically altered participants' ability to discriminate perceptual stimuli even 4 s after encoding, whereas task-irrelevant information caused only an acutely transient effect. While continuously present, the size of this shift in discrimination thresholds gradually decreased over time. Concomitantly, the size of the bias in working memory reports increased over time. The opposing directions of threshold and bias effects are consistent with the local maintenance of information in perceptual areas, explained by a temporal decay + visual code account. As the maintained representation decays over time, its ability to alter incoming perceptual signals decreases (reduced threshold effects) while its likelihood of being impacted by those same signals increases (increased bias effects). Altogether, these results suggest that the readout of working memory relies on a sensory representation at a cost of increased interference by ongoing perception.

Attention and platypuses.

This perspective piece discusses a set of attentional phenomena that are not easily accommodated within current theories of attentional selection. We call these phenomena attentional platypuses, as they allude to an observation that within biological taxonomies the platypus does not fit into either mammal or bird categories. Similarly, attentional phenomena that do not fit neatly within current attentional models suggest that current models are in need of a revision. We list a few instances of the "attentional platypuses" and then offer a new approach, that we term dynamically weighted prioritization, stipulating that multiple factors impinge onto the attentional priority map, each with a corresponding weight. The interaction between factors and their corresponding weights determines the current state of the priority map which subsequently constrains/guides attentional allocation. We propose that this new approach should be considered as a supplement to existing models of attention, especially those that emphasize categorical organizations. This article is categorized under: Psychology > Attention Psychology > Perception and Psychophysics Neuroscience > Cognition.

Behavioral asymmetries in visual short-term memory occur in retinotopic coordinates.

Visual short-term memory (VSTM) is an essential store that creates continuous representations from disjointed visual input. However, severe capacity limits exist, reflecting constraints in supporting brain networks. VSTM performance shows spatial biases predicted by asymmetries in the brain based upon the location of the remembered object. Visual representations are retinotopic, or relative to location of the representation on the retina. It therefore stands to reason that memory performance may also show retinotopic biases. Here, eye position was manipulated to tease apart retinotopic coordinates from spatiotopic coordinates, or location relative to the external world. Memory performance was measured while participants performed a color change-detection task for items presented across the visual field while subjects fixated central or peripheral position. VSTM biases reflected the location of the stimulus on the retina, regardless of where the stimulus appeared on the screen. Therefore, spatial biases occur in retinotopic coordinates in VSTM and suggest a fundamental link between behavioral VSTM measures and visual representations.

Effective connectivity underlying neural and behavioral components of prism adaptation.

Prism adaptation (PA) is a form of visuomotor training that produces both sensorimotor and cognitive aftereffects depending on the direction of the visual displacement. Recently, a neural framework explaining both types of PA-induced aftereffects has been proposed, but direct evidence for it is lacking. We employed Structural Equation Modeling (SEM), a form of effective connectivity analysis, to establish directionality among connected nodes of the brain network thought to subserve PA. The findings reveal two distinct network branches: (1) a loop involving connections from the parietal cortices to the right parahippocampal gyrus, and (2) a branch linking the lateral premotor cortex to the parahippocampal gyrus via the cerebellum. Like the sensorimotor aftereffects, the first branch exhibited qualitatively different modulations for left versus right PA, and critically, changes in these connections were correlated with the magnitude of the sensorimotor aftereffects. Like the cognitive aftereffects, changes in the second branch were qualitatively similar for left and right PA, with greater change for left PA and a trend correlation with cognitive aftereffects. These results provide direct evidence that PA is supported by two functionally distinct subnetworks, a parietal-temporal network responsible for sensorimotor aftereffects and a fronto-cerebellar network responsible for cognitive aftereffects.

Mugs and Plants: Object Semantic Knowledge Alters Perceptual Processing With Behavioral Ramifications.

Neural processing of objects with action associations recruits dorsal visual regions more than the neural processing of objects without such associations. We hypothesized that because the dorsal and ventral visual pathways have differing proportions of magno- and parvocellular input, there should be behavioral differences in perceptual tasks between manipulable and nonmanipulable objects. This hypothesis was tested in college-age adults across five experiments (Ns = 26, 26, 30, 25, and 25) using a gap-detection task, suited to the spatial resolution of parvocellular processing, and an object-flicker-discrimination task, suited to the temporal resolution of magnocellular processing. Directly predicted from the cellular composition of each pathway, a strong nonmanipulable-object advantage was observed in gap detection, and a small manipulable-object advantage was observed in flicker discrimination. Additionally, these effects were modulated by reducing object recognition through inversion and by suppressing magnocellular processing using red light. These results establish perceptual differences between objects dependent on semantic knowledge.

How much is a cow like a meow? A novel database of human judgements of audiovisual semantic relatedness.

Semantic information about objects, events, and scenes influences how humans perceive, interact with, and navigate the world. The semantic information about any object or event can be highly complex and frequently draws on multiple sensory modalities, which makes it difficult to quantify. Past studies have primarily relied on either a simplified binary classification of semantic relatedness based on category or on algorithmic values based on text corpora rather than human perceptual experience and judgement. With the aim to further accelerate research into multisensory semantics, we created a constrained audiovisual stimulus set and derived similarity ratings between items within three categories (animals, instruments, household items). A set of 140 participants provided similarity judgments between sounds and images. Participants either heard a sound (e.g., a meow) and judged which of two pictures of objects (e.g., a picture of a dog and a duck) it was more similar to, or saw a picture (e.g., a picture of a duck) and selected which of two sounds it was more similar to (e.g., a bark or a meow). Judgements were then used to calculate similarity values of any given cross-modal pair. An additional 140 participants provided word judgement to calculate similarity of word-word pairs. The derived and reported similarity judgements reflect a range of semantic similarities across three categories and items, and highlight similarities and differences among similarity judgments between modalities. We make the derived similarity values available in a database format to the research community to be used as a measure of semantic relatedness in cognitive psychology experiments, enabling more robust studies of semantics in audiovisual environments.

From "satisfaction of search" to "subsequent search misses": a review of multiple-target search errors across radiology and cognitive science.

For over 50 years, the satisfaction of search effect has been studied within the field of radiology. Defined as a decrease in detection rates for a subsequent target when an initial target is found within the image, these multiple target errors are known to underlie errors of omission (e.g., a radiologist is more likely to miss an abnormality if another abnormality is identified). More recently, they have also been found to underlie lab-based search errors in cognitive science experiments (e.g., an observer is more likely to miss a target 'T' if a different target 'T' was detected). This phenomenon was renamed the subsequent search miss (SSM) effect in cognitive science. Here we review the SSM literature in both radiology and cognitive science and discuss: (1) the current SSM theories (i.e., satisfaction, perceptual set, and resource depletion theories), (2) the eye movement errors that underlie the SSM effect, (3) the existing efforts tested to alleviate SSM errors, and (4) the evolution of methodologies and analyses used when calculating the SSM effect. Finally, we present the attentional template theory, a novel mechanistic explanation for SSM errors, which ties together our current understanding of SSM errors and the attentional template literature.

Task-Irrelevant Semantic Properties of Objects Impinge on Sensory Representations within the Early Visual Cortex.

Objects can be described in terms of low-level (e.g., boundaries) and high-level properties (e.g., object semantics). While recent behavioral findings suggest that the influence of semantic relatedness between objects on attentional allocation can be independent of task-relevance, the underlying neural substrate of semantic influences on attention remains ill-defined. Here, we employ behavioral and functional magnetic resonance imaging measures to uncover the mechanism by which semantic information increases visual processing efficiency. We demonstrate that the strength of the semantic relatedness signal decoded from the left inferior frontal gyrus: 1) influences attention, producing behavioral semantic benefits; 2) biases spatial attention maps in the intraparietal sulcus, subsequently modulating early visual cortex activity; and 3) directly predicts the magnitude of behavioral semantic benefit. Altogether, these results identify a specific mechanism driving task-independent semantic influences on attention.

Callosal anisotropy predicts attentional network changes after parietal inhibitory stimulation.

Hemispatial neglect is thought to result from disruption of interhemispheric equilibrium. Right hemisphere lesions deactivate the right frontoparietal network and hyperactivate the left via release from interhemispheric inhibition. Support for this putative mechanism comes from neuropsychological evidence as well as transcranial magnetic stimulation (TMS) studies in healthy subjects, in whom right posterior parietal cortex (PPC) inhibition causes neglect-like, rightward, visuospatial bias. Concurrent TMS and fMRI after right PPC TMS show task-dependent changes but may fail to identify effects of stimulation in areas not directly activated by the specific task, complicating interpretations. We used resting-state functional connectivity (RSFC) after inhibitory TMS over the right PPC to examine changes in the networks underlying visuospatial attention and used diffusion-weighted imaging to measure the structural properties of relevant white matter pathways. In a crossover experiment in healthy individuals, we delivered continuous theta burst TMS to the right PPC and vertex as control condition. We hypothesized that PPC inhibitory stimulation would result in a rightward visuospatial bias, decrease frontoparietal RSFC, and increase the PPC RSFC with the attentional network in the left hemisphere. We also expected that individual differences in fractional anisotropy (FA) of the frontoparietal network and the callosal pathway between the PPCs would account for variability of the TMS-induced RSFC changes. As hypothesized, TMS over the right PPC caused a rightward shift in line bisection judgment and increased RSFC between the right PPC and the left superior temporal gyrus. This effect was inversely related to FA in the posterior corpus callosum. Local inhibition of the right PPC reshapes connectivity in the attentional network and depends significantly on interhemispheric connections.

Target frequency modulates object-based attention.

Decades of research have provided evidence that object representations contribute to attentional selection. However, most evidence for object-based attentional allocation is drawn from studies employing the two-rectangle paradigm where the target distribution is biased towards the cued object. It is thus unclear whether object-based attentional selection is from object representations or a consequence of spatial attention based on statistical imbalances. Here, we investigate the extent to which target frequency modulates object-based attention by systematically manipulating the frequency of target appearance in a particular spatial location within objects to equate spatial allocation, bias specific spatial locations, or bias objects. In four experiments, participants were presented with a variant of the two-rectangle paradigm in which one end of a rectangle was cued and performed a target discrimination task. Critically, the target location probabilities were parametrically manipulated. The target could appear equally in all ends within the objects (valid, invalid within-object, invalid between-object, diagonal) (Experiment 1) or with overall equality between objects but biased towards the invalid locations (Experiment 2). The target could also appear in three locations (valid, invalid within-object, invalid between-object) distributed equally between objects but biased towards the invalid between-object location (Experiment 3) or with an overall bias towards the invalid between-object location (Experiment 4). We observed that while objects bias attention, spatial biases are prioritized over object representations. Combined results suggest that object-based contribution to attentional guidance is the result of both spatial probabilities and object representations.

Neural Correlates of Perceptual Grouping Under Conditions of Inattention and Divided Attention.

Grouping local elements of the visual environment together is crucial for meaningful perception. While our attentional system facilitates perception, it is limited in that we are unaware of some aspects of our environment that can still influence how we experience it. In this study, the neural mechanisms underlying the Ponzo illusion were examined under inattention and divided-attention conditions using functional magnetic resonance imaging to investigate the brain regions responsible for accessing visual stimuli. A line discrimination task was performed in which two horizontal lines were superimposed on a background of black and white dots that, on occasion, induced the Ponzo illusion if perceptually grouped together. Our findings revealed activation for perceptual grouping in the frontal, parietal, and occipital regions of the brain and activation in the bilateral frontal, temporal, and cingulate gyrus in response to divided attention compared with inattention trials. A direct comparison between grouping and attention showed involvement of the right supramarginal gyrus in grouping specifically under conditions of inattention, suggesting that even during implicit grouping complex visual processing occurs. Given that much of the visual world is not represented in conscious perception, these findings provide crucial information about how we make sense of visual scenes in the world.

Prism Adaptation Modulates Connectivity of the Intraparietal Sulcus with Multiple Brain Networks.

Prism adaptation (PA) alters spatial cognition according to the direction of visual displacement by temporarily modifying sensorimotor mapping. Right-shifting prisms (right PA) improve neglect of left visual field in patients, possibly by decreasing activity in the left hemisphere and increasing it in the right. Left PA shifts attention rightward in healthy individuals by an opposite mechanism. However, functional imaging studies of PA are inconsistent, perhaps because of differing activation tasks. We measured resting-state functional connectivity (RSFC) in healthy individuals before and after PA. When contrasted, right versus left PA decreased RSFC in the spatial navigation network defined by the right posterior parietal cortex (PPC), hippocampus, and cerebellum. Within-PA-direction comparisons showed that right PA increased RSFC in subregions of the PPCs and between the PPCs and the right middle frontal gyrus and left PA decreased RSFC between these regions. Both right and left PA decreased RSFC between the PPCs and bilateral temporal areas. In summary, right PA increases connectivity in the right frontoparietal network and left PA produces essentially opposite effects. Furthermore, right, compared with left, PA modulates RSFC in the right hemisphere navigation network.

Invalidly cued targets are well localized when detected.

Considerable attention has been devoted to understanding how objects are localized when there is ample time and attention to detect them. However, in the real world, we often must react to, or act upon, objects that we have glimpsed only briefly and are not directly at the focus of our attention. This paper describes two experiments examining the role of attentional constraints on 2-D (directional) localization, particularly in cases in which targets have been detected but are not within the spatial focus of attention. Targets were asterisks presented briefly (34-150 ms) above or below a central fixation point. Just prior to the target's appearance, a cue directed attention toward, or away from, the target. Participants indicated whether or not they saw the target, and then used a mouse to indicate the target's location. The impact of guessing was mitigated by removing trials that participants had flagged as not detected. Longer glimpses generally benefitted localization; by contrast, cue validity had very little effect on response sensitivity, bias or precision. At very brief durations, invalid cueing did result in a small increase in foveal bias. These results indicate that the directional location of objects can be extracted reasonably well from brief glimpses even with reduced attention. This directional information provides an important basis for 3-D localization of objects on the ground, via their angular declination. The current studies suggest that egocentric distance perception might be similarly robust to reduced attention when localization is based primarily on a target's angular declination.

Attention scales according to inferred real-world object size.

Natural scenes consist of objects of varying shapes and sizes. The impact of object size on visual perception has been well-demonstrated, from classic mental imagery experiments1, to recent studies of object representations reporting topographic organization of object size in the occipito-temporal cortex2. While the role of real-world physical size in perception is clear, the effect of inferred size on attentional selection is ill-defined. Here, we investigate whether inferred real-world object size influences attentional allocation. Across five experiments, attentional allocation was measured in objects of equal retinal size, but varied in inferred real-world size (for example, domino, bulldozer). Following each experiment, participants rated the real-world size of each object. We hypothesized that, if inferred real-world size influences attention, selection in retinal size-matched objects should be less efficient in larger objects. This effect should increase with greater attentional demand. Predictions were supported by faster identified targets in objects inferred to be small than large, with costlier attentional shifting in large than small objects when attentional demand was high. Critically, there was a direct correlation between the rated size of individual objects and response times (and shifting costs). Finally, systematic degradation of size inference proportionally reduced object size effect. It is concluded that, along with retinal size, inferred real-world object size parametrically modulates attention. These findings have important implications for models of attentional control and invite sensitivity to object size for future studies that use real-world images in psychological research.

Intrusive effects of task-irrelevant information on visual selective attention: semantics and size.

Attentional selection is a mechanism by which incoming sensory information is prioritized for further, detailed, and more effective, processing. Given that attended information is privileged by the sensory system, understanding and predicting what information is granted prioritization becomes an important endeavor. It has been argued that salient events as well as information that is related to the current goal of the organism (i.e., task-relevant) receive such a priority. Here, we propose that attentional prioritization is not limited to task-relevance, and discuss evidence showing that task-irrelevant, non-salient, high-level properties of unattended objects, namely object meaning and size, influence attentional allocation. Such an intrusion of non-salient, task-irrelevant, high-level information points to the need to re-conceptualize and formally modify current models of attentional guidance.

Left-shifting prism adaptation boosts reward-based learning.

Visuospatial cognition has an inherent lateralized bias. Individual differences in the direction and magnitude of this bias are associated with asymmetrical D2/3 dopamine binding and dopamine system genotypes. Dopamine level affects feedback-based learning and dopamine signaling asymmetry is related to differential learning from reward and punishment. High D2 binding in the left hemisphere is associated with preference for reward. Prism adaptation (PA) is a simple sensorimotor technique, which modulates visuospatial bias according to the direction of the deviation. Left-deviating prism adaptation (LPA) induces rightward bias in healthy subjects. It is therefore possible that the right side of space increases in saliency along with left hemisphere dopaminergic activity. Right-deviating prism adaptation (RPA) has been used mainly as a control condition because it does not modulate behavior in healthy individuals. Since LPA induces a rightward visuospatial bias as a result of left hemisphere modulation, and higher dopaminergic activity in the left hemisphere is associated with preference for rewarding events we hypothesized that LPA would increase the preference for learning with reward. Healthy volunteers performed a computer-based probabilistic classification task before and after LPA or RPA. Consistent with our predictions, PA altered the preference for rewarded versus punished learning, with the LPA group exhibiting increased learning from reward. These results suggest that PA modulates dopaminergic activity in a lateralized fashion.