Incorporating simulated spatial context information improves the effectiveness of contrastive learning models.

Visual learning often occurs in a specific context, where an agent acquires skills through exploration and tracking of its location in a consistent environment. The historical spatial context of the agent provides a similarity signal for self-supervised contrastive learning. We present a unique approach, termed environmental spatial similarity (ESS), that complements existing contrastive learning methods. Using images from simulated, photorealistic environments as an experimental setting, we demonstrate that ESS outperforms traditional instance discrimination approaches. Moreover, sampling additional data from the same environment substantially improves accuracy and provides new augmentations. ESS allows remarkable proficiency in room classification and spatial prediction tasks, especially in unfamiliar environments. This learning paradigm has the potential to enable rapid visual learning in agents operating in new environments with unique visual characteristics. Potentially transformative applications span from robotics to space exploration. Our proof of concept demonstrates improved efficiency over methods that rely on extensive, disconnected datasets.

Transient Attention Gates Access Consciousness: Coupling N2pc and P3 Latencies Using Dynamic Time Warping.

The N2pc and P3 event-related potentials (ERPs), used to index selective attention and access to working memory and conscious awareness, respectively, have been important tools in cognitive sciences. Although it is likely that these two components and the underlying cognitive processes are temporally and functionally linked, such links have not yet been convincingly demonstrated. Adopting a novel methodological approach based on dynamic time warping (DTW), we provide evidence that the N2pc and P3 ERP components are temporally linked. We analyzed data from an experiment where 23 participants (16 women) monitored bilateral rapid serial streams of letters and digits in order to report a target digit indicated by a shape cue, separately for trials with correct responses and trials where a temporally proximal distractor was reported instead (distractor intrusion). DTW analyses revealed that N2pc and P3 latencies were correlated in time, both when the target or a distractor was reported. Notably, this link was weaker on distractor intrusion trials. This N2pc-P3 association is discussed with respect to the relationship between attention and access consciousness. Our results demonstrate that our novel method provides a valuable approach for assessing temporal links between two cognitive processes and their underlying modulating factors. This method allows to establish links and their modulator for any two time-series across all domains of the field (general-purpose MATLAB functions and a Python module are provided alongside this paper).

Conceptual information of meaningful objects is stored incidentally.

Prior research has shown that visual working memory capacity is enhanced for meaningful stimuli (i.e., real-world objects) compared to abstract shapes (i.e., colored circles). Here, we hypothesized that the shape of meaningful objects would be better remembered incidentally than the shape of nonmeaningful objects in a color memory task where the shape of the objects is task-irrelevant. We used a surprise-trial paradigm in which participants performed a color memory task for several trials before being probed with a surprise trial that asked them about the shape of the last object they saw. Across three experiments, we found a memory advantage for recognizable shapes relative to scrambled versions of these shapes (Experiment 1) that was robust across different encoding times (Experiment 2), and the addition of a verbal suppression task (Experiment 3). Interestingly, this advantage disappeared when all objects were from the same category (Experiment 4), suggesting that people are incidentally encoding broad conceptual information about object identities, but not visual details. Finally, when we asked about the location of objects in a surprise trial, we did not observe any difference between the two stimulus types (Experiment 5). Overall, these results show that conceptual information about the categories of meaningful objects is incidentally encoded into working memory even when task-irrelevant. This privilege for meaningful information does not exhibit a trade-off with location memory, suggesting that meaningful features influence representations of visual working memory in higher-level visual regions without altering the use of spatial reference frames at the lower level. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Attention with or without working memory: mnemonic reselection of attended information.

Attention has been regarded as the 'gatekeeper' controlling what information gets selected into working memory. However, a new perspective has emerged with the discovery of attribute amnesia, a phenomenon revealing that people are frequently unable to report information they have just attended to moments ago. This report failure is thought to stem from a lack of consolidating the attended information into working memory, indicating a dissociation between attention and working memory. Building on these findings, a new concept called memory reselection is proposed to describe a secondary round of selection among the attended information. These discoveries challenge the conventional view of how attention and working memory are related and shed new light onto modeling attention and memory as dissociable processes.

Surprise! Draw the scene: Visual recall reveals poor incidental working memory following visual search in natural scenes.

Searching within natural scenes can induce incidental encoding of information about the scene and the target, particularly when the scene is complex or repeated. However, recent evidence from attribute amnesia (AA) suggests that in some situations, searchers can find a target without building a robust incidental memory of it's task relevant features. Through drawing-based visual recall and an AA search task, we investigated whether search in natural scenes necessitates memory encoding. Participants repeatedly searched for and located an easily detected item in novel scenes for numerous trials before being unexpectedly prompted to draw either the entire scene (Experiment 1) or their search target (Experiment 2) directly after viewing the search image. Naïve raters assessed the similarity of the drawings to the original information. We found that surprise-trial drawings of the scene and search target were both poorly recognizable, but the same drawers produced highly recognizable drawings on the next trial when they had an expectation to draw the image. Experiment 3 further showed that the poor surprise trial memory could not merely be attributed to interference from the surprising event. Our findings suggest that even for searches done in natural scenes, it is possible to locate a target without creating a robust memory of either it or the scene it was in, even if attended to just a few seconds prior. This disconnection between attention and memory might reflect a fundamental property of cognitive computations designed to optimize task performance and minimize resource use.

Slipping through the cracks: The peril of unexpected interruption on the contents of working memory.

Working memory allows us to hold specific pieces of information in an active and easily retrieved state, but what happens to that information during an unexpected interruption between study and test? To answer this question, we used a surprise trial paradigm in which an unexpected event precedes a probe of the observer's memory for a search target. In the first set of experiments, participants were tasked to report the identity of the target letter before unexpectedly being asked to read a task-irrelevant passage. We observed that the introduction of this passage interfered with the observer's memory of the target letter, but this interference only occurred after participants had experience completing the task without interruption. However, a remember cue placed just prior to the reading prompt reduced this cost, suggesting that participants can rapidly reinforce information about the target in working memory to resist the interference. We then used this same cuing manipulation to test whether information in an attribute amnesia paradigm, which unexpectedly probes an attribute relevant to target selection but irrelevant to participant's response expectations, could also be protected against unexpected interference. Using this paradigm, we observed that a remember cue did not improve performance following the surprising event, which supports theories that attribute amnesia is not caused solely by interference. These results reveal both the vulnerability and flexibility of working memory and demonstrate the importance of understanding how task experience establishes expectations that impact the underlying cognitive representations formed by the observer. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Location has a privilege, but it is limited: Evidence from probing task-irrelevant location.

We investigated the extents of automaticity in location and orientation encoding in visual working memory (VWM) by manipulating their task relevance and assessing the amount of resource recruited by their encoding. Across five experiments, participants were surprised with a location report trial (Experiment 1A, 2A, and 3) or an orientation report trial (Experiment 2A and 2B) at a point when only the item's color had been task-relevant. This was followed by control trials to assess the memory quality of color when location or orientation had become task-relevant. We found the surprise trial performance to be significantly worse than the first control trial for both location and orientation, although to a greater extent for orientation for which there was virtually no measurable information from the subjects' reports. This was the case even when encoding was the only incidental memory process before the control trials (Experiment 2A and 2B), and the surprise memory costs cannot be attributed to the unexpectedness inherent to the surprise question (Experiment 3). The control trials revealed a consistent reduction of color memory only in the orientation experiments. These results suggest that although location encoding is more automatic than orientation, neither is encoded in a fully automatic manner. Our results show that incidentally encoded location is only coarse-grained, constraining the spatial precision of space-based indexing systems. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

In Defense of Modular Thinking.

In this issue, Pessoa emphasizes the importance of viewing neural activity from a perspective that functional networks form dynamically in a way that dramatically changes the functional contribution of individual brain areas. In this response, I argue that we should strive toward pluralism in understanding neural activity at both the emergent network and modular levels, on the bases that a purely emergent understanding would be incomplete, and that there are computational advantages to anatomically stable modularity.

The influence of category representativeness on the low prevalence effect in visual search.

Visual search is greatly affected by the appearance rate of given target types, such that low-prevalence items are harder to detect, which has consequences for real-world search tasks where target frequency cannot be balanced. However, targets that are highly representative of a categorically defined task set are also easier to find. We hypothesized that targets that are highly representative are less vulnerable to low-prevalence effects because an observer's attentional set prioritizes guidance toward them even when they are rare. We assessed this hypothesis by first determining the categorical structure of "prohibited carry-ons" via an exemplar-naming task, and used this structure to assess how category representativeness interacted with prevalence. Specifically, from the exemplar-naming task we selected a commonly named (knives) and rarely named (gas cans) target for a search task in which one of the targets was shown infrequently. As predicted, highly representative targets were found more easily than their less representative counterparts, but they also were less affected by prevalence manipulations. Experiment 1b replicated the results with targets matched for emotional valence (water bottles and fireworks). These findings demonstrate the powerful explanatory power of theories of attentional guidance that incorporate the dynamic influence of recent experience with the knowledge that comes from life experience to better predict behavioral outcomes associated with high-stakes search environments.

What the Flip? What the P-N Flip Can Tell Us about Proactive Suppression.

It has been debated whether salient distractors in visual search can be proactively suppressed to completely prevent attentional capture, as the occurrence of proactive suppression implies that the initial shift of attention is not entirely driven by physical salience. While the presence of a Pd component in the EEG (associated with suppression) without a preceding N2pc component (associated with selection) has been used as evidence for proactive suppression, the link between these ERPs and the underlying mechanisms is not always clear. This is exemplified in two recent articles that observed the same waveform pattern, where an early Pd-like component flipped to a N2pc-like component, but provided vastly different interpretations (Drisdelle, B. L., & Eimer, E. PD components and distractor inhibition in visual search: New evidence for the signal suppression hypothesis. Psychophysiology, 58, e13898, 2021; Kerzel, D., & Burra, N. Capture by context elements, not attentional suppression of distractors, explains the PD with small search displays. Journal of Cognitive Neuroscience, 32, 1170-1183, 2020). Using RAGNAROC (Wyble et al., Understanding visual attention with RAGNAROC: A Reflexive Attention Gradient through Neural AttRactOr Competition. Psychological Review, 127, 1163-1198, 2020), a computational model of reflexive attention, we successfully simulated this ERP pattern with minimal changes to its existing architecture, providing a parsimonious and mechanistic explanation for this flip in the EEG that is unique from both of the previous interpretations. Our account supports the occurrence of proactive suppression and demonstrates the benefits of incorporating computational modeling into theory building.

The early attentional pancake: Minimal selection in depth for rapid attentional cueing.

There have been conflicting findings on the degree to which rapidly deployed visual attention is selective for depth, and this issue has important implications for attention models. Previous findings have attempted to find depth-based cueing effects on such attention using reaction time (RT) measures for stimuli presented in stereo goggles with a display screen. Results stemming from such approaches have been mixed, depending on whether target/distractor discrimination was required. To help clarify the existence of such depth effects, we have developed a paradigm that measures accuracy rather than RT in an immersive virtual-reality environment, providing a more appropriate context of depth. Three modified Posner Cueing paradigms were run to test for depth-specific rapid attentional selectivity. Participants fixated a cross while attempting to identify a rapidly masked black letter preceded by a red cue that could be valid in depth, side, or both. In Experiment 1a, a potent cueing effect was found for lateral cueing validity, but a weak effect was found for depth despite an extreme difference in virtual depth (1 vs. 300 m). In Experiment 1b, a near-replication of 1a, the lateral effect replicated while the depth effect did not. Finally, in Experiment 2, to increase the depth cue's effectiveness, the letter matched the cue's color, and the presentation duration was increased; however, again only a minimal depth-based cueing effect - no greater than that of Experiment 1a - was observed. Thus, we conclude that rapidly deployed attention is driven largely by spatiotopic rather than depth-based information.

A model of working memory for latent representations.

We propose a mechanistic explanation of how working memories are built and reconstructed from the latent representations of visual knowledge. The proposed model features a variational autoencoder with an architecture that corresponds broadly to the human visual system and an activation-based binding pool of neurons that links latent space activities to tokenized representations. The simulation results revealed that new pictures of familiar types of items can be encoded and retrieved efficiently from higher levels of the visual hierarchy, whereas truly novel patterns are better stored using only early layers. Moreover, a given stimulus in working memory can have multiple codes, which allows representation of visual detail in addition to categorical information. Finally, we validated our model's assumptions by testing a series of predictions against behavioural results obtained from working memory tasks. The model provides a demonstration of how visual knowledge yields compact visual representation for efficient memory encoding.

The Learning Salon: Toward a new participatory science.

The Learning Salon is an online weekly forum for discussing points of contention and common ground in biological and artificial learning. Hosting neuroscientists, computer scientists, AI researchers, and philosophers, the Salon promotes short talks and long discussions, committed to an ethos of participation, horizontality, and inclusion.

And like that, they were gone: A failure to remember recently attended unique faces.

Attribute amnesia (AA) is a phenomenon in which participants have difficulty answering an unexpected question about an attended attribute of the most recent target stimulus. A similar situation can occur in cases of real-life eyewitness identification when the eyewitness did not explicitly try to remember the alleged perpetrator's face despite having attended to it. We found that AA is generalizable to novel faces, such that when participants were unexpectedly asked to identify a face, performance was poor, even though they had just attended to that face seconds ago (N = 40 each in an initial experiment and its replication). This finding shows that unexpected face identification is inaccurate even when the face had just been attended to and suffered minimal decay and interference, implying that AA can explain some cases of failure of eyewitness identification that cannot be attributed to a lack of attention or post-event interference.

No explicit memory for individual trial display configurations in a visual search task.

Previous evidence demonstrated that individuals can recall a target's location in a search display even if location information is completely task-irrelevant. This finding raises the question: does this ability to automatically encode a single item's location into a reportable memory trace extend to other aspects of spatial information as well? We tested this question using a paradigm designed to elicit attribute amnesia (Chen & Wyble, Psychological Science, 26(2) 203-210, 2015a). Participants were initially asked to report the location of a target letter among digits with stimuli arranged to form one of two or four spatial configurations varying randomly across trials. After completing numerous trials that matched their expectations, participants were surprised with a series of unexpected questions probing their memory for various aspects of the display they had just viewed. Participants had a profound inability to report which spatial configuration they had just perceived when the target's location was not unique to a specific configuration (i.e., orthogonal). Despite being unable to report the most recent configuration, answer choices on the surprise trial were focused around previously seen configurations, rather than novel configurations. Thus, there were clear memories of the set of configurations that had been viewed during the experiment but not of the specific configuration from the most recent trial. This finding helps to set boundary conditions on previous findings regarding the automatic encoding of location information into memory.

Towards Democratizing and Automating Online Conferences: Lessons from the Neuromatch Conferences.

Legacy conferences are costly and time consuming, and exclude scientists lacking various resources or abilities. During the 2020 pandemic, we created an online conference platform, Neuromatch Conferences (NMC), aimed at developing technological and cultural changes to make conferences more democratic, scalable, and accessible. We discuss the lessons we learned.

I tried a bunch of things: The dangers of unexpected overfitting in classification of brain data.

Machine learning has enhanced the abilities of neuroscientists to interpret information collected through EEG, fMRI, and MEG data. With these powerful techniques comes the danger of overfitting of hyperparameters which can render results invalid. We refer to this problem as 'overhyping' and show that it is pernicious despite commonly used precautions. Overhyping occurs when analysis decisions are made after observing analysis outcomes and can produce results that are partially or even completely spurious. It is commonly assumed that cross-validation is an effective protection against overfitting or overhyping, but this is not actually true. In this article, we show that spurious results can be obtained on random data by modifying hyperparameters in seemingly innocuous ways, despite the use of cross-validation. We recommend a number of techniques for limiting overhyping, such as lock boxes, blind analyses, pre-registrations, and nested cross-validation. These techniques, are common in other fields that use machine learning, including computer science and physics. Adopting similar safeguards is critical for ensuring the robustness of machine-learning techniques in the neurosciences.

Understanding visual attention with RAGNAROC: A reflexive attention gradient through neural AttRactOr competition.

A quintessential challenge for any perceptual system is the need to focus on task-relevant information without being blindsided by unexpected, yet important information. The human visual system incorporates several solutions to this challenge, 1 of which is a reflexive covert attention system that is rapidly responsive to both the physical salience and the task-relevance of new information. This article presents a model that simulates behavioral and neural correlates of reflexive attention as the product of brief neural attractor states that are formed across the visual hierarchy when attention is engaged. Such attractors emerge from an attentional gradient distributed over a population of topographically organized neurons and serve to focus processing at 1 or more locations in the visual field, while inhibiting the processing of lower priority information. The model moves toward a resolution of key debates about the nature of reflexive attention, such as whether it is parallel or serial, and whether suppression effects are distributed in a spatial surround, or selectively at the location of distractors. The model also develops a framework for understanding the neural mechanisms of visual attention as a spatiotopic decision process within a hierarchy and links them to observable correlates such as accuracy, reaction time (RT), and the N2pc and PD components of the electroencephalogram (EEG). This last contribution is the most crucial for repairing the disconnect that exists between our understanding of behavioral and neural correlates of attention. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Memories of Visual Events Can Be Formed Without Specific Spatial Coordinates.

To what extent does specific spatiotopic location accompany the remembered representation of a visual event? Feature integration theory suggests that identifying a multi-feature object requires focusing on its spatial location to integrate those features. Moreover, single unit data from anterior ventral stream neurons that fire preferentially to complex objects indicates that they have retinotopic receptive fields. It can, therefore, be predicted that location information of features of a complex stimulus is inherent in the memory of a perceived visual stimulus' representation. To evaluate this prediction, we presented participants with a brief array of characters with instructions to identify and locate the solitary letter among a set of digits. Surprisingly, analysis of trials in which the target identity was accurately reported indicated that in more than 15% of trials (i.e., in Experiments 2b & 2c) participants were almost completely uninformed about the location of the letter that they had just identified. Further analysis showed that there were two main sources of these location errors; misbinding the target to the distractors' locations and extremely poor spatial representation of the target's location to an extent that was indistinguishable from guessing. The latter finding indicates that consciously accessible representations of visual events can form despite being untethered to robust and spatially-specific representations, implying that the specific location was either not quite encoded into working memory, or was rapidly forgotten. However, when the target was marked by a single feature (color), there was no evidence of remembering the target identity without remembering its location even with strong masking.

A delay in sampling information from temporally autocorrelated visual stimuli.

Much of our world changes smoothly in time, yet the allocation of attention is typically studied with sudden changes - transients. A sizeable lag in selecting feature information is seen when stimuli change smoothly. Yet this lag is not seen with temporally uncorrelated rapid serial visual presentation (RSVP) stimuli. This suggests that temporal autocorrelation of a feature paradoxically increases the latency at which information is sampled. To test this, participants are asked to report the color of a disk when a cue was presented. There is an increase in selection latency when the disk's color changed smoothly compared to randomly. This increase is due to the smooth color change presented after the cue rather than extrapolated predictions based on the color changes presented before. These results support an attentional drag theory, whereby attentional engagement is prolonged when features change smoothly. A computational model provides insights into the potential underlying neural mechanisms.