Working Memory Swap Errors Have Identifiable Neural Representations.

Working memory is an essential component of cognition that facilitates goal-directed behavior. Famously, it is severely limited and performance suffers when memory load exceeds an individual's capacity. Modeling of visual working memory responses has identified two likely types of errors: guesses and swaps. Swap errors may arise from a misbinding between the features of different items. Alternatively, these errors could arise from memory noise in the feature dimension used for cueing a to-be-tested memory item, resulting in the wrong item being selected. Finally, it is possible that so-called swap errors actually reflect informed guessing, which could occur at the time of a cue, or alternatively, at the time of the response. Here, we combined behavioral response modeling and fMRI pattern analysis to test the hypothesis that swap errors involve the active maintenance of an incorrect memory item. After the encoding of six spatial locations, a retro-cue indicated which location would be tested after memory retention. On accurate trials, we could reconstruct a memory representation of the cued location in both early visual cortex and intraparietal sulcus. On swap error trials identified with mixture modeling, we were able to reconstruct a representation of the swapped location, but not of the cued location, suggesting the maintenance of the incorrect memory item before response. Moreover, participants subjectively responded with some level of confidence, rather than complete guessing, on a majority of swap error trials. Together, these results suggest that swap errors are not mere response-phase guesses, but instead result from failures of selection in working memory, contextual binding errors, or informed guesses, which produce active maintenance of incorrect memory representations.

Multi-scale neural decoding and analysis.

Objective. Complex spatiotemporal neural activity encodes rich information related to behavior and cognition. Conventional research has focused on neural activity acquired using one of many different measurement modalities, each of which provides useful but incomplete assessment of the neural code. Multi-modal techniques can overcome tradeoffs in the spatial and temporal resolution of a single modality to reveal deeper and more comprehensive understanding of system-level neural mechanisms. Uncovering multi-scale dynamics is essential for a mechanistic understanding of brain function and for harnessing neuroscientific insights to develop more effective clinical treatment.Approach. We discuss conventional methodologies used for characterizing neural activity at different scales and review contemporary examples of how these approaches have been combined. Then we present our case for integrating activity across multiple scales to benefit from the combined strengths of each approach and elucidate a more holistic understanding of neural processes.Main results. We examine various combinations of neural activity at different scales and analytical techniques that can be used to integrate or illuminate information across scales, as well the technologies that enable such exciting studies. We conclude with challenges facing future multi-scale studies, and a discussion of the power and potential of these approaches.Significance. This roadmap will lead the readers toward a broad range of multi-scale neural decoding techniques and their benefits over single-modality analyses. This Review article highlights the importance of multi-scale analyses for systematically interrogating complex spatiotemporal mechanisms underlying cognition and behavior.

Distraction in Visual Working Memory: Resistance is Not Futile.

Over half a century of research focused on understanding how working memory is capacity constrained has overshadowed the fact that it is also remarkably resistant to interference. Protecting goal-relevant information from distraction is a cornerstone of cognitive function that involves a multifaceted collection of control processes and storage mechanisms. Here, we discuss recent advances in cognitive psychology and neuroscience that have produced new insights into the nature of visual working memory and its ability to resist distraction. We propose that distraction resistance should be an explicit component in any model of working memory and that understanding its behavioral and neural correlates is essential for building a comprehensive understanding of real-world memory function.

Dissociable neural mechanisms underlie currently-relevant, future-relevant, and discarded working memory representations.

In daily life, we use visual working memory (WM) to guide our actions. While attending to currently-relevant information, we must simultaneously maintain future-relevant information, and discard information that is no longer relevant. However, the neural mechanisms by which unattended, but future-relevant, information is maintained in working memory, and future-irrelevant information is discarded, are not well understood. Here, we investigated representations of these different information types, using functional magnetic resonance imaging in combination with multivoxel pattern analysis and computational modeling based on inverted encoding model simulations. We found that currently-relevant WM information in the focus of attention was maintained through representations in visual, parietal and posterior frontal brain regions, whereas deliberate forgetting led to suppression of the discarded representations in early visual cortex. In contrast, future-relevant information was neither inhibited nor actively maintained in these areas. These findings suggest that different neural mechanisms underlie the WM representation of currently- and future-relevant information, as compared to information that is discarded from WM.

Flexible Coding of Visual Working Memory Representations during Distraction.

Visual working memory (VWM) recruits a broad network of brain regions, including prefrontal, parietal, and visual cortices. Recent evidence supports a "sensory recruitment" model of VWM, whereby precise visual details are maintained in the same stimulus-selective regions responsible for perception. A key question in evaluating the sensory recruitment model is how VWM representations persist through distracting visual input, given that the early visual areas that putatively represent VWM content are susceptible to interference from visual stimulation.To address this question, we used a functional magnetic resonance imaging inverted encoding model approach to quantitatively assess the effect of distractors on VWM representations in early visual cortex and the intraparietal sulcus (IPS), another region previously implicated in the storage of VWM information. This approach allowed us to reconstruct VWM representations for orientation, both before and after visual interference, and to examine whether oriented distractors systematically biased these representations. In our human participants (both male and female), we found that orientation information was maintained simultaneously in early visual areas and IPS in anticipation of possible distraction, and these representations persisted in the absence of distraction. Importantly, early visual representations were susceptible to interference; VWM orientations reconstructed from visual cortex were significantly biased toward distractors, corresponding to a small attractive bias in behavior. In contrast, IPS representations did not show such a bias. These results provide quantitative insight into the effect of interference on VWM representations, and they suggest a dynamic tradeoff between visual and parietal regions that allows flexible adaptation to task demands in service of VWM.SIGNIFICANCE STATEMENT Despite considerable evidence that stimulus-selective visual regions maintain precise visual information in working memory, it remains unclear how these representations persist through subsequent input. Here, we used quantitative model-based fMRI analyses to reconstruct the contents of working memory and examine the effects of distracting input. Although representations in the early visual areas were systematically biased by distractors, those in the intraparietal sulcus appeared distractor-resistant. In contrast, early visual representations were most reliable in the absence of distraction. These results demonstrate the dynamic, adaptive nature of visual working memory processes, and provide quantitative insight into the ways in which representations can be affected by interference. Further, they suggest that current models of working memory should be revised to incorporate this flexibility.

The Effect of Disruption of Prefrontal Cortical Function with Transcranial Magnetic Stimulation on Visual Working Memory.

It is proposed that feedback signals from the prefrontal cortex (PFC) to extrastriate cortex are essential for goal-directed processing, maintenance, and selection of information in visual working memory (VWM). In a previous study, we found that disruption of PFC function with transcranial magnetic stimulation (TMS) in healthy individuals impaired behavioral performance on a face/scene matching task and decreased category-specific tuning in extrastriate cortex as measured with functional magnetic resonance imaging (fMRI). In this study, we investigated the effect of disruption of left inferior frontal gyrus (IFG) function on the fidelity of neural representations of two distinct information codes: (1) the stimulus category and (2) the goal-relevance of viewed stimuli. During fMRI scanning, subjects were presented face and scene images in pseudo-random order and instructed to remember either faces or scenes. Within both anatomical and functional regions of interest (ROIs), a multi-voxel pattern classifier was used to quantitatively assess the fidelity of activity patterns representing stimulus category: whether a face or a scene was presented on each trial, and goal relevance, whether the presented image was task relevant (i.e., a face is relevant in a "Remember Faces" block, but irrelevant in a "Remember Scenes" block). We found a reduction in the fidelity of the stimulus category code in visual cortex after left IFG disruption, providing causal evidence that lateral PFC modulates object category codes in visual cortex during VWM. In addition, we found that IFG disruption caused a reduction in the fidelity of the goal relevance code in a distributed set of brain regions. These results suggest that the IFG is involved in determining the task-relevance of visual input and communicating that information to a network of regions involved in further processing during VWM. Finally, we found that participants who exhibited greater fidelity of the goal relevance code in the non-disrupted right IFG after TMS performed the task with the highest accuracy.

Expertise for upright faces improves the precision but not the capacity of visual working memory.

Considerable research has focused on how basic visual features are maintained in working memory, but little is currently known about the precision or capacity of visual working memory for complex objects. How precisely can an object be remembered, and to what extent might familiarity or perceptual expertise contribute to working memory performance? To address these questions, we developed a set of computer-generated face stimuli that varied continuously along the dimensions of age and gender, and we probed participants' memories using a method-of-adjustment reporting procedure. This paradigm allowed us to separately estimate the precision and capacity of working memory for individual faces, on the basis of the assumptions of a discrete capacity model, and to assess the impact of face inversion on memory performance. We found that observers could maintain up to four to five items on average, with equally good memory capacity for upright and upside-down faces. In contrast, memory precision was significantly impaired by face inversion at every set size tested. Our results demonstrate that the precision of visual working memory for a complex stimulus is not strictly fixed but, instead, can be modified by learning and experience. We find that perceptual expertise for upright faces leads to significant improvements in visual precision, without modifying the capacity of working memory.