Temporally Dissociable Mechanisms of Spatial, Feature, and Motor Selection during Working Memory-guided Behavior.

Working memory (WM) is a capacity- and duration-limited system that forms a temporal bridge between fleeting sensory phenomena and possible actions. But how are the contents of WM used to guide behavior? A recent high-profile study reported evidence for simultaneous access to WM content and linked motor plans during WM-guided behavior, challenging serial models where task-relevant WM content is first selected and then mapped on to a task-relevant motor response. However, the task used in that study was not optimized to distinguish the selection of spatial versus nonspatial visual information stored in memory, nor to distinguish whether or how the chronometry of selecting nonspatial visual information stored in memory might differ from the selection of linked motor plans. Here, we revisited the chronometry of spatial, feature, and motor selection during WM-guided behavior using a task optimized to disentangle these processes. Concurrent EEG and eye position recordings revealed clear evidence for temporally dissociable spatial, feature, and motor selection during this task. Thus, our data reveal the existence of multiple WM selection mechanisms that belie conceptualizations of WM-guided behavior based on purely serial or parallel visuomotor processing.

Internal selective attention is delayed by competition between endogenous and exogenous factors.

External attention is mediated by competition between endogenous (goal-driven) and exogenous (stimulus-driven) factors, with the balance of competition determining which stimuli are selected. Occasionally, exogenous factors "win" this competition and drive the selection of task-irrelevant stimuli. Endogenous and exogenous selection mechanisms may also compete to control the selection of internal representations (e.g., those stored in working memory), but whether this competition is resolved in the same way as external attention is unknown. Here, we leveraged the high temporal resolution of human EEG to determine how competition between endogenous and exogenous factors influences the selection of internal representations. Unlike external attention, competition did not prompt the selection of task-irrelevant working memory content. Instead, it delayed the endogenous selection of task-relevant working memory content by several hundred milliseconds. Thus, competition between endogenous and exogenous factors influences internal selective attention, but in a different way than external selective attention.

The entorhinal-DG/CA3 pathway in the medial temporal lobe retains visual working memory of a simple surface feature.

Classic models consider working memory (WM) and long-term memory as distinct mental faculties that are supported by different neural mechanisms. Yet, there are significant parallels in the computation that both types of memory require. For instance, the representation of precise item-specific memory requires the separation of overlapping neural representations of similar information. This computation has been referred to as pattern separation, which can be mediated by the entorhinal-DG/CA3 pathway of the medial temporal lobe (MTL) in service of long-term episodic memory. However, although recent evidence has suggested that the MTL is involved in WM, the extent to which the entorhinal-DG/CA3 pathway supports precise item-specific WM has remained elusive. Here, we combine an established orientation WM task with high-resolution fMRI to test the hypothesis that the entorhinal-DG/CA3 pathway retains visual WM of a simple surface feature. Participants were retrospectively cued to retain one of the two studied orientation gratings during a brief delay period and then tried to reproduce the cued orientation as precisely as possible. By modeling the delay-period activity to reconstruct the retained WM content, we found that the anterior-lateral entorhinal cortex (aLEC) and the hippocampal DG/CA3 subfield both contain item-specific WM information that is associated with subsequent recall fidelity. Together, these results highlight the contribution of MTL circuitry to item-specific WM representation.

Changes in behavioral priority influence the accessibility of working memory content.

Evolving behavioral goals require the existence of selection mechanisms that prioritize task-relevant working memory (WM) content for action. Selecting an item stored in WM is known to blunt and/or reverse information loss in stimulus-specific representations of that item reconstructed from human brain activity, but extant studies have focused on all-or-none circumstances that allow or disallow an agent to select one of several items stored in WM. Conversely, behavioral studies suggest that humans can flexibly assign different levels of priority to different items stored in WM, but how doing so influences neural representations of WM content is unclear. One possibility is that assigning different levels of priority to items in WM influences the quality of those representations, resulting in more robust neural representations of high- vs. low-priority WM content. A second - and non-exclusive - possibility is that asymmetries in behavioral priority influence how rapidly neural representations of high- vs. low-priority WM content can be selected and reported. We tested these possibilities in two experiments by decoding high- and low-priority WM content from EEG recordings obtained while human volunteers performed a retrospectively cued WM task. Probabilistic changes in the behavioral relevance of a remembered item had no effect on our ability to decode it from EEG signals; instead, these changes influenced the latency at which above-chance decoding performance was reached. Thus, our results indicate that probabilistic changes in the behavioral relevance of WM content influence the ease with which memories can be selected independently of their strength.

Retrospective cue benefits in visual working memory are limited to a single location at a time.

Working memory (WM) performance can be improved by an informative cue presented during storage. This effect, termed a retrocue benefit, can be used to study limits on how human observers prioritize information stored in WM for behavioral output. There is disagreement about whether retrocue benefits extend to multiple WM locations. Here, we hypothesized that multiple retrocues may improve some aspects of memory performance (e.g., a reduction in random guessing) while worsening others (e.g., an increase in the probability of reporting a feature presented at a non-probed location). We tested this possibility in three experiments. Participants remembered arrays of four orientations or colors over a brief delay, and spatial retrocues instructed participants to prioritize zero, one, two, or all four remembered orientations for possible report. At the end of the trial, participants recalled the orientation that appeared at one location. The results of this study revealed that participants' recall errors were lower during cue-one relative to cue-two and cue-four trials, and this benefit was driven primarily by a reduction in random guessing during cue-one trials. We found no evidence suggesting that multiple spatial cues (i.e., during cue-two trials) induced a trade-off between memory precision, random guessing, and non-target reports compared to neutral trials (i.e., cue-zero or cue-four). Thus, cuing participants to prioritize information appearing at multiple unique spatial positions led to no improvement in memory performance compared to neutral or no-cue trials, providing additional support for the view that retrocue benefits on WM performance are limited to a single spatial location at a time.

The Roles of Salience and Value in Inattention Among Children With Attention-Deficit/Hyperactivity Disorder: A Pilot Investigation.

Although inattention is a key symptom subdomain of attention-deficit/hyperactivity disorder (ADHD), the mechanisms underlying this subdomain and related symptoms remain unclear. There is a need for more granular approaches that allow for greater specificity in linking disruptions in specific domains of cognitive performance (e.g., executive function and reward processing) with behavioral manifestations of ADHD. Such approaches may inform the development of more targeted therapeutic interventions. Here, we describe the results of a pilot study of elementary-aged children (ages 6-12years) with ADHD (n=50) and typically developing children (n=48) utilizing a cognitive science task designed to target two dissociable mechanisms of attentional selection: a goal-driven mechanism (i.e., reward/value-driven) and a salience-driven mechanism. Participants were asked to optimally extract and combine information about stimulus salience and value to maximize rewards. While results of this pilot study are ambiguous due to the small sample size and limited number of task trials, data suggest that neither participants with ADHD nor typically developing participants performed optimally to maximize rewards, though typically developing participants were somewhat more successful at the task (i.e., more likely to report high-value targets) regardless of task condition. Further, the manuscript examines several follow-up questions regarding group differences in task response times and group differences in task performance as related to sustained attention across the duration of the task. Finally, the manuscript examines follow-up questions related to heterogeneity in the ADHD group (i.e., age, DSM 5 presentation, and comorbid diagnosis) in predicting task performance.

Recovery of information from latent memory stores decreases over time.

Working memory (WM) performance can be enhanced by an informative cue presented during storage. This effect, termed a retrocue benefit, can be used to explore how observers prioritize information stored in WM. Recent studies have demonstrated that neural representations of task-relevant memoranda are strengthened following a retrocue, suggesting that participants can supplement active memory traces with information from other memory stores. We sought to better understand these additional store(s) by asking whether they are subject to the same temporal degradation seen in active memory representations during storage. We tested this possibility by reconstructing and quantifying representations of remembered positions from EEG activity while varying the interval separating an encoding display and retrocue during a spatial WM task. We observed a significant increase in the quality of location-specific representations following a retrocue, but the magnitude of this benefit was linearly and inversely related to the timing of the retrocue such that later cues yielded smaller increases. This result suggests that participants' ability to supplement active memory representations with information from additional memory stores is not static: the information maintained in these stores may be subject to temporal degradation, or these stores may become more difficult to access over time.

Categorical Biases in Human Occipitoparietal Cortex.

Categorization allows organisms to generalize existing knowledge to novel stimuli and to discriminate between physically similar yet conceptually different stimuli. Humans, nonhuman primates, and rodents can readily learn arbitrary categories defined by low-level visual features, and learning distorts perceptual sensitivity for category-defining features such that differences between physically similar yet categorically distinct exemplars are enhanced, whereas differences between equally similar but categorically identical stimuli are reduced. We report a possible basis for these distortions in human occipitoparietal cortex. In three experiments, we used an inverted encoding model to recover population-level representations of stimuli from multivoxel and multielectrode patterns of human brain activity while human participants (both sexes) classified continuous stimulus sets into discrete groups. In each experiment, reconstructed representations of to-be-categorized stimuli were systematically biased toward the center of the appropriate category. These biases were largest for exemplars near a category boundary, predicted participants' overt category judgments, emerged shortly after stimulus onset, and could not be explained by mechanisms of response selection or motor preparation. Collectively, our findings suggest that category learning can influence processing at the earliest stages of cortical visual processing.SIGNIFICANCE STATEMENT Category learning enhances perceptual sensitivity for physically similar yet categorically different stimuli. We report a possible mechanism for these changes in human occipitoparietal cortex. In three experiments, we used an inverted encoding model to recover population-level representations of stimuli from multivariate patterns in occipitoparietal cortex while participants categorized sets of continuous stimuli into discrete groups. The recovered representations were systematically biased by category membership, with larger biases for exemplars adjacent to a category boundary. These results suggest that mechanisms of categorization shape information processing at the earliest stages of the visual system.

Retrospective Cues Mitigate Information Loss in Human Cortex during Working Memory Storage.

Working memory (WM) enables the flexible representation of information over short intervals. It is well established that WM performance can be enhanced by a retrospective cue presented during storage, yet the neural mechanisms responsible for this benefit are unclear. Here, we tested several explanations for retrospective cue benefits by quantifying changes in spatial WM representations reconstructed from alpha-band (8-12 Hz) EEG activity recorded from human participants (both sexes) before and after the presentation of a retrospective cue. This allowed us to track cue-related changes in WM representations with high temporal resolution (tens of milliseconds). Participants encoded the locations of two colored discs for subsequent report. During neutral trials, an uninformative cue instructed participants to remember the locations of both discs across a blank delay, and we observed a monotonic decrease in the fidelity of reconstructed spatial WM representations with time. During valid trials, a 100% reliable cue indicated that the color of the disc participants would be probed to report. Critically, valid cues were presented immediately after the termination of the encoding display ["valid early" (VE) trials] or midway through the delay period ["valid late" (VL) trials]. During VE trials, the gradual loss of location-specific information observed during neutral trials was eliminated, while during VL trials it was partially reversed. Our findings suggest that retrospective cues engage several different mechanisms that together serve to mitigate information loss during WM storage.SIGNIFICANCE STATEMENT Working memory (WM) performance can be improved by a cue presented during storage. This effect, termed a retrospective cue benefit, has been used to explore the limitations of attentional prioritization in WM. However, the mechanisms responsible for retrospective cue benefits are unclear. Here we tested several explanations for retrospective cue benefits by examining how they influence WM representations reconstructed from human EEG activity. This approach allowed us to visualize, quantify, and track the effects of retrospective cues with high temporal resolution (on the order of tens of milliseconds). We show that under different circumstances retrospective cues can both eliminate and even partially reverse information loss during WM storage, suggesting that retrospective cue benefits have manifold origins.

Task-irrelevant distractors in the delay period interfere selectively with visual short-term memory for spatial locations.

Visual short-term memory (VSTM) enables the representation of information in a readily accessible state. VSTM is typically conceptualized as a form of "active" storage that is resistant to interference or disruption, yet several recent studies have shown that under some circumstances task-irrelevant distractors may indeed disrupt performance. Here, we investigated how task-irrelevant visual distractors affected VSTM by asking whether distractors induce a general loss of remembered information or selectively interfere with memory representations. In a VSTM task, participants recalled the spatial location of a target visual stimulus after a delay in which distractors were presented on 75% of trials. Notably, the distractor's eccentricity always matched the eccentricity of the target, while in the critical conditions the distractor's angular position was shifted either clockwise or counterclockwise relative to the target. We then computed estimates of recall error for both eccentricity and polar angle. A general interference model would predict an effect of distractors on both polar angle and eccentricity errors, while a selective interference model would predict effects of distractors on angle but not on eccentricity errors. Results showed that for stimulus angle there was an increase in the magnitude and variability of recall errors. However, distractors had no effect on estimates of stimulus eccentricity. Our results suggest that distractors selectively interfere with VSTM for spatial locations.

Restoring Latent Visual Working Memory Representations in Human Cortex.

Working memory (WM) enables the storage and manipulation of limited amounts of information over short periods. Prominent models posit that increasing the number of remembered items decreases the spiking activity dedicated to each item via mutual inhibition, which irreparably degrades the fidelity of each item's representation. We tested these models by determining if degraded memory representations could be recovered following a post-cue indicating which of several items in spatial WM would be recalled. Using an fMRI-based image reconstruction technique, we identified impaired behavioral performance and degraded mnemonic representations with elevated memory load. However, in several cortical regions, degraded mnemonic representations recovered substantially following a post-cue, and this recovery tracked behavioral performance. These results challenge pure spike-based models of WM and suggest that remembered items are additionally encoded within latent or hidden neural codes that can help reinvigorate active WM representations.

Feature-Selective Attentional Modulations in Human Frontoparietal Cortex.

UNLABELLED: Control over visual selection has long been framed in terms of a dichotomy between "source" and "site," where top-down feedback signals originating in frontoparietal cortical areas modulate or bias sensory processing in posterior visual areas. This distinction is motivated in part by observations that frontoparietal cortical areas encode task-level variables (e.g., what stimulus is currently relevant or what motor outputs are appropriate), while posterior sensory areas encode continuous or analog feature representations. Here, we present evidence that challenges this distinction. We used fMRI, a roving searchlight analysis, and an inverted encoding model to examine representations of an elementary feature property (orientation) across the entire human cortical sheet while participants attended either the orientation or luminance of a peripheral grating. Orientation-selective representations were present in a multitude of visual, parietal, and prefrontal cortical areas, including portions of the medial occipital cortex, the lateral parietal cortex, and the superior precentral sulcus (thought to contain the human homolog of the macaque frontal eye fields). Additionally, representations in many-but not all-of these regions were stronger when participants were instructed to attend orientation relative to luminance. Collectively, these findings challenge models that posit a strict segregation between sources and sites of attentional control on the basis of representational properties by demonstrating that simple feature values are encoded by cortical regions throughout the visual processing hierarchy, and that representations in many of these areas are modulated by attention. SIGNIFICANCE STATEMENT: Influential models of visual attention posit a distinction between top-down control and bottom-up sensory processing networks. These models are motivated in part by demonstrations showing that frontoparietal cortical areas associated with top-down control represent abstract or categorical stimulus information, while visual areas encode parametric feature information. Here, we show that multivariate activity in human visual, parietal, and frontal cortical areas encode representations of a simple feature property (orientation). Moreover, representations in several (though not all) of these areas were modulated by feature-based attention in a similar fashion. These results provide an important challenge to models that posit dissociable top-down control and sensory processing networks on the basis of representational properties.

Parietal and Frontal Cortex Encode Stimulus-Specific Mnemonic Representations during Visual Working Memory.

Working memory (WM) enables the storage and manipulation of information in an active state. WM storage has long been associated with sustained increases in activation across a network of frontal and parietal cortical regions. However, recent evidence suggests that these regions primarily encode information related to general task goals rather than feature-selective representations of specific memoranda. These goal-related representations are thought to provide top-down feedback that coordinates the representation of fine-grained details in early sensory areas. Here, we test this model using fMRI-based reconstructions of remembered visual details from region-level activation patterns. We could reconstruct high-fidelity representations of a remembered orientation based on activation patterns in occipital visual cortex and in several sub-regions of frontal and parietal cortex, independent of sustained increases in mean activation. These results challenge models of WM that postulate disjoint frontoparietal "top-down control" and posterior sensory "feature storage" networks.

Substitution and pooling in visual crowding induced by similar and dissimilar distractors.

Visual crowding refers to a phenomenon whereby objects that appear in the periphery of the visual field are more difficult to identify when embedded within clutter. Pooling models assert that crowding results from an obligatory averaging or other combination of target and distractor features that occurs prior to awareness. One well-known manifestation of pooling is feature averaging, with which the features of target and nontarget stimuli are combined at an early stage of visual processing. Conversely, substitution models assert that crowding results from binding a target and nearby distractors to incorrect spatial locations. Recent evidence suggests that substitution predominates when target-flanker feature similarity is low, but it is unclear whether averaging or substitution best explains crowding when similarity is high. Here, we examined participants' orientation report errors for targets crowded by similar or dissimilar flankers. In two experiments, we found evidence inconsistent with feature averaging regardless of target-flanker similarity. However, the observed data could be accommodated by a probabilistic substitution model in which participants occasionally "swap" a target for a distractor. Thus, we conclude that-at least for the displays used here-crowding likely results from a probabilistic substitution of targets and distractors, regardless of target-distractor feature similarity.

Sensory gain outperforms efficient readout mechanisms in predicting attention-related improvements in behavior.

Spatial attention has been postulated to facilitate perceptual processing via several different mechanisms. For instance, attention can amplify neural responses in sensory areas (sensory gain), mediate neural variability (noise modulation), or alter the manner in which sensory signals are selectively read out by postsensory decision mechanisms (efficient readout). Even in the context of simple behavioral tasks, it is unclear how well each of these mechanisms can account for the relationship between attention-modulated changes in behavior and neural activity because few studies have systematically mapped changes between stimulus intensity, attentional focus, neural activity, and behavioral performance. Here, we used a combination of psychophysics, event-related potentials (ERPs), and quantitative modeling to explicitly link attention-related changes in perceptual sensitivity with changes in the ERP amplitudes recorded from human observers. Spatial attention led to a multiplicative increase in the amplitude of an early sensory ERP component (the P1, peaking ∼80-130 ms poststimulus) and in the amplitude of the late positive deflection component (peaking ∼230-330 ms poststimulus). A simple model based on signal detection theory demonstrates that these multiplicative gain changes were sufficient to account for attention-related improvements in perceptual sensitivity, without a need to invoke noise modulation. Moreover, combining the observed multiplicative gain with a postsensory readout mechanism resulted in a significantly poorer description of the observed behavioral data. We conclude that, at least in the context of relatively simple visual discrimination tasks, spatial attention modulates perceptual sensitivity primarily by modulating the gain of neural responses during early sensory processing.

Reconstructions of information in visual spatial working memory degrade with memory load.

Working memory (WM) enables the maintenance and manipulation of information relevant to behavioral goals. Variability in WM ability is strongly correlated with IQ [1], and WM function is impaired in many neurological and psychiatric disorders [2, 3], suggesting that this system is a core component of higher cognition. WM storage is thought to be mediated by patterns of activity in neural populations selective for specific properties (e.g., color, orientation, location, and motion direction) of memoranda [4-13]. Accordingly, many models propose that differences in the amplitude of these population responses should be related to differences in memory performance [14, 15]. Here, we used functional magnetic resonance imaging and an image reconstruction technique based on a spatial encoding model [16] to visualize and quantify population-level memory representations supported by multivoxel patterns of activation within regions of occipital, parietal and frontal cortex while participants precisely remembered the location(s) of zero, one, or two small stimuli. We successfully reconstructed images containing representations of the remembered-but not forgotten-locations within regions of occipital, parietal, and frontal cortex using delay-period activation patterns. Critically, the amplitude of representations of remembered locations and behavioral performance both decreased with increasing memory load. These results suggest that differences in visual WM performance between memory load conditions are mediated by changes in the fidelity of large-scale population response profiles distributed across multiple areas of human cortex.

Electrophysiological evidence for failures of item individuation in crowded visual displays.

Visual perception is strongly impaired when peripheral targets are surrounded by nearby distractors, a phenomenon known as visual crowding. One common behavioral signature of visual crowding is an increased tendency for observers to mistakenly report the features of nearby distractors instead of the target item. Here, our goal was to distinguish between two possible explanations of such substitution errors. On the one hand, crowding may have its effects after the deployment of attention toward-and individuation of-targets and flankers, such that multiple individuated perceptual representations compete to guide the behavioral response. On the other hand, crowding may prevent the individuation of closely spaced stimuli, thereby reducing the number of apprehended items. We attempted to distinguish these alternatives using the N2pc, an ERP that has been shown to track the deployment of spatial attention and index the number of individuated items within a hemifield. N2pc amplitude increased monotonically with set size in uncrowded displays, but this set size effect was abolished in crowded visual displays. Moreover, these crowding-induced declines in N2pc amplitude predicted individual differences in the rate of substitution errors. Thus, crowding-induced confusions between targets and distractors may be a consequence of failures to individuate target and distractor stimuli during early stages of visual selection.

Variability in visual working memory ability limits the efficiency of perceptual decision making.

The ability to make rapid and accurate decisions based on limited sensory information is a critical component of visual cognition. Available evidence suggests that simple perceptual discriminations are based on the accumulation and integration of sensory evidence over time. However, the memory system(s) mediating this accumulation are unclear. One candidate system is working memory (WM), which enables the temporary maintenance of information in a readily accessible state. Here, we show that individual variability in WM capacity is strongly correlated with the speed of evidence accumulation in speeded two-alternative forced choice tasks. This relationship generalized across different decision-making tasks, and could not be easily explained by variability in general arousal or vigilance. Moreover, we show that performing a difficult discrimination task while maintaining a concurrent memory load has a deleterious effect on the latter, suggesting that WM storage and decision making are directly linked.

Evidence for a fixed capacity limit in attending multiple locations.

A classic question concerns whether humans can attend multiple locations or objects at once. Although it is generally agreed that the answer to this question is "yes," the limits on this ability are subject to extensive debate. According to one view, attentional resources can be flexibly allocated to a variable number of locations, with an inverse relationship between the number of selected locations and the quality of information processing at each location. Alternatively, these resources might be quantized in a "discrete" fashion that enables concurrent access to a small number of locations. Here, we report a series of experiments comparing these alternatives. In each experiment, we cued participants to attend a variable number of spatial locations and asked them to report the orientation of a single, briefly presented target. In all experiments, participants' orientation report errors were well-described by a model that assumes a fixed upper limit in the number of locations that can be attended. Conversely, report errors were poorly described by a flexible-resource model that assumes no fixed limit on the number of locations that can be attended. Critically, we showed that these discrete limits were predicted by cue-evoked neural activity elicited before the onset of the target array, suggesting that performance was limited by selection processes that began prior to subsequent encoding and memory storage. Together, these findings constitute novel evidence supporting the hypothesis that human observers can attend only a small number of discrete locations at an instant.