Systematic differences in visual working memory performance are not caused by differences in working memory storage.

People vary in their performance on visual working memory tasks, and these individual differences covary with a wide range of higher-level cognitive processes including fluid intelligence. Performance also varies across study displays, purportedly driven by both low- and higher-level processes. Understanding what causes these sources of systematic variability has been crucial for developing theories of working memory. However, here we find that all such variability in performance on a test of visual working memory can be accounted for by concurrent variability in visual iconic memory: A person with relatively high working memory capacity will have high iconic memory capacity, and a particularly easy working memory display will also be easy under iconic memory conditions. These results are supported by a nonparametric factor analysis and hierarchical Bayesian model comparison. In a second experiment the relationship between iconic and working memory holds even when they are measured with substantially different experimental paradigms, and a third experiment suggests that the relationship between tests of iconic and working memory is driven by mechanisms other than iconic or working memory storage, such as variation in perceptual or attentional processes. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Local motion pooling is continuous, global motion perception is discrete.

Perceiving the motion of an object is thought to involve two stages: Local motion energy is measured at each point in space, and these signals are then pooled across space to build coherent global motion. There are several theories of how local-to-global pooling occurs, but they all predict that global motion perception is a continuous process, such that increasing the strength of motion energy should gradually increase the precision of perceived motion directions. We test this prediction against the alternative that global motion perception is discrete: Motion is either perceived with high precision or fails to be perceived altogether. Data from human observers provides clear evidence that, whereas pooling local motion energy is continuous, the segmentation of local signals into coherent global motion patterns is a discrete process. This result adds motion perception to the growing list of processes that exhibit evidence of all-or-none visual awareness. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Eriksen flanker delta plot shapes depend on the stimulus.

Several experimental paradigms are purported to measure response conflict, including the Stroop, Simon, and Eriksen flanker tasks. Although these tasks are often treated as being similar, delta plot analyses of response time distributions have revealed marked differences across them. Several theories have been proposed to explain these differences, however, assessing their veracity is difficult given the numerous differences across tasks. To explore what might cause delta plots to differ in a more controlled manner, here stimulus materials were manipulated across four Eriksen flanker tasks. The results reveal substantially different delta plot shapes for different stimuli: positive-going functions when color or motion served as the target and flankers, and delta plots with negative-going components when stimuli were arrows or orientated gratings. These results cast doubt on the proposal that negative-going delta plots occur only when spatial location serves as the interfering stimulus dimension. Moreover, because targets and flankers were always of the same stimulus type, the results also suggest that differences in materials across the relevant and irrelevant dimensions do not determine delta plot shapes. Instead, we propose that the delta plot shape is determined by several factors, including how early the interfering information is processed in the visual cortex.

Set size effects on working memory precision are not due to an averaging of slots.

Visual working memory is often characterized as a discrete system, where an item is either stored in memory or it is lost completely. As this theory predicts, increasing memory load primarily affects the probability that an item is in memory. However, the precision of items successfully stored in memory also decreases with memory load. The prominent explanation for this effect is the "slots-plus-averaging" model, which proposes that an item can be stored in replicate across multiple memory slots. Here, however, precision declined with set size even in iconic memory tasks that did not require working memory storage, ruling out such storage accounts. Moreover, whereas the slots-plus-averaging model predicts that precision effects should plateau at working memory capacity limits, precision continued to decline well beyond these limits in an iconic memory task, where the number of items available at test was far greater than working memory capacity. Precision also declined in tasks that did not require study items to be encoded simultaneously, ruling out perceptual limitations as the cause of set size effects on memory precision. Taken together, these results imply that set size effects on working memory precision do not stem from working memory storage processes, such as an averaging of slots, and are not due to perceptual limitations. This rejection of the prominent slots-plus-averaging model has implications for how contemporary models of discrete capacities theories can be improved, and how they might be rejected.

Swap errors in spatial working memory are guesses.

In typical visual working memory tasks, participants report the color of a previously studied item at some probed location. Alternatively, in some recent studies, a color is probed and participants must report the item's location. There is a surprising difference between these tasks: in location reports participants almost never guess randomly as they do when reporting color, but often incorrectly report the locations of non-probed items. This finding has been taken as evidence for feature binding errors in memory, and evidence against discrete capacity models, which predict that pure guessing should occur. We test an alternative possibility: that non-target responses are guesses, but intelligent ones. In particular, when asked to report the location of an item for which participants have no memory, they may guess near locations where they know something was presented. Here we present false-probe trials in which a color is probed that was not actually studied, and find that the responses, which are necessarily guesses, are nonetheless centered around studied locations. Moreover, we find that the confidence ratings for non-target responses are low, and similar to confidence for uniformly distributed guesses. In a second experiment, we find that manipulating the retention interval, which is known to affect guess rates, changes the rate of these low-confidence non-target responses. These results suggest that the tendency to report locations of non-probed items reflects a good guessing strategy; not something fundamental about how features and objects are represented in working memory.

Iconic Memories Die a Sudden Death.

Iconic memory is characterized by its large storage capacity and brief storage duration, whereas visual working memory is characterized by its small storage capacity. The limited information stored in working memory is often modeled as an all-or-none process in which studied information is either successfully stored or lost completely. This view raises a simple question: If almost all viewed information is stored in iconic memory, yet one second later most of it is completely absent from working memory, what happened to it? Here, I characterized how the precision and capacity of iconic memory changed over time and observed a clear dissociation: Iconic memory suffered from a complete loss of visual items, while the precision of items retained in memory was only marginally affected by the passage of time. These results provide new evidence for the discrete-capacity view of working memory and a new characterization of iconic memory decay.

Integrating Theoretical Models with Functional Neuroimaging.

The development of mathematical models to characterize perceptual and cognitive processes dates back almost to the inception of the field of psychology. Since the 1990s, human functional neuroimaging has provided for rapid empirical and theoretical advances across a variety of domains in cognitive neuroscience. In more recent work, formal modeling and neuroimaging approaches are being successfully combined, often producing models with a level of specificity and rigor that would not have been possible by studying behavior alone. In this review, we highlight examples of recent studies that utilize this combined approach to provide novel insights into the mechanisms underlying human cognition. The studies described here span domains of perception, attention, memory, categorization, and cognitive control, employing a variety of analytic and model-inspired approaches. Across these diverse studies, a common theme is that individually tailored, creative solutions are often needed to establish compelling links between multi-parameter models and complex sets of neural data. We conclude that future developments in model-based cognitive neuroscience will have great potential to advance our theoretical understanding and ability to model both low-level and high-level cognitive processes.

Accounting for stimulus-specific variation in precision reveals a discrete capacity limit in visual working memory.

If we view a visual scene that contains many objects, then momentarily close our eyes, some details persist while others seem to fade. Discrete models of visual working memory (VWM) assume that only a few items can be actively maintained in memory, beyond which pure guessing will emerge. Alternatively, continuous resource models assume that all items in a visual scene can be stored with some precision. Distinguishing between these competing models is challenging, however, as resource models that allow for stochastically variable precision (across items and trials) can produce error distributions that resemble random guessing behavior. Here, we evaluated the hypothesis that a major source of variability in VWM performance arises from systematic variation in precision across the stimuli themselves; such stimulus-specific variability can be incorporated into both discrete-capacity and variable-precision resource models. Participants viewed multiple oriented gratings, and then reported the orientation of a cued grating from memory. When modeling the overall distribution of VWM errors, we found that the variable-precision resource model outperformed the discrete model. However, VWM errors revealed a pronounced "oblique effect," with larger errors for oblique than cardinal orientations. After this source of variability was incorporated into both models, we found that the discrete model provided a better account of VWM errors. Our results demonstrate that variable precision across the stimulus space can lead to an unwarranted advantage for resource models that assume stochastically variable precision. When these deterministic sources are adequately modeled, human working memory performance reveals evidence of a discrete capacity limit. (PsycINFO Database Record

Radial bias is not necessary for orientation decoding.

Multivariate pattern analysis can be used to decode the orientation of a viewed grating from fMRI signals in early visual areas. Although some studies have reported identifying multiple sources of the orientation information that make decoding possible, a recent study argued that orientation decoding is only possible because of a single source: a coarse-scale retinotopically organized preference for radial orientations. Here we aim to resolve these discrepant findings. We show that there were subtle, but critical, experimental design choices that led to the erroneous conclusion that a radial bias is the only source of orientation information in fMRI signals. In particular, we show that the reliance on a fast temporal-encoding paradigm for spatial mapping can be problematic, as effects of space and time become conflated and lead to distorted estimates of a voxel's orientation or retinotopic preference. When we implement minor changes to the temporal paradigm or to the visual stimulus itself, by slowing the periodic rotation of the stimulus or by smoothing its contrast-energy profile, we find significant evidence of orientation information that does not originate from radial bias. In an additional block-paradigm experiment where space and time were not conflated, we apply a formal model comparison approach and find that many voxels exhibit more complex tuning properties than predicted by radial bias alone or in combination with other known coarse-scale biases. Our findings support the conclusion that radial bias is not necessary for orientation decoding. In addition, our study highlights potential limitations of using temporal phase-encoded fMRI designs for characterizing voxel tuning properties.

Sensory uncertainty decoded from visual cortex predicts behavior.

Bayesian theories of neural coding propose that sensory uncertainty is represented by a probability distribution encoded in neural population activity, but direct neural evidence supporting this hypothesis is currently lacking. Using fMRI in combination with a generative model-based analysis, we found that probability distributions reflecting sensory uncertainty could reliably be estimated from human visual cortex and, moreover, that observers appeared to use knowledge of this uncertainty in their perceptual decisions.

Attention alters orientation processing in the human lateral geniculate nucleus.

Orientation selectivity is a cornerstone property of vision, commonly believed to emerge in the primary visual cortex. We found that reliable orientation information could be detected even earlier, in the human lateral geniculate nucleus, and that attentional feedback selectively altered these orientation responses. This attentional modulation may allow the visual system to modify incoming feature-specific signals at the earliest possible processing site.

Spatial specificity of working memory representations in the early visual cortex.

Recent fMRI decoding studies have demonstrated that early retinotopic visual areas exhibit similar patterns of activity during the perception of a stimulus and during the maintenance of that stimulus in working memory. These findings provide support for the sensory recruitment hypothesis that the mechanisms underlying perception serve as a foundation for visual working memory. However, a recent study by Ester, Serences, and Awh (2009) found that the orientation of a peripheral grating maintained in working memory could be classified from both the contralateral and ipsilateral regions of the primary visual cortex (V1), implying that, unlike perception, feature-specific information was maintained in a nonretinotopic manner. Here, we evaluated the hypothesis that early visual areas can maintain information in a spatially specific manner and will do so if the task encourages the binding of feature information to a specific location. To encourage reliance on spatially specific memory, our experiment required observers to retain the orientations of two laterally presented gratings. Multivariate pattern analysis revealed that the orientation of each remembered grating was classified more accurately based on activity patterns in the contralateral than in the ipsilateral regions of V1 and V2. In contrast, higher extrastriate areas exhibited similar levels of performance across the two hemispheres. A time-resolved analysis further indicated that the retinotopic specificity of the working memory representation in V1 and V2 was maintained throughout the retention interval. Our results suggest that early visual areas provide a cortical basis for actively maintaining information about the features and locations of stimuli in visual working memory.

Expertise for upright faces improves the precision but not the capacity of visual working memory.

Considerable research has focused on how basic visual features are maintained in working memory, but little is currently known about the precision or capacity of visual working memory for complex objects. How precisely can an object be remembered, and to what extent might familiarity or perceptual expertise contribute to working memory performance? To address these questions, we developed a set of computer-generated face stimuli that varied continuously along the dimensions of age and gender, and we probed participants' memories using a method-of-adjustment reporting procedure. This paradigm allowed us to separately estimate the precision and capacity of working memory for individual faces, on the basis of the assumptions of a discrete capacity model, and to assess the impact of face inversion on memory performance. We found that observers could maintain up to four to five items on average, with equally good memory capacity for upright and upside-down faces. In contrast, memory precision was significantly impaired by face inversion at every set size tested. Our results demonstrate that the precision of visual working memory for a complex stimulus is not strictly fixed but, instead, can be modified by learning and experience. We find that perceptual expertise for upright faces leads to significant improvements in visual precision, without modifying the capacity of working memory.

How attention extracts objects from noise.

The visual system is remarkably proficient at extracting relevant object information from noisy, cluttered environments. Although attention is known to enhance sensory processing, the mechanisms by which attention extracts relevant information from noise are not well understood. According to the perceptual template model, attention may act to amplify responses to all visual input, or it may act as a noise filter, dampening responses to irrelevant visual noise. Amplification allows for improved performance in the absence of visual noise, whereas a noise-filtering mechanism can only improve performance if the target stimulus appears in noise. Here, we used fMRI to investigate how attention modulates cortical responses to objects at multiple levels of the visual pathway. Participants viewed images of faces, houses, chairs, and shoes, presented in various levels of visual noise. We used multivoxel pattern analysis to predict the viewed object category, for attended and unattended stimuli, from cortical activity patterns in individual visual areas. Early visual areas, V1 and V2, exhibited a benefit of attention only at high levels of visual noise, suggesting that attention operates via a noise-filtering mechanism at these early sites. By contrast, attention led to enhanced processing of noise-free images (i.e., amplification) only in higher visual areas, including area V4, fusiform face area, mid-Fusiform area, and the lateral occipital cortex. Together, these results suggest that attention improves people's ability to discriminate objects by de-noising visual input in early visual areas and amplifying this noise-reduced signal at higher stages of visual processing.

Assessing the dissociability of recollection and familiarity in recognition memory.

Recognition memory is often modeled as constituting 2 separate processes, recollection and familiarity, rather than as constituting a single process mediated by a generic latent strength. One way of stating evidence for the more complex 2-process model is to show dissociations with select manipulations, in which one manipulation affects recollection more than the second and the second affects familiarity more than the first. One of the best paradigms for assessing such dissociations is the confidence-ratings paradigm, because within it criterial and mnemonic effects may be separately estimated. There is, unfortunately, a relative lack of easily interpretable dissociation experiments in the confidence ratings paradigm, and those that exist do not show clear evidence for dissociations. We report the results of 3 experiments with conventional manipulations designed to maximally dissociate recollection and familiarity. To provide valid assessment, without recourse to aggregation over items or participants, we develop a hierarchical version of Yonelinas' (1994) dual-process model and a novel test of dissociability for state-trace plots. The data do not provide evidence that recollection and familiarity are dissociable. Instead, estimates of recollection and familiarity are positively related across conditions and experiments. On balance, performance is more parsimoniously accounted for by a single mnemonic process that drives both recollection and familiarity estimates.

Decoding patterns of human brain activity.

Considerable information about mental states can be decoded from noninvasive measures of human brain activity. Analyses of brain activity patterns can reveal what a person is seeing, perceiving, attending to, or remembering. Moreover, multidimensional models can be used to investigate how the brain encodes complex visual scenes or abstract semantic information. Such feats of "brain reading" or "mind reading," though impressive, raise important conceptual, methodological, and ethical issues. What does successful decoding reveal about the cognitive functions performed by a brain region? How should brain signals be spatially selected and mathematically combined to ensure that decoding reflects inherent computations of the brain rather than those performed by the decoder? We highlight recent advances and describe how multivoxel pattern analysis can provide a window into mind-brain relationships with unprecedented specificity, when carefully applied. However, as brain-reading technology advances, issues of neuroethics and mental privacy will be important to consider.

On perfect working-memory performance with large numbers of items.

Many popular models conceptualize working memory as consisting of three or four discrete slots or bins. This conceptualization, however, has been seemingly refuted by Bays and Husain (2009), who reported perfect performance on a working memory task with a large number of very simple items. We show, however, that this perfect-performance result likely reflects a design flaw rather than mnemonic structure. The flaw is that the test array itself in Bays and Husain's study provides information about the correct answer without recourse to working memory. We show perfect performance on eight items for 18 participants when this information is present. We show that performance is poorer, however, when this information is removed. Hence, the Bays and Husain result does not threaten models that stipulate that working memory is composed of limited slots.

Gradual growth versus shape invariance in perceptual decision making.

A dominant theme in modeling human perceptual judgments is that sensory neural activity is summed or integrated until a critical bound is reached. Such models predict that, in general, the shape of response time distributions change across conditions, although in practice, this shape change may be subtle. An alternative view is that response time distributions are shape invariant across conditions or groups. Shape invariance is predicted by some race models in which the first of several parallel fibers to communicate the signal determines the response. We competitively assess a specific gradual growth model, the one-bound diffusion model, against a natural shape-invariant competitor: shape invariance in an inverse Gaussian distribution. Assessment of subtle shape change versus shape invariance of response time distributions is aided by a Bayesian approach that allows the pooling of information across multiple participants. We find, conditional on reasonable distributional assumptions, subtle shape changes in response time that are highly concordant with a simple diffusion gradual growth model and discordant with shape invariance.

Exploring the differences in distributional properties between Stroop and Simon effects using delta plots.

Stroop and Simon tasks are logically similar and are often used to investigate cognitive control and inhibition processes. We compare the distributional properties of Stroop and Simon effects with delta plots and find different although stable patterns. Stroop effects across a variety of conditions are smallest for fast responses and increase as responses slow. Simon effects across a variety of conditions, however, are largest for fast responses but decrease, and even reverse, as responses slow. We show in three experiments that these diverging patterns hold within participants and even when the stimulus materials are identical across the tasks. These stable differences in time course serve as bedrock phenomena for building and testing theories of cognitive control and inhibition. The results of two additional experiments suggest that the determinant of time course is not simply whether the distracting information is location.

Latent mnemonic strengths are latent: a comment on Mickes, Wixted, and Wais (2007).

Mickes, Wixted, and Wais (2007) proposed a simple test of latent strength variability in recognition memory. They asked participants to rate their confidence using either a 20-point or a 99-point strength scale and plotted distributions of the resulting ratings. They found 25% more variability in ratings for studied than for new items, which they interpreted as providing evidence that latent mnemonic strength distributions are 25% more variable for studied than for new items. We show here that this conclusion is critically dependent on assumptions--so much so that these assumptions determine the conclusions. In fact, opposite conclusions, such that study does not affect the variability of latent strength, may be reached by making different but equally plausible assumptions. Because all measurements of mnemonic strength variability are critically dependent on untestable assumptions, all are arbitrary. Hence, there is no principled method for assessing the relative variability of latent mnemonic strength distributions.