Adding Meaning to Memories: How Parietal Cortex Combines Semantic Content with Episodic Experience.

Neuroimaging studies of human memory have consistently found that univariate responses in parietal cortex track episodic experience with stimuli (whether stimuli are 'old' or 'new'). More recently, pattern-based fMRI studies have shown that parietal cortex also carries information about the semantic content of remembered experiences. However, it is not well understood how memory-based and content-based signals are integrated within parietal cortex. Here, in humans (males and females), we used voxel-wise encoding models and a recognition memory task to predict the fMRI activity patterns evoked by complex natural scene images based on (1) the episodic history and (2) the semantic content of each image. Models were generated and compared across distinct subregions of parietal cortex and for occipitotemporal cortex. We show that parietal and occipitotemporal regions each encode memory and content information, but they differ in how they combine this information. Among parietal subregions, angular gyrus was characterized by robust and overlapping effects of memory and content. Moreover, subject-specific semantic tuning functions revealed that successful recognition shifted the amplitude of tuning functions in angular gyrus but did not change the selectivity of tuning. In other words, effects of memory and content were additive in angular gyrus. This pattern of data contrasted with occipitotemporal cortex where memory and content effects were interactive: memory effects were preferentially expressed by voxels tuned to the content of a remembered image. Collectively, these findings provide unique insight into how parietal cortex combines information about episodic memory and semantic content.SIGNIFICANCE STATEMENT Neuroimaging studies of human memory have identified multiple brain regions that not only carry information about "whether" a visual stimulus is successfully recognized but also "what" the content of that stimulus includes. However, a fundamental and open question concerns how the brain integrates these two types of information (memory and content). Here, using a powerful combination of fMRI analysis methods, we show that parietal cortex, particularly the angular gyrus, robustly combines memory- and content-related information, but these two forms of information are represented via additive, independent signals. In contrast, memory effects in high-level visual cortex critically depend on (and interact with) content representations. Together, these findings reveal multiple and distinct ways in which the brain combines memory- and content-related information.

Pupillometry signatures of sustained attention and working memory.

There exists an intricate relationship between attention and working memory. Recent work has further established that attention and working memory fluctuate synchronously, by tightly interleaving sustained attention and working memory tasks. This work has raised many open questions about physiological signatures underlying these behavioral fluctuations. Across two experiments, we explore pupil dynamics using real-time triggering in conjunction with an interleaved sustained attention and working memory task. In Experiment 1, we use behavioral real-time triggering and replicate recent findings from our lab (deBettencourt et al., 2019) that sustained attention fluctuates concurrently with the number of items maintained in working memory. Furthermore, highly attentive moments, detected via behavior, also exhibited larger pupil sizes. In Experiment 2, we develop a novel real-time pupil-triggering technique to track pupil size fluctuations in real time and trigger working memory probes. We show that this pupil triggering procedure reveals differences in sustained attention, as indexed by response time. These experiments reflect methodological advances in real-time triggering and further disentangle the relationship among general arousal, sustained attention, and working memory.

Real-time triggering reveals concurrent lapses of attention and working memory.

Attention and working memory are clearly intertwined, as shown by co-variations in individual ability and the recruitment of similar neural substrates. Both processes fluctuate over time1-5, and these fluctuations may be a key determinant of individual variations in ability6,7. If these fluctuations are due to the waxing and waning of a common cognitive resource, attention and working memory should co-vary on a moment-to-moment basis. To test this, we developed a hybrid task that interleaved a sustained attention task and a whole-report working memory task. Experiment 1 established that performance fluctuations on these tasks correlated across and within participants: attention lapses led to worse working memory performance. Experiment 2 extended this finding using a real-time triggering procedure that monitored attention fluctuations to probe working memory during optimal (high-attention) or suboptimal (low-attention) moments. In low-attention moments, participants stored fewer items in working memory. Experiment 3 ruled out task-general fluctuations as an explanation for these co-variations by showing that the precision of colour memory was unaffected by variations in attention state. In summary, we demonstrate that attention and working memory lapse together, providing additional evidence for the tight integration of these cognitive processes.