Multiple memory systems for efficient temporal order memory.

We report distinct contributions of multiple memory systems to the retrieval of the temporal order of events. The neural dynamics related to the retrieval of movie scenes revealed that recalling the temporal order of close events elevates hippocampal theta power, like that observed for recalling close spatial relationships. In contrast, recalling far events increases beta power in the orbitofrontal cortex, reflecting recall based on the overall movie structure.

Quantifying the narrative flow of imagined versus autobiographical stories.

Lifelong experiences and learned knowledge lead to shared expectations about how common situations tend to unfold. Such knowledge of narrative event flow enables people to weave together a story. However, comparable computational tools to evaluate the flow of events in narratives are limited. We quantify the differences between autobiographical and imagined stories by introducing sequentiality, a measure of narrative flow of events, drawing probabilistic inferences from a cutting-edge large language model (GPT-3). Sequentiality captures the flow of a narrative by comparing the probability of a sentence with and without its preceding story context. We applied our measure to study thousands of diary-like stories, collected from crowdworkers, about either a recent remembered experience or an imagined story on the same topic. The results show that imagined stories have higher sequentiality than autobiographical stories and that the sequentiality of autobiographical stories increases when the memories are retold several months later. In pursuit of deeper understandings of how sequentiality measures the flow of narratives, we explore proportions of major and minor events in story sentences, as annotated by crowdworkers. We find that lower sequentiality is associated with higher proportions of major events. The methods and results highlight opportunities to use cutting-edge computational analyses, such as sequentiality, on large corpora of matched imagined and autobiographical stories to investigate the influences of memory and reasoning on language generation processes.

Event segmentation reveals working memory forgetting rate.

We encounter the world as a continuous flow and effortlessly segment sequences of events into episodes. This process of event segmentation engages working memory (WM) for tracking the flow of events and impacts subsequent memory accuracy. WM is limited in how much information (i.e., WM capacity) and for how long the information is retained (i.e., forgetting rate). In this study, across multiple tasks, we estimated participants' WM capacity and forgetting rate in a dynamic context and evaluated their relationship to event segmentation. A U-shaped relationship across tasks shows that individuals who segmented the movie more finely or coarsely than the average have a faster WM forgetting rate. A separate task assessing long-term memory retrieval revealed that the coarse-segmenters have better recognition of temporal order of events compared to the fine-segmenters. These findings show that event segmentation employs dissociable memory strategies and correlates with how long information is retained in WM.

Medial Orbitofrontal Cortex, Dorsolateral Prefrontal Cortex, and Hippocampus Differentially Represent the Event Saliency.

Two primary functions attributed to the hippocampus and prefrontal cortex (PFC) network are retaining the temporal and spatial associations of events and detecting deviant events. It is unclear, however, how these two functions converge into one mechanism. Here, we tested whether increased activity with perceiving salient events is a deviant detection signal or contains information about the event associations by reflecting the magnitude of deviance (i.e., event saliency). We also tested how the deviant detection signal is affected by the degree of anticipation. We studied regional neural activity when people watched a movie that had varying saliency of a novel or an anticipated flow of salient events. Using intracranial electroencephalography from 10 patients, we observed that high-frequency activity (50-150 Hz) in the hippocampus, dorsolateral PFC, and medial OFC tracked event saliency. We also observed that medial OFC activity was stronger when the salient events were anticipated than when they were novel. These results suggest that dorsolateral PFC and medial OFC, as well as the hippocampus, signify the saliency magnitude of events, reflecting the hierarchical structure of event associations.

Working Memory Replay Prioritizes Weakly Attended Events.

One view of working memory posits that maintaining a series of events requires their sequential and equal mnemonic replay. Another view is that the content of working memory maintenance is prioritized by attention. We decoded the dynamics for retaining a sequence of items using magnetoencephalography, wherein participants encoded sequences of three stimuli depicting a face, a manufactured object, or a natural item and maintained them in working memory for 5000 ms. Memory for sequence position and stimulus details were probed at the end of the maintenance period. Decoding of brain activity revealed that one of the three stimuli dominated maintenance independent of its sequence position or category; and memory was enhanced for the selectively replayed stimulus. Analysis of event-related responses during the encoding of the sequence showed that the selectively replayed stimuli were determined by the degree of attention at encoding. The selectively replayed stimuli had the weakest initial encoding indexed by weaker visual attention signals at encoding. These findings do not rule out sequential mnemonic replay but reveal that attention influences the content of working memory maintenance by prioritizing replay of weakly encoded events. We propose that the prioritization of weakly encoded stimuli protects them from interference during the maintenance period, whereas the more strongly encoded stimuli can be retrieved from long-term memory at the end of the delay period.

Human hippocampal pre-activation predicts behavior.

The response to an upcoming salient event is accelerated when the event is expected given the preceding events - i.e. a temporal context effect. For example, naming a picture following a strongly constraining temporal context is faster than naming a picture after a weakly constraining temporal context. We used sentences as naturalistic stimuli to manipulate expectations on upcoming pictures without prior training. Here, using intracranial recordings from the human hippocampus we found more power in the high-frequency band prior to high-expected pictures than weakly expected ones. We applied pattern similarity analysis on the temporal pattern of hippocampal high-frequency band activity in single hippocampal contacts. We found that greater similarity in the pattern of hippocampal field potentials between pre-picture interval and expected picture interval in the high-frequency band predicted picture-naming latencies. Additional pattern similarity analysis indicated that the hippocampal representations follow a semantic map. The results suggest that hippocampal pre-activation of expected stimuli is a facilitating mechanism underlying the powerful contextual behavioral effect.

Familiarity expands space and contracts time.

When humans draw maps, or make judgments about travel-time, their responses are rarely accurate and are often systematically distorted. Distortion effects on estimating time to arrival and the scale of sketch-maps reveal the nature of mental representation of time and space. Inspired by data from rodent entorhinal grid cells, we predicted that familiarity to an environment would distort representations of the space by expanding the size of it. We also hypothesized that travel-time estimation would be distorted in the same direction as space-size, if time and space rely on the same cognitive map. We asked international students, who had lived at a college in London for 9 months, to sketch a south-up map of their college district, estimate travel-time to destinations within the area, and mark their everyday walking routes. We found that while estimates for sketched space were expanded with familiarity, estimates of the time to travel through the space were contracted with familiarity. Thus, we found dissociable responses to familiarity in representations of time and space. © 2016 The Authors Hippocampus Published by Wiley Periodicals, Inc.

Replay of very early encoding representations during recollection.

Long-term memories are linked to cortical representations of perceived events, but it is unclear which types of representations can later be recollected. Using magnetoencephalography-based decoding, we examined which brain activity patterns elicited during encoding are later replayed during recollection in the human brain. The results show that the recollection of images depicting faces and scenes is associated with a replay of neural representations that are formed at very early (180 ms) stages of encoding. This replay occurs quite rapidly, ~500 ms after the onset of a cue that prompts recollection and correlates with source memory accuracy. Therefore, long-term memories are rapidly replayed during recollection and involve representations that were formed at very early stages of encoding. These findings indicate that very early representational information can be preserved in the memory engram and can be faithfully and rapidly reinstated during recollection. These novel insights into the nature of the memory engram provide constraints for mechanistic models of long-term memory function.

Population level inference for multivariate MEG analysis.

Multivariate analysis is a very general and powerful technique for analysing Magnetoencephalography (MEG) data. An outstanding problem however is how to make inferences that are consistent over a group of subjects as to whether there are condition-specific differences in data features, and what are those features that maximise these differences. Here we propose a solution based on Canonical Variates Analysis (CVA) model scoring at the subject level and random effects Bayesian model selection at the group level. We apply this approach to beamformer reconstructed MEG data in source space. CVA estimates those multivariate patterns of activation that correlate most highly with the experimental design; the order of a CVA model is then determined by the number of significant canonical vectors. Random effects Bayesian model comparison then provides machinery for inferring the optimal order over the group of subjects. Absence of a multivariate dependence is indicated by the null model being the most likely. This approach can also be applied to CVA models with a fixed number of canonical vectors but supplied with different feature sets. We illustrate the method by identifying feature sets based on variable-dimension MEG power spectra in the primary visual cortex and fusiform gyrus that are maximally discriminative of data epochs before versus after visual stimulation.