Event Integration and Temporal Differentiation: How Hierarchical Knowledge Emerges in Hippocampal Subfields through Learning.

Everyday life is composed of events organized by changes in contexts, with each event containing an unfolding sequence of occurrences. A major challenge facing our memory systems is how to integrate sequential occurrences within events while also maintaining their details and avoiding over-integration across different contexts. We asked if and how distinct hippocampal subfields come to hierarchically and, in parallel, represent both event context and subevent occurrences with learning. Female and male human participants viewed sequential events defined as sequences of objects superimposed on shared color frames while undergoing high-resolution fMRI. Importantly, these events were repeated to induce learning. Event segmentation, as indexed by increased reaction times at event boundaries, was observed in all repetitions. Temporal memory decisions were quicker for items from the same event compared to across different events, indicating that events shaped memory. With learning, hippocampal CA3 multivoxel activation patterns clustered to reflect the event context, with more clustering correlated with behavioral facilitation during event transitions. In contrast, in the dentate gyrus (DG), temporally proximal items that belonged to the same event became associated with more differentiated neural patterns. A computational model explained these results by dynamic inhibition in the DG. Additional similarity measures support the notion that CA3 clustered representations reflect shared voxel populations, while DG's distinct item representations reflect different voxel populations. These findings suggest an interplay between temporal differentiation in the DG and attractor dynamics in CA3. They advance our understanding of how knowledge is structured through integration and separation across time and context.

Predictions transform memories: How expected versus unexpected events are integrated or separated in memory.

Our brains constantly generate predictions about the environment based on prior knowledge. Many of the events we experience are consistent with these predictions, while others might be inconsistent with prior knowledge and thus violate our predictions. To guide future behavior, the memory system must be able to strengthen, transform, or add to existing knowledge based on the accuracy of our predictions. We synthesize recent evidence suggesting that when an event is consistent with our predictions, it leads to neural integration between related memories, which is associated with enhanced associative memory, as well as memory biases. Prediction errors, in turn, can promote both neural integration and separation, and lead to multiple mnemonic outcomes. We review these findings and how they interact with factors such as memory reactivation, prediction error strength, and task goals, to offer insight into what determines memory for events that violate our predictions. In doing so, this review brings together recent neural and behavioral research to advance our understanding of how predictions shape memory, and why.

Mnemonic prediction errors promote detailed memories.

When our experience violates our predictions, it is adaptive to update our knowledge to promote a more accurate representation of the world and facilitate future predictions. Theoretical models propose that these mnemonic prediction errors should be encoded into a distinct memory trace to prevent interference with previous, conflicting memories. We investigated this proposal by repeatedly exposing participants to pairs of sequentially presented objects (A → B), thus evoking expectations. Then, we violated participants' expectations by replacing the second object in the pairs with a novel object (A → C). The following item memory test required participants to discriminate between identical old items and similar lures, thus testing detailed and distinctive item memory representations. In two experiments, mnemonic prediction errors enhanced item memory: Participants correctly identified more old items as old when those items violated expectations during learning, compared with items that did not violate expectations. This memory enhancement for C items was only observed when participants later showed intact memory for the related A → B pairs, suggesting that strong predictions are required to facilitate memory for violations. Following up on this, a third experiment reduced prediction strength prior to violation and subsequently eliminated the memory advantage of violations. Interestingly, mnemonic prediction errors did not increase gist-based mistakes of identifying old items as similar lures or identifying similar lures as old. Enhanced item memory in the absence of gist-based mistakes suggests that violations enhanced memory for items' details, which could be mediated via distinct memory traces. Together, these results advance our knowledge of how mnemonic prediction errors promote memory formation.

Publisher Correction: Mnemonic prediction errors bias hippocampal states.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Prior knowledge promotes hippocampal separation but cortical assimilation in the left inferior frontal gyrus.

An adaptive memory system rarely learns information tabula rasa, but rather builds on prior knowledge to facilitate learning. How prior knowledge influences the neural representation of novel associations remains unknown. Here, participants associated pairs of faces in two conditions: a famous, highly familiar face with a novel face or two novel faces while undergoing fMRI. We examine multivoxel activity patterns corresponding to individual faces before and after learning. The activity patterns representing members of famous-novel pairs becomes separated in the hippocampus, that is, more distinct from one another through learning, in striking contrast to paired novel faces that become similar. In the left inferior frontal gyrus, however, prior knowledge leads to integration, and in a specific direction: the representation of the novel face becomes similar to that of the famous face after learning, suggesting assimilation of new into old memories. We propose that hippocampal separation might resolve interference between existing and newly learned information, allowing cortical assimilation. Thus, associative learning with versus without prior knowledge relies on radically different computations.

Mnemonic prediction errors bias hippocampal states.

When our experience violates our predictions, it is adaptive to upregulate encoding of novel information, while down-weighting retrieval of erroneous memory predictions to promote an updated representation of the world. We asked whether mnemonic prediction errors promote hippocampal encoding versus retrieval states, as marked by distinct network connectivity between hippocampal subfields. During fMRI scanning, participants were cued to internally retrieve well-learned complex room-images and were then presented with either an identical or a modified image (0-4 changes). In the left hemisphere, we find that CA1-entorhinal connectivity increases, and CA1-CA3 connectivity decreases, with the number of changes. Further, in the left CA1, the similarity between activity patterns during cued-retrieval of the learned room and during the image is lower when the image includes changes, consistent with a prediction error signal in CA1. Our findings provide a mechanism by which mnemonic prediction errors may drive memory updating-by biasing hippocampal states.

Face Selective Neural Activity: Comparisons Between Fixed and Free Viewing.

Event Related Potentials (ERPs) are widely used to study category-selective EEG responses to visual stimuli, such as the face-selective N170 component. Typically, this is done by flashing stimuli at the point of static gaze fixation. While allowing for good experimental control, these paradigms ignore the dynamic role of eye-movements in natural vision. Fixation-related potentials (FRPs), obtained using simultaneous EEG and eye-tracking, overcome this limitation. Various studies have used FRPs to study processes such as lexical processing, target detection and attention allocation. The goal of this study was to carefully compare face-sensitive activity time-locked to an abrupt stimulus onset at fixation, with that time-locked to a self-generated fixation on a stimulus. Twelve participants participated in three experimental conditions: Free-viewing (FRPs), Cued-viewing (FRPs) and Control (ERPs). We used a multiple regression approach to disentangle overlapping activity components. Our results show that the N170 face-effect is evident for the first fixation on a stimulus, whether it follows a self-generated saccade or stimulus appearance at fixation point. The N170 face-effect has similar topography across viewing conditions, but there were major differences within each stimulus category. We ascribe these differences to an overlap of the fixation-related lambda response and the N170. We tested the plausibility of this account using dipole simulations. Finally, the N170 exhibits category-specific adaptation in free viewing. This study establishes the comparability of the free-viewing N170 face-effect with the classic event-related effect, while highlighting the importance of accounting for eye-movement related effects.

Distinct Neural Suppression and Encoding Effects for Conceptual Novelty and Familiarity.

Like yin and yang, novelty and familiarity are typically described as separate-yet-complementary aspects of an experience, two ends of a single continuum. However, novelty and familiarity are also multifaceted. For instance, novelty can sometimes result in enhanced mnemonic performance, whereas at other times familiarity is better remembered. As previous investigations focused primarily on the experimental aspect of novelty, the mechanisms supporting conceptual novelty (the novel combination of two previously unrelated existing concepts) remain unclear. Importantly, conceptual novelty can be recognized as such only when compared with preexperimental familiar knowledge, regardless of experimental status. Here we applied a combined repetition suppression/subsequent memory fMRI paradigm, focusing on the conceptual aspect of novelty and familiarity as the subject matter. Conceptual novelty was characterized by sustained neural activity; familiarity, on the other hand, exhibited repetition effects in multiple cortical regions, a subset of which was modulated by successful encoding. Subsequent memory of novelty was associated only with activation differences in a distinct set of regions, including the hippocampus and medial cortical regions. These results suggest that conceptual novelty (a) does not (easily) trigger the repetition suppression phenomenon but requires sustained neural recruitment and (b) activates dedicated encoding mechanisms. Conceptual familiarity, in contrast, allows rapid neural processing that depends upon existing neural representations. Overall, these findings challenge the definition of novelty as a unitary concept. Furthermore, they bear important implications for research into the neural bases of knowledge representation and recognition memory.

Delineating the effect of semantic congruency on episodic memory: the role of integration and relatedness.

A fundamental challenge in the study of learning and memory is to understand the role of existing knowledge in the encoding and retrieval of new episodic information. The importance of prior knowledge in memory is demonstrated in the congruency effect-the robust finding wherein participants display better memory for items that are compatible, rather than incompatible, with their pre-existing semantic knowledge. Despite its robustness, the mechanism underlying this effect is not well understood. In four studies, we provide evidence that demonstrates the privileged explanatory power of the elaboration-integration account over alternative hypotheses. Furthermore, we question the implicit assumption that the congruency effect pertains to the truthfulness/sensibility of a subject-predicate proposition, and show that congruency is a function of semantic relatedness between item and context words.

Prior knowledge influences on hippocampus and medial prefrontal cortex interactions in subsequent memory.

Prior knowledge is known to influence the encoding of new events. Specifically, recent theoretical frameworks suggest that positively correlated hippocampus (HC)-medial prefrontal cortex (mPFC) activity is involved in creating enduring traces of events inconsistent with our prior knowledge. Events that are consistent with our schemas are suggested to be encoded via mPFC-cortical interactions. Previous studies examined differences in functional connectivity between subsequently remembered and forgotten items, but the source of these differences was not addressed. Therefore, the involvement of the inter-regions functional connectivity in subsequent remembering or subsequent forgetting of events is unknown. In this study, in addition to probing for a remembered-forgotten difference in functional connectivity, we also examined how connectivity differed from baseline in each of the memory conditions. At encoding, the participants were presented with pairs of semantically related (schema-consistent) and semantically unrelated (schema-inconsistent) words. A surprise recognition test was administered, and a subsequent memory analysis evaluating potential interactions with the HC and mPFC was conducted. Consistent with the suggested frameworks, subsequent memory modulated HC-mPFC connectivity only in schema-inconsistent events. Importantly, the HC and mPFC were positively correlated with respect to subsequently remembered schema-inconsistent items, whereas the subsequently forgotten schema-inconsistent events did not differ from baseline. We also found that positively correlated activity of the mPFC with visual and parietal regions mediated subsequent memory of schema-inconsistent items. Therefore, inconsistent events may be encoded by a network of cortical and medial temporal lobe regions.

Explicit memory for unattended words: the importance of being in the "no".

Recognition tests in which participants indicate whether they recognize items using binary yes/no response options have typically yielded "yes" responses at equal rates for unattended old items and new items. Because most responses to unattended stimuli in such tests are "no" responses, we reasoned that a closer examination of "no" responses might reveal memory for unattended items. We modified a classic paradigm to allow participants to indicate high and low levels of confidence in their responses. As in earlier studies, the overall proportion of "yes" responses did not differ between unattended old items and new items. However, there was a crossover effect in the "no" responses: More high-confidence "no" responses were given for new items than for unattended old items, whereas more low-confidence "no" responses were given for unattended old items than for new items. These results indicate explicit memory for unattended material presented under high perceptual load. Our findings suggest that the null effects obtained in previous studies may not have stemmed from failures of perception or memory, but rather may have been due to insufficiently sensitive memory assessment.