The influence of emotion on temporal context models.

Temporal context models (TCMs) have been influential in understanding episodic memory and its neural underpinnings. Recently, TCMs have been extended to explain emotional memory effects, one of the most clinically important findings in the field of memory research. This review covers recent advances in hypotheses for the neural representation of spatiotemporal context through the lens of TCMs, including their ability to explain the influence of emotion on episodic and temporal memory. In recent years, simplifying assumptions of "classical" TCMs - with exponential trace decay and the mechanism by which temporal context is recovered - have become increasingly clear. The review also outlines how recent advances could be incorporated into a future TCM, beyond classical assumptions, to integrate emotional modulation.

Ramping cells in rodent mPFC encode time to past and future events via real Laplace transform.

In interval reproduction tasks, animals must remember the event starting the interval and anticipate the time of the planned response to terminate the interval. The interval reproduction task thus allows for studying both memory for the past and anticipation of the future. We analyzed previously published recordings from rodent mPFC (Henke et al., 2021) during an interval reproduction task and identified two cell groups by modeling their temporal receptive fields using hierarchical Bayesian models. The firing in the "past cells" group peaked at the start of the interval and relaxed exponentially back to baseline. The firing in the "future cells" group increased exponentially and peaked right before the planned action at the end of the interval. Contrary to the previous assumption that timing information in the brain has one or two time scales for a given interval, we found strong evidence for a continuous distribution of the exponential rate constants for both past and future cell populations. The real Laplace transformation of time predicts exponential firing with a continuous distribution of rate constants across the population. Therefore, the firing pattern of the past cells can be identified with the Laplace transform of time since the past event while the firing pattern of the future cells can be identified with the Laplace transform of time until the planned future event.

Foundations of a temporal RL.

Recent advances in neuroscience and psychology show that the brain has access to timelines of both the past and the future. Spiking across populations of neurons in many regions of the mammalian brain maintains a robust temporal memory, a neural timeline of the recent past. Behavioral results demonstrate that people can estimate an extended temporal model of the future, suggesting that the neural timeline of the past could extend through the present into the future. This paper presents a mathematical framework for learning and expressing relationships between events in continuous time. We assume that the brain has access to a temporal memory in the form of the real Laplace transform of the recent past. Hebbian associations with a diversity of synaptic time scales are formed between the past and the present that record the temporal relationships between events. Knowing the temporal relationships between the past and the present allows one to predict relationships between the present and the future, thus constructing an extended temporal prediction for the future. Both memory for the past and the predicted future are represented as the real Laplace transform, expressed as the firing rate over populations of neurons indexed by different rate constants s. The diversity of synaptic timescales allows for a temporal record over the much larger time scale of trial history. In this framework, temporal credit assignment can be assessed via a Laplace temporal difference. The Laplace temporal difference compares the future that actually follows a stimulus to the future predicted just before the stimulus was observed. This computational framework makes a number of specific neurophysiological predictions and, taken together, could provide the basis for a future iteration of RL that incorporates temporal memory as a fundamental building block.

Internally generated time in the rodent hippocampus is logarithmically compressed.

The Weber-Fechner law proposes that our perceived sensory input increases with physical input on a logarithmic scale. Hippocampal 'time cells' carry a record of recent experience by firing sequentially during a circumscribed period of time after a triggering stimulus. Different cells have 'time fields' at different delays up to at least tens of seconds. Past studies suggest that time cells represent a compressed timeline by demonstrating that fewer time cells fire late in the delay and their time fields are wider. This paper asks whether the compression of time cells obeys the Weber-Fechner Law. Time cells were studied with a hierarchical Bayesian model that simultaneously accounts for the firing pattern at the trial level, cell level, and population level. This procedure allows separate estimates of the within-trial receptive field width and the across-trial variability. After isolating across-trial variability, time field width increased linearly with delay. Further, the time cell population was distributed evenly along a logarithmic time axis. These findings provide strong quantitative evidence that the neural temporal representation in rodent hippocampus is logarithmically compressed and obeys a neural Weber-Fechner Law.

Scanning a compressed ordered representation of the future.

Several authors have suggested a deep symmetry between the psychological processes that underlie our ability to remember the past and make predictions about the future. The judgment of recency (JOR) task measures temporal order judgments for the past by presenting pairs of probe stimuli; participants choose the probe that was presented more recently. We performed a short-term relative JOR task and introduced a novel judgment of imminence (JOI) task to study temporal order judgments for the future. In the JOR task, participants were presented with a sequence of stimuli and asked to choose which of two probe stimuli was presented closer to the present. In the JOI task, participants were trained on a probabilistic sequence. After training, the sequence was interrupted with probe stimuli. Participants were asked to choose which of two probe stimuli was expected to be presented closer to the present. Replicating prior work on JOR, we found that RT results supported a backward self-terminating search model operating on a temporally organized representation of the past. We also showed that RT distributions are consistent with this model and that the temporally organized representation is compressed. Critically, results for the JOI task probing expectations of the future suggest a forward self-terminating search model operating on a temporally organized representation of the future. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Consistent population activity on the scale of minutes in the mouse hippocampus.

Neurons in the hippocampus fire in consistent sequence over the timescale of seconds during the delay period of some memory experiments. For longer timescales, the firing of hippocampal neurons also changes slowly over minutes within experimental sessions. It was thought that these slow dynamics are caused by stochastic drift or a continuous change in the representation of the episode, rather than consistent sequences unfolding over minutes. This paper studies the consistency of contextual drift in three chronic calcium imaging recordings from the hippocampus CA1 region in mice. Computational measures of consistency show reliable sequences within experimental trials at the scale of seconds as one would expect from time cells or place cells during the trial, as well as across experimental trials on the scale of minutes within a recording session. Consistent sequences in the hippocampus are observed over a wide range of time scales, from seconds to minutes. The hippocampal activity could reflect a scale-invariant spatiotemporal context as suggested by theories of memory from cognitive psychology.

Predicting the Future With a Scale-Invariant Temporal Memory for the Past.

In recent years, it has become clear that the brain maintains a temporal memory of recent events stretching far into the past. This letter presents a neurally inspired algorithm to use a scale-invariant temporal representation of the past to predict a scale-invariant future. The result is a scale-invariant estimate of future events as a function of the time at which they are expected to occur. The algorithm is time-local, with credit assigned to the present event by observing how it affects the prediction of the future. To illustrate the potential utility of this approach, we test the model on simultaneous renewal processes with different timescales. The algorithm scales well on these problems despite the fact that the number of states needed to describe them as a Markov process grows exponentially.

Scale-Dependent Relationships in Natural Language.

Language, like other natural sequences, exhibits statistical dependencies at a wide range of scales (Lin & Tegmark, 2016). However, many statistical learning models applied to language impose a sampling scale while extracting statistical structure. For instance, Word2Vec creates vector embeddings by sampling context in a window around each word, the size of which defines a strong scale; relationships over much larger temporal scales would be invisible to the algorithm. This paper examines the family of Word2Vec embeddings generated while systematically manipulating the size of the context window. The primary result is that different linguistic relationships are preferentially encoded at different scales. Different scales emphasize different syntactic and semantic relations between words, as assessed both by analogical reasoning tasks in the Google Analogies test set and human similarity rating datasets WordSim-353 and SimLex-999. Moreover, the neighborhoods of a given word in the embeddings change considerably depending on the scale. These results suggest that sampling at any individual scale can only identify a subset of the meaningful relationships a word might have, and point toward the importance of developing scale-free models of semantic meaning.

Conjunctive representation of what and when in monkey hippocampus and lateral prefrontal cortex during an associative memory task.

Adaptive memory requires the organism to form associations that bridge between events separated in time. Many studies show interactions between hippocampus (HPC) and prefrontal cortex (PFC) during formation of such associations. We analyze neural recording from monkey HPC and PFC during a memory task that requires the monkey to associate stimuli separated by about a second in time. After the first stimulus was presented, large numbers of units in both HPC and PFC fired in sequence. Many units fired only when a particular stimulus was presented at a particular time in the past. These results indicate that both HPC and PFC maintain a temporal record of events that could be used to form associations across time. This temporal record of the past is a key component of the temporal coding hypothesis, a hypothesis in psychology that memory not only encodes what happened, but when it happened.

A temporal record of the past with a spectrum of time constants in the monkey entorhinal cortex.

Episodic memory is believed to be intimately related to our experience of the passage of time. Indeed, neurons in the hippocampus and other brain regions critical to episodic memory code for the passage of time at a range of timescales. The origin of this temporal signal, however, remains unclear. Here, we examined temporal responses in the entorhinal cortex of macaque monkeys as they viewed complex images. Many neurons in the entorhinal cortex were responsive to image onset, showing large deviations from baseline firing shortly after image onset but relaxing back to baseline at different rates. This range of relaxation rates allowed for the time since image onset to be decoded on the scale of seconds. Further, these neurons carried information about image content, suggesting that neurons in the entorhinal cortex carry information about not only when an event took place but also, the identity of that event. Taken together, these findings suggest that the primate entorhinal cortex uses a spectrum of time constants to construct a temporal record of the past in support of episodic memory.

Generation of Scale-Invariant Sequential Activity in Linear Recurrent Networks.

Sequential neural activity has been observed in many parts of the brain and has been proposed as a neural mechanism for memory. The natural world expresses temporal relationships at a wide range of scales. Because we cannot know the relevant scales a priori, it is desirable that memory, and thus the generated sequences, is scale invariant. Although recurrent neural network models have been proposed as a mechanism for generating sequences, the requirements for scale-invariant sequences are not known. This letter reports the constraints that enable a linear recurrent neural network model to generate scale-invariant sequential activity. A straightforward eigendecomposition analysis results in two independent conditions that are required for scale invariance for connectivity matrices with real, distinct eigenvalues. First, the eigenvalues of the network must be geometrically spaced. Second, the eigenvectors must be related to one another via translation. These constraints are easily generalizable for matrices that have complex and distinct eigenvalues. Analogous albeit less compact constraints hold for matrices with degenerate eigenvalues. These constraints, along with considerations on initial conditions, provide a general recipe to build linear recurrent neural networks that support scale-invariant sequential activity.

Time-conjunctive representations of future events.

It is widely accepted that people can predict the relative imminence of future events. However, it is unknown whether the timing of future events is represented using only a "strength-like" estimate or if future events are represented conjunctively with their position on a mental timeline. We examined how people judge temporal relationships among anticipated future events using the novel Judgment of Anticipated Co-Occurrence (JACO) task. Participants were initially trained on a stream of letters sampled from a probabilistically repeating sequence. During test trials, the stream was interrupted with pairs of probe letters and the participants' task was to choose the probe letter they expected to appear in the stream during a lagged target window 4-6 items (4.3-8.5 s) in the future. Participants performed above chance as they gained experience with the task. Because the correct item was sometimes the more imminent probe letter and other times the less imminent probe letter, these results rule out the possibility that participants relied solely on thresholding a strength-like estimate of temporal imminence. Rather, these results suggest that participants held (1) temporally organized predictions of the future letters in the stream, (2) a temporal estimate of the lagged target window, and (3) some means to compare the two and evaluate their temporal alignment. Response time increased with the lag to the more imminent probe letter, suggesting that participants accessed the future sequentially in a manner that mirrors scanning processes previously proposed to operate on memory representations in the short-term judgment of recency task.

In a Temporally Segmented Experience Hippocampal Neurons Represent Temporally Drifting Context But Not Discrete Segments.

There is widespread agreement that episodic memory is organized into a timeline of past experiences. Recent work suggests that the hippocampus may parse the flow of experience into discrete episodes separated by event boundaries. A complementary body of work suggests that context changes gradually as experience unfolds. We recorded from hippocampal neurons as male Long-Evans rats performed 6 blocks of an object discrimination task in sets of 15 trials. Each block was separated by removal from the testing chamber for a delay to enable segmentation. The reward contingency reversed from one block to the next to incentivize segmentation. We expected animals to hold two distinct, recurring representations of context to match the two distinct rule contingencies. Instead, we found that overtrained rats began each block neither above nor below chance but by guessing randomly. While many units had clear firing fields selective to the conjunction of objects in places, a significant population also reflected a continuously drifting code both within block and across blocks. Despite clear boundaries between blocks, we saw no neural evidence for event segmentation in this experiment. Rather, the hippocampal ensemble drifted continuously across time. This continuous drift in the neural representation was consistent with the lack of segmentation observed in behavior.SIGNIFICANCE STATEMENT The neuroscience literature yet to reach consensus on how the hippocampus supports the organization of events across time in episodic memory. Initial studies reported stable hippocampal maps segmented by remapping events. However, it remains unclear whether segmentation is an artifact of cue responsivity. Recently, research has shown that the hippocampal code exhibits continuous drift. Drift may represent a continually evolving context; however, it is unclear whether this is an artifact of changing experiences. We recorded dCA1 in rats performing an object discrimination task designed to segment time. Overtrained rats could not anticipate upcoming context switches but used context boundaries to their advantage. Hippocampal ensembles showed neither evidence of alternating between stable contexts nor sensitivity to boundaries, but showed robust temporal drift.

Estimating Scale-Invariant Future in Continuous Time.

Natural learners must compute an estimate of future outcomes that follow from a stimulus in continuous time. Widely used reinforcement learning algorithms discretize continuous time and estimate either transition functions from one step to the next (model-based algorithms) or a scalar value of exponentially discounted future reward using the Bellman equation (model-free algorithms). An important drawback of model-based algorithms is that computational cost grows linearly with the amount of time to be simulated. An important drawback of model-free algorithms is the need to select a timescale required for exponential discounting. We present a computational mechanism, developed based on work in psychology and neuroscience, for computing a scale-invariant timeline of future outcomes. This mechanism efficiently computes an estimate of inputs as a function of future time on a logarithmically compressed scale and can be used to generate a scale-invariant power-law-discounted estimate of expected future reward. The representation of future time retains information about what will happen when. The entire timeline can be constructed in a single parallel operation that generates concrete behavioral and neural predictions. This computational mechanism could be incorporated into future reinforcement learning algorithms.

Medial Temporal Lobe Amnesia Is Associated with a Deficit in Recovering Temporal Context.

Medial-temporal lobe (MTL) lesions are associated with severe impairments in episodic memory. In the framework of the temporal context model, the hypothesized mechanism for episodic memory is the reinstatement of a prior experienced context (i.e., "jump back in time"), which relies upon the MTL [Howard, M. W., Fotedar, M. S., Datey, A. V., & Hasselmo, M. E. The temporal context model in spatial navigation and relational learning: Toward a common explanation of medial temporal lobe function across domains. Psychological Review, 112, 75-116, 2005]. This hypothesis has proven difficult to test in amnesia because of the floor-level performance by patients in recall tasks. To circumvent this issue, in this study, we used a "looped-list" format, in which a set of verbal stimuli was presented multiple times in a consistent order. This allowed for comparison of statistical properties such as probability of first recall and lag-conditional response probability (lag-CRP) between amnesic patients and healthy controls. Results revealed that the lag-CRP, but not the probability of first recall, is altered in amnesia, suggesting a selective disruption of temporal contiguity. To further characterize the results, we fit a scale-invariant version of the temporal context model [Howard, M. W., Shankar, K. H., Aue, W. R., & Criss, A. H. A distributed representation of internal time. Psychological Review, 122, 24-53, 2015] to the probability of first recall and lag-CRP curves. The modeling results suggested that the deficit in temporal contiguity in amnesia is best described as a failure to recover temporal context. These results provide the first direct evidence for an impairment in a jump-back-in-time mechanism in patients with MTL amnesia.

A neural microcircuit model for a scalable scale-invariant representation of time.

Scale-invariant timing has been observed in a wide range of behavioral experiments. The firing properties of recently described time cells provide a possible neural substrate for scale-invariant behavior. Earlier neural circuit models do not produce scale-invariant neural sequences. In this article, we present a biologically detailed network model based on an earlier mathematical algorithm. The simulations incorporate exponentially decaying persistent firing maintained by the calcium-activated nonspecific (CAN) cationic current and a network structure given by the inverse Laplace transform to generate time cells with scale-invariant firing rates. This model provides the first biologically detailed neural circuit for generating scale-invariant time cells. The circuit that implements the inverse Laplace transform merely consists of off-center/on-surround receptive fields. Critically, rescaling temporal sequences can be accomplished simply via cortical gain control (changing the slope of the f-I curve).

The Same Hippocampal CA1 Population Simultaneously Codes Temporal Information over Multiple Timescales.

It has long been hypothesized that a primary function of the hippocampus is to discover and exploit temporal relationships between events. Previously, it has been reported that sequences of "time cells" in the hippocampus extend for tens of seconds. Other studies have shown that neuronal firing in the hippocampus fluctuates over hours and days. Both of these mechanisms could enable temporal encoding of events over very different timescales. However, thus far, these two classes of phenomena have never been observed simultaneously, which is necessary to ascribe broad-range temporal coding to the hippocampus. Using in vivo calcium imaging in unrestrained mice, we observed sequences of hippocampal neurons that bridged a 10 s delay. Similar sequences were observed over multiple days, but the set of neurons participating in those sequences changed gradually. Thus, the same population of neurons that encodes temporal information over seconds can also be used to distinguish periods of time over much longer timescales. These results unify two previously separate paradigms of temporal processing in the hippocampus that support episodic memory.

Human Episodic Memory Retrieval Is Accompanied by a Neural Contiguity Effect.

Cognitive psychologists have long hypothesized that experiences are encoded in a temporal context that changes gradually over time. When an episodic memory is retrieved, the state of context is recovered-a jump back in time. We recorded from single units in the medial temporal lobe of epilepsy patients performing an item recognition task. The population vector changed gradually over minutes during presentation of the list. When a probe from the list was remembered with high confidence, the population vector reinstated the temporal context of the original presentation of that probe during study, a neural contiguity effect that provides a possible mechanism for behavioral contiguity effects. This pattern was only observed for well remembered probes; old probes that were not well remembered showed an anti-contiguity effect. These results constitute the first direct evidence that recovery of an episodic memory in humans is associated with retrieval of a gradually changing state of temporal context, a neural "jump back in time" that parallels the act of remembering.SIGNIFICANCE STATEMENT Episodic memory is the ability to relive a specific experience from one's life. For decades, researchers have hypothesized that, unlike other forms of memory that can be described as simple associations between stimuli, episodic memory depends on the recovery of a neural representation of spatiotemporal context. During study of a sequence of stimuli, the brain state of epilepsy patients changed slowly over at least a minute. When the participant remembered a particular event from the list, this gradually changing state was recovered. This provides direct confirmation of the prediction from computational models of episodic memory. The resolution of this point means that the study of episodic memory can focus on the mechanisms by which this representation of spatiotemporal context is maintained and sometimes recovered.

Compressed Timeline of Recent Experience in Monkey Lateral Prefrontal Cortex.

Cognitive theories suggest that working memory maintains not only the identity of recently presented stimuli but also a sense of the elapsed time since the stimuli were presented. Previous studies of the neural underpinnings of working memory have focused on sustained firing, which can account for maintenance of the stimulus identity, but not for representation of the elapsed time. We analyzed single-unit recordings from the lateral prefrontal cortex of macaque monkeys during performance of a delayed match-to-category task. Each sample stimulus triggered a consistent sequence of neurons, with each neuron in the sequence firing during a circumscribed period. These sequences of neurons encoded both stimulus identity and elapsed time. The encoding of elapsed time became less precise as the sample stimulus receded into the past. These findings suggest that working memory includes a compressed timeline of what happened when, consistent with long-standing cognitive theories of human memory.

Is working memory stored along a logarithmic timeline? Converging evidence from neuroscience, behavior and models.

A growing body of evidence suggests that short-term memory does not only store the identity of recently experienced stimuli, but also information about when they were presented. This representation of 'what' happened 'when' constitutes a neural timeline of recent past. Behavioral results suggest that people can sequentially access memories for the recent past, as if they were stored along a timeline to which attention is sequentially directed. In the short-term judgment of recency (JOR) task, the time to choose between two probe items depends on the recency of the more recent probe but not on the recency of the more remote probe. This pattern of results suggests a backward self-terminating search model. We review recent neural evidence from the macaque lateral prefrontal cortex (lPFC) (Tiganj, Cromer, Roy, Miller, & Howard, in press) and behavioral evidence from human JOR task (Singh & Howard, 2017) bearing on this question. Notably, both lines of evidence suggest that the timeline is logarithmically compressed as predicted by Weber-Fechner scaling. Taken together, these findings provide an integrative perspective on temporal organization and neural underpinnings of short-term memory.