Segmentation of spatial experience by hippocampal θ sequences.

The encoding and storage of experience by the hippocampus is essential for the formation of episodic memories and the transformation of individual experiences into semantic structures such as maps and schemas. The rodent hippocampus compresses ongoing experience into repeating theta sequences, but the factors determining the content of theta sequences are not understood. Here we first show that the spatial paths represented by theta sequences in rats extend farther in front of the rat during acceleration and higher running speeds and begin farther behind the rat during deceleration. Second, the length of the path is directly related to the length of the theta cycle and the number of gamma cycles in it. Finally, theta sequences represent the environment in segments or 'chunks'. These results imply that information encoded in theta sequences is subject to powerful modulation by behavior and task variables. Furthermore, these findings suggest a potential mechanism for the cognitive 'chunking' of experience.

Hippocampal replay is not a simple function of experience.

Replay of behavioral sequences in the hippocampus during sharp wave ripple complexes (SWRs) provides a potential mechanism for memory consolidation and the learning of knowledge structures. Current hypotheses imply that replay should straightforwardly reflect recent experience. However, we find these hypotheses to be incompatible with the content of replay on a task with two distinct behavioral sequences (A and B). We observed forward and backward replay of B even when rats had been performing A for >10 min. Furthermore, replay of nonlocal sequence B occurred more often when B was infrequently experienced. Neither forward nor backward sequences preferentially represented highly experienced trajectories within a session. Additionally, we observed the construction of never-experienced novel-path sequences. These observations challenge the idea that sequence activation during SWRs is a simple replay of recent experience. Instead, replay reflected all physically available trajectories within the environment, suggesting a potential role in active learning and maintenance of the cognitive map.

Context learning in the rodent hippocampus.

We present a Bayesian statistical theory of context learning in the rodent hippocampus. While context is often defined in an experimental setting in relation to specific background cues or task demands, we advance a single, more general notion of context that suffices for a variety of learning phenomena. Specifically, a context is defined as a statistically stationary distribution of experiences, and context learning is defined as the problem of how to form contexts out of groups of experiences that cluster together in time. The challenge of context learning is solving the model selection problem: How many contexts make up the rodent's world? Solving this problem requires balancing two opposing goals: minimize the variability of the distribution of experiences within a context and minimize the likelihood of transitioning between contexts. The theory provides an understanding of why hippocampal place cell remapping sometimes develops gradually over many days of experience and why even consistent landmark differences may need to be relearned after other environmental changes. The theory provides an explanation for progressive performance improvements in serial reversal learning, based on a clear dissociation between the incremental process of context learning and the relatively abrupt context selection process. The impact of partial reinforcement on reversal learning is also addressed. Finally, the theory explains why alternating sequence learning does not consistently result in unique context-dependent sequence representations in hippocampus.

Representation and timing in theories of the dopamine system.

Although the responses of dopamine neurons in the primate midbrain are well characterized as carrying a temporal difference (TD) error signal for reward prediction, existing theories do not offer a credible account of how the brain keeps track of past sensory events that may be relevant to predicting future reward. Empirically, these shortcomings of previous theories are particularly evident in their account of experiments in which animals were exposed to variation in the timing of events. The original theories mispredicted the results of such experiments due to their use of a representational device called a tapped delay line. Here we propose that a richer understanding of history representation and a better account of these experiments can be given by considering TD algorithms for a formal setting that incorporates two features not originally considered in theories of the dopaminergic response: partial observability (a distinction between the animal's sensory experience and the true underlying state of the world) and semi-Markov dynamics (an explicit account of variation in the intervals between events). The new theory situates the dopaminergic system in a richer functional and anatomical context, since it assumes (in accord with recent computational theories of cortex) that problems of partial observability and stimulus history are solved in sensory cortex using statistical modeling and inference and that the TD system predicts reward using the results of this inference rather than raw sensory data. It also accounts for a range of experimental data, including the experiments involving programmed temporal variability and other previously unmodeled dopaminergic response phenomena, which we suggest are related to subjective noise in animals' interval timing. Finally, it offers new experimental predictions and a rich theoretical framework for designing future experiments.

Bayesian theories of conditioning in a changing world.

The recent flowering of Bayesian approaches invites the re-examination of classic issues in behavior, even in areas as venerable as Pavlovian conditioning. A statistical account can offer a new, principled interpretation of behavior, and previous experiments and theories can inform many unexplored aspects of the Bayesian enterprise. Here we consider one such issue: the finding that surprising events provoke animals to learn faster. We suggest that, in a statistical account of conditioning, surprise signals change and therefore uncertainty and the need for new learning. We discuss inference in a world that changes and show how experimental results involving surprise can be interpreted from this perspective, and also how, thus understood, these phenomena help constrain statistical theories of animal and human learning.

A spin glass model of path integration in rat medial entorhinal cortex.

Electrophysiological recording studies in the dorsocaudal region of medial entorhinal cortex (dMEC) of the rat reveal cells whose spatial firing fields show a remarkably regular hexagonal grid pattern (Fyhn et al., 2004; Hafting et al., 2005). We describe a symmetric, locally connected neural network, or spin glass model, that spontaneously produces a hexagonal grid of activity bumps on a two-dimensional sheet of units. The spatial firing fields of the simulated cells closely resemble those of dMEC cells. A collection of grids with different scales and/or orientations forms a basis set for encoding position. Simulations show that the animal's location can easily be determined from the population activity pattern. Introducing an asymmetry in the model allows the activity bumps to be shifted in any direction, at a rate proportional to velocity, to achieve path integration. Furthermore, information about the structure of the environment can be superimposed on the spatial position signal by modulation of the bump activity levels without significantly interfering with the hexagonal periodicity of firing fields. Our results support the conjecture of Hafting et al. (2005) that an attractor network in dMEC may be the source of path integration information afferent to hippocampus.

Influence of path integration versus environmental orientation on place cell remapping between visually identical environments.

To assess the effects of interactions between angular path integration and visual landmarks on the firing of hippocampal neurons, we recorded from CA1 pyramidal cells as rats foraged in two identical boxes with polarizing internal cues. In the same-orientation condition, following an earlier experiment by Skaggs and McNaughton, the boxes were oriented identically and connected by a corridor. In the opposite-orientation condition, the boxes were abutted by rotating them 90 degrees in opposite directions, so that their orientations differed by 180 degrees . After 16-23 days of pretraining on the same-orientation condition, three rats experienced both conditions in counterbalanced order on each of two consecutive days. On the third day they ran two opposite-orientation trials. Although Skaggs and McNaughton observed stable partial "remapping" of place fields, none of the fields in this experiment remapped in the same-orientation condition. In the opposite-orientation condition, place fields in the first box were isomorphic with those in the same-orientation condition, whereas in the second box the rats eventually exhibited completely different fields. The rats differed as to the trial in which this first occurred. Once the second box exhibited different fields, it continued to do so in all subsequent opposite-orientation trials, yet fields remained the same in subsequent same-orientation trials. The results demonstrate that when animals move actively between environments, and are thus potentially able to maintain their inertial angular orientation, discordance between environmental orientation and the rat's idiothetic direction sense can profoundly affect the hippocampal map-either immediately, or as a result of cumulative experience.

Deforming the hippocampal map.

To investigate conjoint stimulus control over place cells, Fenton et al. (J Gen Physiol 116:191-209, 2000a) recorded while rats foraged in a cylinder with 45 degrees black and white cue cards on the wall. Card centers were 135 degrees apart. In probe trials, the cards were rotated together or apart by 25 degrees . Firing field centers shifted during these trials, stretching and shrinking the cognitive map. Fenton et al. (2000b) described this deformation with an ad hoc vector field equation. We consider what sorts of neural network mechanisms might be capable of accounting for their observations. In an abstract, maximum likelihood formulation, the rat's location is estimated by a conjoint probability density function of landmark positions. In an attractor neural network model, recurrent connections produce a bump of activity over a two-dimensional array of cells; the bump's position is influenced by landmark features such as distances or bearings. If features are chosen with appropriate care, the attractor network and maximum likelihood models yield similar results, in accord with previous demonstrations that recurrent neural networks can efficiently implement maximum likelihood computations (Pouget et al. Neural Comput 10:373-401, 1998; Deneve et al. Nat Neurosci 4:826-831, 2001).

Long-term reward prediction in TD models of the dopamine system.

This article addresses the relationship between long-term reward predictions and slow-timescale neural activity in temporal difference (TD) models of the dopamine system. Such models attempt to explain how the activity of dopamine (DA) neurons relates to errors in the prediction of future rewards. Previous models have been mostly restricted to short-term predictions of rewards expected during a single, somewhat artificially defined trial. Also, the models focused exclusively on the phasic pause-and-burst activity of primate DA neurons; the neurons' slower, tonic background activity was assumed to be constant. This has led to difficulty in explaining the results of neurochemical experiments that measure indications of DA release on a slow timescale, results that seem at first glance inconsistent with a reward prediction model. In this article, we investigate a TD model of DA activity modified so as to enable it to make longer-term predictions about rewards expected far in the future. We show that these predictions manifest themselves as slow changes in the baseline error signal, which we associate with tonic DA activity. Using this model, we make new predictions about the behavior of the DA system in a number of experimental situations. Some of these predictions suggest new computational explanations for previously puzzling data, such as indications from microdialysis studies of elevated DA activity triggered by aversive events.

Sequence Manipulation Using Parallel Mapping Networks.

We describe a parallel mapping matrix that performs several types of sequence manipulations that are the building blocks of well-known phonological processes. Our results indicate that human phonological behavior can by modeled by a highly constrained connectionist architecture, one that uses purely feedforward circuitry and imposes tight limits on depth of derivations.