Memory Retrieval from First Principles.

The dilemma that neurotheorists face is that (1) detailed biophysical models that can be constrained by direct measurements, while being of great importance, offer no immediate insights into cognitive processes in the brain, and (2) high-level abstract cognitive models, on the other hand, while relevant for understanding behavior, are largely detached from neuronal processes and typically have many free, experimentally unconstrained parameters that have to be tuned to a particular data set and, hence, cannot be readily generalized to other experimental paradigms. In this contribution, we propose a set of "first principles" for neurally inspired cognitive modeling of memory retrieval that has no biologically unconstrained parameters and can be analyzed mathematically both at neuronal and cognitive levels. We apply this framework to the classical cognitive paradigm of free recall. We show that the resulting model accounts well for puzzling behavioral data on human participants and makes predictions that could potentially be tested with neurophysiological recording techniques.

Dynamics of population rate codes in ensembles of neocortical neurons.

Information processing in neocortex can be very fast, indicating that neuronal ensembles faithfully transmit rapidly changing signals to each other. Apart from signal-to-noise issues, population codes are fundamentally constrained by the neuronal dynamics. In particular, the biophysical properties of individual neurons and collective phenomena may substantially limit the speed at which a graded signal can be represented by the activity of an ensemble. These implications of the neuronal dynamics are rarely studied experimentally. Here, we combine theoretical analysis and whole cell recordings to show that encoding signals in the variance of uncorrelated synaptic inputs to a neocortical ensemble enables faithful transmission of graded signals with high temporal resolution. In contrast, the encoding of signals in the mean current is subject to low-pass filtering.

Intracortical origin of visual maps.

Previous experiments indicate that the shape of maps of preferred orientation in the primary visual cortex does not depend on visual experience. We propose a network model that demonstrates that the orientation and direction selectivity of individual units and the structure of the corresponding angle maps could emerge from local recurrent connections. Our model reproduces the structure of preferred orientation and direction maps, and explains the origin of their interrelationship. The model also provides an explanation for the correlation between position shifts of receptive fields and changes of preferred orientations of single neurons across the surface of the cortex.

An algorithm for modifying neurotransmitter release probability based on pre- and postsynaptic spike timing.

The precise times of occurrence of individual pre- and postsynaptic action potentials are known to play a key role in the modification of synaptic efficacy. Based on stimulation protocols of two synaptically connected neurons, we infer an algorithm that reproduces the experimental data by modifying the probability of vesicle discharge as a function of the relative timing of spikes in the pre- and postsynaptic neurons. The primary feature of this algorithm is an asymmetry with respect to the direction of synaptic modification depending on whether the presynaptic spikes precede or follow the postsynaptic spike. Specifically, if the presynaptic spike occurs up to 50 ms before the postsynaptic spike, the probability of vesicle discharge is upregulated, while the probability of vesicle discharge is downregulated if the presynaptic spike occurs up to 50 ms after the postsynaptic spike. When neurons fire irregularly with Poisson spike trains at constant mean firing rates, the probability of vesicle discharge converges toward a characteristic value determined by the pre- and postsynaptic firing rates. On the other hand, if the mean rates of the Poisson spike trains slowly change with time, our algorithm predicts modifications in the probability of release that generalize Hebbian and Bienenstock-Cooper-Munro rules. We conclude that the proposed spike-based synaptic learning algorithm provides a general framework for regulating neurotransmitter release probability.

Synchrony generation in recurrent networks with frequency-dependent synapses.

Throughout the neocortex, groups of neurons have been found to fire synchronously on the time scale of several milliseconds. This near coincident firing of neurons could coordinate the multifaceted information of different features of a stimulus. The mechanisms of generating such synchrony are not clear. We simulated the activity of a population of excitatory and inhibitory neurons randomly interconnected into a recurrent network via synapses that display temporal dynamics in their transmission; surprisingly, we found a behavior of the network where action potential activity spontaneously self-organized to produce highly synchronous bursts involving virtually the entire network. These population bursts were also triggered by stimuli to the network in an all-or-none manner. We found that the particular intensities of the external stimulus to specific neurons were crucial to evoke population bursts. This topographic sensitivity therefore depends on the spectrum of basal discharge rates across the population and not on the anatomical individuality of the neurons, because this was random. These results suggest that networks in which neurons are even randomly interconnected via frequency-dependent synapses could exhibit a novel form of reflex response that is sensitive to the nature of the stimulus as well as the background spontaneous activity.

Linking spontaneous activity of single cortical neurons and the underlying functional architecture.

The relation between the activity of a single neocortical neuron and the dynamics of the network in which it is embedded was explored by single-unit recordings and real-time optical imaging. The firing rate of a spontaneously active single neuron strongly depends on the instantaneous spatial pattern of ongoing population activity in a large cortical area. Very similar spatial patterns of population activity were observed both when the neuron fired spontaneously and when it was driven by its optimal stimulus. The evoked patterns could be used to reconstruct the spontaneous activity of single neurons.

Attractor neural network models of spatial maps in hippocampus.

Hippocampal pyramidal neurons in rats are selectively activated at specific locations in an environment (O'Keefe and Dostrovsky, Brain Res 1971;34:171-175). Different cells are active in different places, therefore providing a faithful representation of the environment in which every spatial location is mapped to a particular population state of activity of place cells (Wilson and McNaughton, Science 1993;261:1055-1058; Zhang et al., J Neurosci 1998;79:1017-1044). We describe a theory of the hippocampus, according to which the map results from the cooperative dynamics of network, in which the strength of synaptic interaction between the neurons depends on the distance between their place fields. This synaptic structure guarantees that the network possesses a quasi-continuous set of stable states (attractors) that are localized in the space of neuronal variables reflecting their synaptic interactions, rather than their physical location in the hippocampus. As a consequence of the stable states, the network can exhibit place selective activity even without relying on input from external sensory cues.

Relation between retinotopical and orientation maps in visual cortex.

A recent study of cat visual cortex reported abrupt changes in the positions of the receptive fields of adjacent neurons whose preferred orientations strongly differed (Das & Gilbert, 1997). Using a simple cortical model, we show that this covariation of discontinuities in maps of orientation preference and local distortions in maps of visual space reflects collective effects of the lateral cortical feedback.

Information processing with frequency-dependent synaptic connections.

The efficacy of synaptic transmission between two neurons changes as a function of the history of previous activations of the synaptic connection. This history dependence can be characterized by examining the dependence of transmission on the frequency of stimulation. In this framework synaptic plasticity can also be examined in terms of changes in the frequency dependence of transmission and not merely in terms of synaptic strength which constitutes only a linear scaling mechanism. Recent work shows that the frequency dependence of transmission determines the content of information transmitted between neurons and that synaptic modifications can change the content of information transmitted. Multipatch-clamp recordings revealed that the frequency dependence of transmission is potentially unique for each synaptic connection made by a single axon and that the class of pre-postsynaptic neuron determines the class of frequency dependence (activity independent), while the unique activity relationship between any two neurons could determine the precise values of the parameters within a specific class (activity dependent). The content of information transmitted between neurons is also formalized to provide synaptic transfer functions which can be used to determine the role of the synaptic connection within a network of neurons. It is proposed that deriving synaptic transfer functions is crucial in order to understand the link between synaptic transmission and information processing within networks of neurons and to understand the link between synaptic plasticity and learning and memory.

Potential for multiple mechanisms, phenomena and algorithms for synaptic plasticity at single synapses.

Recent experimental evidence indicates that in the neocortex, the manner in which each synapse releases neurotransmitter in response to trains of presynaptic action potentials is potentially unique. These unique transmission characteristics arise because of a large heterogeneity in various synaptic properties that determine frequency dependence of transmission such as those governing the rates of synaptic depression and facilitation. A theoretical analysis was therefore undertaken to explore the phenomenologies of changes in the values of these synaptic parameters. The results illustrate how the change in any one of several synaptic parameters produces a distinctive effect on synaptic transmission and how these distinctive effects can point to the most likely biophysical mechanisms. These results could therefore be useful in studies of synaptic plasticity in order to obtain a full characterization of the phenomenologies of synaptic modifications and to isolate potential biophysical mechanisms. Based on this theoretical analysis and experimental data, it is proposed that there exists multiple mechanisms, phenomena and algorithms for synaptic plasticity at single synapses. Finally, it is shown that the impact of changing the values of synaptic parameters depends on the values of the other parameters. This may indicate that the various mechanisms, phenomena and algorithms are interlinked in a 'synaptic plasticity code'.

Differential signaling via the same axon of neocortical pyramidal neurons.

The nature of information stemming from a single neuron and conveyed simultaneously to several hundred target neurons is not known. Triple and quadruple neuron recordings revealed that each synaptic connection established by neocortical pyramidal neurons is potentially unique. Specifically, synaptic connections onto the same morphological class differed in the numbers and dendritic locations of synaptic contacts, their absolute synaptic strengths, as well as their rates of synaptic depression and recovery from depression. The same axon of a pyramidal neuron innervating another pyramidal neuron and an interneuron mediated frequency-dependent depression and facilitation, respectively, during high frequency discharges of presynaptic action potentials, suggesting that the different natures of the target neurons underlie qualitative differences in synaptic properties. Facilitating-type synaptic connections established by three pyramidal neurons of the same class onto a single interneuron, were all qualitatively similar with a combination of facilitation and depression mechanisms. The time courses of facilitation and depression, however, differed for these convergent connections, suggesting that different pre-postsynaptic interactions underlie quantitative differences in synaptic properties. Mathematical analysis of the transfer functions of frequency-dependent synapses revealed supra-linear, linear, and sub-linear signaling regimes in which mixtures of presynaptic rates, integrals of rates, and derivatives of rates are transferred to targets depending on the precise values of the synaptic parameters and the history of presynaptic action potential activity. Heterogeneity of synaptic transfer functions therefore allows multiple synaptic representations of the same presynaptic action potential train and suggests that these synaptic representations are regulated in a complex manner. It is therefore proposed that differential signaling is a key mechanism in neocortical information processing, which can be regulated by selective synaptic modifications.

Reading neuronal synchrony with depressing synapses.

A recent experiment showed that neurons in the primary auditory cortex of the monkey do not change their mean firing rate during an ongoing tone stimulus. The only change was an enhanced correlation among the individual spike trains during the tone. We show that there is an easy way to extract this coherence information in the cortical cell population by projecting the spike trains through depressing synapses onto a postsynaptic neuron.

Neural networks with dynamic synapses.

Transmission across neocortical synapses depends on the frequency of presynaptic activity (Thomson & Deuchars, 1994). Interpyramidal synapses in layer V exhibit fast depression of synaptic transmission, while other types of synapses exhibit facilitation of transmission. To study the role of dynamic synapses in network computation, we propose a unified phenomenological model that allows computation of the postsynaptic current generated by both types of synapses when driven by an arbitrary pattern of action potential (AP) activity in a presynaptic population. Using this formalism, we analyze different regimes of synaptic transmission and demonstrate that dynamic synapses transmit different aspects of the presynaptic activity depending on the average presynaptic frequency. The model also allows for derivation of mean-field equations, which govern the activity of large, interconnected networks. We show that the dynamics of synaptic transmission results in complex sets of regular and irregular regimes of network activity.

Associative memory and segmentation in an oscillatory neural model of the olfactory bulb.

We discuss the first few stages of olfactory processing in the framework of a layered neural network. Its central component is an oscillatory associative memory, describing the external plexiform layer, that consists of inhibitory and excitatory neurons with dendrodendritic interactions. We explore the computational properties of this neural network and point out its possible functional role in the olfactory bulb. When receiving a complex input that is composed of several odors, the network segments it into its components. This is done in two stages. First, multiple odor input is preprocessed in the glomerular layer via a decorrelation mechanism that relies on temporal independence of odor sources. Second, as the recall process of a pattern consists of associative convergence to an oscillatory attractor, multiple inputs are identified by alternate dominance of memory patterns during different sniff cycles. This could explain how quick analysis of mixed odors is subserved by the rapid sniffing behavior of highly olfactory animals. When one of the odors is much stronger than the rest, the network converges onto it, thus displaying odor masking.

Excitatory-inhibitory network in the visual cortex: psychophysical evidence.

At early stages in visual processing cells respond to local stimuli with specific features such as orientation and spatial frequency. Although the receptive fields of these cells have been thought to be local and independent, recent physiological and psychophysical evidence has accumulated, indicating that the cells participate in a rich network of local connections. Thus, these local processing units can integrate information over much larger parts of the visual field; the pattern of their response to a stimulus apparently depends on the context presented. To explore the pattern of lateral interactions in human visual cortex under different context conditions we used a novel chain lateral masking detection paradigm, in which human observers performed a detection task in the presence of different length chains of high-contrast-flanked Gabor signals. The results indicated a nonmonotonic relation of the detection threshold with the number of flankers. Remote flankers had a stronger effect on target detection when the space between them was filled with other flankers, indicating that the detection threshold is caused by dynamics of large neuronal populations in the neocortex, with a major interplay between excitation and inhibition. We considered a model of the primary visual cortex as a network consisting of excitatory and inhibitory cell populations, with both short- and long-range interactions. The model exhibited a behavior similar to the experimental results throughout a range of parameters. Experimental and modeling results indicated that long-range connections play an important role in visual perception, possibly mediating the effects of context.

Paradoxical effects of external modulation of inhibitory interneurons.

The neocortex, hippocampus, and several other brain regions contain populations of excitatory principal cells with recurrent connections and strong interactions with local inhibitory interneurons. To improve our understanding of the interactions among these cell types, we modeled the dynamic behavior of this type of network, including external inputs. A surprising finding was that increasing the direct external inhibitory input to the inhibitory interneurons, without directly affecting any other part of the network, can, in some circumstances, cause the interneurons to increase their firing rates. The main prerequisite for this paradoxical response to external input is that the recurrent connections among the excitatory cells are strong enough to make the excitatory network unstable when feedback inhibition is removed. Because this requirement is met in the neocortex and several regions of the hippocampus, these observations have important implications for understanding the responses of interneurons to a variety of pharmacological and electrical manipulations. The analysis can be extended to a scenario with periodically varying external input, where it predicts a systematic relationship between the phase shift and depth of modulation for each interneuron. This prediction was tested by recording from interneurons in the CA1 region of the rat hippocampus in vivo, and the results broadly confirmed the model. These findings have further implications for the function of inhibitory and neuromodulatory circuits, which can be tested experimentally.

The role of inhibition in an associative memory model of the olfactory bulb.

The external plexiform layer is where the interactions between the mitral (excitatory) and granule (inhibitory) cells of the olfactory bulb (OB) take place. Two outstanding features of these interactions are that they are dendrodendritic and that there seem to be none between excitatory cells. The latter are usually credited with the role of forming Hebbian cell assemblies. Hence, it would seem that this structure lacks the necessary ingredients for an associative memory system. In this article we show that in spite of these two properties this system can serve as an associative memory. Our model incorporates the essential anatomical characteristics of the OB. The memories in our system, defined by Hebbian mitral assemblies, are activated via the interactions with the inhibitory granule cells. The nonlinearity is introduced in our model via a sigmoid function that describes neurotransmitter release in reciprocal dendrodendritic synapses. The capacity (maximal number of odors that can be memorized) depends on the sparseness of coding that is being used. For very low memory activities, the capacity grows as a fractional power of the number of neurons. We validate the theoretical results by numerical simulations. An interesting result of our model is that its capacity increases as a function of the ratio of inhibitory to excitatory populations. This may provide an explanation for the dominance of inhibitory cells in the olfactory bulb.

The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability.

Although signaling between neurons is central to the functioning of the brain, we still do not understand how the code used in signaling depends on the properties of synaptic transmission. Theoretical analysis combined with patch clamp recordings from pairs of neocortical pyramidal neurons revealed that the rate of synaptic depression, which depends on the probability of neurotransmitter release, dictates the extent to which firing rate and temporal coherence of action potentials within a presynaptic population are signaled to the postsynaptic neuron. The postsynaptic response primarily reflects rates of firing when depression is slow and temporal coherence when depression is fast. A wide range of rates of synaptic depression between different pairs of pyramidal neurons was found, suggesting that the relative contribution of rate and temporal signals varies along a continuum. We conclude that by setting the rate of synaptic depression, release probability is an important factor in determining the neural code.

Redistribution of synaptic efficacy between neocortical pyramidal neurons.

Experience-dependent potentiation and depression of synaptic strength has been proposed to subserve learning and memory by changing the gain of signals conveyed between neurons. Here we examine synaptic plasticity between individual neocortical layer-5 pyramidal neurons. We show that an increase in the synaptic response, induced by pairing action-potential activity in pre- and postsynaptic neurons, was only observed when synaptic input occurred at low frequencies. This frequency-dependent increase in synaptic responses arises because of a redistribution of the available synaptic efficacy and not because of an increase in the efficacy. Redistribution of synaptic efficacy could represent a mechanism to change the content, rather than the gain, of signals conveyed between neurons.

Population dynamics and theta rhythm phase precession of hippocampal place cell firing: a spiking neuron model.

O'Keefe and Recce ([1993] Hippocampus 68:317-330) have observed that the spatially selective firing of pyramidal cells in the CA1 field of the rat hippocampus tends to advance to earlier phases of the electroencephalogram theta rhythm as a rat passes through the place field of a cell. We present here a neural network model based on integrate- and-fire neurons that accounts for this effect. In this model, place selectivity in the hippocampus is a consequence of synaptic interactions between pyramidal neurons together with weakly selective external input. The phase shift of neuronal spiking arises in the model as result of asymmetric spread of activation through the network, caused by asymmetry in the synaptic interactions. Several experimentally observed properties of the phase shift effect follow naturally from the model, including 1) the observation that the first spikes a cell fires appear near the theta phase corresponding to minimal population activity, 2) the overall advance is less than 360 degrees, and 3) the location of the rat within the place field of the cell is the primary correlate of the firing phase, not the time the rat has been in the field. The model makes several predictions concerning the emergence of place fields during the earliest stages of exploration in a novel environment. It also suggests new experiments that could provide further constraints on a possible explanation of the phase precession effect.