Memory, modelling and Marr: a commentary on Marr (1971) 'Simple memory: a theory of archicortex'.

David Marr's theory of the archicortex, a brain structure now more commonly known as the hippocampus and hippocampal formation, is an epochal contribution to theoretical neuroscience. Addressing the problem of how information about 10 000 events could be stored in the archicortex during the day so that they can be retrieved using partial information and then transferred to the neocortex overnight, the paper presages a whole wealth of later empirical and theoretical work, proving impressively prescient. Despite this impending success, Marr later apparently grew dissatisfied with this style of modelling, but he went on to make seminal suggestions that continue to resonate loudly throughout the field of theoretical neuroscience. We describe Marr's theory of the archicortex and his theory of theories, setting them into their original and a contemporary context, and assessing their impact. This commentary was written to celebrate the 350th anniversary of the journal Philosophical Transactions of the Royal Society.

Development of nerve connections under the control of neurotrophic factors: parallels with consumer-resource systems in population biology.

The development of connections between neurons and their target cells involves competition between axons for target-derived neurotrophic factors. Although the notion of competition is commonly used in neurobiology, the process is not well understood, and only a few formal models exist. In population biology, in contrast, the concept of competition is well developed and has been studied by means of many formal models of consumer-resource systems. Here we show that a recently formulated model of axonal competition can be rewritten as a general consumer-resource system. This allows neurobiological phenomena to be interpreted in population biological terms and, conversely, results from population biology to be applied to neurobiology. Using findings from population biology, we have studied two extensions of our axonal competition model. In the first extension, the spatial dimension of the target is explicitly taken into account. We show that distance between axons on their target mitigates competition and permits the coexistence of axons. The model can account for the fact that in many types of neurons a positive correlation exists between the size of the dendritic tree and the number of innervating axons surviving into adulthood. In the second extension, axons are allowed to respond to more than one neurotrophic factor. We show that this permits competitive exclusion among axons of one type, while at the same time there is coexistence with axons of another type innervating the same target. The model offers an explanation for the innervation pattern found on cerebellar Purkinje cells, where climbing fibres compete with each other until only a single one remains, which coexists with parallel fibre input to the same Purkinje cell.

Lateral cell movement driven by dendritic interactions is sufficient to form retinal mosaics.

The formation of retinal mosaics is thought to involve lateral movement of retinal cells from their clonal column of origin. The forces underlying this lateral cell movement are currently unknown. We have used a model of neurite outgrowth combined with cell movement to investigate the hypothesis that lateral cell movement is guided by dendritic interactions. We have assumed that cells repel each other in proportion to the degree of dendritic overlap between neighbouring cells. Our results first show that small cell movements are sufficient to transform random cell distributions into regular mosaics, and that all cells within the population move. When dendritic fields are allowed to grow, the model produces regular mosaics across all cell densities tested. We also find that the model can produce constant coverage of visual space over varying cell densities. However, if dendritic field sizes are fixed, mosaic regularity is proportional to the cell density and dendritic field size. Our model suggests that dendritic mechanisms may therefore provide sufficient information for rearrangement of cells into regular mosaics. We conclude by mentioning possible future experiments that might suggest whether dendritic interactions are adaptive or fixed during mosaic formation.

Competition for neurotrophic factor in the development of nerve connections.

The development of nerve connections is thought to involve competition among axons for survival promoting factors, or neurotrophins, which are released by the cells that are innervated by the axons. Although the notion of competition is widely used within neurobiology, there is little understanding of the nature of the competitive process and the underlying mechanisms. We present a new theoretical model to analyse competition in the development of nerve connections. According to the model, the precise manner in which neurotrophins regulate the growth of axons, in particular the growth of the amount of neurotrophin receptor, determines what patterns of target innervation can develop. The regulation of neurotrophin receptors is also involved in the degeneration and regeneration of connections. Competition in our model can be influenced by factors dependent on and independent of neuronal electrical activity. Our results point to the need to measure directly the specific form of the regulation by neurotrophins of their receptors.

Poly- and mononeuronal innervation in a model for the development of neuromuscular connections.

In the normal development of connections between motor neurons and muscle fibres, an initial stage of polyneuronal innervation is followed by withdrawal of connections until each muscle fibre is innervated by a single axon. However, polyneuronal innervation has been found to persist after prolonged nerve conduction block, in spite of the resumption of normal neuromuscular activity. Here we analyse in detail a model proposed for the withdrawal of nerve connections in developing muscle, based on competition between nerve terminals. The model combines competition for a pre-synaptic resource with competition for a post-synaptic resource. Using bifurcation and phase space analysis, we show that polyneuronal innervation, as well as mononeuronal innervation, can be stable. The model accounts for the development of mononeuronal innervation and for persistent polyneuronal innervation after prolonged nerve conduction block, which appears as a consequence of the general competitive interactions operating during normal development.

A massively connected subthalamic nucleus leads to the generation of widespread pulses.

A composite model of the subthalamic nucleus is developed from physiological and anatomical considerations. First, study of a geometric model of the anatomical arrangements of projection neurons within the nucleus indicates that they form a massively connected network. Second, given the excitatory nature of these neurons, their threshold and peak firing rates, a simple model of neuron responses reveals that large regions of this highly interconnected nucleus can respond to excitatory input in the form of a wide-spread uniform pulse. Such widespread pulses of activity may act as a braking signal that resets the major basal ganglia output nuclei.

Short-term interactions between microtubules and actin filaments underlie long-term behaviour in neuronal growth cones.

We present two new computational models of microtubule dynamics in the neuronal growth cone. These extend previous models of microtubule dynamics, which have neglected the effect of microtubule interactions with one another and with F-actin in the growth cone. Ultimately, these interactions determine whether the nerve cell makes the right target connections. In the first model, analysis of the effect of microtubule bundling on axonal elongation shows that small interaction effects between individual microtubules can be amplified within the microtubule bundle to significantly alter the rate of axonal growth. The second model concerns the effect of interactions between microtubules and F-actin on growth-cone turning. The model simulates microtubule invasion into the growth cone after contact with a target cell. Results suggest that microtubules do not randomly invade the growth cone, which supports the recent view that microtubules play a more active role in pathfinding than previously expected. Our results indicate that microtubule interactions with F-actin and with other microtubules play a fundamental role in axonal elongation and growth-cone turning.

A new approach to Kanerva's sparse distributed memory.

The sparse distributed memory (SDM) was originally developed to tackle the problem of storing large binary data patterns. The model succeeded well in storing random input data. However, its efficiency, particularly in handling nonrandom data, was poor. In its original form it is a static and inflexible system. Most of the recent work on the SDM has concentrated on improving the efficiency of a modified form of the SDM which treats the memory as a single-layer neural network. This paper introduces an alternative SDM, the SDM signal model which retains the essential characteristics of the original SDM, while providing the memory with a greater scope for plasticity and self-evolution. By removing many of the problematic features of the original SDM the new model is not as dependent upon a priori input values. This gives it an increased robustness to learn either random or correlated input patterns. The improvements in this new SDM signal model should be also of benefit to modified SDM neural network models.

An evaluation of the use of multidimensional scaling for understanding brain connectivity.

A large amount of data is now available about the pattern of connections between brain regions. Computational methods are increasingly relevant for uncovering structure in such datasets. There has been recent interest in the use of non-metric multidimensional scaling (NMDS) for such analysis. NMDS produces a spatial representation of the 'dissimilarities' between a number of entities. Normally, it is applied to data matrices containing a large number of levels of dissimilarity, whereas for brain connectivity data there is a very small number. We address the suitability of NMDS for this case. Systematic numerical studies are presented to evaluate the ability of this method to reconstruct known geometrical configurations from dissimilarity data processing few levels. In this case there is a strong bias for NMDS to produce annular configurations, whether or not such structure exists in the original data. For the case of a connectivity dataset derived from the primate cortical visual system, we demonstrate that great caution is needed in interpreting the resulting configuration. Application of an independent method that we developed also strongly suggests that the visual system NMDS configuration is affected by an annular bias. We question the strength of support that an NMDS analysis of the visual system data provides for the two streams view of visual processing.

Presynaptic and postsynaptic competition in models for the development of neuromuscular connections.

In the establishment of connections between nerve and muscle there is an initial stage when each muscle fibre is innervated by several different motor axons. Withdrawal of connections then takes place until each fibre has contact from just a single axon. The evidence suggests that the withdrawal process involves competition between nerve terminals. We examine in formal models several types of competitive mechanism that have been proposed for this phenomenon. We show that a model which combines competition for a presynaptic resource with competition for a postsynaptic resource is superior to others. This model accounts for many anatomical and physiological findings and has a biologically plausible implementation. Intrinsic withdrawal appears to be a side effect of the competitive mechanism rather than a separate non-competitive feature. The model's capabilities are confirmed by theoretical analysis and full scale computer simulations.

Optimising synaptic learning rules in linear associative memories.

Associative matrix memories with real-valued synapses have been studied in many incarnations. We consider how the signal/noise ratio for associations depends on the form of the learning rule, and we show that a covariance rule is optimal. Two other rules, which have been suggested in the neurobiology literature, are asymptotically optimal in the limit of sparse coding. The results appear to contradict a line of reasoning particularly prevalent in the physics community. It turns out that the apparent conflict is due to the adoption of different underlying models. Ironically, they perform identically at their co-incident optima. We give details of the mathematical results, and discuss some other possible derivations and definitions of the signal/noise ratio.

An assessment of Marr's theory of the hippocampus as a temporary memory store.

The recent reawakened interest in 'neural' networks begs the question of their relevance to the analysis of real nervous systems. Network models have been criticized for the lack of realism of their individual components, and because the architectures required by some neural-network algorithms do not seem to exist in real nervous systems. In three related papers published in the 1970s, David Marr proposed that the cerebellum, the neocortex and the hippocampus each acts as a memorizing device. These theories were intended to satisfy the biological constraints, but in computational terms they are undetermined. In this paper we reassess Marr's theory of the hippocampus as a temporary memory store. We give a complete computational account of the theory and we show that Marr's computational arguments do not sufficiently constrain his choice of model. We discuss Marr's specific model of temporary memory with reference to the neurophysiology and neuroanatomy of the mammalian hippocampus. Our analysis is supported by simulation studies done on various memory models built according to the principles advocated by Marr.

The discontinuous visual projections on the Xenopus optic tectum following regeneration after unilateral nerve section.

The establishment of retinotectal projections following transection of one optic nerve in developing Xenopus has been investigated. Between 3 weeks and 11 months after the operation, the nerve fibre tracer horseradish peroxidase (HRP) was applied to either the operated or the unoperated nerve, and the brains were prepared for examination as whole mounts. In most cases fibres from the operated nerve innervated both tecta, with the result that one tectum was doubly innervated and one tectum singly innervated. Two months after transection of the optic nerve in tadpole life, between stages 50 and 54, this nerve usually made a uniform projection on the contralateral tectum and a striped projection on the ipsilateral, doubly innervated, tectum. The projection made by the unoperated nerve on this tectum was a similar pattern of stripes, which ran generally rostrocaudally. Two months after transection of the optic nerve of newly metamorphosed animals, the projection formed by the operated nerve on the doubly innervated tectum was usually a pattern of spots or spots mixed together with stripes in no particular orientation superimposed on a roughly uniform background. In a small number of cases the projections made by the same nerve on the two tecta were approximately complementary; that is, the presence of label on one tectum corresponded with its absence on the other tectum. The results are examined in the context of the development of the retina and of the tectum. It is suggested that the consistently oriented stripes which result from nerve transection at a stage at which only a small proportion of the retinal fibres had reached the tectum are formed by the interaction of two equally matched sets of developing fibres, stripe orientation being determined by the mode of growth of the optic tectum. The formation of patterns of spots or spots mixed together with stripes following nerve transection after the end of the main phase of tectal histogenesis, and when 50% of the optic fibres had already reached the tectum, is attributed to an unequal competition between the two sets of fibres.

The distribution of fibres in the optic tract after contralateral translocation of an eye in Xenopus.

In Xenopus embryos of stage 30 the right eye was translocated, without rotation, to a left host orbit. Shortly after metamorphosis the visuotectal projection through the operated eye was mapped electrophysiologically and shown to be normal dorsoventrally but reversed nasotemporally. Labelling of small groups of retinal axons with HRP showed that the fibre trajectories from dorsal and ventral retina were normal, whereas fibres from nasally placed retina had diencephalic pathways and tectal terminations typical and temporal fibres, and fibres from temporally placed retina had diencephalic pathways and tectal terminations typical of nasal fibres. Thus from just beyond the chiasma the fibres had already achieved the major uniaxial rearrangement necessary to establish a normal tract distribution despite the eye translocation. The fibre rearrangement required to permit the formation of a nasotemporally inverted visuotectal projection appears, therefore, to occur not on the tectum or in the optic tract, but either within the nerve or at the chiasma.

The visuotectal projections made by Xenopus 'pie slice' compound eyes.

In xenopus embryos at stages 28-32 one quarter to one third of the left eye rudiment was replaced by a similarly sized piece from a different position in a right eye rudiment. Three groups of operations were performed: (1) temporal tissue was placed in a nasal position; (2) nasal tissue was placed temporally; (3) ventral tissue was placed dorsally. The visuotectal projections made by these 'pie-slice' compound eyes were assessed electrophysiologically at 1 week to 6 months after metamorphosis. Of 97 animals, 71 yielded interpretable projections. In most cases two projections could be identified in each map. One, ascribed to the host part of the retina, extended over the entire tectal surface mapped. The other, identified as that from tissue derived from the pie-slice graft, projected to the tectum in register with that part of the host retina which matched the pie-slice in origin. Both projections were well ordered, and in the orientation expected if the corresponding piece of retinal tissue had participated in a normal projection. Consistent differences in pie-slice size and tectal coverage between the three groups were found. Pie-slices of nasal origin gave maps showing that they came from a relatively large portion of the retina and projected to a relatively large amount of the tectum; those of temporal origin occupied relatively small amounts of field and tectum. It was concluded that these results are further evidence for the existence of positional markers in the retina which are used for the assembly of the retinotectal map.

The establishment and the subsequent elimination of polyneuronal innervation of developing muscle: theoretical considerations.

An analysis is given of the polyneuronal innervation of embryonic skeletal muscle and its subsequent elimination during development. The amount of polyneuronal innervation that has been observed is consistent with the notion that initially each motor neuron distributes its contacts at random among the available fibres of a particular muscle. The idea that the elimination of excess innervation proceeds through interactions between terminals is placed on quantitative basis. Each motor neuron is presumed to have a finite capacity for maintaining the structure and activity of its terminals, which is shared out among them; a survival strength can be assigned to each terminal. Survival strengths undergo a process of continual adjustment. A terminal with above average strength for its endplate is strengthened at the expense of the weaker terminals, subject to the total survival strength available to each motor neuron remaining constant. It is proved that this scheme will transform the initial pattern of innervation into one in which each muscle fibre has contact from a single axon. Interpretations of the following results are given: the decrease in the spread of motor unit size during the development of innervation of the rat soleus muscle; the time course of superinnervation; the effects of neonatal partial denervation. Various suggestions are made for future experimental approaches.