Encoding Temporal Regularities and Information Copying in Hippocampal Circuits.

Discriminating, extracting and encoding temporal regularities is a critical requirement in the brain, relevant to sensory-motor processing and learning. However, the cellular mechanisms responsible remain enigmatic; for example, whether such abilities require specific, elaborately organized neural networks or arise from more fundamental, inherent properties of neurons. Here, using multi-electrode array technology, and focusing on interval learning, we demonstrate that sparse reconstituted rat hippocampal neural circuits are intrinsically capable of encoding and storing sub-second-order time intervals for over an hour timescale, represented in changes in the spatial-temporal architecture of firing relationships among populations of neurons. This learning is accompanied by increases in mutual information and transfer entropy, formal measures related to information storage and flow. Moreover, temporal relationships derived from previously trained circuits can act as templates for copying intervals into untrained networks, suggesting the possibility of circuit-to-circuit information transfer. Our findings illustrate that dynamic encoding and stable copying of temporal relationships are fundamental properties of simple in vitro networks, with general significance for understanding elemental principles of information processing, storage and replication.

Insect Bio-inspired Neural Network Provides New Evidence on How Simple Feature Detectors Can Enable Complex Visual Generalization and Stimulus Location Invariance in the Miniature Brain of Honeybees.

The ability to generalize over naturally occurring variation in cues indicating food or predation risk is highly useful for efficient decision-making in many animals. Honeybees have remarkable visual cognitive abilities, allowing them to classify visual patterns by common features despite having a relatively miniature brain. Here we ask the question whether generalization requires complex visual recognition or whether it can also be achieved with relatively simple neuronal mechanisms. We produced several simple models inspired by the known anatomical structures and neuronal responses within the bee brain and subsequently compared their ability to generalize achromatic patterns to the observed behavioural performance of honeybees on these cues. Neural networks with just eight large-field orientation-sensitive input neurons from the optic ganglia and a single layer of simple neuronal connectivity within the mushroom bodies (learning centres) show performances remarkably similar to a large proportion of the empirical results without requiring any form of learning, or fine-tuning of neuronal parameters to replicate these results. Indeed, a model simply combining sensory input from both eyes onto single mushroom body neurons returned correct discriminations even with partial occlusion of the patterns and an impressive invariance to the location of the test patterns on the eyes. This model also replicated surprising failures of bees to discriminate certain seemingly highly different patterns, providing novel and useful insights into the inner workings facilitating and limiting the utilisation of visual cues in honeybees. Our results reveal that reliable generalization of visual information can be achieved through simple neuronal circuitry that is biologically plausible and can easily be accommodated in a tiny insect brain.

Primordial evolvability: Impasses and challenges.

While it is generally agreed that some kind of replicating non-living compounds were the precursors of life, there is much debate over their possible chemical nature. Metabolism-first approaches propose that mutually catalytic sets of simple organic molecules could be capable of self-replication and rudimentary chemical evolution. In particular, the graded autocatalysis replication domain (GARD) model, depicting assemblies of amphiphilic molecules, has received considerable interest. The system propagates compositional information across generations and is suggested to be a target of natural selection. However, evolutionary simulations indicate that the system lacks selectability (i.e. selection has negligible effect on the equilibrium concentrations). We elaborate on the lessons learnt from the example of the GARD model and, more widely, on the issue of evolvability, and discuss the implications for similar metabolism-first scenarios. We found that simple incorporation-type chemistry based on non-covalent bonds, as assumed in GARD, is unlikely to result in alternative autocatalytic cycles when catalytic interactions are randomly distributed. An even more serious problem stems from the lognormal distribution of catalytic factors, causing inherent kinetic instability of such loops, due to the dominance of efficiently catalyzed components that fail to return catalytic aid. Accordingly, the dynamics of the GARD model is dominated by strongly catalytic, but not auto-catalytic, molecules. Without effective autocatalysis, stable hereditary propagation is not possible. Many repetitions and different scaling of the model come to no rescue. Despite all attempts to show the contrary, the GARD model is not evolvable, in contrast to reflexively autocatalytic networks, complemented by rare uncatalyzed reactions and compartmentation. The latter networks, resting on the creation and breakage of chemical bonds, can generate novel ('mutant') autocatalytic loops from a given set of environmentally available compounds. Real chemical reactions that make or break covalent bonds, rather than mere incorporation of components, are necessary for open-ended evolvability. The issue of whether or not several concrete chemical systems (rather than singular curiosities) could realize reflexively autocatalytic macromolecular networks will ultimately determine the relevance of metabolism-first approaches to the origin of life, as stepping stones towards true open-endedness that requires the combination of rich combinatorial chemistry controlled by information stored in template replicators.

From blickets to synapses: inferring temporal causal networks by observation.

How do human infants learn the causal dependencies between events? Evidence suggests that this remarkable feat can be achieved by observation of only a handful of examples. Many computational models have been produced to explain how infants perform causal inference without explicit teaching about statistics or the scientific method. Here, we propose a spiking neuronal network implementation that can be entrained to form a dynamical model of the temporal and causal relationships between events that it observes. The network uses spike-time dependent plasticity, long-term depression, and heterosynaptic competition rules to implement Rescorla-Wagner-like learning. Transmission delays between neurons allow the network to learn a forward model of the temporal relationships between events. Within this framework, biologically realistic synaptic plasticity rules account for well-known behavioral data regarding cognitive causal assumptions such as backwards blocking and screening-off. These models can then be run as emulators for state inference. Furthermore, this mechanism is capable of copying synaptic connectivity patterns between neuronal networks by observing the spontaneous spike activity from the neuronal circuit that is to be copied, and it thereby provides a powerful method for transmission of circuit functionality between brain regions.

Evolution of associative learning in chemical networks.

Organisms that can learn about their environment and modify their behaviour appropriately during their lifetime are more likely to survive and reproduce than organisms that do not. While associative learning - the ability to detect correlated features of the environment - has been studied extensively in nervous systems, where the underlying mechanisms are reasonably well understood, mechanisms within single cells that could allow associative learning have received little attention. Here, using in silico evolution of chemical networks, we show that there exists a diversity of remarkably simple and plausible chemical solutions to the associative learning problem, the simplest of which uses only one core chemical reaction. We then asked to what extent a linear combination of chemical concentrations in the network could approximate the ideal Bayesian posterior of an environment given the stimulus history so far? This Bayesian analysis revealed the 'memory traces' of the chemical network. The implication of this paper is that there is little reason to believe that a lack of suitable phenotypic variation would prevent associative learning from evolving in cell signalling, metabolic, gene regulatory, or a mixture of these networks in cells.

What is comparable in comparative cognition?

To understand how complex, or 'advanced' various forms of cognition are, and to compare them between species for evolutionary studies, we need to understand the diversity of neural-computational mechanisms that may be involved, and to identify the genetic changes that are necessary to mediate changes in cognitive functions. The same overt cognitive capacity might be mediated by entirely different neural circuitries in different species, with a many-to-one mapping between behavioural routines, computations and their neural implementations. Comparative behavioural research needs to be complemented with a bottom-up approach in which neurobiological and molecular-genetic analyses allow pinpointing of underlying neural and genetic bases that constrain cognitive variation. Often, only very minor differences in circuitry might be needed to generate major shifts in cognitive functions and the possibility that cognitive traits arise by convergence or parallel evolution needs to be taken seriously. Hereditary variation in cognitive traits between individuals of a species might be extensive, and selection experiments on cognitive traits might be a useful avenue to explore how rapidly changes in cognitive abilities occur in the face of pertinent selection pressures.

Selectionist and evolutionary approaches to brain function: a critical appraisal.

We consider approaches to brain dynamics and function that have been claimed to be Darwinian. These include Edelman's theory of neuronal group selection, Changeux's theory of synaptic selection and selective stabilization of pre-representations, Seung's Darwinian synapse, Loewenstein's synaptic melioration, Adam's selfish synapse, and Calvin's replicating activity patterns. Except for the last two, the proposed mechanisms are selectionist but not truly Darwinian, because no replicators with information transfer to copies and hereditary variation can be identified in them. All of them fit, however, a generalized selectionist framework conforming to the picture of Price's covariance formulation, which deliberately was not specific even to selection in biology, and therefore does not imply an algorithmic picture of biological evolution. Bayesian models and reinforcement learning are formally in agreement with selection dynamics. A classification of search algorithms is shown to include Darwinian replicators (evolutionary units with multiplication, heredity, and variability) as the most powerful mechanism for search in a sparsely occupied search space. Examples are given of cases where parallel competitive search with information transfer among the units is more efficient than search without information transfer between units. Finally, we review our recent attempts to construct and analyze simple models of true Darwinian evolutionary units in the brain in terms of connectivity and activity copying of neuronal groups. Although none of the proposed neuronal replicators include miraculous mechanisms, their identification remains a challenge but also a great promise.

Evolvable physical self-replicators.

Building an evolvable physical self-replicating machine is a grand challenge. The main problem is that the device must be capable of hereditary variation, that is, replicating in many configurations-configurations into which it enters unpredictably by mutation. Template replication is the solution found by nature. A scalable device must also be capable of miniaturization, and so have few or no moving and electronic parts. Here a significant step toward this goal is presented in the form of a physical template replicator made from small plastic pieces containing embedded magnets that float on an air-hockey-type table and undergo stochastic motion. Our units replicate by a process analogous to the replication of DNA, except without the involvement of enzymes. Building a physical rather than a computational model forces us to confront several problems that have analogues on the nano scale. In particular, replication must be maintained by preventing side reactions such as spontaneous ligation, cyclization, product inhibition, and elongation at staggered ends. The last of these results in ever-lengthening sequences in a process known as the elongation catastrophe. The extreme specificity of structure required by the monomers is indirect evidence that some kind of natural selection took place prior to the existence of nucleotide analogues during the origin of life.

Evolution before genes.

BACKGROUND: Our current understanding of evolution is so tightly linked to template-dependent replication of DNA and RNA molecules that the old idea from Oparin of a self-reproducing 'garbage bag' ('coacervate') of chemicals that predated fully-fledged cell-like entities seems to be farfetched to most scientists today. However, this is exactly the kind of scheme we propose for how Darwinian evolution could have occurred prior to template replication. RESULTS: We cannot confirm previous claims that autocatalytic sets of organic polymer molecules could undergo evolution in any interesting sense by themselves. While we and others have previously imagined inhibition would result in selectability, we found that it produced multiple attractors in an autocatalytic set that cannot be selected for. Instead, we discovered that if general conditions are satisfied, the accumulation of adaptations in chemical reaction networks can occur. These conditions are the existence of rare reactions producing viable cores (analogous to a genotype), that sustains a molecular periphery (analogous to a phenotype). CONCLUSIONS: We conclude that only when a chemical reaction network consists of many such viable cores, can it be evolvable. When many cores are enclosed in a compartment there is competition between cores within the same compartment, and when there are many compartments, there is between-compartment competition due to the phenotypic effects of cores and their periphery at the compartment level. Acquisition of cores by rare chemical events, and loss of cores at division, allows macromutation, limited heredity and selectability, thus explaining how a poor man's natural selection could have operated prior to genetic templates. This is the only demonstration to date of a mechanism by which pre-template accumulation of adaptation could occur.

Evolvable neuronal paths: a novel basis for information and search in the brain.

We propose a previously unrecognized kind of informational entity in the brain that is capable of acting as the basis for unlimited hereditary variation in neuronal networks. This unit is a path of activity through a network of neurons, analogous to a path taken through a hidden Markov model. To prove in principle the capabilities of this new kind of informational substrate, we show how a population of paths can be used as the hereditary material for a neuronally implemented genetic algorithm, (the swiss-army knife of black-box optimization techniques) which we have proposed elsewhere could operate at somatic timescales in the brain. We compare this to the same genetic algorithm that uses a standard 'genetic' informational substrate, i.e. non-overlapping discrete genotypes, on a range of optimization problems. A path evolution algorithm (PEA) is defined as any algorithm that implements natural selection of paths in a network substrate. A PEA is a previously unrecognized type of natural selection that is well suited for implementation by biological neuronal networks with structural plasticity. The important similarities and differences between a standard genetic algorithm and a PEA are considered. Whilst most experiments are conducted on an abstract network model, at the conclusion of the paper a slightly more realistic neuronal implementation of a PEA is outlined based on Izhikevich spiking neurons. Finally, experimental predictions are made for the identification of such informational paths in the brain.

Symbol manipulation and rule learning in spiking neuronal networks.

It has been claimed that the productivity, systematicity and compositionality of human language and thought necessitate the existence of a physical symbol system (PSS) in the brain. Recent discoveries about temporal coding suggest a novel type of neuronal implementation of a physical symbol system. Furthermore, learning classifier systems provide a plausible algorithmic basis by which symbol re-write rules could be trained to undertake behaviors exhibiting systematicity and compositionality, using a kind of natural selection of re-write rules in the brain, We show how the core operation of a learning classifier system, namely, the replication with variation of symbol re-write rules, can be implemented using spike-time dependent plasticity based supervised learning. As a whole, the aim of this paper is to integrate an algorithmic and an implementation level description of a neuronal symbol system capable of sustaining systematic and compositional behaviors. Previously proposed neuronal implementations of symbolic representations are compared with this new proposal.

The neuronal replicator hypothesis.

We propose that replication (with mutation) of patterns of neuronal activity can occur within the brain using known neurophysiological processes. Thereby evolutionary algorithms implemented by neuro- nal circuits can play a role in cognition. Replication of structured neuronal representations is assumed in several cognitive architectures. Replicators overcome some limitations of selectionist models of neuronal search. Hebbian learning is combined with replication to structure exploration on the basis of associations learned in the past. Neuromodulatory gating of sets of bistable neurons allows patterns of activation to be copied with mutation. If the probability of copying a set is related to the utility of that set, then an evolutionary algorithm can be implemented at rapid timescales in the brain. Populations of neuronal replicators can undertake a more rapid and stable search than can be achieved by serial modification of a single solution. Hebbian learning added to neuronal replication allows a powerful structuring of variability capable of learning the location of a global optimum from multiple previously visited local optima. Replication of solutions can solve the problem of catastrophic forgetting in the stability-plasticity dilemma. In short, neuronal replication is essential to explain several features of flexible cognition. Predictions are made for the experimental validation of the neuronal replicator hypothesis.

A model of the mechanisms of language extinction and revitalization strategies to save endangered languages.

Why and how have languages died out? We have devised a mathematical model to help us understand how languages go extinct. We use the model to ask whether language extinction can be prevented in the future and why it may have occurred in the past. A growing number of mathematical models of language dynamics have been developed to study the conditions for language coexistence and death, yet their phenomenological approach compromises their ability to influence language revitalization policy. In contrast, here we model the mechanisms underlying language competition and look at how these mechanisms are influenced by specific language revitalization interventions, namely, private interventions to raise the status of the language and thus promote language learning at home, public interventions to increase the use of the minority language, and explicit teaching of the minority language in schools. Our model reveals that it is possible to preserve a minority language but that continued long-term interventions will likely be necessary. We identify the parameters that determine which interventions work best under certain linguistic and societal circumstances. In this way the efficacy of interventions of various types can be identified and predicted. Although there are qualitative arguments for these parameter values (e.g., the responsiveness of children to learning a language as a function of the proportion of conversations heard in that language, the relative importance of conversations heard in the family and elsewhere, and the amplification of spoken to heard conversations of the high-status language because of the media), extensive quantitative data are lacking in this field. We propose a way to measure these parameters, allowing our model, as well as others models in the field, to be validated.

In silico biology.

Rather than studying existent living systems, we can increasingly produce computer models that capture the salient aspects of life. This provides us with unprecedented opportunities to examine, manipulate, and explore biological phenomena, allowing us to investigate some of the deepest issues in biology.

Molecular circuits for associative learning in single-celled organisms.

We demonstrate how a single-celled organism could undertake associative learning. Although to date only one previous study has found experimental evidence for such learning, there is no reason in principle why it should not occur. We propose a gene regulatory network that is capable of associative learning between any pre-specified set of chemical signals, in a Hebbian manner, within a single cell. A mathematical model is developed, and simulations show a clear learned response. A preliminary design for implementing this model using plasmids within Escherichia coli is presented, along with an alternative approach, based on double-phosphorylated protein kinases.

Copying and evolution of neuronal topology.

We propose a mechanism for copying of neuronal networks that is of considerable interest for neuroscience for it suggests a neuronal basis for causal inference, function copying, and natural selection within the human brain. To date, no model of neuronal topology copying exists. We present three increasingly sophisticated mechanisms to demonstrate how topographic map formation coupled with Spike-Time Dependent Plasticity (STDP) can copy neuronal topology motifs. Fidelity is improved by error correction and activity-reverberation limitation. The high-fidelity topology-copying operator is used to evolve neuronal topologies. Possible roles for neuronal natural selection are discussed.

The origin of autonomous agents by natural selection.

We propose conditions in which an autonomous agent could arise, and increase in complexity. It is assumed that on the primitive Earth there arose a recycling flow-reactor containing spontaneously formed oil droplets or lipid aggregates. These droplets grew at a basal rate by simple incorporation of lipid phase material, and divided by external agitation. This type of system was able to implement a natural selection algorithm once heredity was added. Macroevolution became possible by selection for rarely occurring chemical reactions that produced holistic autocatalytic molecular replicators (contained within the aggregate) capable of doubling at least as fast as the lipid aggregate, and which were also capable of benefiting the growth of its lipid aggregate container. No nucleotides or monomers capable of modular heredity were required at the outset. To explicitly state this hypothesis, a computer model was developed that employed an artificial chemistry, exhibiting conservation of mass and energy, incorporated within each individual of a population of lipid aggregates. This model evolved increasingly complex self-sustaining processes of constitution, a result that is also expected in real chemistry.

Natural selection in chemical evolution.

We propose that chemical evolution can take place by natural selection if a geophysical process is capable of heterotrophic formation of liposomes that grow at some base rate, divide by external agitation, and are subject to stochastic chemical avalanches, in the absence of nucleotides or any monomers capable of modular heredity. We model this process using a simple hill-climbing algorithm, and an artificial chemistry that is unique in exhibiting conservation of mass and energy in an open thermodynamic system. Selection at the liposome level results in the stabilization of rarely occurring molecular autocatalysts that either catalyse or are consumed in reactions that confer liposome level fitness; typically they contribute in parallel to an increasingly conserved intermediary metabolism. Loss of competing autocatalysts can sometimes be adaptive. Steady-state energy flux by the individual increases due to the energetic demands of growth, but also of memory, i.e. maintaining variations in the chemical network. Self-organizing principles such as those proposed by Kauffman, Fontana, and Morowitz have been hypothesized as an ordering principle in chemical evolution, rather than chemical evolution by natural selection. We reject those notions as either logically flawed or at best insufficient in the absence of natural selection. Finally, a finite population model without elitism shows the practical evolutionary constraints for achieving chemical evolution by natural selection in the lab.

A stochastic model of nonenzymatic nucleic acid replication: "elongators" sequester replicators.

The origin of nucleic acid template replication is a major unsolved problem in science. A novel stochastic model of nucleic acid chemistry was developed to allow rapid prototyping of chemical experiments designed to discover sufficient conditions for template replication. Experiments using the model brought to attention a robust property of nucleic acid template populations, the tendency for elongation to outcompete replication. Externally imposed denaturation-renaturation cycles did not reverse this tendency. For example, it has been proposed that fast tidal cycling could establish a TCR (tidal chain reaction) analogous to a PCR (polymerase chain reaction) acting on nucleic acid polymers, allowing their self-replication. However, elongating side-reactions that would have been prevented by the polymerase in the PCR still occurred in the simulation of the TCR. The same finding was found with temperature and monomer cycles. We propose that if cycling reactors are to allow template replication, oligonucleotide phenotypes that are capable of favorably altering the flux ratio between replication and elongation, for example, by facilitating sequence-specific cleavage within templates, are necessary; accordingly the minimal replicase ribozyme may have possessed restriction functionality.