Spin-wave interference patterns created by spin-torque nano-oscillators for memory and computation.

Magnetization dynamics in nanomagnets has attracted broad interest since it was predicted that a dc current flowing through a thin magnetic layer can create spin-wave excitations. These excitations are due to spin momentum transfer, a transfer of spin angular momentum between conduction electrons and the background magnetization, that enables new types of information processing. Here we show how arrays of spin-torque nano-oscillators can create propagating spin-wave interference patterns of use for memory and computation. Memristic transponders distributed on the thin film respond to threshold tunnel magnetoresistance values, thereby allowing spin-wave detection and creating new excitation patterns. We show how groups of transponders create resonant (reverberating) spin-wave interference patterns that may be used for polychronous wave computation and information storage.

Cooperation of spike timing-dependent and heterosynaptic plasticities in neural networks: a Fokker-Planck approach.

It is believed that both Hebbian and homeostatic mechanisms are essential in neural learning. While Hebbian plasticity selectively modifies synaptic connectivity according to activity experienced, homeostatic plasticity constrains this change so that neural activity is always within reasonable physiological limits. Recent experiments reveal spike timing-dependent plasticity (STDP) as a new type of Hebbian learning with high time precision and heterosynaptic plasticity (HSP) as a new homeostatic mechanism acting directly on synapses. Here, we study the effect of STDP and HSP on randomly connected neural networks. Despite the reported successes of STDP to account for neural activities at the single-cell level, we find that, surprisingly, at the network level, networks trained using STDP alone cannot seem to generate realistic neural activities. For instance, STDP would stipulate that past sensory experience be maintained forever if it is no longer activated. To overcome this difficulty, motivated by the fact that HSP can induce strong competition between sensory experiences, we propose a biophysically plausible learning rule by combining STDP and HSP. Based on the Fokker-Planck theory and extensive numerical computations, we demonstrate that HSP and STDP operated on different time scales can complement each other, resulting in more realistic network activities. Our finding may provide fresh insight into the learning mechanism of the brain.

Characterization of neural interaction during learning and adaptation from spike-train data.

A basic task in understanding the neural mechanism of learning and adaptation is to detect and characterize neural interactions and their changes in response to new experiences. Recent experimental work has indicated that neural interactions in the primary motor cortex of the monkey brain tend to change their preferred directions during adaptation to an external force field. To quantify such changes, it is necessary to develop computational methodology for data analysis. Given that typical experimental data consist of spike trains recorded from individual neurons, probing the strength of neural interactions and their changes is extremely challenging. We recently reported in a brief communication [Zhu et al., Neural Computations 15, 2359 (2003)] a general procedure to detect and quantify the causal interactions among neurons, which is based on the method of directed transfer function derived from a class of multivariate, linear stochastic models. The procedure was applied to spike trains from neurons in the primary motor cortex of the monkey brain during adaptation, where monkeys were trained to learn a new skill by moving their arms to reach a target under external perturbations. Our computation and analysis indicated that the adaptation tends to alter the connection topology of the underlying neural network, yet the average interaction strength in the network is approximately conserved before and after the adaptation. The present paper gives a detailed account of this procedure and its applicability to spike-train data in terms of the hypotheses, theory, computational methods, control test, and extensive analysis of experimental data.

Capacity of oscillatory associative-memory networks with error-free retrieval.

Networks of coupled periodic oscillators (similar to the Kuramoto model) have been proposed as models of associative memory. However, error-free retrieval states of such oscillatory networks are typically unstable, resulting in a near zero capacity. This puts the networks at disadvantage as compared with the classical Hopfield network. Here we propose a simple remedy for this undesirable property and show rigorously that the error-free capacity of our oscillatory, associative-memory networks can be made as high as that of the Hopfield network. They can thus not only provide insights into the origin of biological memory, but can also be potentially useful for applications in information science and engineering.

Probing changes in neural interaction during adaptation.

A procedure is developed to probe the changes in the functional interactions among neurons in primary motor cortex of the monkey brain during adaptation. A monkey is trained to learn a new skill, moving its arm to reach a target under the influence of external perturbations. The spike trains of multiple neurons in the primary motor cortex are recorded simultaneously. We utilize the methodology of directed transfer function, derived from a class of linear stochastic models, to quantify the causal interactions between the neurons. We find that the coupling between the motor neurons tends to increase during the adaptation but return to the original level after the adaptation. Furthermore, there is evidence that adaptation tends to affect the topology of the neural network, despite the approximate conservation of the average coupling strength in the network before and after the adaptation.

Heterogeneity in oscillator networks: are smaller worlds easier to synchronize?

Small-world and scale-free networks are known to be more easily synchronized than regular lattices, which is usually attributed to the smaller network distance between oscillators. Surprisingly, we find that networks with a homogeneous distribution of connectivity are more synchronizable than heterogeneous ones, even though the average network distance is larger. We present numerical computations and analytical estimates on synchronizability of the network in terms of its heterogeneity parameters. Our results suggest that some degree of homogeneity is expected in naturally evolved structures, such as neural networks, where synchronizability is desirable.

Numerical and experimental investigation of the effect of filtering on chaotic symbolic dynamics.

Motivated by the practical consideration of the measurement of chaotic signals in experiments or the transmission of these signals through a physical medium, we investigate the effect of filtering on chaotic symbolic dynamics. We focus on the linear, time-invariant filters that are used frequently in many applications, and on the two quantities characterizing chaotic symbolic dynamics: topological entropy and bit-error rate. Theoretical consideration suggests that the topological entropy is invariant under filtering. Since computation of this entropy requires that the generating partition for defining the symbolic dynamics be known, in practical situations the computed entropy may change as a filtering parameter is changed. We find, through numerical computations and experiments with a chaotic electronic circuit, that with reasonable care the computed or measured entropy values can be preserved for a wide range of the filtering parameter.

Bursts as a unit of neural information: selective communication via resonance.

What is the functional significance of generating a burst of spikes, as opposed to a single spike? A dominant point of view is that bursts are needed to increase the reliability of communication between neurons. Here, we discuss the alternative, but complementary, hypothesis: bursts with specific resonant interspike frequencies are more likely to cause a postsynaptic cell to fire than are bursts with higher or lower frequencies. Such a frequency preference might occur at the level of individual synapses because of the interplay between short-term synaptic depression and facilitation, or at the postsynaptic cell level because of subthreshold membrane potential oscillations and resonance. As a result, the same burst could resonate for some synapses or cells and not resonate for others, depending on their natural resonance frequencies. This observation suggests that, in addition to increasing reliability of synaptic transmission, bursts of action potentials might provide effective mechanisms for selective communication between neurons.

Smallest small-world network.

Efficiency in passage times is an important issue in designing networks, such as transportation or computer networks. The small-world networks have structures that yield high efficiency, while keeping the network highly clustered. We show that among all networks with the small-world structure, the most efficient ones have a "single center" node, from which all shortcuts are connected to uniformly distributed nodes over the network. The networks with several centers and a connected subnetwork of shortcuts are shown to be "almost" as efficient. Genetic-algorithm simulations further support our results.