Impact of time-history terms on reservoir dynamics and prediction accuracy in echo state networks.

The echo state network (ESN) is an excellent machine learning model for processing time-series data. This model, utilising the response of a recurrent neural network, called a reservoir, to input signals, achieves high training efficiency. Introducing time-history terms into the neuron model of the reservoir is known to improve the time-series prediction performance of ESN, yet the reasons for this improvement have not been quantitatively explained in terms of reservoir dynamics characteristics. Therefore, we hypothesised that the performance enhancement brought about by time-history terms could be explained by delay capacity, a recently proposed metric for assessing the memory performance of reservoirs. To test this hypothesis, we conducted comparative experiments using ESN models with time-history terms, namely leaky integrator ESNs (LI-ESN) and chaotic echo state networks (ChESN). The results suggest that compared with ESNs without time-history terms, the reservoir dynamics of LI-ESN and ChESN can maintain diversity and stability while possessing higher delay capacity, leading to their superior performance. Explaining ESN performance through dynamical metrics are crucial for evaluating the numerous ESN architectures recently proposed from a general perspective and for the development of more sophisticated architectures, and this study contributes to such efforts.

Maximal Memory Capacity Near the Edge of Chaos in Balanced Cortical E-I Networks.

We examine the efficiency of information processing in a balanced excitatory and inhibitory (E-I) network during the developmental critical period, when network plasticity is heightened. A multimodule network composed of E-I neurons was defined, and its dynamics were examined by regulating the balance between their activities. When adjusting E-I activity, both transitive chaotic synchronization with a high Lyapunov dimension and conventional chaos with a low Lyapunov dimension were found. In between, the edge of high-dimensional chaos was observed. To quantify the efficiency of information processing, we applied a short-term memory task in reservoir computing to the dynamics of our network. We found that memory capacity was maximized when optimal E-I balance was realized, underscoring both its vital role and vulnerability during critical periods of brain development.

Acetylcholine-mediated top-down attention improves the response to bottom-up inputs by deformation of the attractor landscape.

To understand the effect of attention on neuronal dynamics, we propose a multi-module network, with each module consisting of fully interconnected groups of excitatory and inhibitory neurons. This network shows transitive dynamics among quasi-attractors as its typical dynamics. When the release of acetylcholine onto the network is simulated by attention, the transitive dynamics change into stable dynamics in which the system converges to an attractor. We found that this network can reproduce three experimentally observed properties of attention-dependent response modulation, namely an increase in the firing rate, a decrease in the Fano factor of the firing rate, and a decrease in the correlation coefficients between the firing rates of pairs of neurons. Moreover, we also showed theoretically that the release of acetylcholine increases the sensitivity to bottom-up inputs by changing the response function.

Quantifying Strength of Chaos in the Population Firing Rate of Neurons.

In this study, I considered quantifying the strength of chaos in the population firing rate of a pulse-coupled neural network. In particular, I considered the dynamics where the population firing rate is chaotic and the firing of each neuron is stochastic. I calculated a time histogram of firings to show the variation in the population firing rate over time. To smooth this histogram, I used Bayesian adaptive regression splines and a gaussian filter. The nonlinear prediction method, based on reconstruction, was applied to a sequence of interpeak intervals in the smoothed time histogram of firings. I propose the use of the sum of nonlinearity as a quantifier of the strength of chaos. When applying this method to the firings of a pulse-coupled neural network, the sum of nonlinearity was seen to satisfy three properties for quantifying the strength of chaos. First, it can be calculated from spiking data alone. Second, it takes large values when applied to firings that are confirmed, theoretically or numerically, to be chaotic. Third, it reflects the strength of chaos of the original dynamics.

Chaotic Pattern Alternations Can Reproduce Properties of Dominance Durations in Multistable Perception.

We propose a pulse neural network that exhibits chaotic pattern alternations among stored patterns as a model of multistable perception, which is reflected in phenomena such as binocular rivalry and perceptual ambiguity. When we regard the mixed state of patterns as a part of each pattern, the durations of the retrieved pattern obey unimodal distributions. We confirmed that no chaotic properties are observed in the time series of durations, consistent with the findings of previous psychological studies. Moreover, it is shown that our model also reproduces two properties of multistable perception that characterize the relationship between the contrast of inputs and the durations.

Deformation of attractor landscape via cholinergic presynaptic modulations: a computational study using a phase neuron model.

Corticopetal acetylcholine (ACh) is released transiently from the nucleus basalis of Meynert (NBM) into the cortical layers and is associated with top-down attention. Recent experimental data suggest that this release of ACh disinhibits layer 2/3 pyramidal neurons (PYRs) via muscarinic presynaptic effects on inhibitory synapses. Together with other possible presynaptic cholinergic effects on excitatory synapses, this may result in dynamic and temporal modifications of synapses associated with top-down attention. However, the system-level consequences and cognitive relevance of such disinhibitions are poorly understood. Herein, we propose a theoretical possibility that such transient modifications of connectivity associated with ACh release, in addition to top-down glutamatergic input, may provide a neural mechanism for the temporal reactivation of attractors as neural correlates of memories. With baseline levels of ACh, the brain returns to quasi-attractor states, exhibiting transitive dynamics between several intrinsic internal states. This suggests that top-down attention may cause the attention-induced deformations between two types of attractor landscapes: the quasi-attractor landscape (Q-landscape, present under low-ACh, non-attentional conditions) and the attractor landscape (A-landscape, present under high-ACh, top-down attentional conditions). We present a conceptual computational model based on experimental knowledge of the structure of PYRs and interneurons (INs) in cortical layers 1 and 2/3 and discuss the possible physiological implications of our results.

Rewiring-induced chaos in pulse-coupled neural networks.

The dependence of the dynamics of pulse-coupled neural networks on random rewiring of excitatory and inhibitory connections is examined. When both excitatory and inhibitory connections are rewired, periodic synchronization emerges with a Hopf-like bifurcation and a subsequent period-doubling bifurcation; chaotic synchronization is also observed. When only excitatory connections are rewired, periodic synchronization emerges with a saddle node-like bifurcation, and chaotic synchronization is also observed. This result suggests that randomness in the system does not necessarily contaminate the system, and sometimes it even introduces rich dynamics to the system such as chaos.

Roles of inhibitory neurons in rewiring-induced synchronization in pulse-coupled neural networks.

The roles of inhibitory neurons in synchronous firing are examined in a network of excitatory and inhibitory neurons with Watts and Strogatz's rewiring. By examining the persistence of the synchronous firing that exists in the random network, it was found that there is a probability of rewiring at which a transition between the synchronous state and the asynchronous state takes place, and the dynamics of the inhibitory neurons play an important role in determining this probability.

Stochastic synchrony of chaos in a pulse-coupled neural network with both chemical and electrical synapses among inhibitory neurons.

The synchronous firing of neurons in a pulse-coupled neural network composed of excitatory and inhibitory neurons is analyzed. The neurons are connected by both chemical synapses and electrical synapses among the inhibitory neurons. When electrical synapses are introduced, periodically synchronized firing as well as chaotically synchronized firing is widely observed. Moreover, we find stochastic synchrony where the ensemble-averaged dynamics shows synchronization in the network but each neuron has a low firing rate and the firing of the neurons seems to be stochastic. Stochastic synchrony of chaos corresponding to a chaotic attractor is also found.

Chaotic pattern transitions in pulse neural networks.

In models of associative memory composed of pulse neurons, chaotic pattern transitions where the pattern retrieved by the network changes chaotically were found. The network is composed of multiple modules of pulse neurons, and when the inter-module connection strength decreased, the stability of pattern retrieval changed from stable to chaotic. It was found that the mixed pattern of stored patterns plays an important role in chaotic pattern transitions.

Analysis of synchronization between two modules of pulse neural networks with excitatory and inhibitory connections.

To study the synchronized oscillations among distant neurons in the visual cortex, we analyzed the synchronization between two modules of pulse neural networks using the phase response function. It was found that the intermodule connections from excitatory to excitatory ensembles tend to stabilize the antiphase synchronization and that the intermodule connections from excitatory to inhibitory ensembles tend to stabilize the in-phase synchronization. It was also found that the intermodule synchronization was more noticeable when the inner-module synchronization was weak.

Detecting chaotic structures in noisy pulse trains based on interspike interval reconstruction.

The nonlinear prediction method based on the interspike interval (ISI) reconstruction is applied to the ISI sequence of noisy pulse trains and the detection of the deterministic structure is performed. It is found that this method cannot discriminate between the noisy periodic pulse train and the noisy chaotic one when noise-induced pulses exist. When the noise-induced pulses are eliminated by the grouping of ISI sequence with the genetic algorithm, the chaotic structure of the chaotic firings becomes clear, and the noisy chaotic pulse train could be discriminated from the periodic one.

Synchronized firings in the networks of class 1 excitable neurons with excitatory and inhibitory connections and their dependences on the forms of interactions.

Synchronized firings in the networks of class 1 excitable neurons with excitatory and inhibitory connections are investigated, and their dependences on the forms of interactions are analyzed. As the forms of interactions, we treat the double exponential coupling and the interactions derived from it: pulse coupling, exponential coupling, and alpha coupling. It is found that the bifurcation structure of the networks depends mainly on the decay time of the synaptic interaction and the effect of the rise time is smaller than that of the decay time.

An analysis of globally connected active rotators with excitatory and inhibitory connections having different time constants using the nonlinear Fokker-Planck equations.

The globally connected active rotators with excitatory and inhibitory connections having different time constants under noise are analyzed using the nonlinear Fokker-Planck equation, and their oscillatory phenomena are investigated. Based on numerically calculated bifurcation diagrams, both periodic solutions and chaotic solutions are found. The periodic firings are classified based on the firing period, the coefficient of variation, and the correlation coefficient, and weakly synchronized periodic firings which are often observed in physiological experiments are found.

Analysis of globally connected active rotators with excitatory and inhibitory connections using the Fokker-Planck equation.

The globally connected active rotators with excitatory and inhibitory connections are analyzed using the nonlinear Fokker-Planck equation. The bifurcation diagram of the system is obtained numerically, and both periodic solutions and chaotic solutions are found. By observing the interspike interval, the coefficient of variance, and the correlation coefficient of the system, the relationship of our model to the biological data is discussed.

Array-enhanced coherence resonance and forced dynamics in coupled FitzHugh-Nagumo neurons with noise.

Nonlinear dynamics of coupled FitzHugh-Nagumo neurons subject to independent noise is analyzed. A kind of self-sustained global oscillation with almost synchronous firing is generated by array-enhanced coherence resonance. Further, forced dynamics of the self-sustained global oscillation stimulated by sinusoidal input is analyzed and classified as synchronized, quasiperiodic, and chaotic responses just like the forced oscillations in nerve membranes observed by in vitro experiments with squid giant axons. Possible physiological importance of such forced oscillations is also discussed.