Optimizing memory in reservoir computers.

A reservoir computer is a way of using a high dimensional dynamical system for computation. One way to construct a reservoir computer is by connecting a set of nonlinear nodes into a network. Because the network creates feedback between nodes, the reservoir computer has memory. If the reservoir computer is to respond to an input signal in a consistent way (a necessary condition for computation), the memory must be fading; that is, the influence of the initial conditions fades over time. How long this memory lasts is important for determining how well the reservoir computer can solve a particular problem. In this paper, I describe ways to vary the length of the fading memory in reservoir computers. Tuning the memory can be important to achieve optimal results in some problems; too much or too little memory degrades the accuracy of the computation.

Low dimensional manifolds in reservoir computers.

A reservoir computer is a complex dynamical system, often created by coupling nonlinear nodes in a network. The nodes are all driven by a common driving signal. Reservoir computers can contain hundreds to thousands of nodes, resulting in a high dimensional dynamical system, but the reservoir computer variables evolve on a lower dimensional manifold in this high dimensional space. This paper describes how this manifold dimension depends on the parameters of the reservoir computer, and how the manifold dimension is related to the performance of the reservoir computer at a signal estimation task. It is demonstrated that increasing the coupling between nodes while controlling the largest Lyapunov exponent of the reservoir computer can optimize the reservoir computer performance. It is also noted that the sparsity of the reservoir computer network does not have any influence on performance.

Do reservoir computers work best at the edge of chaos?

It has been demonstrated that cellular automata had the highest computational capacity at the edge of chaos [N. H. Packard, in Dynamic Patterns in Complex Systems, edited by J. A. S. Kelso, A. J. Mandell, and M. F. Shlesinger (World Scientific, Singapore, 1988), pp. 293-301; C. G. Langton, Physica D 42(1), 12-37 (1990); J. P. Crutchfield and K. Young, in Complexity, Entropy, and the Physics of Information, edited by W. H. Zurek (Addison-Wesley, Redwood City, CA, 1990), pp. 223-269], the parameter at which their behavior transitioned from ordered to chaotic. This same concept has been applied to reservoir computers; a number of researchers have stated that the highest computational capacity for a reservoir computer is at the edge of chaos, although others have suggested that this rule is not universally true. Because many reservoir computers do not show chaotic behavior but merely become unstable, it is felt that a more accurate term for this instability transition is the "edge of stability." Here, I find two examples where the computational capacity of a reservoir computer decreases as the edge of stability is approached: in one case because generalized synchronization breaks down and in the other case because the reservoir computer is a poor match to the problem being solved. The edge of stability as an optimal operating point for a reservoir computer is not in general true, although it may be true in some cases.

Path length statistics in reservoir computers.

Because reservoir computers are high dimensional dynamical systems, designing a good reservoir computer is difficult. In many cases, the designer must search a large nonlinear parameter space, and each step of the search requires simulating the full reservoir computer. In this work, I show that a simple statistic based on the mean path length between nodes in the reservoir computer is correlated with better reservoir computer performance. The statistic predicts the diversity of signals produced by the reservoir computer, as measured by the covariance matrix of the reservoir computer. This statistic by itself is not sufficient to predict reservoir computer performance because not only must the reservoir computer produce a diverse set of signals, it must be well matched to the training signals. Nevertheless, this path length statistic allows the designer to eliminate some network configurations from consideration without having to actually simulate the reservoir computer, reducing the complexity of the design process.

Dimension of reservoir computers.

A reservoir computer is a complex dynamical system, often created by coupling nonlinear nodes in a network. The nodes are all driven by a common driving signal. In this work, three dimension estimation methods, false nearest neighbor, covariance dimension, and Kaplan-Yorke dimension, are used to estimate the dimension of the reservoir dynamical system. It is shown that the signals in the reservoir system exist on a relatively low dimensional surface. Changing the spectral radius of the reservoir network can increase the fractal dimension of the reservoir signals, leading to an increase in a testing error.

Network structure effects in reservoir computers.

A reservoir computer is a complex nonlinear dynamical system that has been shown to be useful for solving certain problems, such as prediction of chaotic signals, speech recognition, or control of robotic systems. Typically, a reservoir computer is constructed by connecting a large number of nonlinear nodes in a network, driving the nodes with an input signal and using the node outputs to fit a training signal. In this work, we set up reservoirs where the edges (or connections) between all the network nodes are either +1 or 0 and proceed to alter the network structure by flipping some of these edges from +1 to -1. We use this simple network because it turns out to be easy to characterize; we may use the fraction of edges flipped as a measure of how much we have altered the network. In some cases, the network can be rearranged in a finite number of ways without changing its structure; these rearrangements are symmetries of the network, and the number of symmetries is also useful for characterizing the network. We find that changing the number of edges flipped in the network changes the rank of the covariance of a matrix consisting of the time series from the different nodes in the network and speculate that this rank is important for understanding the reservoir computer performance.

Testing dynamical system variables for reconstruction.

Analyzing data from dynamical systems often begins with creating a reconstruction of the trajectory based on one or more variables, but not all variables are suitable for reconstructing the trajectory. The concept of nonlinear observability has been investigated as a way to determine if a dynamical system can be reconstructed from one signal or a combination of signals [L. A. Aguirre, IEEE Trans. Educ. 38, 33 (1995); C. Letellier, L. A. Aguirre, and J. Maquet, Phys. Rev. E 71, 066213 (2005); L. A. Aguirre, S. B. Bastos, M. A. Alves, and C. Letellier, Chaos 18, 013123 (2008); L. A. Aguirre and C. Letellier, Phys. Rev. E 83, 066209 (2011); and E. Bianco-Martinez, M. S. Baptista, and C. Letellier, Phys. Rev. E 91, 062912 (2015)]; however, nonlinear observability can be difficult to calculate for a high dimensional system. In this work, I compare the results from nonlinear observability to a continuity statistic that indicates the likelihood that there is a continuous function between two sets of multidimensional points-in this case, two different reconstructions of the same attractor from different signals are simultaneously measured. Without a metric against which to test the ability to reconstruct a system, the predictions of nonlinear observability and continuity are ambiguous. As an additional test on how well different signals can predict the ability to reconstruct a dynamical system, I use the fitting error from training a reservoir computer.

Dimension from covariance matrices.

We describe a method to estimate embedding dimension from a time series. This method includes an estimate of the probability that the dimension estimate is valid. Such validity estimates are not common in algorithms for calculating the properties of dynamical systems. The algorithm described here compares the eigenvalues of covariance matrices created from an embedded signal to the eigenvalues for a covariance matrix of a Gaussian random process with the same dimension and number of points. A statistical test gives the probability that the eigenvalues for the embedded signal did not come from the Gaussian random process.

Chaotic sequences for noisy environments.

There have been many attempts to apply chaotic signals to communications or radar, but one obstacle has been that there is no effective way to recover chaotic signals from noise larger than the signal. In this work, we create "pseudo-chaotic" signals by concatenating dictionary sequences generated from a chaotic attractor. Because the number of dictionary sequences is finite, these pseudo-chaotic signals are not actually chaotic, but they can still contain some of the desirable properties of chaos. Using dictionary sequences allows the pseudo-chaotic signal to be recovered from noise using a correlation detector and a Viterbi decoder, so the signal can be recovered from noise or interference that is larger than the signal itself.

Grid-based partitioning for comparing attractors.

Stationary dynamical systems have invariant measures (or densities) that are characteristic of the particular dynamical system. We develop a method to characterize this density by partitioning the attractor into the smallest regions in phase space that contain information about the structure of the attractor. To accomplish this, we develop a statistic that tells us if we get more information about our data by dividing a set of data points into partitions rather than just lumping all the points together. We use this method to show that not only can we detect small changes in an attractor from a circuit experiment, but we can also distinguish between a large set of numerically generated chaotic attractors designed by Sprott. These comparisons are not limited to chaotic attractors-they should work for signals from any finite-dimensional dynamical system.

Attractor comparisons based on density.

Recognizing a chaotic attractor can be seen as a problem in pattern recognition. Some feature vector must be extracted from the attractor and used to compare to other attractors. The field of machine learning has many methods for extracting feature vectors, including clustering methods, decision trees, support vector machines, and many others. In this work, feature vectors are created by representing the attractor as a density in phase space and creating polynomials based on this density. Density is useful in itself because it is a one dimensional function of phase space position, but representing an attractor as a density is also a way to reduce the size of a large data set before analyzing it with graph theory methods, which can be computationally intensive. The density computation in this paper is also fast to execute. In this paper, as a demonstration of the usefulness of density, the density is used directly to construct phase space polynomials for comparing attractors. Comparisons between attractors could be useful for tracking changes in an experiment when the underlying equations are too complicated for vector field modeling.

Using filtering effects to identify objects.

Reflecting signals off of targets is a method widely used to locate objects, but the reflected signal also contains information that can be used to identify the object. In radar or sonar, the signal amplitudes used are small enough that only linear effects are present, so we can consider the effect of the target on the signal as a linear filter. Using the known effects of linear filters on chaotic signals, we can create a reference that allows us to match a particular target to a particular reflected signal. Furthermore, if some parts of this "filter" vary only slowly as the aspect angle of the object changes, we can produce a reference that averages out the parts that are highly angle dependent so that one reference can be used to identify the target over a range of angles.

Detecting variation in chaotic attractors.

If the output of an experiment is a chaotic signal, it may be useful to detect small changes in the signal, but there are a limited number of ways to compare signals from chaotic systems, and most known methods are not robust in the presence of noise. One may calculate dimension or Lyapunov exponents from the signal, or construct a synchronizing model, but all of these are only useful in low noise situations. I introduce a method for detecting small variations in a chaotic attractor based on directly calculating the difference between vector fields in phase space. The differences are found by comparing close strands in phase space, rather than close neighbors. The use of strands makes the method more robust to noise and more sensitive to small attractor differences.

Detecting recursive and nonrecursive filters using chaos.

Filtering a chaotic signal through a recursive [or infinite impulse response (IIR)] filter has been shown to increase the dimension of chaos under certain conditions. Filtering with a nonrecursive [or finite impulse response (FIR)] filter should not increase dimension, but it has been shown that if the FIR filter has a long tail, measurements of actual signals may appear to show a dimension increase. I simulate IIR and FIR filters that correspond to naturally occurring resonant objects, and I show that using dimension measurements, I can distinguish the filter type. These measurements could be used to detect resonances using radar, sonar, or laser signals, or to determine if a resonance is due to an IIR or an FIR filter.

Phase space correlation to improve detection accuracy.

The standard method used for detecting signals in radar or sonar is cross correlation. The accuracy of the detection with cross correlation is limited by the bandwidth of the signals. We show that by calculating the cross correlation based on points that are nearby in phase space rather than points that are simultaneous in time, the detection accuracy is improved. The phase space correlation technique works for some standard radar signals, but it is especially well suited to chaotic signals because trajectories that are adjacent in phase space move apart from each other at an exponential rate.

Phase space method for identification of driven nonlinear systems.

We seek in this paper to differentiate driven nonlinear systems using only a single output signal from the driven system. We do not have access to the driving signal. We demonstrate the phase space identification techniques with an experimental model of a radio transmitter. We restrict the driving signals to nearly periodic signals, because these types of signals are the most common signals used in real transmitters. We find that by studying our transmitter as a driven nonlinear system, we are able to distinguish one transmitter from another. This work may have consequences for real transmitters.

Optimizing chaos-based signals for complex radar targets.

There has been interest in the use of chaotic signals for radar, but most researchers consider only a few chaotic systems and how these signals perform for the detection of point targets. The range of possible chaotic signals is far greater than what most of these researchers consider, so to demonstrate this, I use a chaotic map whose parameters may be adjusted by a numerical optimization routine, producing different chaotic signals that are modulated onto a carrier and optimized for different situations. It is also suggested that any advantage for these chaos-based signals may come in the detection of complex targets, not point targets, and I compare the performance of chaos-based signals to a standard radar signal, the linear frequency modulated chirp. I find that I can optimize a chaos-based signal to increase the cross-correlation with the reflection from one complex target compared to the cross-correlation with the reflection from a different target, thus allowing the identification of a complex target. I am also able to increase the cross-correlation of the reflection from a complex target compared with the cross-correlation with the reflection from spatially extended clutter. I show that a larger output signal-to-noise ratio is possible if I cross-correlate with a reference signal that is different from the transmitted signal, and I justify my results by showing how the ambiguity diagram for a chaos-based signal can be different than the ambiguity diagram for a noise signal.

A nonlinear dynamics method for signal identification.

When a radio frequency signal is radiated by a transmitter, the properties of the transmitter itself affect the properties of the signal. These transmitter-induced changes are known as unintentional modulation, to differentiate them from intentional modulation used to add information to the signal. The unintentional modulation can be used to identify which transmitter produced a signal. This paper shows how phase space analysis based on nonlinear dynamics ideas can be used to determine which of several amplifiers produced a signal.

The role of pigmentation, ultraviolet radiation tolerance, and leaf colonization strategies in the epiphytic survival of phyllosphere bacteria.

Phenotypic mechanisms that enhance bacterial UVR survival typically include pigmentation and DNA repair mechanisms which provide protection from UVA and UVB wavelengths, respectively. In this study, we examined the contribution of pigmentation to field survival in Clavibacter michiganensis and evaluated differences in population dynamics and leaf colonization strategies. Two C. michiganensis pigment-deficient mutants were significantly reduced in UVA radiation survival in vitro; one of these mutants also exhibited reduced field populations on peanut when compared to the wild-type strain over the course of replicate 25-day experiments. The UVR-tolerant C. michiganensis strains G7.1 and G11.1 maintained larger epiphytic field populations on peanut compared to the UVR-sensitive C. michiganensis T5.1. Epiphytic field populations of C. michiganensis utilized the strategy of solar UVR avoidance during leaf colonization resulting in increased strain survival on leaves after UVC irradiation. These results further demonstrate the importance of UVR tolerance in the ability of bacterial strains to maintain population size in the phyllosphere. However, an examination of several bacterial species from the peanut phyllosphere and a collection of environmental Pseudomonas spp. revealed that sensitivity to UVA and UVC radiation was correlated in some but not all of these bacteria. These results underscore a need to further understand the biological effects of different solar wavelength groups on microbial ecology.

Chaotic control and synchronization for system identification.

Research into applications of synchronized chaotic systems assumes that it will be necessary to build many different drive-response pairs, but little is known in general about designing higher dimensional chaotic flows. In this paper, I do not add any design techniques, but I show that it is possible to create multiple drive-response pairs from one chaotic system by applying chaos control techniques to the drive and response systems. If one can design one chaotic system with the desired properties, then many drive-response pairs can be built from this system, so that it is not necessary to solve the design problem more than once. I show both numerical simulations and experimental work with chaotic circuits. I also test the response systems for ability to overcome noise or other interference.