Hybridizing traditional and next-generation reservoir computing to accurately and efficiently forecast dynamical systems.

Reservoir computers (RCs) are powerful machine learning architectures for time series prediction. Recently, next generation reservoir computers (NGRCs) have been introduced, offering distinct advantages over RCs, such as reduced computational expense and lower training data requirements. However, NGRCs have their own practical difficulties, including sensitivity to sampling time and type of nonlinearities in the data. Here, we introduce a hybrid RC-NGRC approach for time series forecasting of dynamical systems. We show that our hybrid approach can produce accurate short-term predictions and capture the long-term statistics of chaotic dynamical systems in situations where the RC and NGRC components alone are insufficient, e.g., due to constraints from limited computational resources, sub-optimal hyperparameters, sparsely sampled training data, etc. Under these conditions, we show for multiple model chaotic systems that the hybrid RC-NGRC method with a small reservoir can achieve prediction performance approaching that of a traditional RC with a much larger reservoir, illustrating that the hybrid approach can offer significant gains in computational efficiency over traditional RCs while simultaneously addressing some of the limitations of NGRCs. Our results suggest that the hybrid RC-NGRC approach may be particularly beneficial in cases when computational efficiency is a high priority and an NGRC alone is not adequate.

Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics.

We examine the efficiency of Recurrent Neural Networks in forecasting the spatiotemporal dynamics of high dimensional and reduced order complex systems using Reservoir Computing (RC) and Backpropagation through time (BPTT) for gated network architectures. We highlight advantages and limitations of each method and discuss their implementation for parallel computing architectures. We quantify the relative prediction accuracy of these algorithms for the long-term forecasting of chaotic systems using as benchmarks the Lorenz-96 and the Kuramoto-Sivashinsky (KS) equations. We find that, when the full state dynamics are available for training, RC outperforms BPTT approaches in terms of predictive performance and in capturing of the long-term statistics, while at the same time requiring much less training time. However, in the case of reduced order data, large scale RC models can be unstable and more likely than the BPTT algorithms to diverge. In contrast, RNNs trained via BPTT show superior forecasting abilities and capture well the dynamics of reduced order systems. Furthermore, the present study quantifies for the first time the Lyapunov Spectrum of the KS equation with BPTT, achieving similar accuracy as RC. This study establishes that RNNs are a potent computational framework for the learning and forecasting of complex spatiotemporal systems.

Analysis of multiple physical parameters for mechanical phenotyping of living cells.

Since the cytoskeleton is known to regulate many cell functions, an increasing amount of effort to characterize cells by their mechanical properties has occured. Despite the structural complexity and dynamics of the multicomponent cytoskeleton, mechanical measurements on single cells are often fit to simple models with two to three parameters, and those parameters are recorded and reported. However, different simple models are likely needed to capture the distinct mechanical cell states, and additional parameters may be needed to capture the ability of cells to actively deform. Our new approach is to capture a much larger set of possibly redundant parameters from cells' mechanical measurement using multiple rheological models as well as dynamic deformation and image data. Principal component analysis and network-based approaches are used to group parameters to reduce redundancies and develop robust biomechanical phenotyping. Network representation of parameters allows for visual exploration of cells' complex mechanical system, and highlights unexpected connections between parameters. To demonstrate that our biomechanical phenotyping approach can detect subtle mechanical differences, we used a Microfluidic Optical Cell Stretcher to mechanically stretch circulating human breast tumor cells bearing genetically-engineered alterations in c-src tyrosine kinase activation, which is known to influence reattachment and invasion during metastasis.

Robustness of network measures to link errors.

In various applications involving complex networks, network measures are employed to assess the relative importance of network nodes. However, the robustness of such measures in the presence of link inaccuracies has not been well characterized. Here we present two simple stochastic models of false and missing links and study the effect of link errors on three commonly used node centrality measures: degree centrality, betweenness centrality, and dynamical importance. We perform numerical simulations to assess robustness of these three centrality measures. We also develop an analytical theory, which we compare with our simulations, obtaining very good agreement.

Path to fracture in granular flows: dynamics of contact networks.

Capturing the dynamics of granular flows at intermediate length scales can often be difficult. We propose studying the dynamics of contact networks as a new tool to study fracture at intermediate scales. Using experimental three-dimensional flow fields with particle-scale resolution, we calculate the time evolving broken-links network and find that a giant component of this network is formed as shear is applied to this system. We implement a model of link breakages where the probability of a link breaking is proportional to the average rate of longitudinal strain (elongation) in the direction of the edge and find that the model demonstrates qualitative agreement with the data when studying the onset of the giant component. We note, however, that the broken-links network formed in the model is less clustered than our experimental observations, indicating that the model reflects less localized breakage events and does not fully capture the dynamics of the granular flow.

External periodic driving of large systems of globally coupled phase oscillators.

Large systems of coupled oscillators subjected to a periodic external drive occur in many situations in physics and biology. Here the simple paradigmatic case of equal strength, all-to-all sine coupling of phase oscillators subject to a sinusoidal external drive, is considered. The stationary states and their stability are determined. Using the stability information and numerical experiments, parameter space phase diagrams showing when different types of system behavior apply are constructed, and the bifurcations marking transitions between different types of behavior are delineated. The analysis is supported by results of direct numerical simulation of an ensemble of oscillators.

Bacterial diversity promotes community stability and functional resilience after perturbation.

The relationships between bacterial community diversity and stability were investigated by perturbing soils, with naturally differing levels of diversity, to equivalent toxicity using copper sulfate and benzene. Benzene amendment led to large decreases in total bacterial numbers and biomass in both soils. Benzene amendment of an organo-mineral/improved pasture soil altered total soil bacterial community structure but, unlike amendment of the mineral/arable soil, maintained genetic diversity, based on polymerase chain reaction-denaturing gradient gel electrophoresis (PCR-DGGE) analysis targeting DNA and RNA, until week 9 of the perturbation experiment. Assuming equivalent toxicity, the genetic diversity of the naturally more diverse soil was more resistant to benzene perturbation than the less diverse soil. The broad scale function (mineralization of 14C-labelled wheat shoot) of both benzene- and copper-treated soil communities was unaffected. However, narrow niche function (mineralization of 14C-labelled 2,4-dichlorophenol) was impaired for both benzene-polluted soils. The organo-mineral soil recovered this function by the end of the experiment but the mineral soil did not, suggesting greater resilience in the more diverse soil. Despite a large reduction in bacterial numbers and biomass in the copper-treated soils, only small differences in bacterial community diversity were observed by week 9 in the copper-polluted soils. The overall community structure was little altered and functionality, measured by mineralization rates, remained unchanged. This suggested a non-selective pressure and a degree of genetic and functional resistance to copper perturbation, despite a significant reduction in bacterial numbers and biomass. However, initial shifts in physiological profiles of both copper-polluted soils were observed but rapidly returned to those of the controls. This apparent functional recovery, accompanied by an increase in culturability, possibly reflects adaptation by the surviving communities to perturbation. The findings indicate that, although soil communities may be robust, relationships between diversity and stability need to be considered in developing a predictive understanding of response to environmental perturbations.

Soil microbial community response to land use change in an agricultural landscape of western Kenya.

Tropical agroecosystems are subject to degradation processes such as losses in soil carbon, nutrient depletion, and reduced water holding capacity that occur rapidly resulting in a reduction in soil fertility that can be difficult to reverse. In this research, a polyphasic methodology has been used to investigate changes in microbial community structure and function in a series of tropical soils in western Kenya. These soils have different land usage with both wooded and agricultural soils at Kakamega and Ochinga, whereas at Ochinga, Leuro, Teso, and Ugunja a replicated field experiment compared traditional continuous maize cropping against an improved N-fixing fallow system. For all sites, principal component analysis of 16S rRNA gene denaturing gradient gel electrophoresis (DGGE) profiles revealed that soil type was the key determinant of total bacterial community structure, with secondary variation found between wooded and agricultural soils. Similarly, phospholipid fatty acid (PLFA) analysis also separated wooded from agricultural soils, primarily on the basis of higher abundance of monounsaturated fatty acids, anteiso- and iso-branched fatty acids, and methyl-branched fatty acids in the wooded soils. At Kakamega and Ochinga wooded soils had between five 5 and 10-fold higher levels of soil carbon and microbial biomass carbon than agricultural soils from the same location, whereas total enzyme activities were also lower in the agricultural sites. Soils with woody vegetation had a lower percentage of phosphatase activity and higher cellulase and chitinase activities than the agricultural soils. BIOLOG analysis showed woodland soils to have the greatest substrate diversity. Throughout the study the two functional indicators (enzyme activity and BIOLOG), however, showed lower specificity with respect to soil type and land usage than did the compositional indicators (DGGE and PLFA). In the field experiment comparing two types of maize cropping, both the maize yields and total microbial biomass were found to increase with the fallow system. Moreover, 16S rRNA gene and PLFA analyses revealed shifts in the total microbial community in response to the different management regimes, indicating that deliberate management of soils can have considerable impact on microbial community structure and function in tropical soils.

Finding and evaluating community structure in networks.

We propose and study a set of algorithms for discovering community structure in networks-natural divisions of network nodes into densely connected subgroups. Our algorithms all share two definitive features: first, they involve iterative removal of edges from the network to split it into communities, the edges removed being identified using any one of a number of possible "betweenness" measures, and second, these measures are, crucially, recalculated after each removal. We also propose a measure for the strength of the community structure found by our algorithms, which gives us an objective metric for choosing the number of communities into which a network should be divided. We demonstrate that our algorithms are highly effective at discovering community structure in both computer-generated and real-world network data, and show how they can be used to shed light on the sometimes dauntingly complex structure of networked systems.

Community structure in social and biological networks.

A number of recent studies have focused on the statistical properties of networked systems such as social networks and the Worldwide Web. Researchers have concentrated particularly on a few properties that seem to be common to many networks: the small-world property, power-law degree distributions, and network transitivity. In this article, we highlight another property that is found in many networks, the property of community structure, in which network nodes are joined together in tightly knit groups, between which there are only looser connections. We propose a method for detecting such communities, built around the idea of using centrality indices to find community boundaries. We test our method on computer-generated and real-world graphs whose community structure is already known and find that the method detects this known structure with high sensitivity and reliability. We also apply the method to two networks whose community structure is not well known--a collaboration network and a food web--and find that it detects significant and informative community divisions in both cases.

Structure of growing social networks.

We propose some simple models of the growth of social networks, based on three general principles: (1). meetings take place between pairs of individuals at a rate that is high if a pair has one or more mutual friends and low otherwise; (2). acquaintances between pairs of individuals who rarely meet decay over time; (3). there is an upper limit on the number of friendships an individual can maintain. Using computer simulations, we find that models that incorporate all of these features reproduce many of the features of real social networks, including high levels of clustering or network transitivity and strong community structure in which individuals have more links to others within their community than to individuals from other communities.