Black and gray box learning of amplitude equations: Application to phase field systems.

We present a data-driven approach to learning surrogate models for amplitude equations and illustrate its application to interfacial dynamics of phase field systems. In particular, we demonstrate learning effective partial differential equations describing the evolution of phase field interfaces from full phase field data. We illustrate this on a model phase field system, where analytical approximate equations for the dynamics of the phase field interface (a higher-order eikonal equation and its approximation, the Kardar-Parisi-Zhang equation) are known. For this system, we discuss data-driven approaches for the identification of equations that accurately describe the front interface dynamics. When the analytical approximate models mentioned above become inaccurate, as we move beyond the region of validity of the underlying assumptions, the data-driven equations outperform them. In these regimes, going beyond black box identification, we explore different approaches to learning data-driven corrections to the analytically approximate models, leading to effective gray box partial differential equations.

Learning emergent partial differential equations in a learned emergent space.

We propose an approach to learn effective evolution equations for large systems of interacting agents. This is demonstrated on two examples, a well-studied system of coupled normal form oscillators and a biologically motivated example of coupled Hodgkin-Huxley-like neurons. For such types of systems there is no obvious space coordinate in which to learn effective evolution laws in the form of partial differential equations. In our approach, we accomplish this by learning embedding coordinates from the time series data of the system using manifold learning as a first step. In these emergent coordinates, we then show how one can learn effective partial differential equations, using neural networks, that do not only reproduce the dynamics of the oscillator ensemble, but also capture the collective bifurcations when system parameters vary. The proposed approach thus integrates the automatic, data-driven extraction of emergent space coordinates parametrizing the agent dynamics, with machine-learning assisted identification of an emergent PDE description of the dynamics in this parametrization.

An explanation of how mutant and wild-type mitochondria might stably co-exist in inherited mitochondrial diseases.

Mitochondria are cellular organelles of crucial relevance for the survival of metazoan organisms. Damage to the mitochondrial DNA can give rise to a variety of mitochondrial diseases and is thought also to be involved in the aging process. The fate of mtDNA mutants is controlled by their synthesis as well as degradation and mathematical models can help to better understand this complex interplay. We present here a model that combines a replicative advantage for mtDNA mutants with selective degradation enabled by mitochondrial fission and fusion processes. The model not only shows that the cell has efficient means to deal with (many) types of mutants but, surprisingly, also predicts that under certain conditions a stable co-existence of mutant and wild-type mtDNAs is possible. We discuss how this new finding might explain how mitochondria can be at the heart of processes with such different phenotypes as mitochondrial diseases and aging.

Initializing LSTM internal states via manifold learning.

We present an approach, based on learning an intrinsic data manifold, for the initialization of the internal state values of long short-term memory (LSTM) recurrent neural networks, ensuring consistency with the initial observed input data. Exploiting the generalized synchronization concept, we argue that the converged, "mature" internal states constitute a function on this learned manifold. The dimension of this manifold then dictates the length of observed input time series data required for consistent initialization. We illustrate our approach through a partially observed chemical model system, where initializing the internal LSTM states in this fashion yields visibly improved performance. Finally, we show that learning this data manifold enables the transformation of partially observed dynamics into fully observed ones, facilitating alternative identification paths for nonlinear dynamical systems.

Global and local reduced models for interacting, heterogeneous agents.

Large collections of coupled, heterogeneous agents can manifest complex dynamical behavior presenting difficulties for simulation and analysis. However, if the collective dynamics lie on a low-dimensional manifold, then the original agent-based model may be approximated with a simplified surrogate model on and near the low-dimensional space where the dynamics live. Analytically identifying such simplified models can be challenging or impossible, but here we present a data-driven coarse-graining methodology for discovering such reduced models. We consider two types of reduced models: globally based models that use global information and predict dynamics using information from the whole ensemble and locally based models that use local information, that is, information from just a subset of agents close (close in heterogeneity space, not physical space) to an agent, to predict the dynamics of an agent. For both approaches, we are able to learn laws governing the behavior of the reduced system on the low-dimensional manifold directly from time series of states from the agent-based system. These laws take the form of either a system of ordinary differential equations (ODEs), for the globally based approach, or a partial differential equation (PDE) in the locally based case. For each technique, we employ a specialized artificial neural network integrator that has been templated on an Euler time stepper (i.e., a ResNet) to learn the laws of the reduced model. As part of our methodology, we utilize the proper orthogonal decomposition (POD) to identify the low-dimensional space of the dynamics. Our globally based technique uses the resulting POD basis to define a set of coordinates for the agent states in this space and then seeks to learn the time evolution of these coordinates as a system of ODEs. For the locally based technique, we propose a methodology for learning a partial differential equation representation of the agents; the PDE law depends on the state variables and partial derivatives of the state variables with respect to model heterogeneities. We require that the state variables are smooth with respect to model heterogeneities, which permit us to cast the discrete agent-based problem as a continuous one in heterogeneity space. The agents in such a representation bear similarity to the discretization points used in typical finite element/volume methods. As an illustration of the efficacy of our techniques, we consider a simplified coupled neuron model for rhythmic oscillations in the pre-Bötzinger complex and demonstrate how our data-driven surrogate models are able to produce dynamics comparable to the dynamics of the full system. A nontrivial conclusion is that the dynamics can be equally well reproduced by an all-to-all coupled and by a locally coupled model of the same agents.

Lyapunov spectra and collective modes of chimera states in globally coupled Stuart-Landau oscillators.

Oscillatory systems with long-range or global coupling offer promising insight into the interplay between high-dimensional (or microscopic) chaotic motion and collective interaction patterns. Within this paper, we use Lyapunov analysis to investigate whether chimera states in globally coupled Stuart-Landau (SL) oscillators exhibit collective degrees of freedom. We compare two types of chimera states, which emerge in SL ensembles with linear and nonlinear global coupling, respectively, the latter introducing a constraint that conserves the oscillation of the mean. Lyapunov spectra reveal that for both chimera states the Lyapunov exponents split into several groups with different convergence properties in the limit of large system size. Furthermore, in both cases the Lyapunov dimension is found to scale extensively and the localization properties of covariant Lypunov vectors manifest the presence of collective Lyapunov modes. Here, however, we find qualitative differences between the two types of chimera states: Whereas the ones in the system under nonlinear global coupling exhibit only slow collective modes corresponding to Lyapunov exponents equal or close to zero, those which experience the linear mean-field coupling exhibit also faster collective modes associated with Lyapunov exponents with large positive or negative values. Furthermore, for the fastest collective mode we showed that it spreads across both synchonous and incoherent oscillators.

Cluster singularity: The unfolding of clustering behavior in globally coupled Stuart-Landau oscillators.

The ubiquitous occurrence of cluster patterns in nature still lacks a comprehensive understanding. It is known that the dynamics of many such natural systems is captured by ensembles of Stuart-Landau oscillators. Here, we investigate clustering dynamics in a mean-coupled ensemble of such limit-cycle oscillators. In particular, we show how clustering occurs in minimal networks and elaborate how the observed 2-cluster states crowd when increasing the number of oscillators. Using persistence, we discuss how this crowding leads to a continuous transition from balanced cluster states to synchronized solutions via the intermediate unbalanced 2-cluster states. These cascade-like transitions emerge from what we call a cluster singularity. At this codimension-2 point, the bifurcations of all 2-cluster states collapse and the stable balanced cluster state bifurcates into the synchronized solution supercritically. We confirm our results using numerical simulations and discuss how our conclusions apply to spatially extended systems.

Symmetries of Chimera States.

Symmetry broken states arise naturally in oscillatory networks. In this Letter, we investigate chaotic attractors in an ensemble of four mean-coupled Stuart-Landau oscillators with two oscillators being synchronized. We report that these states with partially broken symmetry, so-called chimera states, have different setwise symmetries in the incoherent oscillators, and in particular, some are and some are not invariant under a permutation symmetry on average. This allows for a classification of different chimera states in small networks. We conclude our report with a discussion of related states in spatially extended systems, which seem to inherit the symmetry properties of their counterparts in small networks.

An Emergent Space for Distributed Data with Hidden Internal Order through Manifold Learning.

Manifold-learning techniques are routinely used in mining complex spatiotemporal data to extract useful, parsimonious data representations/parametrizations; these are, in turn, useful in nonlinear model identification tasks. We focus here on the case of time series data that can ultimately be modelled as a spatially distributed system (e.g. a partial differential equation, PDE), but where we do not know the space in which this PDE should be formulated. Hence, even the spatial coordinates for the distributed system themselves need to be identified - to "emerge from"-the data mining process. We will first validate this "emergent space" reconstruction for time series sampled without space labels in known PDEs; this brings up the issue of observability of physical space from temporal observation data, and the transition from spatially resolved to lumped (order-parameter-based) representations by tuning the scale of the data mining kernels. We will then present actual emergent space "discovery" illustrations. Our illustrative examples include chimera states (states of coexisting coherent and incoherent dynamics), and chaotic as well as quasiperiodic spatiotemporal dynamics, arising in partial differential equations and/or in heterogeneous networks. We also discuss how data-driven "spatial" coordinates can be extracted in ways invariant to the nature of the measuring instrument. Such gauge-invariant data mining can go beyond the fusion of heterogeneous observations of the same system, to the possible matching of apparently different systems. For an older version of this article, including other examples, see https://arxiv.org/abs/1708.05406.

A classification scheme for chimera states.

We present a universal characterization scheme for chimera states applicable to both numerical and experimental data sets. The scheme is based on two correlation measures that enable a meaningful definition of chimera states as well as their classification into three categories: stationary, turbulent, and breathing. In addition, these categories can be further subdivided according to the time-stationarity of these two measures. We demonstrate that this approach is both consistent with previously recognized chimera states and enables us to classify states as chimeras which have not been categorized as such before. Furthermore, the scheme allows for a qualitative and quantitative comparison of experimental chimeras with chimeras obtained through numerical simulations.