Reconstruction, forecasting, and stability of chaotic dynamics from partial data.

The forecasting and computation of the stability of chaotic systems from partial observations are tasks for which traditional equation-based methods may not be suitable. In this computational paper, we propose data-driven methods to (i) infer the dynamics of unobserved (hidden) chaotic variables (full-state reconstruction); (ii) time forecast the evolution of the full state; and (iii) infer the stability properties of the full state. The tasks are performed with long short-term memory (LSTM) networks, which are trained with observations (data) limited to only part of the state: (i) the low-to-high resolution LSTM (LH-LSTM), which takes partial observations as training input, and requires access to the full system state when computing the loss; and (ii) the physics-informed LSTM (PI-LSTM), which is designed to combine partial observations with the integral formulation of the dynamical system's evolution equations. First, we derive the Jacobian of the LSTMs. Second, we analyze a chaotic partial differential equation, the Kuramoto-Sivashinsky, and the Lorenz-96 system. We show that the proposed networks can forecast the hidden variables, both time-accurately and statistically. The Lyapunov exponents and covariant Lyapunov vectors, which characterize the stability of the chaotic attractors, are correctly inferred from partial observations. Third, the PI-LSTM outperforms the LH-LSTM by successfully reconstructing the hidden chaotic dynamics when the input dimension is smaller or similar to the Kaplan-Yorke dimension of the attractor. The performance is also analyzed against noisy data. This work opens new opportunities for reconstructing the full state, inferring hidden variables, and computing the stability of chaotic systems from partial data.

Multimodal Batch-Wise Change Detection.

We address the problem of detecting distribution changes in a novel batch-wise and multimodal setup. This setup is characterized by a stationary condition where batches are drawn from potentially different modalities among a set of distributions in [Formula: see text] represented in the training set. Existing change detection (CD) algorithms assume that there is a unique-possibly multipeaked-distribution characterizing stationary conditions, and in batch-wise multimodal context exhibit either low detection power or poor control of false positives. We present MultiModal QuantTree (MMQT), a novel CD algorithm that uses a single histogram to model the batch-wise multimodal stationary conditions. During testing, MMQT automatically identifies which modality has generated the incoming batch and detects changes by means of a modality-specific statistic. We leverage the theoretical properties of QuantTree to: 1) automatically estimate the number of modalities in a training set and 2) derive a principled calibration procedure that guarantees false-positive control. Our experiments show that MMQT achieves high detection power and accurate control over false positives in synthetic and real-world multimodal CD problems. Moreover, we show the potential of MMQT in Stream Learning applications, where it proves effective at detecting concept drifts and the emergence of novel classes by solely monitoring the input distribution.

Stability analysis of chaotic systems from data.

The prediction of the temporal dynamics of chaotic systems is challenging because infinitesimal perturbations grow exponentially. The analysis of the dynamics of infinitesimal perturbations is the subject of stability analysis. In stability analysis, we linearize the equations of the dynamical system around a reference point and compute the properties of the tangent space (i.e. the Jacobian). The main goal of this paper is to propose a method that infers the Jacobian, thus, the stability properties, from observables (data). First, we propose the echo state network (ESN) with the Recycle validation as a tool to accurately infer the chaotic dynamics from data. Second, we mathematically derive the Jacobian of the echo state network, which provides the evolution of infinitesimal perturbations. Third, we analyse the stability properties of the Jacobian inferred from the ESN and compare them with the benchmark results obtained by linearizing the equations. The ESN correctly infers the nonlinear solution and its tangent space with negligible numerical errors. In detail, we compute from data only (i) the long-term statistics of the chaotic state; (ii) the covariant Lyapunov vectors; (iii) the Lyapunov spectrum; (iv) the finite-time Lyapunov exponents; (v) and the angles between the stable, neutral, and unstable splittings of the tangent space (the degree of hyperbolicity of the attractor). This work opens up new opportunities for the computation of stability properties of nonlinear systems from data, instead of equations. Supplementary Information: The online version contains supplementary material available at 10.1007/s11071-023-08285-1.

Robust Optimization and Validation of Echo State Networks for learning chaotic dynamics.

An approach to the time-accurate prediction of chaotic solutions is by learning temporal patterns from data. Echo State Networks (ESNs), which are a class of Reservoir Computing, can accurately predict the chaotic dynamics well beyond the predictability time. Existing studies, however, also showed that small changes in the hyperparameters may markedly affect the network's performance. The overarching aim of this paper is to improve the robustness in the selection of hyperparameters in Echo State Networks for the time-accurate prediction of chaotic solutions. We define the robustness of a validation strategy as its ability to select hyperparameters that perform consistently between validation and test sets. The goal is three-fold. First, we investigate routinely used validation strategies. Second, we propose the Recycle Validation, and the chaotic versions of existing validation strategies, to specifically tackle the forecasting of chaotic systems. Third, we compare Bayesian optimization with the traditional grid search for optimal hyperparameter selection. Numerical tests are performed on prototypical nonlinear systems that have chaotic and quasiperiodic solutions, such as the Lorenz and Lorenz-96 systems, and the Kuznetsov oscillator. Both model-free and model-informed Echo State Networks are analysed. By comparing the networks' performance in learning chaotic (unpredictable) versus quasiperiodic (predictable) solutions, we highlight fundamental challenges in learning chaotic solutions. The proposed validation strategies, which are based on the dynamical systems properties of chaotic time series, are shown to outperform the state-of-the-art validation strategies. Because the strategies are principled - they are based on chaos theory such as the Lyapunov time - they can be applied to other Recurrent Neural Networks architectures with little modification. This work opens up new possibilities for the robust design and application of Echo State Networks, and Recurrent Neural Networks, to the time-accurate prediction of chaotic systems.

Using adjoint-based optimization to enhance ignition in non-premixed jets.

Gradient-based optimization is used to reliably and optimally induce ignition in three examples of laminar non-premixed mixture configurations. Using time-integrated heat release as a cost functional, the non-convex optimization problem identified optimal energy source locations that coincide with the stoichiometric local mixture fraction surface for short optimization horizons, while for longer horizons, the hydrodynamics plays an increasingly important role and a balance between flow and chemistry features determines non-trivial optimal ignition locations. Rather than identifying a single optimal ignition location, the results of this study show that there may be several equally good ignition locations in a given flow configuration.