Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control.

Sparse model identification enables the discovery of nonlinear dynamical systems purely from data; however, this approach is sensitive to noise, especially in the low-data limit. In this work, we leverage the statistical approach of bootstrap aggregating (bagging) to robustify the sparse identification of the nonlinear dynamics (SINDy) algorithm. First, an ensemble of SINDy models is identified from subsets of limited and noisy data. The aggregate model statistics are then used to produce inclusion probabilities of the candidate functions, which enables uncertainty quantification and probabilistic forecasts. We apply this ensemble-SINDy (E-SINDy) algorithm to several synthetic and real-world datasets and demonstrate substantial improvements to the accuracy and robustness of model discovery from extremely noisy and limited data. For example, E-SINDy uncovers partial differential equations models from data with more than twice as much measurement noise as has been previously reported. Similarly, E-SINDy learns the Lotka Volterra dynamics from remarkably limited data of yearly lynx and hare pelts collected from 1900 to 1920. E-SINDy is computationally efficient, with similar scaling as standard SINDy. Finally, we show that ensemble statistics from E-SINDy can be exploited for active learning and improved model predictive control.

Signal recovery from stimulation artifacts in intracranial recordings with dictionary learning.

OBJECTIVE: Electrical stimulation of the human brain is commonly used for eliciting and inhibiting neural activity for clinical diagnostics, modifying abnormal neural circuit function for therapeutics, and interrogating cortical connectivity. However, recording electrical signals with concurrent stimulation results in dominant electrical artifacts that mask the neural signals of interest. Here we develop a method to reproducibly and robustly recover neural activity during concurrent stimulation. We concentrate on signal recovery across an array of electrodes without channel-wise fine-tuning of the algorithm. Our goal includes signal recovery with trains of stimulation pulses, since repeated, high-frequency pulses are often required to induce desired effects in both therapeutic and research domains. We have made all of our code and data publicly available. APPROACH: We developed an algorithm that automatically detects templates of artifacts across many channels of recording, creating a dictionary of learned templates using unsupervised clustering. The artifact template that best matches each individual artifact pulse is subtracted to recover the underlying activity. To assess the success of our method, we focus on whether it extracts physiologically interpretable signals from real recordings. MAIN RESULTS: We demonstrate our signal recovery approach on invasive electrophysiologic recordings from human subjects during stimulation. We show the recovery of meaningful neural signatures in both electrocorticographic (ECoG) arrays and deep brain stimulation (DBS) recordings. In addition, we compared cortical responses induced by the stimulation of primary somatosensory (S1) by natural peripheral touch, as well as motor cortex activity with and without concurrent S1 stimulation. SIGNIFICANCE: Our work will enable future advances in neural engineering with simultaneous stimulation and recording.

Model selection for hybrid dynamical systems via sparse regression.

Hybrid systems are traditionally difficult to identify and analyse using classical dynamical systems theory. Moreover, recently developed model identification methodologies largely focus on identifying a single set of governing equations solely from measurement data. In this article, we develop a new methodology, Hybrid-Sparse Identification of Nonlinear Dynamics, which identifies separate nonlinear dynamical regimes, employs information theory to manage uncertainty and characterizes switching behaviour. Specifically, we use the nonlinear geometry of data collected from a complex system to construct a set of coordinates based on measurement data and augmented variables. Clustering the data in these measurement-based coordinates enables the identification of nonlinear hybrid systems. This methodology broadly empowers nonlinear system identification without constraining the data locally in time and has direct connections to hybrid systems theory. We demonstrate the success of this method on numerical examples including a mass-spring hopping model and an infectious disease model. Characterizing complex systems that switch between dynamic behaviours is integral to overcoming modern challenges such as eradication of infectious diseases, the design of efficient legged robots and the protection of cyber infrastructures.

The control structure of the nematode Caenorhabditis elegans: Neuro-sensory integration and proprioceptive feedback.

We develop a biophysically realistic model of the nematode C. elegans that includes: (i) its muscle structure and activation, (ii) key connectomic activation circuitry, and (iii) a weighted and time-dynamic proprioception. In combination, we show that these model components can reproduce the complex waveforms exhibited in C. elegans locomotive behaviors, chiefly omega turns. This is achieved via weighted, time-dependent suppression of the proprioceptive signal. Though speculative, such dynamics are biologically plausible due to the presence of neuromodulators which have recently been experimentally implicated in the escape response, which includes an omega turn. This is the first integrated neuromechanical model to reveal a mechanism capable of generating the complex waveforms observed in the behavior of C. elegans, thus contributing to a mathematical framework for understanding how control decisions can be executed at the connectome level in order to produce the full repertoire of observed behaviors.

Sparse identification of nonlinear dynamics for model predictive control in the low-data limit.

Data-driven discovery of dynamics via machine learning is pushing the frontiers of modelling and control efforts, providing a tremendous opportunity to extend the reach of model predictive control (MPC). However, many leading methods in machine learning, such as neural networks (NN), require large volumes of training data, may not be interpretable, do not easily include known constraints and symmetries, and may not generalize beyond the attractor where models are trained. These factors limit their use for the online identification of a model in the low-data limit, for example following an abrupt change to the system dynamics. In this work, we extend the recent sparse identification of nonlinear dynamics (SINDY) modelling procedure to include the effects of actuation and demonstrate the ability of these models to enhance the performance of MPC, based on limited, noisy data. SINDY models are parsimonious, identifying the fewest terms in the model needed to explain the data, making them interpretable and generalizable. We show that the resulting SINDY-MPC framework has higher performance, requires significantly less data, and is more computationally efficient and robust to noise than NN models, making it viable for online training and execution in response to rapid system changes. SINDY-MPC also shows improved performance over linear data-driven models, although linear models may provide a stopgap until enough data is available for SINDY. SINDY-MPC is demonstrated on a variety of dynamical systems with different challenges, including the chaotic Lorenz system, a simple model for flight control of an F8 aircraft, and an HIV model incorporating drug treatment.

Model selection for dynamical systems via sparse regression and information criteria.

We develop an algorithm for model selection which allows for the consideration of a combinatorially large number of candidate models governing a dynamical system. The innovation circumvents a disadvantage of standard model selection which typically limits the number of candidate models considered due to the intractability of computing information criteria. Using a recently developed sparse identification of nonlinear dynamics algorithm, the sub-selection of candidate models near the Pareto frontier allows feasible computation of Akaike information criteria (AIC) or Bayes information criteria scores for the remaining candidate models. The information criteria hierarchically ranks the most informative models, enabling the automatic and principled selection of the model with the strongest support in relation to the time-series data. Specifically, we show that AIC scores place each candidate model in the strong support, weak support or no support category. The method correctly recovers several canonical dynamical systems, including a susceptible-exposed-infectious-recovered disease model, Burgers' equation and the Lorenz equations, identifying the correct dynamical system as the only candidate model with strong support.

Preventing Neurodegenerative Memory Loss in Hopfield Neuronal Networks Using Cerebral Organoids or External Microelectronics.

Developing technologies have made significant progress towards linking the brain with brain-machine interfaces (BMIs) which have the potential to aid damaged brains to perform their original motor and cognitive functions. We consider the viability of such devices for mitigating the deleterious effects of memory loss that is induced by neurodegenerative diseases and/or traumatic brain injury (TBI). Our computational study considers the widely used Hopfield network, an autoassociative memory model in which neurons converge to a stable state pattern after receiving an input resembling the given memory. In this study, we connect an auxiliary network of neurons, which models the BMI device, to the original Hopfield network and train it to converge to its own auxiliary memory patterns. Injuries to the original Hopfield memory network, induced through neurodegeneration, for instance, can then be analyzed with the goal of evaluating the ability of the BMI to aid in memory retrieval tasks. Dense connectivity between the auxiliary and Hopfield networks is shown to promote robustness of memory retrieval tasks for both optimal and nonoptimal memory sets. Our computations estimate damage levels and parameter ranges for which full or partial memory recovery is achievable, providing a starting point for novel therapeutic strategies.

Mode-locked X-wave lasers.

We theoretically demonstrate X-waves as global attractors that enable mode-locking of a laser cavity operating in the normal dispersion regime. This result is based upon a fully comprehensive physical model of the laser cavity, where the nonlinear discrete diffraction dynamics of a waveguide array mediates the spontaneous periodic generation of spatio-temporal X-waves.

Stability of attractive Bose-Einstein condensates in a periodic potential.

Using a standing light wave potential, a stable quasi-one-dimensional attractive dilute-gas Bose-Einstein condensate can be realized. In a mean-field approximation, this phenomenon is modeled by the cubic nonlinear Schrödinger equation with attractive nonlinearity and an elliptic function potential of which a standing light wave is a special case. New families of stationary solutions are presented. Some of these solutions have neither an analog in the linear Schrödinger equation nor in the integrable nonlinear Schrödinger equation. Their stability is examined using analytic and numerical methods. Trivial-phase solutions are experimentally stable provided they have nodes and their density is localized in the troughs of the potential. Stable time-periodic solutions are also examined.

Stability of stationary states in the cubic nonlinear Schrödinger equation: applications to the Bose-Einstein condensate.

The cubic nonlinear Schrödinger equation is the quasi-one-dimensional limit of the mean-field theory which models dilute gas Bose-Einstein condensates. Stationary solutions of this equation can be characterized as soliton trains. It is demonstrated that for repulsive nonlinearity a soliton train is stable to initial stochastic perturbation, while for attractive nonlinearity its behavior depends on the spacing between individual solitons in the train. Toroidal and harmonic confinement, both of experimental interest for Bose-Einstein condensates, are considered.

Stability of repulsive Bose-Einstein condensates in a periodic potential.

The cubic nonlinear Schrödinger equation with repulsive nonlinearity and an elliptic function potential models a quasi-one-dimensional repulsive dilute gas Bose-Einstein condensate trapped in a standing light wave. New families of stationary solutions are presented. Some of these solutions have neither an analog in the linear Schrödinger equation nor in the integrable nonlinear Schrödinger equation. Their stability is examined using analytical and numerical methods. All trivial-phase stable solutions are deformations of the ground state of the linear Schrödinger equation. Our results show that a large number of condensed atoms is sufficient to form a stable, periodic condensate. Physically, this implies stability of states near the Thomas-Fermi limit.

Bose-Einstein condensates in standing waves: the cubic nonlinear Schrödinger equation with a periodic potential.

We present a new family of stationary solutions to the cubic nonlinear Schrödinger equation with an elliptic function potential. In the limit of a sinusoidal potential our solutions model a quasi-one-dimensional dilute gas Bose-Einstein condensate trapped in a standing light wave. Provided that the ratio of the height of the variations of the condensate to its dc offset is small enough, both trivial phase and nontrivial phase solutions are shown to be stable. Recent developments allow for experimental investigation of these predictions.

Observation of the breakup of a prechirped N-soliton in an optical fiber.

We present what is believed to be the first experimental evidence showing the breakup of a chirped N-soliton pulse into an ordered train of fundamental solitons, as predicted by theory. We also present numerical experiments that confirm this phenomenon. Implications for optical communications systems that use chirped pulses are discussed.

Noise-induced amplitude and chirp jitter in dispersion-managed soliton communications.

An analytic theory is presented that demonstrates that noise-induced amplitude and quadratic chirp jitter in a dispersion-managed soliton system can impose a fundamental transmission limit. Using a variational method, we show that the nonlinear amplitude and chirp dynamics are well approximated by a two-dimensional random-walk process.

Dispersion-managed breathers with average normal dispersion.

Analytic and numerical evidence is presented that demonstrates that a dispersion-managed breather can be supported in an optical fiber even when the average dispersion is in the normal regime. This nonlinear behavior, which is contrary to guiding-center theory, is shown to originate from the reversible dynamics associated with the strong quadratic chirp that is generated in both the anomalous and the normal dispersion regimes.

Enhanced power transfer and mode coupling in spun twin-core optical fibers.

In a twin-core optical fiber a propagating light pulse periodically transfers power between its two cores. Experiments by Tjugiarto et al. [Opt. Lett. 17, 1058 (1992)] have demonstrated that this coupling length is considerably reduced when the fiber is also spun. A coupled-mode analysis reveals a pitch resonance that couples a cladding mode with the circularly polarized core mode whose handedness matches that of the helical twist of the cores. This resonance mechanism explains the observation of enhanced coupling and claddingmode cross coupling.

Long-distance pulse propagation in nonlinear optical fibers by using periodically spaced parametric amplifiers.

We analyze pulse propagation in a nonlinear optical fiber in which linear loss in the fiber is balanced by a chain of periodically spaced, phase-sensitive, degenerate parametric amplifiers. Our analysis shows that no pulse evolution occurs over a soliton period owing to attenuation in the quadrature orthogonal to the amplified quadrature. Evidence is presented that indicates that stable pulse solutions exist on length scales much longer than the soliton period. These pulses are governed by a nonlinear fourth-order evolution equation, which describes the exponential decay of arbitrary initial pulses (within the stability regime) onto stable, steady-state, solitonlike pulses.