Transformers for modeling physical systems.

Transformers are widely used in natural language processing due to their ability to model longer-term dependencies in text. Although these models achieve state-of-the-art performance for many language related tasks, their applicability outside of the natural language processing field has been minimal. In this work, we propose the use of transformer models for the prediction of dynamical systems representative of physical phenomena. The use of Koopman based embeddings provides a unique and powerful method for projecting any dynamical system into a vector representation which can then be predicted by a transformer. The proposed model is able to accurately predict various dynamical systems and outperform classical methods that are commonly used in the scientific machine learning literature.1.

Predictive collective variable discovery with deep Bayesian models.

Extending spatio-temporal scale limitations of models for complex atomistic systems considered in biochemistry and materials science necessitates the development of enhanced sampling methods. The potential acceleration in exploring the configurational space by enhanced sampling methods depends on the choice of collective variables (CVs). In this work, we formulate the discovery of CVs as a Bayesian inference problem and consider the CVs as hidden generators of the full-atomistic trajectory. The ability to generate samples of the fine-scale atomistic configurations using limited training data allows us to compute estimates of observables as well as our probabilistic confidence on them. The methodology is based on emerging methodological advances in machine learning and variational inference. The discovered CVs are related to physicochemical properties which are essential for understanding mechanisms especially in unexplored complex systems. We provide a quantitative assessment of the CVs in terms of their predictive ability for alanine dipeptide (ALA-2) and ALA-15 peptide.

A stochastic optimization approach to coarse-graining using a relative-entropy framework.

Relative entropy has been shown to provide a principled framework for the selection of coarse-grained potentials. Despite the intellectual appeal of it, its application has been limited by the fact that it requires the solution of an optimization problem with noisy gradients. When using deterministic optimization schemes, one is forced to either decrease the noise by adequate sampling or to resolve to ad hoc modifications in order to avoid instabilities. The former increases the computational demand of the method while the latter is of questionable validity. In order to address these issues and make relative entropy widely applicable, we propose alternative schemes for the solution of the optimization problem using stochastic algorithms.