Deep learning probability flows and entropy production rates in active matter.

Active matter systems, from self-propelled colloids to motile bacteria, are characterized by the conversion of free energy into useful work at the microscopic scale. They involve physics beyond the reach of equilibrium statistical mechanics, and a persistent challenge has been to understand the nature of their nonequilibrium states. The entropy production rate and the probability current provide quantitative ways to do so by measuring the breakdown of time-reversal symmetry. Yet, their efficient computation has remained elusive, as they depend on the system's unknown and high-dimensional probability density. Here, building upon recent advances in generative modeling, we develop a deep learning framework to estimate the score of this density. We show that the score, together with the microscopic equations of motion, gives access to the entropy production rate, the probability current, and their decomposition into local contributions from individual particles. To represent the score, we introduce a spatially local transformer network architecture that learns high-order interactions between particles while respecting their underlying permutation symmetry. We demonstrate the broad utility and scalability of the method by applying it to several high-dimensional systems of active particles undergoing motility-induced phase separation (MIPS). We show that a single network trained on a system of 4,096 particles at one packing fraction can generalize to other regions of the phase diagram, including to systems with as many as 32,768 particles. We use this observation to quantify the spatial structure of the departure from equilibrium in MIPS as a function of the number of particles and the packing fraction.

How microscopic epistasis and clonal interference shape the fitness trajectory in a spin glass model of microbial long-term evolution.

The adaptive dynamics of evolving microbial populations takes place on a complex fitness landscape generated by epistatic interactions. The population generically consists of multiple competing strains, a phenomenon known as clonal interference. Microscopic epistasis and clonal interference are central aspects of evolution in microbes, but their combined effects on the functional form of the population's mean fitness are poorly understood. Here, we develop a computational method that resolves the full microscopic complexity of a simulated evolving population subject to a standard serial dilution protocol. Through extensive numerical experimentation, we find that stronger microscopic epistasis gives rise to fitness trajectories with slower growth independent of the number of competing strains, which we quantify with power-law fits and understand mechanistically via a random walk model that neglects dynamical correlations between genes. We show that increasing the level of clonal interference leads to fitness trajectories with faster growth (in functional form) without microscopic epistasis, but leaves the rate of growth invariant when epistasis is sufficiently strong, indicating that the role of clonal interference depends intimately on the underlying fitness landscape. The simulation package for this work may be found at https://github.com/nmboffi/spin_glass_evodyn.

Implicit Regularization and Momentum Algorithms in Nonlinearly Parameterized Adaptive Control and Prediction.

Stable concurrent learning and control of dynamical systems is the subject of adaptive control. Despite being an established field with many practical applications and a rich theory, much of the development in adaptive control for nonlinear systems revolves around a few key algorithms. By exploiting strong connections between classical adaptive nonlinear control techniques and recent progress in optimization and machine learning, we show that there exists considerable untapped potential in algorithm development for both adaptive nonlinear control and adaptive dynamics prediction. We begin by introducing first-order adaptation laws inspired by natural gradient descent and mirror descent. We prove that when there are multiple dynamics consistent with the data, these non-Euclidean adaptation laws implicitly regularize the learned model. Local geometry imposed during learning thus may be used to select parameter vectors-out of the many that will achieve perfect tracking or prediction-for desired properties such as sparsity. We apply this result to regularized dynamics predictor and observer design, and as concrete examples, we consider Hamiltonian systems, Lagrangian systems, and recurrent neural networks. We subsequently develop a variational formalism based on the Bregman Lagrangian. We show that its Euler Lagrange equations lead to natural gradient and mirror descent-like adaptation laws with momentum, and we recover their first-order analogues in the infinite friction limit. We illustrate our analyses with simulations demonstrating our theoretical results.

Coordinate transformation methodology for simulating quasistatic elastoplastic solids.

Molecular dynamics simulations frequently employ periodic boundary conditions where the positions of the periodic images are manipulated in order to apply deformation to the material sample. For example, Lees-Edwards conditions use moving periodic images to apply simple shear. Here, we examine the problem of precisely comparing this type of simulation to continuum solid mechanics. We employ a hypoelastoplastic mechanical model, and develop a projection method to enforce quasistatic equilibrium. We introduce a simulation framework that uses a fixed Cartesian computational grid on a reference domain, and which imposes deformation via a time-dependent coordinate transformation to the physical domain. As a test case for our method, we consider the evolution of shear bands in a bulk metallic glass using the shear transformation zone theory of amorphous plasticity. We examine the growth of shear bands in simple shear and pure shear conditions as a function of the initial preparation of the bulk metallic glass.

A Continuous-Time Analysis of Distributed Stochastic Gradient.

We analyze the effect of synchronization on distributed stochastic gradient algorithms. By exploiting an analogy with dynamical models of biological quorum sensing, where synchronization between agents is induced through communication with a common signal, we quantify how synchronization can significantly reduce the magnitude of the noise felt by the individual distributed agents and their spatial mean. This noise reduction is in turn associated with a reduction in the smoothing of the loss function imposed by the stochastic gradient approximation. Through simulations on model nonconvex objectives, we demonstrate that coupling can stabilize higher noise levels and improve convergence. We provide a convergence analysis for strongly convex functions by deriving a bound on the expected deviation of the spatial mean of the agents from the global minimizer for an algorithm based on quorum sensing, the same algorithm with momentum, and the elastic averaging SGD (EASGD) algorithm. We discuss extensions to new algorithms that allow each agent to broadcast its current measure of success and shape the collective computation accordingly. We supplement our theoretical analysis with numerical experiments on convolutional neural networks trained on the CIFAR-10 data set, where we note a surprising regularizing property of EASGD even when applied to the non-distributed case. This observation suggests alternative second-order in time algorithms for nondistributed optimization that are competitive with momentum methods.

Efficient Computation of the Hartree-Fock Exchange in Real-Space with Projection Operators.

We describe an efficient projection-based real-space implementation of the nonlocal single-determinant exchange operator. Through a matrix representation of the projected operator, we show that this scheme works equally well for both occupied and virtual states. Our scheme reaches a speedup of 2 orders of magnitude and has no significant loss of accuracy compared to an implementation of the full nonlocal single-determinant exchange operator. We find excellent agreement upon comparing Hartree-Fock eigenvalues, dipoles, and polarizabilities of selected molecules calculated using our method to values in the literature. To illustrate the efficiency of this scheme we perform calculations on systems with up to 240 carbon atoms.

Asymptotic behavior and interpretation of virtual states: The effects of confinement and of basis sets.

A real-space high order finite difference method is used to analyze the effect of spherical domain size on the Hartree-Fock (and density functional theory) virtual eigenstates. We show the domain size dependence of both positive and negative virtual eigenvalues of the Hartree-Fock equations for small molecules. We demonstrate that positive states behave like a particle in spherical well and show how they approach zero. For the negative eigenstates, we show that large domains are needed to get the correct eigenvalues. We compare our results to those of Gaussian basis sets and draw some conclusions for real-space, basis-sets, and plane-waves calculations.

The role of dimensionality in the decay of surface effects.

We computationally investigate the decay of surface effects in one-, two-, and three-dimensional materials using two-band tight-binding models. These general models facilitate a direct comparison between materials of differing dimensionality, which reveals that material dimensionality (not material-specific chemistry/physics) is the primary factor controlling the decay of surface effects. Our results corroborate more sophisticated, material-specific studies, finding that surface effects decay after ∼10, ∼25, and ≳ 100 layers in three-dimensional, two-dimensional, and one-dimensional materials, respectively. Physically, higher-dimensional materials screen surface effects more efficiently, as theoretically described by integration over each layer's Brillouin zone. Finally, we discuss several implications of these results.