Machine learning without a processor: Emergent learning in a nonlinear analog network.

Standard deep learning algorithms require differentiating large nonlinear networks, a process that is slow and power-hungry. Electronic contrastive local learning networks (CLLNs) offer potentially fast, efficient, and fault-tolerant hardware for analog machine learning, but existing implementations are linear, severely limiting their capabilities. These systems differ significantly from artificial neural networks as well as the brain, so the feasibility and utility of incorporating nonlinear elements have not been explored. Here, we introduce a nonlinear CLLN-an analog electronic network made of self-adjusting nonlinear resistive elements based on transistors. We demonstrate that the system learns tasks unachievable in linear systems, including XOR (exclusive or) and nonlinear regression, without a computer. We find our decentralized system reduces modes of training error in order (mean, slope, curvature), similar to spectral bias in artificial neural networks. The circuitry is robust to damage, retrainable in seconds, and performs learned tasks in microseconds while dissipating only picojoules of energy across each transistor. This suggests enormous potential for fast, low-power computing in edge systems like sensors, robotic controllers, and medical devices, as well as manufacturability at scale for performing and studying emergent learning.

Physical effects of learning.

Interacting many-body physical systems ranging from neural networks in the brain to folding proteins to self-modifying electrical circuits can learn to perform diverse tasks. This learning, both in nature and in engineered systems, can occur through evolutionary selection or through dynamical rules that drive active learning from experience. Here, we show that learning in linear physical networks with weak input signals leaves architectural imprints on the Hessian of a physical system. Compared to a generic organization of the system components, (a) the effective physical dimension of the response to inputs decreases, (b) the response of physical degrees of freedom to random perturbations (or system "susceptibility") increases, and (c) the low-eigenvalue eigenvectors of the Hessian align with the task. Overall, these effects embody the typical scenario for learning processes in physical systems in the weak input regime, suggesting ways of discovering whether a physical network may have been trained.

Identifying microscopic factors that influence ductility in disordered solids.

There are empirical strategies for tuning the degree of strain localization in disordered solids, but they are system-specific and no theoretical framework explains their effectiveness or limitations. Here, we study three model disordered solids: a simulated atomic glass, an experimental granular packing, and a simulated polymer glass. We tune each system using a different strategy to exhibit two different degrees of strain localization. In tandem, we construct structuro-elastoplastic (StEP) models, which reduce descriptions of the systems to a few microscopic features that control strain localization, using a machine learning-based descriptor, softness, to represent the stability of the disordered local structure. The models are based on calculated correlations of softness and rearrangements. Without additional parameters, the models exhibit semiquantitative agreement with observed stress-strain curves and softness statistics for all systems studied. Moreover, the StEP models reveal that initial structure, the near-field effect of rearrangements on local structure, and rearrangement size, respectively, are responsible for the changes in ductility observed in the three systems. Thus, StEP models provide microscopic understanding of how strain localization depends on the interplay of structure, plasticity, and elasticity.

Discontinuous instabilities in disordered solids.

Under a sufficiently large load, a solid material will flow via rearrangements, where particles change neighbors. Such plasticity is most easily described in the athermal, quasistatic limit of zero temperature and infinitesimal loading rate, where rearrangements occur only when the system becomes mechanically unstable. For disordered solids, the instabilities marking the onset of rearrangements have long been believed to be fold instabilities, in which an energy barrier disappears and the frequency of a normal mode of vibration vanishes continuously. Here, we report that there exists another, anomalous, type of instability caused by the breaking of a "stabilizing bond," whose removal creates an unstable vibrational mode. For commonly studied systems, such as those with harmonic finite-range interparticle interactions, such "discontinuous instabilities" are not only inevitable, they often dominate the modes of failure. Stabilizing bonds are a subset of all the bonds in the system and are prevalent in disordered solids generally. Although they do not trigger discontinuous instabilities in systems with vanishing stiffness at the interaction cutoff, they are, even in those cases, local indicators of incipient mechanical failure. They therefore provide an accurate structural predictor of instabilities not only of the discontinuous type but of the fold type as well.

Minimal vertex model explains how the amnioserosa avoids fluidization during Drosophila dorsal closure.

Dorsal closure is a process that occurs during embryogenesis of Drosophila melanogaster. During dorsal closure, the amnioserosa (AS), a one-cell thick epithelial tissue that fills the dorsal opening, shrinks as the lateral epidermis sheets converge and eventually merge. During this process, the aspect ratio of amnioserosa cells increases markedly. The standard 2-dimensional vertex model, which successfully describes tissue sheet mechanics in multiple contexts, would in this case predict that the tissue should fluidize via cell neighbor changes. Surprisingly, however, the amnioserosa remains an elastic solid with no such events. We here present a minimal extension to the vertex model that explains how the amnioserosa can achieve this unexpected behavior. We show that continuous shrink-age of the preferred cell perimeter and cell perimeter polydispersity lead to the retention of the solid state of the amnioserosa. Our model accurately captures measured cell shape and orientation changes and predicts non-monotonic junction tension that we confirm with laser ablation experiments.

Minimal vertex model explains how the amnioserosa avoids fluidization during Drosophila dorsal closure.

Dorsal closure is a process that occurs during embryogenesis of Drosophila melanogaster . During dorsal closure, the amnioserosa (AS), a one-cell thick epithelial tissue that fills the dorsal opening, shrinks as the lateral epidermis sheets converge and eventually merge. During this process, the aspect ratio of amnioserosa cells increases markedly. The standard 2-dimensional vertex model, which successfully describes tissue sheet mechanics in multiple contexts, would in this case predict that the tissue should fluidize via cell neighbor changes. Surprisingly, however, the amnioserosa remains an elastic solid with no such events. We here present a minimal extension to the vertex model that explains how the amnioserosa can achieve this unexpected behavior. We show that continuous shrinkage of the preferred cell perimeter and cell perimeter polydispersity lead to the retention of the solid state of the amnioserosa. Our model accurately captures measured cell shape and orientation changes and predicts non-monotonic junction tension that we confirm with laser ablation experiments. Significance Statement: During embryogenesis, cells in tissues can undergo significant shape changes. Many epithelial tissues fluidize, i.e. cells exchange neighbors, when the average cell aspect ratio increases above a threshold value, consistent with the standard vertex model. During dorsal closure in Drosophila melanogaster , however, the amnioserosa tissue remains solid even as the average cell aspect ratio increases well above threshold. We introduce perimeter polydispersity and allow the preferred cell perimeters, usually held fixed in vertex models, to decrease linearly with time as seen experimentally. With these extensions to the standard vertex model, we capture experimental observations quantitatively. Our results demonstrate that vertex models can describe the behavior of the amnioserosa in dorsal closure by allowing normally fixed parameters to vary with time.

Fragility in glassy liquids: A structural approach based on machine learning.

The rapid rise of viscosity or relaxation time upon supercooling is a universal hallmark of glassy liquids. The temperature dependence of viscosity, however, is quite nonuniversal for glassy liquids and is characterized by the system's "fragility," with liquids with nearly Arrhenius temperature-dependent relaxation times referred to as strong liquids and those with super-Arrhenius behavior referred to as fragile liquids. What makes some liquids strong and others fragile is still not well understood. Here, we explore this question in a family of harmonic spheres that range from extremely strong to extremely fragile, using "softness," a structural order parameter identified by machine learning to be highly correlated with dynamical rearrangements. We use a support vector machine to identify softness as the same linear combination of structural quantities across the entire family of liquids studied. We then use softness to identify the factors controlling fragility.

Probing Gardner Physics in an Active Quasithermal Pressure-Controlled Granular System of Noncircular Particles.

To search for experimental signals of the Gardner crossover, an active quasithermal granular glass is constructed using a monolayer of air-fluidized star-shaped particles. The pressure of the system is controlled by adjusting the tension exerted on an enclosing boundary. Velocity distributions of the internal particles and the scaling of the pressure, density, effective temperature, and relaxation time are examined, demonstrating that the system has key features of a thermal system. Using a pressure-based quenching protocol that brings the system into deeper glassy states, signals of the Gardner crossover are detected via cage size and separation order parameters for both particle positions and orientations, offering experimental evidence of Gardner physics for a system of anisotropic quasithermal particles in a low spatial dimension.

Transient learning degrees of freedom for introducing function in materials.

SignificanceMany protocols used in material design and training have a common theme: they introduce new degrees of freedom, often by relaxing away existing constraints, and then evolve these degrees of freedom based on a rule that leads the material to a desired state at which point these new degrees of freedom are frozen out. By creating a unifying framework for these protocols, we can now understand that some protocols work better than others because the choice of new degrees of freedom matters. For instance, introducing particle sizes as degrees of freedom to the minimization of a jammed particle packing can lead to a highly stable state, whereas particle stiffnesses do not have nearly the same impact.

Correlation of plastic events with local structure in jammed packings across spatial dimensions.

In frictionless jammed packings, existing evidence suggests a picture in which localized physics dominates in low spatial dimensions, d = 2, 3, but quickly loses relevance as d rises, replaced by spatially extended mean-field behavior. For example, quasilocalized low-energy vibrational modes and low-coordination particles associated with deviation from mean-field behavior (rattlers and bucklers) all vanish rapidly with increasing d. These results suggest that localized rearrangements, which are associated with low-energy vibrational modes, correlated with local structure, and dominant in low dimensions, should give way in higher d to extended rearrangements uncorrelated with local structure. Here, we use machine learning to analyze simulations of jammed packings under athermal, quasistatic shear, identifying a local structural variable, softness, that correlates with rearrangements in dimensions d = 2 to d = 5. We find that softness­and even just the local coordination number Z­is essentially equally predictive of rearrangements in all d studied. This result provides direct evidence that local structure plays an important role in higher d, suggesting a modified picture for the dimensional cross-over to mean-field theory.

Elastoplasticity Mediates Dynamical Heterogeneity Below the Mode Coupling Temperature.

As liquids approach the glass transition temperature, dynamical heterogeneity emerges as a crucial universal feature of their behavior. Dynamic facilitation, where local motion triggers further motion nearby, plays a major role in this phenomenon. Here we show that long-ranged, elastically mediated facilitation appears below the mode coupling temperature, adding to the short-range component present at all temperatures. Our results suggest deep connections between the supercooled liquid and glass states, and pave the way for a deeper understanding of dynamical heterogeneity in glassy systems.

Quantifying the link between local structure and cellular rearrangements using information in models of biological tissues.

Machine learning techniques have been used to quantify the relationship between local structural features and variations in local dynamical activity in disordered glass-forming materials. To date these methods have been applied to an array of standard (Arrhenius and super-Arrhenius) glass formers, where work on "soft spots" indicates a connection between the linear vibrational response of a configuration and the energy barriers to non-linear deformations. Here we study the Voronoi model, which takes its inspiration from dense epithelial monolayers and which displays anomalous, sub-Arrhenius scaling of its dynamical relaxation time with decreasing temperature. Despite these differences, we find that the likelihood of rearrangements can nevertheless vary by several orders of magnitude within the model tissue and extract a local structural quantity, "softness," that accurately predicts the temperature dependence of the relaxation time. We use an information-theoretic measure to quantify the extent to which softness determines impending topological rearrangements; we find that softness captures nearly all of the information about rearrangements that is obtainable from structure, and that this information is large in the solid phase of the model and decreases rapidly as state variables are varied into the fluid phase.

Hidden Topological Structure of Flow Network Functionality.

The ability to reroute and control flow is vital to the function of venation networks across a wide range of organisms. By modifying individual edges in these networks, either by adjusting edge conductances or creating and destroying edges, organisms robustly control the propagation of inputs to perform specific tasks. However, a fundamental disconnect exists between the structure and function: networks with different local architectures can perform the same functions. Here, we answer the question of how changes at the level of individual edges collectively create functionality at the scale of an entire network. Using persistent homology, we analyze networks tuned to perform complex tasks. We find that the responses of such networks encode a hidden topological structure composed of sectors of nearly uniform pressure. Although these sectors are not apparent in the underlying network structure, they correlate strongly with the tuned function. The connectivity of these sectors, rather than that of individual nodes, provides a quantitative relationship between structure and function in flow networks.

Periodic training of creeping solids.

We consider disordered solids in which the microscopic elements can deform plastically in response to stresses on them. We show that by driving the system periodically, this plasticity can be exploited to train in desired elastic properties, both in the global moduli and in local "allosteric" interactions. Periodic driving can couple an applied "source" strain to a "target" strain over a path in the energy landscape. This coupling allows control of the system's response, even at large strains well into the nonlinear regime, where it can be difficult to achieve control simply by design.

Attractive versus truncated repulsive supercooled liquids: The dynamics is encoded in the pair correlation function.

We compare glassy dynamics in two liquids that differ in the form of their interaction potentials. Both systems have the same repulsive interactions but one has also an attractive part in the potential. These two systems exhibit very different dynamics despite having nearly identical pair correlation functions. We demonstrate that a properly weighted integral of the pair correlation function, which amplifies the subtle differences between the two systems, correctly captures their dynamical differences. The weights are obtained from a standard machine learning algorithm.

Tuning and jamming reduced to their minima.

Inspired by protein folding, we smooth out the complex cost function landscapes of two processes: the tuning of networks and the jamming of ideal spheres. In both processes, geometrical frustration plays a role-tuning pressure differences between pairs of target nodes far from the source in a flow network impedes tuning of nearby pairs more than the reverse process, while unjamming the system in one region can make it more difficult to unjam elsewhere. By modifying the cost functions to control the order in which functions are tuned or regions unjam, we smooth out local minima while leaving global minima unaffected, increasing the success rate for reaching global minima.

Machine learning characterization of structural defects in amorphous packings of dimers and ellipses.

Structural defects within amorphous packings of symmetric particles can be characterized using a machine learning approach that incorporates structure functions of radial distances and angular arrangement. This yields a scalar field, softness, that correlates with the probability that a particle is about to be rearranged. However, when particle shapes are elongated, as in the case of dimers and ellipses, we find that the standard structure functions produce imprecise softness measurements. Moreover, ellipses exhibit deformation profiles in stark contrast to circular particles. In order to account for the effects of orientation and alignment, we introduce structure functions to recover the predictive performance of softness, as well as provide physical insight into local and extended dynamics. We study a model disordered solid, a bidisperse two-dimensional granular pillar, driven by uniaxial compression and composed entirely of monomers, dimers, or ellipses. We demonstrate how the computation of softness via a support vector machine extends to dimers and ellipses with the introduction of orientational structure functions. Then we highlight the spatial extent of rearrangements and defects, as well as their cross correlation, for each particle shape. Finally, we demonstrate how an additional machine learning algorithm, recursive feature elimination, provides an avenue to better understand how softness arises from particular structural aspects. We identify the most crucial structure functions in determining softness and discuss their physical implications.

Inferring statistical properties of 3D cell geometry from 2D slices.

Although cell shape can reflect the mechanical and biochemical properties of the cell and its environment, quantification of 3D cell shapes within 3D tissues remains difficult, typically requiring digital reconstruction from a stack of 2D images. We investigate a simple alternative technique to extract information about the 3D shapes of cells in a tissue; this technique connects the ensemble of 3D shapes in the tissue with the distribution of 2D shapes observed in independent 2D slices. Using cell vertex model geometries, we find that the distribution of 2D shapes allows clear determination of the mean value of a 3D shape index. We analyze the errors that may arise in practice in the estimation of the mean 3D shape index from 2D imagery and find that typically only a few dozen cells in 2D imagery are required to reduce uncertainty below 2%. Even though we developed the method for isotropic animal tissues, we demonstrate it on an anisotropic plant tissue. This framework could also be naturally extended to estimate additional 3D geometric features and quantify their uncertainty in other materials.

Limits of multifunctionality in tunable networks.

Nature is rife with networks that are functionally optimized to propagate inputs to perform specific tasks. Whether via genetic evolution or dynamic adaptation, many networks create functionality by locally tuning interactions between nodes. Here we explore this behavior in two contexts: strain propagation in mechanical networks and pressure redistribution in flow networks. By adding and removing links, we are able to optimize both types of networks to perform specific functions. We define a single function as a tuned response of a single "target" link when another, predetermined part of the network is activated. Using network structures generated via such optimization, we investigate how many simultaneous functions such networks can be programed to fulfill. We find that both flow and mechanical networks display qualitatively similar phase transitions in the number of targets that can be tuned, along with the same robust finite-size scaling behavior. We discuss how these properties can be understood in the context of constraint-satisfaction problems.

Directed aging, memory, and nature's greed.

Disordered materials are often out of equilibrium and evolve very slowly in a rugged and tortuous energy landscape. This slow evolution, referred to as aging, is deemed undesirable as it often leads to material degradation. However, we show that aging also encodes a memory of the stresses imposed during preparation. Because of inhomogeneous local stresses, the material itself decides how to evolve by modifying stressed regions differently from those under less stress. Because material evolution occurs in response to stresses, aging can be "directed" to produce sought-after responses and unusual functionalities that do not inherently exist. Aging obeys a natural "greedy algorithm" as, at each instant, the material simply follows the path of most rapid and accessible relaxation. Our experiments and simulations illustrate directed aging in examples in which the material's elasticity transforms as desired because of an imposed deformation.