Efficient determination of Born-effective charges, LO-TO splitting, and Raman tensors of solids with a real-space atom-centered deep learning approach.

We introduce a deep neural network (DNN) framework called the Real-space Atomic Decomposition NETwork (RADNET), which is capable of making accurate predictions of polarization and of electronic dielectric permittivity tensors in solids. This framework builds on previous, atom-centered approaches while utilizing deep convolutional neural networks. We report excellent accuracies on direct predictions for two prototypical examples: GaAs and BN. We then use automatic differentiation to calculate the Born-effective charges, longitudinal optical-transverse optical (LO-TO) splitting frequencies, and Raman tensors of these materials. We compute the Raman spectra, and find agreement with ab initio results. Lastly, we explore ways to generalize polarization predictions while taking into account periodic boundary conditions and symmetries.

Learning stochastic dynamics and predicting emergent behavior using transformers.

We show that a neural network originally designed for language processing can learn the dynamical rules of a stochastic system by observation of a single dynamical trajectory of the system, and can accurately predict its emergent behavior under conditions not observed during training. We consider a lattice model of active matter undergoing continuous-time Monte Carlo dynamics, simulated at a density at which its steady state comprises small, dispersed clusters. We train a neural network called a transformer on a single trajectory of the model. The transformer, which we show has the capacity to represent dynamical rules that are numerous and nonlocal, learns that the dynamics of this model consists of a small number of processes. Forward-propagated trajectories of the trained transformer, at densities not encountered during training, exhibit motility-induced phase separation and so predict the existence of a nonequilibrium phase transition. Transformers have the flexibility to learn dynamical rules from observation without explicit enumeration of rates or coarse-graining of configuration space, and so the procedure used here can be applied to a wide range of physical systems, including those with large and complex dynamical generators.

Cellular automata can classify data by inducing trajectory phase coexistence.

We show that cellular automata can classify data by inducing a form of dynamical phase coexistence. We use Monte Carlo methods to search for general two-dimensional deterministic automata that classify images on the basis of activity, the number of state changes that occur in a trajectory initiated from the image. When the number of time steps of the automaton is a trainable parameter, the search scheme identifies automata that generate a population of dynamical trajectories displaying high or low activity, depending on initial conditions. Automata of this nature behave as nonlinear activation functions with an output that is effectively binary, resembling an emergent version of a spiking neuron.

Machine Learning Diffusion Monte Carlo Energies.

We present two machine learning methodologies that are capable of predicting diffusion Monte Carlo (DMC) energies with small data sets (≈60 DMC calculations in total). The first uses voxel deep neural networks (VDNNs) to predict DMC energy densities using Kohn-Sham density functional theory (DFT) electron densities as input. The second uses kernel ridge regression (KRR) to predict atomic contributions to the DMC total energy using atomic environment vectors as input (we used atom-centered symmetry functions, atomic environment vectors from the ANI models, and smooth overlap of atomic positions). We first compare the methodologies on pristine graphene lattices, where we find that the KRR methodology performs best in comparison to gradient boosted decision trees, random forest, Gaussian process regression, and multilayer perceptrons. In addition, KRR outperforms VDNNs by an order of magnitude. Afterward, we study the generalizability of KRR to predict the energy barrier associated with a Stone-Wales defect. Lastly, we move from 2D to 3D materials and use KRR to predict total energies of liquid water. In all cases, we find that the KRR models are more accurate than Kohn-Sham DFT and all mean absolute errors are less than chemical accuracy.

High-Throughput Evaluation of Emission and Structure in Reduced-Dimensional Perovskites.

High-throughput experimentation (HTE) seeks to accelerate the exploration of materials space by uniting robotics, combinatorial methods, and parallel processing. HTE is particularly relevant to metal halide perovskites (MHPs), a diverse class of optoelectronic materials with a large chemical space. Here we develop an HTE workflow to synthesize and characterize light-emitting MHP single crystals, allowing us to generate the first reported data set of experimentally derived photoluminescence spectra for low-dimensional MHPs. We leverage the accelerated workflow to optimize the synthesis and emission of a new MHP, methoxy-phenethylammonium lead iodide ((4-MeO-PEAI)2-PbI2). We then synthesize 16 000 MHP single crystals and measure their photoluminescence to study the effects of synthesis parameters and compositional engineering on the emission intensity of 54 distinct MHPs: we achieve an acceleration factor of more than 100 times over previously reported HTE MHP synthesis and characterization methods. Using insights derived from this analysis, we screen an existing database for new, potentially emissive MHPs. On the basis of the Tanimoto similarity of the bright available emitters, we present our top candidates for future exploration. As a proof of concept, we use one of these (3,4-difluorophenylmethanamine) to synthesize an MHP which we find has a photoluminescence quantum yield of 10%.

Toward Orbital-Free Density Functional Theory with Small Data Sets and Deep Learning.

We use voxel deep neural networks to predict energy densities and functional derivatives of electron kinetic energies for the Thomas-Fermi model and Kohn-Sham density functional theory calculations. We show that the ground-state electron density can be found via direct minimization for a graphene lattice without any projection scheme using a voxel deep neural network trained with the Thomas-Fermi model. Additionally, we predict the kinetic energy of a graphene lattice within chemical accuracy after training from only two Kohn-Sham density functional theory (DFT) calculations. We identify an important sampling issue inherent in Kohn-Sham DFT calculations and propose future work to rectify this problem. Furthermore, we demonstrate an alternative, functional derivative-free, Monte Carlo based orbital-free density functional theory algorithm to calculate an accurate two-electron density in a double inverted Gaussian potential with a machine-learned kinetic energy functional.

Correspondence between neuroevolution and gradient descent.

We show analytically that training a neural network by conditioned stochastic mutation or neuroevolution of its weights is equivalent, in the limit of small mutations, to gradient descent on the loss function in the presence of Gaussian white noise. Averaged over independent realizations of the learning process, neuroevolution is equivalent to gradient descent on the loss function. We use numerical simulation to show that this correspondence can be observed for finite mutations, for shallow and deep neural networks. Our results provide a connection between two families of neural-network training methods that are usually considered to be fundamentally different.

Golem: an algorithm for robust experiment and process optimization.

Numerous challenges in science and engineering can be framed as optimization tasks, including the maximization of reaction yields, the optimization of molecular and materials properties, and the fine-tuning of automated hardware protocols. Design of experiment and optimization algorithms are often adopted to solve these tasks efficiently. Increasingly, these experiment planning strategies are coupled with automated hardware to enable autonomous experimental platforms. The vast majority of the strategies used, however, do not consider robustness against the variability of experiment and process conditions. In fact, it is generally assumed that these parameters are exact and reproducible. Yet some experiments may have considerable noise associated with some of their conditions, and process parameters optimized under precise control may be applied in the future under variable operating conditions. In either scenario, the optimal solutions found might not be robust against input variability, affecting the reproducibility of results and returning suboptimal performance in practice. Here, we introduce Golem, an algorithm that is agnostic to the choice of experiment planning strategy and that enables robust experiment and process optimization. Golem identifies optimal solutions that are robust to input uncertainty, thus ensuring the reproducible performance of optimized experimental protocols and processes. It can be used to analyze the robustness of past experiments, or to guide experiment planning algorithms toward robust solutions on the fly. We assess the performance and domain of applicability of Golem through extensive benchmark studies and demonstrate its practical relevance by optimizing an analytical chemistry protocol under the presence of significant noise in its experimental conditions.

Unsupervised hyperspectral stimulated Raman microscopy image enhancement: denoising and segmentation via one-shot deep learning.

Hyperspectral stimulated Raman scattering (SRS) microscopy is a label-free technique for biomedical and mineralogical imaging which can suffer from low signal-to-noise ratios. Here we demonstrate the use of an unsupervised deep learning neural network for rapid and automatic denoising of SRS images: UHRED (Unsupervised Hyperspectral Resolution Enhancement and Denoising). UHRED is capable of "one-shot" learning; only one hyperspectral image is needed, with no requirements for training on previously labelled datasets or images. Furthermore, by applying a k-means clustering algorithm to the processed data, we demonstrate automatic, unsupervised image segmentation, yielding, without prior knowledge of the sample, intuitive chemical species maps, as shown here for a lithium ore sample.

Dynamical Large Deviations of Two-Dimensional Kinetically Constrained Models Using a Neural-Network State Ansatz.

We use a neural-network ansatz originally designed for the variational optimization of quantum systems to study dynamical large deviations in classical ones. We use recurrent neural networks to describe the large deviations of the dynamical activity of model glasses, kinetically constrained models in two dimensions. We present the first finite size-scaling analysis of the large-deviation functions of the two-dimensional Fredrickson-Andersen model, and explore the spatial structure of the high-activity sector of the South-or-East model. These results provide a new route to the study of dynamical large-deviation functions, and highlight the broad applicability of the neural-network state ansatz across domains in physics.

Neural evolution structure generation: High entropy alloys.

We propose a neural evolution structure (NES) generation methodology combining artificial neural networks and evolutionary algorithms to generate high entropy alloy structures. Our inverse design approach is based on pair distribution functions and atomic properties and allows one to train a model on smaller unit cells and then generate a larger cell. With a speed-up factor of ∼1000 with respect to the special quasi-random structures (SQSs), the NESs dramatically reduce computational costs and time, making possible the generation of very large structures (over 40 000 atoms) in few hours. Additionally, unlike the SQSs, the same model can be used to generate multiple structures with the same fractional composition.

Neuroevolutionary Learning of Particles and Protocols for Self-Assembly.

Within simulations of molecules deposited on a surface we show that neuroevolutionary learning can design particles and time-dependent protocols to promote self-assembly, without input from physical concepts such as thermal equilibrium or mechanical stability and without prior knowledge of candidate or competing structures. The learning algorithm is capable of both directed and exploratory design: it can assemble a material with a user-defined property, or search for novelty in the space of specified order parameters. In the latter mode it explores the space of what can be made, rather than the space of structures that are low in energy but not necessarily kinetically accessible.

Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning.

Using a model heat engine, we show that neural-network-based reinforcement learning can identify thermodynamic trajectories of maximal efficiency. We consider both gradient and gradient-free reinforcement learning. We use an evolutionary learning algorithm to evolve a population of neural networks, subject to a directive to maximize the efficiency of a trajectory composed of a set of elementary thermodynamic processes; the resulting networks learn to carry out the maximally efficient Carnot, Stirling, or Otto cycles. When given an additional irreversible process, this evolutionary scheme learns a previously unknown thermodynamic cycle. Gradient-based reinforcement learning is able to learn the Stirling cycle, whereas an evolutionary approach achieves the optimal Carnot cycle. Our results show how the reinforcement learning strategies developed for game playing can be applied to solve physical problems conditioned upon path-extensive order parameters.

Evolutionary reinforcement learning of dynamical large deviations.

We show how to bound and calculate the likelihood of dynamical large deviations using evolutionary reinforcement learning. An agent, a stochastic model, propagates a continuous-time Monte Carlo trajectory and receives a reward conditioned upon the values of certain path-extensive quantities. Evolution produces progressively fitter agents, potentially allowing the calculation of a piece of a large-deviation rate function for a particular model and path-extensive quantity. For models with small state spaces, the evolutionary process acts directly on rates, and for models with large state spaces, the process acts on the weights of a neural network that parameterizes the model's rates. This approach shows how path-extensive physics problems can be considered within a framework widely used in machine learning.

Learning to grow: Control of material self-assembly using evolutionary reinforcement learning.

We show that neural networks trained by evolutionary reinforcement learning can enact efficient molecular self-assembly protocols. Presented with molecular simulation trajectories, networks learn to change temperature and chemical potential in order to promote the assembly of desired structures or choose between competing polymorphs. In the first case, networks reproduce in a qualitative sense the results of previously known protocols, but faster and with higher fidelity; in the second case they identify strategies previously unknown, from which we can extract physical insight. Networks that take as input the elapsed time of the simulation or microscopic information from the system are both effective, the latter more so. The evolutionary scheme we have used is simple to implement and can be applied to a broad range of examples of experimental self-assembly, whether or not one can monitor the experiment as it proceeds. Our results have been achieved with no human input beyond the specification of which order parameter to promote, pointing the way to the design of synthesis protocols by artificial intelligence.

Extensive deep neural networks for transferring small scale learning to large scale systems.

We present a physically-motivated topology of a deep neural network that can efficiently infer extensive parameters (such as energy, entropy, or number of particles) of arbitrarily large systems, doing so with scaling. We use a form of domain decomposition for training and inference, where each sub-domain (tile) is comprised of a non-overlapping focus region surrounded by an overlapping context region. The size of these regions is motivated by the physical interaction length scales of the problem. We demonstrate the application of EDNNs to three physical systems: the Ising model and two hexagonal/graphene-like datasets. In the latter, an EDNN was able to make total energy predictions of a 60 atoms system, with comparable accuracy to density functional theory (DFT), in 57 milliseconds. Additionally EDNNs are well suited for massively parallel evaluation, as no communication is necessary during neural network evaluation. We demonstrate that EDNNs can be used to make an energy prediction of a two-dimensional 35.2 million atom system, over 1.0 µm2 of material, at an accuracy comparable to DFT, in under 25 minutes. Such a system exists on a length scale visible with optical microscopy and larger than some living organisms.

Deep neural networks for direct, featureless learning through observation: The case of two-dimensional spin models.

We demonstrate the capability of a convolutional deep neural network in predicting the nearest-neighbor energy of the 4×4 Ising model. Using its success at this task, we motivate the study of the larger 8×8 Ising model, showing that the deep neural network can learn the nearest-neighbor Ising Hamiltonian after only seeing a vanishingly small fraction of configuration space. Additionally, we show that the neural network has learned both the energy and magnetization operators with sufficient accuracy to replicate the low-temperature Ising phase transition. We then demonstrate the ability of the neural network to learn other spin models, teaching the convolutional deep neural network to accurately predict the long-range interaction of a screened Coulomb Hamiltonian, a sinusoidally attenuated screened Coulomb Hamiltonian, and a modified Potts model Hamiltonian. In the case of the long-range interaction, we demonstrate the ability of the neural network to recover the phase transition with equivalent accuracy to the numerically exact method. Furthermore, in the case of the long-range interaction, the benefits of the neural network become apparent; it is able to make predictions with a high degree of accuracy, and do so 1600 times faster than a CUDA-optimized exact calculation. Additionally, we demonstrate how the neural network succeeds at these tasks by looking at the weights learned in a simplified demonstration.

A randomized comparison between neurostimulation and ultrasound-guided lateral femoral cutaneous nerve block.

BACKGROUND: This prospective, randomized trial compared neurostimulation (NS) and ultrasound (US) guided lateral femoral cutaneous nerve (LFCN) block. We hypothesized that US would result in a shorter total anesthesia-related time (sum of performance and onset times). METHODS: Twenty-one volunteers were enrolled. The right lower limb was randomized to an NS- or US-guided LFCN block. The alternate technique was employed for the left lower limb. With NS, paresthesias were sought in the lateral thigh at a stimulatory threshold of 0.6 mA (pulse width=0.3 ms; frequency=2 Hz) or lower. With US, local anesthetic was deposited under the inguinal ligament, ventral to the iliopsoas muscle. In both groups, 5 mL of lidocaine 2% were used to anesthetize the nerve. During the procedure of the block, the performance time and number of needle passes were recorded. Subsequently, a blinded observer assessed sensory block in the lateral thigh every minute until 20 minutes. Success was defined as loss of pinprick sensation at a point midway between the anterior superior iliac spine and the lateral knee line. The blinded observer also assessed the areas of sensory block in the anterior, medial, lateral, and posterior aspects of the thigh and mapped this distribution onto a corresponding grid. RESULTS: Both modalities provided comparable success rates (76.2%-95.2%), performance times (162.1 to 231.3 seconds), onset times (300.0 to 307.5 seconds) and total anesthesia related-times (480.1 to 554.0 seconds). However US required fewer needle passes (3.2±2.9 vs 9.5±12.2; P=.009). There were no intergroup differences in terms of the distribution of the anesthetized cutaneous areas. However considerable variability was encountered between individuals and between the 2 sides of a same subject. The most common areas of sensory loss included the central lateral two-eighths anteriorly and the central antero-inferior three-eighths laterally. CONCLUSION: Ultrasound guidance and NS provide similar success rates and total anesthesia-related times for LFCN block. The territory of the LFCN displays wide inter- and intra-individual variability.

Emergent rhombus tilings from molecular interactions with M-fold rotational symmetry.

We show that model molecules with particular rotational symmetries can self-assemble into network structures equivalent to rhombus tilings. This assembly happens in an emergent way, in the sense that molecules spontaneously select irregular fourfold local coordination from a larger set of possible local binding geometries. The existence of such networks can be rationalized by simple geometrical arguments, but the same arguments do not guarantee a network's spontaneous self-assembly. This class of structures must in certain regimes of parameter space be able to reconfigure into networks equivalent to triangular tilings.