Acceleration of Graph Neural Network-Based Prediction Models in Chemistry via Co-Design Optimization on Intelligence Processing Units.

Atomic structure prediction and associated property calculations are the bedrock of chemical physics. Since high-fidelity ab initio modeling techniques for computing the structure and properties can be prohibitively expensive, this motivates the development of machine-learning (ML) models that make these predictions more efficiently. Training graph neural networks over large atomistic databases introduces unique computational challenges, such as the need to process millions of small graphs with variable size and support communication patterns that are distinct from learning over large graphs, such as social networks. We demonstrate a novel hardware-software codesign approach to scale up the training of atomistic graph neural networks (GNN) for structure and property prediction. First, to eliminate redundant computation and memory associated with alternative padding techniques and to improve throughput via minimizing communication, we formulate the effective coalescing of the batches of variable-size atomistic graphs as the bin packing problem and introduce a hardware-agnostic algorithm to pack these batches. In addition, we propose hardware-specific optimizations, including a planner and vectorization for the gather-scatter operations targeted for Graphcore's Intelligence Processing Unit (IPU), as well as model-specific optimizations such as merged communication collectives and optimized softplus. Putting these all together, we demonstrate the effectiveness of the proposed codesign approach by providing an implementation of a well-established atomistic GNN on the Graphcore IPUs. We evaluate the training performance on multiple atomistic graph databases with varying degrees of graph counts, sizes, and sparsity. We demonstrate that such a codesign approach can reduce the training time of atomistic GNNs and can improve their performance by up to 1.5× compared to the baseline implementation of the model on the IPUs. Additionally, we compare our IPU implementation with a Nvidia GPU-based implementation and show that our atomistic GNN implementation on the IPUs can run 1.8× faster on average compared to the execution time on the GPUs.

An open database of computed bulk ternary transition metal dichalcogenides.

We present a dataset of structural relaxations of bulk ternary transition metal dichalcogenides (TMDs) computed via plane-wave density functional theory (DFT). We examined combinations of up to two chalcogenides with seven transition metals from groups 4-6 in octahedral (1T) or trigonal prismatic (2H) coordination. The full dataset consists of 672 unique stoichiometries, with a total of 50,337 individual configurations generated during structural relaxation. Our motivations for building this dataset are (1) to develop a training set for the generation of machine and deep learning models and (2) to obtain structural minima over a range of stoichiometries to support future electronic analyses. We provide the dataset as individual VASP xml files as well as all configurations encountered during relaxations collated into an ASE database with the corresponding total energy and atomic forces. In this report, we discuss the dataset in more detail and highlight interesting structural and electronic features of the relaxed structures.

Active sampling for neural network potentials: Accelerated simulations of shear-induced deformation in Cu-Ni multilayers.

Neural network potentials (NNPs) can greatly accelerate atomistic simulations relative to ab initio methods, allowing one to sample a broader range of structural outcomes and transformation pathways. In this work, we demonstrate an active sampling algorithm that trains an NNP that is able to produce microstructural evolutions with accuracy comparable to those obtained by density functional theory, exemplified during structure optimizations for a model Cu-Ni multilayer system. We then use the NNP, in conjunction with a perturbation scheme, to stochastically sample structural and energetic changes caused by shear-induced deformation, demonstrating the range of possible intermixing and vacancy migration pathways that can be obtained as a result of the speedups provided by the NNP. The code to implement our active learning strategy and NNP-driven stochastic shear simulations is openly available at https://github.com/pnnl/Active-Sampling-for-Atomistic-Potentials.

Decoding the protein-ligand interactions using parallel graph neural networks.

Protein-ligand interactions (PLIs) are essential for biochemical functionality and their identification is crucial for estimating biophysical properties for rational therapeutic design. Currently, experimental characterization of these properties is the most accurate method, however, this is very time-consuming and labor-intensive. A number of computational methods have been developed in this context but most of the existing PLI prediction heavily depends on 2D protein sequence data. Here, we present a novel parallel graph neural network (GNN) to integrate knowledge representation and reasoning for PLI prediction to perform deep learning guided by expert knowledge and informed by 3D structural data. We develop two distinct GNN architectures: [Formula: see text] is the base implementation that employs distinct featurization to enhance domain-awareness, while [Formula: see text] is a novel implementation that can predict with no prior knowledge of the intermolecular interactions. The comprehensive evaluation demonstrated that GNN can successfully capture the binary interactions between ligand and protein's 3D structure with 0.979 test accuracy for [Formula: see text] and 0.958 for [Formula: see text] for predicting activity of a protein-ligand complex. These models are further adapted for regression tasks to predict experimental binding affinities and [Formula: see text] crucial for compound's potency and efficacy. We achieve a Pearson correlation coefficient of 0.66 and 0.65 on experimental affinity and 0.50 and 0.51 on [Formula: see text] with [Formula: see text] and [Formula: see text], respectively, outperforming similar 2D sequence based models. Our method can serve as an interpretable and explainable artificial intelligence (AI) tool for predicted activity, potency, and biophysical properties of lead candidates. To this end, we show the utility of [Formula: see text] on SARS-Cov-2 protein targets by screening a large compound library and comparing the prediction with the experimentally measured data.

A look inside the black box: Using graph-theoretical descriptors to interpret a Continuous-Filter Convolutional Neural Network (CF-CNN) trained on the global and local minimum energy structures of neutral water clusters.

We describe a method for the post-hoc interpretation of a neural network (NN) trained on the global and local minima of neutral water clusters. We use the structures recently reported in a newly published database containing over 5 × 106 unique water cluster networks (H2O)N of size N = 3-30. The structural properties were first characterized using chemical descriptors derived from graph theory, identifying important trends in topology, connectivity, and polygon structure of the networks associated with the various minima. The code to generate the molecular graphs and compute the descriptors is available at https://github.com/exalearn/molecular-graph-descriptors, and the graphs are available alongside the original database at https://sites.uw.edu/wdbase/. A Continuous-Filter Convolutional Neural Network (CF-CNN) was trained on a subset of 500 000 networks to predict the potential energy, yielding a mean absolute error of 0.002 ± 0.002 kcal/mol per water molecule. Clusters of sizes not included in the training set exhibited errors of the same magnitude, indicating that the CF-CNN protocol accurately predicts energies of networks for both smaller and larger sizes than those used during training. The graph-theoretical descriptors were further employed to interpret the predictive power of the CF-CNN. Topological measures, such as the Wiener index, the average shortest path length, and the similarity index, suggested that all networks from the test set were within the range of values as the ones from the training set. The graph analysis suggests that larger errors appear when the mean degree and the number of polygons in the cluster lie further from the mean of the training set. This indicates that the structural space, and not just the chemical space, is an important factor to consider when designing training sets, as predictive errors can result when the structural composition is sufficiently different from the bulk of those in the training set. To this end, the developed descriptors are quite effective in explaining the results of the CF-CNN (a.k.a. the "black box") model.

Tracking the Chemical Evolution of Iodine Species Using Recurrent Neural Networks.

We apply recurrent neural networks (RNNs) to predict the time evolution of the concentration profile of multiple species resulting from a set of interconnected chemical reactions. As a proof of concept of our approach, RNNs were trained on a synthetic dataset generated by solving the kinetic equations of a system of aqueous inorganic iodine reactions that can follow after nuclear reactor accidents. We examine the minimum dataset necessary to obtain accurate predictions and explore the ability of RNNs to interpolate and extrapolate when exposed to previously unseen data. We also investigate the limits of our RNN by evaluating the robustness of the training initialization on our dataset.

Ring-Walking of Zerovalent Nickel on Aryl Halides.

While ring-walking is a critical step in transition metal catalyzed cross-coupling reactions, the associated metastable intermediates are often difficult to isolate and characterize. In this work, theoretical structures and energetics for ring-walking and oxidative addition of zerovalent nickel with 1-bromo-2-methylbenzene, 2-bromopyridine, 2-bromo-3-methyl-thiophene, and 2-bromopyrrole were computed at the B3LYP-D3/TZ2P-LANL2TZ(f)-LANL08d level. The mechanisms vary qualitatively with substrate ring size and type-the catalyst weaves along the edges of the benzene and pyridine rings, cuts through the interior of the thiophene ring, and arcs along the bond opposite the nitrogen atom in the pyrrole ring. Analogous computations on the ring-walking and oxidative addition of zerovalent palladium with 1-bromo-2-methylbenzene reveal an energetic profile similar to that of Ni but with much weaker overall binding to the arene. In all cases, dispersion corrections are found to be very important for computing accurate metal-substrate binding energies.

Degradable Polycaprolactone and Polylactide Homopolymer and Block Copolymer Brushes Prepared by Surface-Initiated Polymerization with Triazabicyclodecene and Zirconium Catalysts.

Surface-initiated ring-opening polymerization (SI-ROP) of polycaprolactone (PCL) and polylactide (PLA) polymer brushes with controlled degradation rates were prepared on oxide substrates. PCL brushes were polymerized from hydroxyl-terminated monolayers utilizing triazabicyclodecene (TBD) as the polymerization catalyst. A consistent brush thickness of 40 nm could be achieved with a reproducible unique crystalline morphology. The organocatalyzed PCL brushes were chain extended using lactide in the presence of zirconium n-butoxide to successfully grow PCL/PLA block copolymer (PCL-b-PLA) brushes with a final thickness of 55 nm. The degradation properties of "grafted from" PCL brush and the PCL-b-PLA brush were compared to "grafted to" PCL brushes, and we observed that the brush density plays a major role in degradation kinetics. Solutions of methanol/water at pH 14 were used to better solvate the brushes and increase the kinetics of degradation. This framework enables a control of degradation that allows for the precise removal of these coatings.

Rapid electrochemical reduction of Ni(II) generates reactive monolayers for conjugated polymer brushes in one step.

This article reports the development of a robust, one-step electrochemical technique to generate surface-bound conjugated polymers. The electrochemical reduction of arene diazonium salts at the surface of a gold electrode is used to generate tethered bromobenzene monolayers quickly. The oxidative addition of reactive Ni(0) across the aryl halide bond is achieved in situ through a concerted electrochemical reduction of Ni(dppp)Cl2. This technique limits the diffusion of Ni(0) species away from the surface and overcomes the need for solution deposition techniques which often require multiple steps that result in a loss of surface coverage. With this electrochemical technique, the formation of the reactive monolayer resulted in a surface coverage of 1.29 × 10(14) molecules/cm(2), which is a 6-fold increase over previously reported results using solution deposition techniques.

π-Complexation in nickel-catalyzed cross-coupling reactions.

The kinetic isotope effect (KIE) is used to experimentally elucidate the first irreversible step in oxidative addition reactions of a zerovalent nickel catalyst to a set of haloarene substrates. Halogenated o-methylbenzene, dimethoxybenzene, and thiophene derivatives undergo intramolecular oxidative addition through irreversible π-complexation. Density functional theory computations at the B3LYP-D3/TZ2P-LANL2TZ(f)-LANL08d level predict η(2)-bound π-complexes are generally stable relative to a solvated catalyst plus free substrate and that ring-walking of the Ni(0) catalyst and intramolecular oxidative addition are facile in these intermediates.

Exact ligand cone angles.

Many properties of transition-metal complexes depend on the steric bulk of bound ligands, usually quantified by the Tolman (θ) and solid (Θ) cone angles, which have proven utility but suffer from various limitations and coarse approximations. Here, we present an improved, mathematically rigorous method to determine an exact cone angle (θ°) by solving for the most acute right circular cone that contains the entire ligand. The procedure is applicable to any ligand, planar or nonplanar, monodentate or polydentate, bound to any metal center in any environment, and it is ideal for analyzing structures from quantum chemical computations as well as X-ray crystallography experiments. Exact cone angles were evaluated for a wide array of phosphine and amine ligands bound to palladium, nickel, or platinum by optimizing structures using B3LYP/6-31G* density functional theory with effective core potentials for the transition metals. The mean absolute deviations of the standard θ and Θ parameters from the exact cone angles were 15-25°, mostly caused by distortions from the assumed idealized structures.

Exact Ligand Solid Angles.

Steric demands of a ligand can be quantified by the area occluded by the ligand on the surface of an encompassing sphere centered at the metal atom. When viewed as solid spheres illuminated by the metal center, the ligand atoms generally cast a very complicated collective shadow onto the encompassing sphere, causing mathematical difficulties in computing the subtended solid angle. Herein, an exact, analytic solution to the ligand solid angle integration problem is presented based on a line integral around the multisegmented perimeter of the ligand shadow. The solution, which is valid for any ligand bound to any metal center, provides an excellent method for analyzing geometric structures from quantum chemical computations or X-ray crystallography. Over 275 structures of various metals bound to diverse mono- and multidentate ligands were optimized using B3LYP density functional theory to exhibit exact solid angle (Ω°) computations. Among the intriguing Ω° solutions, Pd(xantphos) and ferrocene exhibit holes in their ligand shadows, and Fe(EDTA)(2-) has a surprisingly simple shadow defined by only four arcs, despite having a multitude of overlaps among individual shadow cones.

Palladium-Mediated Surface-Initiated Kumada Catalyst Polycondensation: A Facile Route Towards Oriented Conjugated Polymers.

Palladium-mediated surface-initiated Kumada catalyst transfer polycondensation is used to generate poly(3-methyl thiophene) films with controlled thickness up to 100 nm. The palladium initiator density is measured using cyclic voltammetry and a ferrocene-capping agent, where the surface density is found to be 55% (1.1 × 10(14) molecules per cm(2)). UV-Vis spectroscopy and AFM show increased aggregation in palladium-initiated films due to the higher grafting density of palladium initiators on the surface. The anisotropy of the P3MT films is determined using polarized UV-Vis spectroscopy, which indicates a degree of orientation perpendicular to the substrate. Evidence that palladium can maintain π-complexation even at elevated temperatures, is also shown through the exclusive intramolecular coupling of both a phenyl and thiophene-based magnesium bromide with different dihaloarenes.

On the Role of Disproportionation Energy in Kumada Catalyst-Transfer Polycondensation.

Kumada catalyst-transfer polycondensation (KCTP) is an effective method for the controlled polymerization of conjugated polymers. Nevertheless, side reactions leading to early termination and unwanted chain coupling cause deviations from the target molecular weight, along with increasing polydispersity and end group variation. The departure from the KCTP cycle stems from a disproportionation reaction that leads to experimentally observed side products. The disproportionation energies for a series of nickel-based initiators containing bidentate phosphino attendant ligands were computed using density functional theory at the B3LYP/DZP level. The initiator was found to be less favorable toward disproportionation by 0.5 kcal mol-1 when ligated by 1,3-bis(diphenylphosphino)propane (dppp) rather than 1,2-bis(diphenylphosphino)ethane (dppe). Trends in disproportionation energy (Edisp) with a variety of bidentate phosphine ligands match experimental observations of decreased polymerization control. Theoretical Edisp values can thus be used to predict the likelihood of disproportionation in cross-coupling reactions and, therefore, aid in catalyst design.