Acceleration of Graph Neural Network-Based Prediction Models in Chemistry via Co-Design Optimization on Intelligence Processing Units.

Atomic structure prediction and associated property calculations are the bedrock of chemical physics. Since high-fidelity ab initio modeling techniques for computing the structure and properties can be prohibitively expensive, this motivates the development of machine-learning (ML) models that make these predictions more efficiently. Training graph neural networks over large atomistic databases introduces unique computational challenges, such as the need to process millions of small graphs with variable size and support communication patterns that are distinct from learning over large graphs, such as social networks. We demonstrate a novel hardware-software codesign approach to scale up the training of atomistic graph neural networks (GNN) for structure and property prediction. First, to eliminate redundant computation and memory associated with alternative padding techniques and to improve throughput via minimizing communication, we formulate the effective coalescing of the batches of variable-size atomistic graphs as the bin packing problem and introduce a hardware-agnostic algorithm to pack these batches. In addition, we propose hardware-specific optimizations, including a planner and vectorization for the gather-scatter operations targeted for Graphcore's Intelligence Processing Unit (IPU), as well as model-specific optimizations such as merged communication collectives and optimized softplus. Putting these all together, we demonstrate the effectiveness of the proposed codesign approach by providing an implementation of a well-established atomistic GNN on the Graphcore IPUs. We evaluate the training performance on multiple atomistic graph databases with varying degrees of graph counts, sizes, and sparsity. We demonstrate that such a codesign approach can reduce the training time of atomistic GNNs and can improve their performance by up to 1.5× compared to the baseline implementation of the model on the IPUs. Additionally, we compare our IPU implementation with a Nvidia GPU-based implementation and show that our atomistic GNN implementation on the IPUs can run 1.8× faster on average compared to the execution time on the GPUs.

Descriptors of water aggregation.

We rely on a total of 23 (cluster size, 8 structural, and 14 connectivity) descriptors to investigate structural patterns and connectivity motifs associated with water cluster aggregation. In addition to the cluster size n (number of molecules), the 8 structural descriptors can be further categorized into (i) one-body (intramolecular): covalent OH bond length (rOH) and HOH bond angle (θHOH), (ii) two-body: OO distance (rOO), OHO angle (θOHO), and HOOX dihedral angle (ϕHOOX), where X lies on the bisector of the HOH angle, (iii) three-body: OOO angle (θOOO), and (iv) many-body: modified tetrahedral order parameter (q) to account for two-, three-, four-, five-coordinated molecules (qm, m = 2, 3, 4, 5) and radius of gyration (Rg). The 14 connectivity descriptors are all many-body in nature and consist of the AD, AAD, ADD, AADD, AAAD, AAADD adjacencies [number of hydrogen bonds accepted (A) and donated (D) by each water molecule], Wiener index, Average Shortest Path Length, hydrogen bond saturation (% HB), and number of non-short-circuited three-membered cycles, four-membered cycles, five-membered cycles, six-membered cycles, and seven-membered cycles. We mined a previously reported database of 4 948 959 water cluster minima for (H2O)n, n = 3-25 to analyze the evolution and correlation of these descriptors for the clusters within 5 kcal/mol of the putative minima. It was found that rOH and % HB correlated strongly with cluster size n, which was identified as the strongest predictor of energetic stability. Marked changes in the adjacencies and cycle count were observed, lending insight into changes in the hydrogen bond network upon aggregation. A Principal Component Analysis (PCA) was employed to identify descriptor dependencies and group clusters into specific structural patterns across different cluster sizes. The results of this study inform our understanding of how water clusters evolve in size and what appropriate descriptors of their structural and connectivity patterns are with respect to system size, stability, and similarity. The approach described in this study is general and can be easily extended to other hydrogen-bonded systems.

Accurate Calculation of Many-Body Energies in Water Clusters Using a Classical Geometry-Dependent Induction Model.

We incorporate geometry-dependent distributed multipole and polarizability surfaces into an induction model that is used to describe the 3- and 4-body terms of the interaction between water molecules. The moment expansion is carried out up to the hexadecapole with the multipoles distributed on the atom sites. Dipole-dipole, dipole-quadrupole, and quadrupole-quadrupole distributed polarizabilities are used to represent the response of the multipoles to an electric field. We compare the model against two large databases consisting of 43,844 3-body terms and 3,603 4-body terms obtained from high level ab initio calculations previously used to fit the MB-pol and q-AQUA classical interaction potentials for water. The classical induction model with no adjustable parameters reproduces the ab initio 3-/4-body terms contained in these two databases with a root-mean-square error (RMSE) of 0.104/0.058 and a mean-absolute error (MAE) of 0.054/0.026 kcal/mol, respectively. These results are on par with the ones obtained by fitting the same data using over 14,000 (for the 3-body) and 200 (for the 4-body) parameters via Permutationally Invariant Polynomials (PIPs). This demonstrates the accuracy of this physically motivated model in describing the 3- and 4-body terms in the interactions between water molecules with no adjustable parameters. The triple-dipole-dispersion energy, included in the calculation of the 3-body energy, was found to be small but not quite negligible. The model represents a practical, efficient, and transferable approach for obtaining accurate nonadditive interactions for multicomponent systems without the need to perform tens of thousands of high level electronic structure calculations and fitting them with PIPs.

Many-Body Effects in Aqueous Systems: Synergies Between Interaction Analysis Techniques and Force Field Development.

Interaction analysis techniques, including the many-body expansion (MBE), symmetry-adapted perturbation theory, and energy decomposition analysis, allow for an intuitive understanding of complex molecular interactions. We review these methods by first providing a historical context for the study of many-body interactions and discussing how nonadditivities emerge from Hamiltonians containing strictly pairwise-additive interactions. We then elaborate on the synergy between these interaction analysis techniques and the development of advanced force fields aimed at accurately reproducing the Born-Oppenheimer potential energy surface. In particular, we focus on ab initio-based force fields that aim to explicitly reproduce many-body terms and are fitted to high-level electronic structure results. These force fields generally incorporate many-body effects through (a) parameterization of distributed multipoles, (b) explicit fitting of the MBE, (c) inclusion of many-atom features in a neural network, and (d) coarse-graining of many-body terms into an effective two-body term. We also discuss the emerging use of the MBE to improve the accuracy and speed of ab initio molecular dynamics.

An extensive assessment of the performance of pairwise and many-body interaction potentials in reproducing ab initio benchmark binding energies for water clusters n = 2-25.

We assess the performance of 7 pairwise additive (TIP3P, TIP4P, TIP4P-ice, TIP5P, OPC, SPC, SPC/E) and 8 families of many-body potentials (q-AQUA, HIPPO, AMOEBA, EFP, TTM, WHBB, MB-pol, MB-UCB) in reproducing high-level ab initio benchmark values, CCSD(T) or MP2 at the complete basis set (CBS) limit for the binding energy and the many-body expansion (MBE) of water clusters n = 2-11, 16-17, 20, 25. By including a large range of cluster sizes having dissimilar hydrogen bonding networks, we obtain an understanding of how these potentials perform for different hydrogen bonding arrangements that are mostly outside of their parameterization range. While it is appropriate to compare the results of ab initio based many-body potentials directly to the electronic binding energies (De's), the pairwise additive ones are compared to the enthalpies at T = 298 K, ΔH(298 K), as the latter class of force fields are parametrized to reproduce enthalpies (implicitly accounting for zero-point energy corrections) rather than binding energies. We find that all pairwise additive potentials considered overestimate the reference ΔH values for the n = 2-25 clusters by >13%. For the water dimer (n = 2) in particular, the errors are in the range 83-119% for the pairwise additive potentials studied since these are based on an effective rather than the true 2-body interaction specifically designed as a means of partially accounting for the missing many-body terms. This stronger 2-body interaction is achieved by an enhanced monomer dipole moment that mimics its increase from the gas phase monomer to the condensed phase value. Indeed, for cluster sizes n ≥ 4 the percent deviations become slightly smaller (albeit all exceeding 13%). In contrast, we find that the many-body potentials perform more accurately in reproducing the electronic binding energies (De's) throughout the entire cluster range (n = 2-25), all reproducing the ab initio benchmark binding energies within ±7% of the respective CBS values. We further assess the ability of a subset of the many-body potentials (MB-UCB, q-AQUA, MB-pol, and TTM2.1-F) to also reproduce the magnitude of the ab initio many-body energy terms for water cluster sizes n = 7, 10, 16 and 17. The potentials show an overall good agreement with the available benchmark values. However, we identify characteristic differences upon comparing the many-body terms at both the ab initio-optimized geometries and the respective potential-optimized geometries to the reference ab initio values. Additionally, by applying this analysis to a wide range of cluster sizes, trends in the MBE of the potentials with increasing cluster size can be identified. Finally, in an attempt to draw a parallel between the pairwise additive and many-body potentials, we report the analysis of the individual molecular dipole moments for water clusters with 1 to ∼4 solvation shells with the TTM2.1-F potential. We find that the internally solvated water molecules have in general a larger molecular dipole moment ranging from 2.6-3.0 D. This justifies the use of an enhanced, with respect to the gas-phase value, molecular dipole moment for the pairwise additive potentials, which is intended to fold in the many body terms into an effective (enhanced) pairwise interaction through the choice of the charges. These results have important implications for the development of future generations of efficient, transferable, and highly accurate classical interaction potentials in both the pairwise additive and many-body categories.

A Formulation of the Many-Body Expansion (MBE) for Periodic Systems: Application to Several Ice Phases.

We introduce a new formulation of the many-body expansion (MBE) for periodic systems and apply it to 7 ice polymorphs (Ih, II, VIII, IX, XIII, XIV, and XV). This new formulation is built via a hierarchical procedure that connects gas-phase clusters that mimic unit cells over finite supercells to infinite solids. For periodic systems, the method is validated by showing that the lattice energies computed up to the 4-body in the MBE reproduce the lattice energies obtained using periodic boundary conditions with an Ewald summation for the 7 ice polymorphs. This development makes it possible to quantify, for the first time, the many-body contributions to the lattice energy of various ice polymorphs, which vary significantly among the 7 ice phases, amounting to between 7 and 24% of the total lattice energies. This development opens the door for obtaining insights into solid-state properties, while leveraging the computational benefits of the MBE.

A critical comparison of CH⋯π versus π⋯π interactions in the benzene dimer: obtaining benchmarks at the CCSD(T) level and assessing the accuracy of lower scaling methods.

We have established CCSD(T)/CBS (Complete Basis Set) limits for 3 stationary points on the benzene dimer potential energy surface, corresponding to the π⋯π (parallel displaced or PD(C2h), minimum) and CH⋯π (T-shaped or T(C2v), transition state) and tilted T-shaped (or TT(Cs), minimum) bonding scenarios considering both the structure and binding energy. The CCSD(T)/CBS binding energies are -2.65 ± 0.02 (PD), -2.74 ± 0.03 (T), and -2.83 ± 0.01 kcal mol-1 (TT). To this end, the CH⋯π is ∼0.2 kcal mol-1 stronger than the π⋯π interaction, whereas the tilting of the CH donating benzene molecule with respect to the other benzene is worth 0.1 kcal mol-1. As previously discussed in the literature, the MP2 level of theory does not provide a close match for either the energy or structure, yet the SCS-MP2 yields structures in excellent agreement with respect to the CCSD(T) result. It is found that the SCS-MI-MP2 also gives optimized structures very close to SCS-MP2 (within ∼0.01 Å of the benchmark). Despite the closer match in structure, the spin-biased MP2 methods (SCS-, SCS-MI-, and SOS-MP2) incorrectly predict the relative stabilities of the isomers. That said, none of the spin biased MP2 methods offers a good compromise between energy and structure for the systems examined. Finally, the CCSD(T)/CBS benchmarks were used to assess the performance of 13 DFT functionals selected from different rungs of Jacob's ladder. Several functionals such as TPSS-D3, B3LYP-D3, B97-D, B97-D3, and B2PLYP-D3 provided a good description of the binding energies for both CH⋯π and π⋯π interactions, yielding values within 6% of the CCSD(T)/CBS benchmark values. Unlike the MP2 methods, these functionals correctly predict the relative stability of the PD(C2h) and T(C2v) dimers. Further, we find that there is no systematic improvement as Jacob's ladder is ascended (increased complexity of functional). The best functionals that result in a good compromise between structure and energy accuracy are B97-D3 and B2PLYP-D3 for both the CH⋯π and π⋯π interaction. Despite the impressive performance of these functionals, a challenge that remains is ensuring the transferability of these density functionals in accurately describing the interaction between dimers of larger aromatic molecules, the latter requiring high-level benchmarks for these systems.

A classical model for three-body interactions in aqueous ionic systems.

We present a classical induction model to evaluate the three-body ion-water-water (I-W-W) and water-water-water (W-W-W) interactions in aqueous ionic systems. The classical description of the induction energy is based on electrostatic distributed multipoles up to hexadecapole and distributed polarizabilities up to quadrupole-quadrupole on the O and H atoms of water. The monatomic ions were described by a point charge and a dipole-dipole polarizability, while for the polyatomic ions, distributed multipoles up to hexadecapole and distributed polarizabilities up to quadrupole-quadrupole were used. The accuracy of the classical model is benchmarked against an accurate dataset of 936 (I-W-W) and 2184 (W-W-W) three-body terms for 13 different monatomic and polyatomic cation and anion systems. The classical model shows excellent agreement with the reference second order Moller-Plesset and coupled-cluster single double and perturbative triple [CCSD(T)] three-body energies. The Root-Mean-Square-Errors (RMSEs) for monatomic cations, monatomic anions, and polyatomic ions were 0.29, 0.25, and 0.12 kcal/mol, respectively. The corresponding RMSE for 1744 CCSD(T)/aVTZ three-body (W-W-W) energies, used to train MB-pol, was 0.12 kcal/mol. The accuracy of the proposed classical model demonstrates that the three-body term for aqueous ionic systems can be accurately modeled classically. This approach provides a fast, efficient, and as-accurate path toward modeling the three-body term in aqueous ionic systems that is fully transferable across systems with different ions without the need to fit to tens of thousands of ab initio calculations for each ion to extend existing many-body force fields to interactions between water and ions.

Guest-Host Interactions in Clathrate Hydrates: Benchmark MP2 and CCSD(T)/CBS Binding Energies of CH4, CO2, and H2S in (H2O)20 Cages.

We present benchmark binding energies of naturally occurring gas molecules CH4, CO2, and H2S in the small cage, namely, the pentagonal dodecahedron (512) (H2O)20, which is one of the constituent cages of the 3 major lattices (structures I, II, and H) of clathrate hydrates. These weak interactions require higher levels of electron correlation and converge slowly with an increasing basis set to the complete basis set (CBS) limit, necessitating the use of large basis sets up to the aug-cc-pV5Z and subsequent correction for basis set superposition error (BSSE). For the host hollow (H2O)20 cages, we have identified a most stable isomer with binding energy of -200.8 ± 2.1 kcal/mol at the CCSD(T)/CBS limit (-199.2 ± 0.5 kcal/mol at the MP2/CBS limit). Additionally, we report converged second order Møller-Plesset (MP2) CBS binding energies for the encapsulation of guests in the (H2O)20 cage of -4.3 ± 0.1 for CH4@(H2O)20, -6.6 ± 0.1 for CO2@(H2O)20, and -8.5 ± 0.1 kcal/mol for H2S@(H2O)20, respectively. For CH4@(H2O)20, exhibiting the weakest encapsulation affinity among the three, we report CCSD(T)/aug-cc-pVTZ binding energies and, based on them, a CCSD(T)/CBS estimate of -4.75 ± 0.1 kcal/mol. To the best of our knowledge, the CCSD(T)/aug-cc-pVTZ calculation for CH4@(H2O)20 is the largest one reported to date (168 valence electrons, 1978 basis functions, and the correlation of 84 doubly occupied and 1873 virtual orbitals) and required a scalable implementation of the (T) module on 6144 nodes (350 208 cores) of the "Cori" supercomputer at the National Energy Research Supercomputing Center (NERSC) for a total execution time of 195 min (for the (T) part). These efficient scalable implementations of highly correlated methods offer the capability to obtain long-lasting benchmarks of intermolecular interactions in complex systems. They also provide a path toward parametrizing classical potentials needed to study the dynamical and transport properties in these complex systems as well as assess the accuracy of lower scaling electronic structure methods such as density functional theory (DFT) and MP2 including its spin-biased variants.

The many-body expansion for aqueous systems revisited: III. Hofmeister ion-water interactions.

We report a Many Body Energy (MBE) analysis of aqueous ionic clusters containing anions and cations at the two opposite ends of the Hofmeister series, viz. the kosmotropes Ca2+ and SO42- and the chaotropes NH4+ and ClO4-, with 9 water molecules to quantify how these ions alter the interaction between the water molecules in their immediate surroundings. We specifically aim at quantifying how various ions (depending on their position in the Hofmeister series) affect the interaction between the surrounding water molecules and probe whether there is a qualitatively different behavior between kosmotropic vs. chaotropic ions. The current results when compared to the ones reported earlier for water clusters [J. P. Heindel and S. S. Xantheas, J. Chem. Theor. Comput., 2020, 16, 6843-6855] as well as for alkali metal and halide ion aqueous clusters of the same size [J. P. Heindel and S. S. Xantheas, J. Chem. Theor. Comput., 2021, 17, 2200-2216], which lie in the middle of the Hofmeister series, offer a complete account of the effect an ion across the Hofmeister series from "kosmotropes" to "chaotropes" has on the interaction between the neighboring water molecules. Through this analysis, noteworthy differences between the MBE of kosmotropes and chaotropes were identified. The MBE of kosmotropes is dominated by ion-water interactions that extend beyond the 4-body term, the rank at which the MBE of pure water converges. The percentage contribution of the 2-B term to the total cluster binding energy is noticeably larger. The disruption of the hydrogen bonded network due to the dominant ion-water interactions results in weak, unfavorable water-water interactions. The MBE for chaotropes, on the other hand, was found to converge more quickly as it more closely resembles that of pure water clusters. Chaotropes exhibit weaker overall binding energies and weaker ion-water interactions in favor of water-water interactions, somewhat recovering the pattern of the 2-4 body terms exemplified by pure water clusters. A remarkable anti-correlation between the 2-B ion-water (I-W) and water-water (W-W) interactions as well as between the 3-B (I-W-W) and (I-W) interactions was found for both kosmotropic and chaotropic ions. This anti-correlation is linear for both monatomic anions and monatomic cations, suggesting the existence of underlying physical mechanisms that were previously unexplored. The consideration of two different structural arrangements (ion inside and outside of a water cluster) suggests that fully solvated (ion inside) chaotropes disrupt the hydrogen bonding network in a similar manner to partially solvated (ion outside) kosmotropes and offers useful insights into the modeling requirements of bulk vs. interfacial ion solvation. It is noteworthy that the 2-B contribution to the total Basis Set Superposition Error (BSSE) correction for both kosmotropic and chaotropic ions follows the universal erf profile vs. intermolecular distance previously reported for pure water, halide ion-water and alkali metal ion-water clusters. When scaled for the corresponding dimer energies and distances, a single profile fits the current results together with all previously reported ones for pure water and halide water clusters. This finding lends further support to schemes for accurately estimating the 2-B BSSE correction in condensed environments.