Accelerating Neural ODEs Using Model Order Reduction.

Embedding nonlinear dynamical systems into artificial neural networks is a powerful new formalism for machine learning. By parameterizing ordinary differential equations (ODEs) as neural network layers, these Neural ODEs are memory-efficient to train, process time series naturally, and incorporate knowledge of physical systems into deep learning (DL) models. However, the practical applications of Neural ODEs are limited due to long inference times because the outputs of the embedded ODE layers are computed numerically with differential equation solvers that can be computationally demanding. Here, we show that mathematical model order reduction (MOR) methods can be used for compressing and accelerating Neural ODEs by accurately simulating the continuous nonlinear dynamics in low-dimensional subspaces. We implement our novel compression method by developing Neural ODEs that integrate the necessary subspace-projection and interpolation operations as layers of the neural network. We validate our approach by comparing it to neuron pruning and singular value decomposition (SVD)-based weight truncation methods from the literature in image and time-series classification tasks. The methods are evaluated by acceleration versus accuracy when adjusting the level of compression. On this spectrum, we achieve a favorable balance over existing methods by using MOR when compressing a convolutional Neural ODE. In compressing a recurrent Neural ODE, SVD-based weight truncation yields good performance. Based on our results, our integration of MOR with Neural ODEs can facilitate efficient, dynamical system-driven DL in resource-constrained applications.

Electronic couplings and rates of excited state charge transfer processes at poly(thiophene-co-quinoxaline)-PC71BM interfaces: two- versus multi-state treatments.

Electronic coupling between adjacent molecules is one of the key parameters determining the charge transfer (CT) rates in bulk heterojunction (BHJ) polymer solar cells (PSCs). We calculate theoretically electronic couplings for exciton dissociation (ED) and charge recombination (CR) processes at local poly(thiophene-co-quinoxaline) (TQ)-PC71BM interfaces. We use eigenstate-based coupling schemes, i.e. the generalized Mulliken-Hush (GMH) and fragment charge difference (FCD) schemes, including 2 to multiple (3-11) states. Moreover, we study the effects of functionals, excited state methods, basis sets, surrounding media, and relative placements of TQ and PC71BM on the coupling values. Generally, both schemes provide consistent couplings with the global hybrid functionals, which yield more charge-localized diabatic states and constant coupling values regardless of the number of states, and so the 2-state schemes may be sufficient. The (non-tuned and optimally tuned) long-range corrected (LRC) functionals result in more notable mixing of the local components with the CT states. Employing multiple states reduces the mixing and thus improves the LRC results, although the method still affects the GMH CR couplings. As the FCD scheme is less sensitive, we recommend combining it with the multi-state treatment for polymer-fullerene systems when using the LRC functionals. Finally, we employ the 11-state FCD couplings to calculate the ED and CR rates, which are consistent with the experimental rates of the polymer-fullerene systems. Our results provide more insight into choosing a suitable eigenstate-based coupling scheme for predicting the electronic couplings and CT rates in photoactive systems.