CoolMomentum: a method for stochastic optimization by Langevin dynamics with simulated annealing.

Deep learning applications require global optimization of non-convex objective functions, which have multiple local minima. The same problem is often found in physical simulations and may be resolved by the methods of Langevin dynamics with Simulated Annealing, which is a well-established approach for minimization of many-particle potentials. This analogy provides useful insights for non-convex stochastic optimization in machine learning. Here we find that integration of the discretized Langevin equation gives a coordinate updating rule equivalent to the famous Momentum optimization algorithm. As a main result, we show that a gradual decrease of the momentum coefficient from the initial value close to unity until zero is equivalent to application of Simulated Annealing or slow cooling, in physical terms. Making use of this novel approach, we propose CoolMomentum-a new stochastic optimization method. Applying Coolmomentum to optimization of Resnet-20 on Cifar-10 dataset and Efficientnet-B0 on Imagenet, we demonstrate that it is able to achieve high accuracies.

Patent citation network analysis: A perspective from descriptive statistics and ERGMs.

Patent Citation Analysis has been gaining considerable traction over the past few decades. In this paper, we collect extensive information on patents and citations and provide a perspective of citation network analysis of patents from a statistical viewpoint. We identify and analyze the most cited patents, the most innovative and the highly cited companies along with the structural properties of the network by providing in-depth descriptive analysis. Furthermore, we employ Exponential Random Graph Models (ERGMs) to analyze the citation networks. ERGMs enables understanding the social perspectives of a patent citation network which has not been studied earlier. We demonstrate that social properties such as homophily (the inclination to cite patents from the same country or in the same language) and transitivity (the inclination to cite references' references) together with the technicalities of the patents (e.g., language, categories), has a significant effect on citations. We also provide an in-depth analysis of citations for sectors in patents and how it is affected by the size of the same. Overall, our paper delves into European patents with the aim of providing new insights and serves as an account for fitting ERGMs on large networks and analyzing them. ERGMs help us model network mechanisms directly, instead of acting as a proxy for unspecified dependence and relationships among the observations.

Hybrid particle-field molecular dynamics under constant pressure.

Hybrid particle-field methods are computationally efficient approaches for modeling soft matter systems. So far, applications of these methodologies have been limited to constant volume conditions. Here, we reformulate particle-field interactions to represent systems coupled to constant external pressure. First, we show that the commonly used particle-field energy functional can be modified to model and parameterize the isotropic contributions to the pressure tensor without interfering with the microscopic forces on the particles. Second, we employ a square gradient particle-field interaction term to model non-isotropic contributions to the pressure tensor, such as in surface tension phenomena. This formulation is implemented within the hybrid particle-field molecular dynamics approach and is tested on a series of model systems. Simulations of a homogeneous water box demonstrate that it is possible to parameterize the equation of state to reproduce any target density for a given external pressure. Moreover, the same parameterization is transferable to systems of similar coarse-grained mapping resolution. Finally, we evaluate the feasibility of the proposed approach on coarse-grained models of phospholipids, finding that the term between water and the lipid hydrocarbon tails is alone sufficient to reproduce the experimental area per lipid in constant-pressure simulations and to produce a qualitatively correct lateral pressure profile.

Simulation of self-heating process on the nanoscale: a multiscale approach for molecular models of nanocomposite materials.

A theoretical-computational protocol to model the Joule heating process in nanocomposite materials is presented. The proposed modeling strategy is based on post processing of trajectories obtained from large scale molecular simulations. This protocol, based on molecular models, is the first one to be applied to organic nanocomposites based on carbon nanotubes (CNT). This strategy allows to keep a microscopic explicit picture of the systems, to directly catch the molecular structure underlying the process under study and, at the same time, to include macroscopic boundary conditions fixed in the experiments. As validation and first application of the proposed strategy, a detailed investigation on CNT based organic composites is reported. The effect of CNT morphologies, concentration and working conditions on Joule heating has been modelled and compared with available experiments. Further experiments are performed also in this work to increase the number of comparisons especially in specific voltage ranges where available references from literature were missing. Simulations are in both qualitative and quantitative agreement with several experiments and trends reported in the recent literature, as well as with experiments performed in this work. The proposed approach combined with large scale hybrid particle-field molecular simulations can give insights and opens to way to a rational design of self-heating nanocomposites.

Fast Maximum Likelihood Estimation via Equilibrium Expectation for Large Network Data.

A major line of contemporary research on complex networks is based on the development of statistical models that specify the local motifs associated with macro-structural properties observed in actual networks. This statistical approach becomes increasingly problematic as network size increases. In the context of current research on efficient estimation of models for large network data sets, we propose a fast algorithm for maximum likelihood estimation (MLE) that affords a significant increase in the size of networks amenable to direct empirical analysis. The algorithm we propose in this paper relies on properties of Markov chains at equilibrium, and for this reason it is called equilibrium expectation (EE). We demonstrate the performance of the EE algorithm in the context of exponential random graph models (ERGMs) a family of statistical models commonly used in empirical research based on network data observed at a single period in time. Thus far, the lack of efficient computational strategies has limited the empirical scope of ERGMs to relatively small networks with a few thousand nodes. The approach we propose allows a dramatic increase in the size of networks that may be analyzed using ERGMs. This is illustrated in an analysis of several biological networks and one social network with 104,103 nodes.

Self-assembly of carbon nanotubes in polymer melts: simulation of structural and electrical behaviour by hybrid particle-field molecular dynamics.

Self-assembly processes of carbon nanotubes (CNTs) dispersed in different polymer phases have been investigated using a hybrid particle-field molecular dynamics technique (MD-SCF). This efficient computational method allowed simulations of large-scale systems (up to ∼1 500 000 particles) of flexible rod-like particles in different matrices made of bead spring chains on the millisecond time scale. The equilibrium morphologies obtained for longer CNTs are in good agreement with those proposed by several experimental studies that hypothesized a two level "multiscale" organization of CNT assemblies. In addition, the electrical properties of the assembled structures have been calculated using a resistor network approach. The calculated behaviour of the conductivities for longer CNTs is consistent with the power laws obtained by numerous experiments. In particular, according to the interpretation established by the systematic studies of Bauhofer and Kovacs, systems close to "statistical percolation" show exponents t ∼ 2 for the power law dependence of the electrical conductivity on the CNT fraction, and systems in which the CNTs reach equilibrium aggregation show exponents t close to 1.7 ("kinetic percolation"). The confinement effects on the assembled structures and their corresponding conductivity behaviour in a non-homogeneous matrix, such as the phase separating block copolymer melt, have also been simulated using different starting configurations. The simulations reported herein contribute to a microscopic interpretation of the literature results, and the proposed modelling procedure may contribute meaningfully to the rational design of strategies aimed at optimizing nanomaterials for improved electrical properties.