Transformer Performance for Chemical Reactions: Analysis of Different Predictive and Evaluation Scenarios.

The prediction of chemical reaction pathways has been accelerated by the development of novel machine learning architectures based on the deep learning paradigm. In this context, deep neural networks initially designed for language translation have been used to accurately predict a wide range of chemical reactions. Among models suited for the task of language translation, the recently introduced molecular transformer reached impressive performance in terms of forward-synthesis and retrosynthesis predictions. In this study, we first present an analysis of the performance of transformer models for product, reactant, and reagent prediction tasks under different scenarios of data availability and data augmentation. We find that the impact of data augmentation depends on the prediction task and on the metric used to evaluate the model performance. Second, we probe the contribution of different combinations of input formats, tokenization schemes, and embedding strategies to model performance. We find that less stable input settings generally lead to better performance. Lastly, we validate the superiority of round-trip accuracy over simpler evaluation metrics, such as top-k accuracy, using a committee of human experts and show a strong agreement for predictions that pass the round-trip test. This demonstrates the usefulness of more elaborate metrics in complex predictive scenarios and highlights the limitations of direct comparisons to a predefined database, which may include a limited number of chemical reaction pathways.

Artificial intelligence achieves easy-to-adapt nonlinear global temperature reconstructions using minimal local data.

Understanding monthly-to-annual climate variability is essential for adapting to future climate extremes. Key ways to do this are through analysing climate field reconstructions and reanalyses. However, producing such reconstructions can be limited by high production costs, unrealistic linearity assumptions, or uneven distribution of local climate records. Here, we present a machine learning-based non-linear climate variability reconstruction method using a Recurrent Neural Network that is able to learn from existing model outputs and reanalysis data. As a proof-of-concept, we reconstructed more than 400 years of global, monthly temperature anomalies based on sparse, realistically distributed pseudo-station data and show the impact of different training data sets. Our reconstructions show realistic temperature patterns and magnitude reproduction costing about 1 hour on a middle-class laptop. We highlight the method's capability in terms of mean statistics compared to more established methods and find that it is also suited to reconstruct specific climate events. This approach can easily be adapted for a wide range of regions, periods and variables.

Selection of optimal proxy locations for temperature field reconstructions using evolutionary algorithms.

In the Era of exponential data generation, increasing the number of paleoclimate records to improve climate field reconstructions might not always be the best strategy. By using pseudo-proxies from different model ensembles, we show how biologically-inspired artificial intelligence can be coupled with different reconstruction methods to minimize the spatial bias induced by the non-homogeneous distribution of available proxies. The results indicate that small subsets of records situated over representative locations can outperform the reconstruction skill of the full proxy network, even in more realistic pseudo-proxy experiments and observational datasets. These locations highlight the importance of high-latitude regions and major teleconnection areas to reconstruct annual global temperature fields and their responses to external forcings and internal variability. However, low frequency temperature variations such as the transition between the Medieval Climate Anomaly and the Little Ice Age are better resolved by records situated at lower latitudes. According to our idealized experiments a careful selection of proxy locations should be performed depending on the targeted time scale of the reconstructed field.