Understanding and overcoming the technical challenges in using in silico predictions in regulatory decisions of complex toxicological endpoints - A pesticide perspective for regulatory toxicologists with a focus on machine learning models.

There are many challenges that must be overcome before in silico toxicity predictions are ripe for regulatory decision-making. Today, mandates in the United States of America and the European Union to avoid animal usage in toxicity testing is driving the need to consider alternative technologies, including Quantitative Structure Activity Relationship (QSAR) models, and read across approaches. However, when adopting new methods, it is critical that both new approach developers as well as regulatory users understand the strengths and challenges with these new approaches. In this paper, we identify potential sources of bias in machine learning methods specific to toxicity predictions, that may impact the overall performance of in silico models. We also discuss ways to mitigate these biases. Based on our experiences, the most prevalent sources of bias include class imbalance (differing numbers of "toxic" vs "nontoxic" compounds), limited numbers of chemicals within a particular chemistry, and biases within the studies that make up the database used for model building, as well as model evaluation biases. While this is already complex for repeated dose toxicity, in reproduction and developmental toxicity a further level of complexity is introduced by the need to evaluate effects on individual animal and litter basis (e.g., a hierarchal structure). We also discuss key considerations developers and regulators need to make when they use machine learning models to predict chemical safety. Our objective is for our paper to serve as a desk reference for model developers and regulators as they evaluate machine learning models and as they make decisions using these models.