Large-scale evaluation of k-fold cross-validation ensembles for uncertainty estimation.

It is insightful to report an estimator that describes how certain a model is in a prediction, additionally to the prediction alone. For regression tasks, most approaches implement a variation of the ensemble method, apart from few exceptions. Instead of a single estimator, a group of estimators yields several predictions for an input. The uncertainty can then be quantified by measuring the disagreement between the predictions, for example by the standard deviation. In theory, ensembles should not only provide uncertainties, they also boost the predictive performance by reducing errors arising from variance. Despite the development of novel methods, they are still considered the "golden-standard" to quantify the uncertainty of regression models. Subsampling-based methods to obtain ensembles can be applied to all models, regardless whether they are related to deep learning or traditional machine learning. However, little attention has been given to the question whether the ensemble method is applicable to virtually all scenarios occurring in the field of cheminformatics. In a widespread and diversified attempt, ensembles are evaluated for 32 datasets of different sizes and modeling difficulty, ranging from physicochemical properties to biological activities. For increasing ensemble sizes with up to 200 members, the predictive performance as well as the applicability as uncertainty estimator are shown for all combinations of five modeling techniques and four molecular featurizations. Useful recommendations were derived for practitioners regarding the success and minimum size of ensembles, depending on whether predictive performance or uncertainty quantification is of more importance for the task at hand.

Machine Learning Applied to the Modeling of Pharmacological and ADMET Endpoints.

The well-known concept of quantitative structure-activity relationships (QSAR) has been gaining significant interest in the recent years. Data, descriptors, and algorithms are the main pillars to build useful models that support more efficient drug discovery processes with in silico methods. Significant advances in all three areas are the reason for the regained interest in these models. In this book chapter we review various machine learning (ML) approaches that make use of measured in vitro/in vivo data of many compounds. We put these in context with other digital drug discovery methods and present some application examples.

Discovery and Characterization of the Potent and Highly Selective 1,7-Naphthyridine-Based Inhibitors BAY-091 and BAY-297 of the Kinase PIP4K2A.

PIP4K2A is an insufficiently studied type II lipid kinase that catalyzes the conversion of phosphatidylinositol-5-phosphate (PI5P) into phosphatidylinositol 4,5-bisphosphate (PI4,5P2). The involvement of PIP4K2A/B in cancer has been suggested, particularly in the context of p53 mutant/null tumors. PIP4K2A/B depletion has been shown to induce tumor growth inhibition, possibly due to hyperactivation of AKT and reactive oxygen species-mediated apoptosis. Herein, we report the identification of the novel potent and highly selective inhibitors BAY-091 and BAY-297 of the kinase PIP4K2A by high-throughput screening and subsequent structure-based optimization. Cellular target engagement of BAY-091 and BAY-297 was demonstrated using cellular thermal shift assay technology. However, inhibition of PIP4K2A with BAY-091 or BAY-297 did not translate into the hypothesized mode of action and antiproliferative activity in p53-deficient tumor cells. Therefore, BAY-091 and BAY-297 serve as valuable chemical probes to study PIP4K2A signaling and its involvement in pathophysiological conditions such as cancer.

Bayer's in silico ADMET platform: a journey of machine learning over the past two decades.

Over the past two decades, an in silico absorption, distribution, metabolism, and excretion (ADMET) platform has been created at Bayer Pharma with the goal to generate models for a variety of pharmacokinetic and physicochemical endpoints in early drug discovery. These tools are accessible to all scientists within the company and can be a useful in assisting with the selection and design of novel leads, as well as the process of lead optimization. Here. we discuss the development of machine-learning (ML) approaches with special emphasis on data, descriptors, and algorithms. We show that high company internal data quality and tailored descriptors, as well as a thorough understanding of the experimental endpoints, are essential to the utility of our models. We discuss the recent impact of deep neural networks and show selected application examples.

Characterization of the Menin-MLL Interaction as Therapeutic Cancer Target.

Inhibiting the interaction of menin with the histone methyltransferase MLL1 (KMT2A) has recently emerged as a novel therapeutic strategy. Beneficial therapeutic effects have been postulated in leukemia, prostate, breast, liver and in synovial sarcoma models. In those indications, MLL1 recruitment by menin was described to critically regulate the expression of disease associated genes. However, most findings so far rely on single study reports. Here we independently evaluated the pathogenic functions of the menin-MLL interaction in a large set of different cancer models with a potent and selective probe inhibitor BAY-155. We characterized the inhibition of the menin-MLL interaction for anti-proliferation, gene transcription effects, and for efficacy in several in vivo xenografted tumor models. We found a specific therapeutic activity of BAY-155 primarily in AML/ALL models. In solid tumors, we observed anti-proliferative effects of BAY-155 in a surprisingly limited fraction of cell line models. These findings were further validated in vivo. Overall, our study using a novel, highly selective and potent inhibitor, shows that the menin-MLL interaction is not essential for the survival of most solid cancer models. We can confirm that disrupting the menin-MLL complex has a selective therapeutic benefit in MLL-fused leukemia. In solid cancers, effects are restricted to single models and more limited than previously claimed.

Modeling Physico-Chemical ADMET Endpoints with Multitask Graph Convolutional Networks.

Simple physico-chemical properties, like logD, solubility, or melting point, can reveal a great deal about how a compound under development might later behave. These data are typically measured for most compounds in drug discovery projects in a medium throughput fashion. Collecting and assembling all the Bayer in-house data related to these properties allowed us to apply powerful machine learning techniques to predict the outcome of those assays for new compounds. In this paper, we report our finding that, especially for predicting physicochemical ADMET endpoints, a multitask graph convolutional approach appears a highly competitive choice. For seven endpoints of interest, we compared the performance of that approach to fully connected neural networks and different single task models. The new model shows increased predictive performance compared to previous modeling methods and will allow early prioritization of compounds even before they are synthesized. In addition, our model follows the generalized solubility equation without being explicitly trained under this constraint.

Discovery and Characterization of the Potent and Selective P2X4 Inhibitor N-[4-(3-Chlorophenoxy)-3-sulfamoylphenyl]-2-phenylacetamide (BAY-1797) and Structure-Guided Amelioration of Its CYP3A4 Induction Profile.

The P2X4 receptor is a ligand-gated ion channel that is expressed on a variety of cell types, especially those involved in inflammatory and immune processes. High-throughput screening led to a new class of P2X4 inhibitors with substantial CYP 3A4 induction in human hepatocytes. A structure-guided optimization with respect to decreased pregnane X receptor (PXR) binding was started. It was found that the introduction of larger and more polar substituents on the ether linker led to less PXR binding while maintaining the P2X4 inhibitory potency. This translated into significantly reduced CYP 3A4 induction for compounds 71 and 73. Unfortunately, the in vivo pharmacokinetic (PK) profiles of these compounds were insufficient for the desired profile in humans. However, BAY-1797 (10) was identified and characterized as a potent and selective P2X4 antagonist. This compound is suitable for in vivo studies in rodents, and the anti-inflammatory and anti-nociceptive effects of BAY-1797 were demonstrated in a mouse complete Freund's adjuvant (CFA) inflammatory pain model.

Discovery and optimization of pyridyl-cycloalkyl-carboxylic acids as inhibitors of microsomal prostaglandin E synthase-1 for the treatment of endometriosis.

Here we report on novel and potent pyridyl-cycloalkyl-carboxylic acid inhibitors of microsomal prostaglandin E synthase-1 (PTGES). PTGES produces, as part of the prostaglandin pathway, prostaglandin E2 which is a well-known driver for pain and inflammation. This fact together with the observed upregulation of PTGES during inflammation suggests that blockade of the enzyme might provide a beneficial treatment option for inflammation related conditions such as endometriosis. Compound 5a, a close analogue of the screening hit, potently inhibited PTGES in vitro, displayed excellent PK properties in vitro and in vivo and demonstrated efficacy in a CFA-induced pain model in mice and in a rat dyspareunia endometriosis model and was therefore selected for further studies.

Identification of a Benzimidazolecarboxylic Acid Derivative (BAY 1316957) as a Potent and Selective Human Prostaglandin E2 Receptor Subtype 4 (hEP4-R) Antagonist for the Treatment of Endometriosis.

The presence and growth of endometrial tissue outside the uterine cavity in endometriosis patients are primarily driven by hormone-dependent and inflammatory processes-the latter being frequently associated with severe, acute, and chronic pelvic pain. The EP4 subtype of prostaglandin E2 (PGE2) receptors (EP4-R) is a particularly promising anti-inflammatory and antinociceptive target as both this receptor subtype and the pathways forming PGE2 are highly expressed in endometriotic lesions. High-throughput screening resulted in the identification of benzimidazole derivatives as novel hEP4-R antagonists. Careful structure-activity relationship investigation guided by rational design identified a methyl substitution adjacent to the carboxylic acid as an appropriate means to accomplish favorable pharmacokinetic properties by reduction of glucuronidation. Further optimization led to the identification of benzimidazolecarboxylic acid BAY 1316957, a highly potent, specific, and selective hEP4-R antagonist with excellent drug metabolism and pharmacokinetics properties. Notably, treatment with BAY 1316957 can be expected to lead to prominent and rapid pain relief and significant improvement of the patient's quality of life.

Mechanistic Reactivity Descriptors for the Prediction of Ames Mutagenicity of Primary Aromatic Amines.

Pharmaceutical products are often synthesized by the use of reactive starting materials and intermediates. These can, either as impurities or through metabolic activation, bind to the DNA. Primary aromatic amines belong to the critical classes that are considered potentially mutagenic in the Ames test, so there is a great need for good prediction models for risk assessment. How primary aromatic amines exert their mutagenic potential can be rationalized by the widely accepted nitrenium ion hypothesis of covalent binding to the DNA of reactive electrophiles formed out of the aromatic amines. Since the reactive chemical species is different in chemical structure from the actual compound, it is difficult to achieve good predictions via classical descriptor or fingerprint-based machine learning. In this approach, we use a combination of different molecular and atomic descriptors that is able to describe different mechanistic aspects of the metabolic transformation leading from the primary aromatic amine to the reactive metabolite that binds to the DNA. Applied to a test set, the combination shows significantly better performance than models that only use one of these descriptors and complemented the general internal Ames mutagenicity prediction model at Bayer.

Efficiency of different measures for defining the applicability domain of classification models.

The goal of defining an applicability domain for a predictive classification model is to identify the region in chemical space where the model's predictions are reliable. The boundary of the applicability domain is defined with the help of a measure that shall reflect the reliability of an individual prediction. Here, the available measures are differentiated into those that flag unusual objects and which are independent of the original classifier and those that use information of the trained classifier. The former set of techniques is referred to as novelty detection while the latter is designated as confidence estimation. A review of the available confidence estimators shows that most of these measures estimate the probability of class membership of the predicted objects which is inversely related to the error probability. Thus, class probability estimates are natural candidates for defining the applicability domain but were not comprehensively included in previous benchmark studies. The focus of the present study is to find the best measure for defining the applicability domain for a given binary classification technique and to determine the performance of novelty detection versus confidence estimation. Six different binary classification techniques in combination with ten data sets were studied to benchmark the various measures. The area under the receiver operating characteristic curve (AUC ROC) was employed as main benchmark criterion. It is shown that class probability estimates constantly perform best to differentiate between reliable and unreliable predictions. Previously proposed alternatives to class probability estimates do not perform better than the latter and are inferior in most cases. Interestingly, the impact of defining an applicability domain depends on the observed area under the receiver operator characteristic curve. That means that it depends on the level of difficulty of the classification problem (expressed as AUC ROC) and will be largest for intermediately difficult problems (range AUC ROC 0.7-0.9). In the ranking of classifiers, classification random forests performed best on average. Hence, classification random forests in combination with the respective class probability estimate are a good starting point for predictive binary chemoinformatic classifiers with applicability domain. Graphical abstract .

Isoform-Selective ATAD2 Chemical Probe with Novel Chemical Structure and Unusual Mode of Action.

ATAD2 (ANCCA) is an epigenetic regulator and transcriptional cofactor, whose overexpression has been linked to the progress of various cancer types. Here, we report a DNA-encoded library screen leading to the discovery of BAY-850, a potent and isoform selective inhibitor that specifically induces ATAD2 bromodomain dimerization and prevents interactions with acetylated histones in vitro, as well as with chromatin in cells. These features qualify BAY-850 as a chemical probe to explore ATAD2 biology.

Comprehensive and Automated Linear Interaction Energy Based Binding-Affinity Prediction for Multifarious Cytochrome P450 Aromatase Inhibitors.

Cytochrome P450 aromatase (CYP19A1) plays a key role in the development of estrogen dependent breast cancer, and aromatase inhibitors have been at the front line of treatment for the past three decades. The development of potent, selective and safer inhibitors is ongoing with in silico screening methods playing a more prominent role in the search for promising lead compounds in bioactivity-relevant chemical space. Here we present a set of comprehensive binding affinity prediction models for CYP19A1 using our automated Linear Interaction Energy (LIE) based workflow on a set of 132 putative and structurally diverse aromatase inhibitors obtained from a typical industrial screening study. We extended the workflow with machine learning methods to automatically cluster training and test compounds in order to maximize the number of explained compounds in one or more predictive LIE models. The method uses protein-ligand interaction profiles obtained from Molecular Dynamics (MD) trajectories to help model search and define the applicability domain of the resolved models. Our method was successful in accounting for 86% of the data set in 3 robust models that show high correlation between calculated and observed values for ligand-binding free energies (RMSE < 2.5 kJ mol-1), with good cross-validation statistics.

Benzoisoquinolinediones as Potent and Selective Inhibitors of BRPF2 and TAF1/TAF1L Bromodomains.

Bromodomains (BD) are readers of lysine acetylation marks present in numerous proteins associated with chromatin. Here we describe a dual inhibitor of the bromodomain and PHD finger (BRPF) family member BRPF2 and the TATA box binding protein-associated factors TAF1 and TAF1L. These proteins are found in large chromatin complexes and play important roles in transcription regulation. The substituted benzoisoquinolinedione series was identified by high-throughput screening, and subsequent structure-activity relationship optimization allowed generation of low nanomolar BRPF2 BD inhibitors with strong selectivity against BRPF1 and BRPF3 BDs. In addition, a strong inhibition of TAF1/TAF1L BD2 was measured for most derivatives. The best compound of the series was BAY-299, which is a very potent, dual inhibitor with an IC50 of 67 nM for BRPF2 BD, 8 nM for TAF1 BD2, and 106 nM for TAF1L BD2. Importantly, no activity was measured for BRD4 BDs. Furthermore, cellular activity was evidenced using a BRPF2- or TAF1-histone H3.3 or H4 interaction assay.

Discovery and Characterization of a Highly Potent and Selective Aminopyrazoline-Based in Vivo Probe (BAY-598) for the Protein Lysine Methyltransferase SMYD2.

Protein lysine methyltransferases have recently emerged as a new target class for the development of inhibitors that modulate gene transcription or signaling pathways. SET and MYND domain containing protein 2 (SMYD2) is a catalytic SET domain containing methyltransferase reported to monomethylate lysine residues on histone and nonhistone proteins. Although several studies have uncovered an important role of SMYD2 in promoting cancer by protein methylation, the biology of SMYD2 is far from being fully understood. Utilization of highly potent and selective chemical probes for target validation has emerged as a concept which circumvents possible limitations of knockdown experiments and, in particular, could result in an improved exploration of drug targets with a complex underlying biology. Here, we report the development of a potent, selective, and cell-active, substrate-competitive inhibitor of SMYD2, which is the first reported inhibitor suitable for in vivo target validation studies in rodents.

Chemogenomic analysis of G-protein coupled receptors and their ligands deciphers locks and keys governing diverse aspects of signalling.

Understanding the molecular mechanism of signalling in the important super-family of G-protein-coupled receptors (GPCRs) is causally related to questions of how and where these receptors can be activated or inhibited. In this context, it is of great interest to unravel the common molecular features of GPCRs as well as those related to an active or inactive state or to subtype specific G-protein coupling. In our underlying chemogenomics study, we analyse for the first time the statistical link between the properties of G-protein-coupled receptors and GPCR ligands. The technique of mutual information (MI) is able to reveal statistical inter-dependence between variations in amino acid residues on the one hand and variations in ligand molecular descriptors on the other. Although this MI analysis uses novel information that differs from the results of known site-directed mutagenesis studies or published GPCR crystal structures, the method is capable of identifying the well-known common ligand binding region of GPCRs between the upper part of the seven transmembrane helices and the second extracellular loop. The analysis shows amino acid positions that are sensitive to either stimulating (agonistic) or inhibitory (antagonistic) ligand effects or both. It appears that amino acid positions for antagonistic and agonistic effects are both concentrated around the extracellular region, but selective agonistic effects are cumulated between transmembrane helices (TMHs) 2, 3, and ECL2, while selective residues for antagonistic effects are located at the top of helices 5 and 6. Above all, the MI analysis provides detailed indications about amino acids located in the transmembrane region of these receptors that determine G-protein signalling pathway preferences.

Benchmark data set for in silico prediction of Ames mutagenicity.

Up to now, publicly available data sets to build and evaluate Ames mutagenicity prediction tools have been very limited in terms of size and chemical space covered. In this report we describe a new unique public Ames mutagenicity data set comprising about 6500 nonconfidential compounds (available as SMILES strings and SDF) together with their biological activity. Three commercial tools (DEREK, MultiCASE, and an off-the-shelf Bayesian machine learner in Pipeline Pilot) are compared with four noncommercial machine learning implementations (Support Vector Machines, Random Forests, k-Nearest Neighbors, and Gaussian Processes) on the new benchmark data set.

A probabilistic approach to classifying metabolic stability.

Metabolic stability is an important property of drug molecules that should-optimally-be taken into account early on in the drug design process. Along with numerous medium- or high-throughput assays being implemented in early drug discovery, a prediction tool for this property could be of high value. However, metabolic stability is inherently difficult to predict, and no commercial tools are available for this purpose. In this work, we present a machine learning approach to predicting metabolic stability that is tailored to compounds from the drug development process at Bayer Schering Pharma. For four different in vitro assays, we develop Bayesian classification models to predict the probability of a compound being metabolically stable. The chosen approach implicitly takes the "domain of applicability" into account. The developed models were validated on recent project data at Bayer Schering Pharma, showing that the predictions are highly accurate and the domain of applicability is estimated correctly. Furthermore, we evaluate the modeling method on a set of publicly available data.

Estimating the domain of applicability for machine learning QSAR models: a study on aqueous solubility of drug discovery molecules.

We investigate the use of different Machine Learning methods to construct models for aqueous solubility. Models are based on about 4000 compounds, including an in-house set of 632 drug discovery molecules of Bayer Schering Pharma. For each method, we also consider an appropriate method to obtain error bars, in order to estimate the domain of applicability (DOA) for each model. Here, we investigate error bars from a Bayesian model (Gaussian Process (GP)), an ensemble based approach (Random Forest), and approaches based on the Mahalanobis distance to training data (for Support Vector Machine and Ridge Regression models). We evaluate all approaches in terms of their prediction accuracy (in cross-validation, and on an external validation set of 536 molecules) and in how far the individual error bars can faithfully represent the actual prediction error.

Estimating the domain of applicability for machine learning QSAR models: a study on aqueous solubility of drug discovery molecules.

We investigate the use of different Machine Learning methods to construct models for aqueous solubility. Models are based on about 4000 compounds, including an in-house set of 632 drug discovery molecules of Bayer Schering Pharma. For each method, we also consider an appropriate method to obtain error bars, in order to estimate the domain of applicability (DOA) for each model. Here, we investigate error bars from a Bayesian model (Gaussian Process (GP)), an ensemble based approach (Random Forest), and approaches based on the Mahalanobis distance to training data (for Support Vector Machine and Ridge Regression models). We evaluate all approaches in terms of their prediction accuracy (in cross-validation, and on an external validation set of 536 molecules) and in how far the individual error bars can faithfully represent the actual prediction error.