The Goldilocks paradigm: comparing classical machine learning, large language models, and few-shot learning for drug discovery applications.

Recent advances in machine learning (ML) have led to newer model architectures including transformers (large language models, LLMs) showing state of the art results in text generation and image analysis as well as few-shot learning (FSLC) models which offer predictive power with extremely small datasets. These new architectures may offer promise, yet the 'no-free lunch' theorem suggests that no single model algorithm can outperform at all possible tasks. Here, we explore the capabilities of classical (SVR), FSLC, and transformer models (MolBART) over a range of dataset tasks and show a 'goldilocks zone' for each model type, in which dataset size and feature distribution (i.e. dataset "diversity") determines the optimal algorithm strategy. When datasets are small ( < 50 molecules), FSLC tend to outperform both classical ML and transformers. When datasets are small-to-medium sized (50-240 molecules) and diverse, transformers outperform both classical models and few-shot learning. Finally, when datasets are of larger and of sufficient size, classical models then perform the best, suggesting that the optimal model to choose likely depends on the dataset available, its size and diversity. These findings may help to answer the perennial question of which ML algorithm is to be used when faced with a new dataset.

Sequential Contrastive and Deep Learning Models to Identify Selective Butyrylcholinesterase Inhibitors.

Butyrylcholinesterase (BChE) is a target of interest in late-stage Alzheimer's Disease (AD) where selective BChE inhibitors (BIs) may offer symptomatic treatment without the harsh side effects of acetylcholinesterase (AChE) inhibitors. In this study, we explore multiple machine learning strategies to identify BIs in silico, optimizing for precision over all other metrics. We compare state-of-the-art supervised contrastive learning (CL) with deep learning (DL) and Random Forest (RF) machine learning, across single and sequential modeling configurations, to identify the best models for BChE selectivity. We used these models to virtually screen a vendor library of 5 million compounds for BIs and tested 20 of these compounds in vitro. Seven of the 20 compounds displayed selectivity for BChE over AChE, reflecting a hit rate of 35% for our model predictions, suggesting a highly efficient strategy for modeling selective inhibition.

In silico ADME/tox comes of age: twenty years later.

In the early 2000s pharmaceutical drug discovery was beginning to use computational approaches for absorption, distribution, metabolism, excretion and toxicity (ADME/Tox, also known as ADMET) prediction. This emphasis on prediction was an effort to reduce the risk of later stage failures from ADME/Tox.Much has been written in the intervening twenty plus years and significant expenditure has occurred in companies developing these in silico capabilities which can be gleaned from publications. It is therefore an appropriate time to briefly reflect on what was proposed then and what the reality is today.20 years ago, we tended to optimise bioactivity and perhaps one ADME/Tox property at a time. Previously pharmaceutical companies needed a whole infrastructure for models - in silico and in vitro experts, IT, champions on a project team, educators and management support. Now we are in the age of generative de novo design where bioactivity and many ADME/Tox properties can be optimised and large language model technologies are available.There are also some challenges such as the focus on very large molecules which may be outside of current ADME/Tox models.We provide an opportunity to look forward with the increasing public data for ADME/Tox as well as expanded types of algorithms available.

Comparing LD50/LC50 Machine Learning Models for Multiple Species.

The lethal dose or concentration which kills 50% of the animals (LD50 or LC50) is an important parameter for scientists to understand the toxicity of chemicals in different scenarios that can be used to make go-no-go decisions, and ultimately assist in the choice of the right personal protective equipment needed for containment. The LD50 assessment process has also required the use of many animals although modern methods have reduced the number of rats needed. Since a compound is usually considered highly toxic when the LD50 is lower than 25 mg/kg, such a classification provides potentially valuable safety information to synthetic chemists and other safety assessment scientists. The need for finding alternative approaches such as computational methods is important to ultimately reduce animal use for this testing further still. We now summarize our efforts to use public data for building in vivo LD50 or LC50 classification and regression machine learning models for various species (rat, mouse, fish and daphnia) and their 5-fold cross validation statistics with different machine learning algorithms as well as an external curated test set for mouse LD50. These datasets consist of different molecule classes, may cover different activity ranges, and also have a range of dataset sizes. The challenges of using such computational models are that their applicability domain will also need to be understood so that they can be used to make reliable predictions for novel molecules. These machine learning models will also need to be backed up with experimental validation. However, such models could also be used for efforts to bridge gaps in individual toxicity datasets. Making such models available also opens them up to potential misuse or dual use. We will summarize these efforts and propose that they could be used for scoring the millions of commercially available molecules, most of which likely do not have a known LD50 or for that matter any data in vitro or in vivo for toxicity.

N-Phenyl-1-(phenylsulfonyl)-1H-1,2,4-triazol-3-amine as a New Class of HIV-1 Non-nucleoside Reverse Transcriptase Inhibitor.

Highly active antiretroviral therapy (HAART) has revolutionized human immunodeficiency virus (HIV) healthcare, turning it from a terminal to a potentially chronic disease, although some patients can develop severe comorbidities. These include neurological complications, such as HIV-associated neurocognitive disorders (HAND), which result in cognitive and/or motor function symptoms. We now describe the discovery, synthesis, and evaluation of a new class of N-phenyl-1-(phenylsulfonyl)-1H-1,2,4-triazol-3-amine HIV-1 non-nucleoside reverse transcriptase inhibitors (NNRTI) aimed at avoiding HAND. The most promising molecule, 12126065, exhibited antiviral activity against wild-type HIV-1 in TZM cells (EC50 = 0.24 nM) with low in vitro cytotoxicity (CC50 = 4.8 µM) as well as retained activity against clinically relevant HIV mutants. 12126065 also demonstrated no in vivo acute or subacute toxicity, good in vivo brain penetration, and minimal neurotoxicity in mouse neurons up to 10 µM, with a 50% toxicity concentration (TC50) of >100 µM, well below its EC50.

Validation of Acetylcholinesterase Inhibition Machine Learning Models for Multiple Species.

Acetylcholinesterase (AChE) is an important enzyme and target for human therapeutics, environmental safety, and global food supply. Inhibitors of this enzyme are also used for pest elimination and can be misused for suicide or chemical warfare. Adverse effects of AChE pesticides on nontarget organisms, such as fish, amphibians, and humans, have also occurred as a result of biomagnifications of these toxic compounds. We have exhaustively curated the public data for AChE inhibition data and developed machine learning classification models for seven different species. Each set of models were built using up to nine different algorithms for each species and Morgan fingerprints (ECFP6) with an activity cutoff of 1 µM. The human (4075 compounds) and eel (5459 compounds) consensus models predicted AChE inhibition activity using external test sets from literature data with 81% and 82% accuracy, respectively, while the reciprocal cross (76% and 82% percent accuracy) was not species-specific. In addition, we also created machine learning regression models for human and eel AChE inhibition to return a predicted IC50 value for a queried molecule. We did observe an improved species specificity in the regression models, where a human support vector regression model of human AChE inhibition (3652 compounds) predicted the IC50s of the human test set to a better extent than the eel regression model (4930 compounds) on the same test set, based on mean absolute percentage error (MAPE = 9.73% vs 13.4%). The predictive power of these models certainly benefits from increasing the chemical diversity of the training set, as evidenced by expanding our human classification model by incorporating data from the Tox21 library of compounds. Of the 10 compounds we tested that were predicted active by this expanded model, two showed >80% inhibition at 100 µM. This machine learning approach therefore offers the ability to rapidly score massive libraries of molecules against the models for AChE inhibition that can then be selected for future in vitro testing to identify potential toxins. It also enabled us to create a public website, MegaAChE, for single-molecule predictions of AChE inhibition using these models at megaache.collaborationspharma.com.

Preventing AI From Creating Biochemical Threats.

We have previously applied our machine learning models for bioactivity and toxicity along with a generative algorithm to develop VX and tens of thousands of analogues. The publication brought attention to the ease of designing chemical warfare agents. In this Viewpoint, we discuss 10 recommendations to prevent future biochemical threats.

Discovery of New Zika Protease and Polymerase Inhibitors through the Open Science Collaboration Project OpenZika.

The Zika virus (ZIKV) is a neurotropic arbovirus considered a global threat to public health. Although there have been several efforts in drug discovery projects for ZIKV in recent years, there are still no antiviral drugs approved to date. Here, we describe the results of a global collaborative crowdsourced open science project, the OpenZika project, from IBM's World Community Grid (WCG), which integrates different computational and experimental strategies for advancing a drug candidate for ZIKV. Initially, molecular docking protocols were developed to identify potential inhibitors of ZIKV NS5 RNA-dependent RNA polymerase (NS5 RdRp), NS3 protease (NS2B-NS3pro), and NS3 helicase (NS3hel). Then, a machine learning (ML) model was built to distinguish active vs inactive compounds for the cytoprotective effect against ZIKV infection. We performed three independent target-based virtual screening campaigns (NS5 RdRp, NS2B-NS3pro, and NS3hel), followed by predictions by the ML model and other filters, and prioritized a total of 61 compounds for further testing in enzymatic and phenotypic assays. This yielded five non-nucleoside compounds which showed inhibitory activity against ZIKV NS5 RdRp in enzymatic assays (IC50 range from 0.61 to 17 µM). Two compounds thermally destabilized NS3hel and showed binding affinity in the micromolar range (Kd range from 9 to 35 µM). Moreover, the compounds LabMol-301 inhibited both NS5 RdRp and NS2B-NS3pro (IC50 of 0.8 and 7.4 µM, respectively) and LabMol-212 thermally destabilized the ZIKV NS3hel (Kd of 35 µM). Both also protected cells from death induced by ZIKV infection in in vitro cell-based assays. However, while eight compounds (including LabMol-301 and LabMol-212) showed a cytoprotective effect and prevented ZIKV-induced cell death, agreeing with our ML model for prediction of this cytoprotective effect, no compound showed a direct antiviral effect against ZIKV. Thus, the new scaffolds discovered here are promising hits for future structural optimization and for advancing the discovery of further drug candidates for ZIKV. Furthermore, this work has demonstrated the importance of the integration of computational and experimental approaches, as well as the potential of large-scale collaborative networks to advance drug discovery projects for neglected diseases and emerging viruses, despite the lack of available direct antiviral activity and cytoprotective effect data, that reflects on the assertiveness of the computational predictions. The importance of these efforts rests with the need to be prepared for future viral epidemic and pandemic outbreaks.

Dual Use of Artificial Intelligence-powered Drug Discovery.

An international security conference explored how artificial intelligence (AI) technologies for drug discovery could be misused for de novo design of biochemical weapons. A thought experiment evolved into a computational proof.

The Commoditization of AI for Molecule Design.

Anyone involved in designing or finding molecules in the life sciences over the past few years has witnessed a dramatic change in how we now work due to the COVID-19 pandemic. Computational technologies like artificial intelligence (AI) seemed to become ubiquitous in 2020 and have been increasingly applied as scientists worked from home and were separated from the laboratory and their colleagues. This shift may be more permanent as the future of molecule design across different industries will increasingly require machine learning models for design and optimization of molecules as they become "designed by AI". AI and machine learning has essentially become a commodity within the pharmaceutical industry. This perspective will briefly describe our personal opinions of how machine learning has evolved and is being applied to model different molecule properties that crosses industries in their utility and ultimately suggests the potential for tight integration of AI into equipment and automated experimental pipelines. It will also describe how many groups have implemented generative models covering different architectures, for de novo design of molecules. We also highlight some of the companies at the forefront of using AI to demonstrate how machine learning has impacted and influenced our work. Finally, we will peer into the future and suggest some of the areas that represent the most interesting technologies that may shape the future of molecule design, highlighting how we can help increase the efficiency of the design-make-test cycle which is currently a major focus across industries.

Machine Learning Models Identify New Inhibitors for Human OATP1B1.

The uptake transporter OATP1B1 (SLC01B1) is largely localized to the sinusoidal membrane of hepatocytes and is a known victim of unwanted drug-drug interactions. Computational models are useful for identifying potential substrates and/or inhibitors of clinically relevant transporters. Our goal was to generate OATP1B1 in vitro inhibition data for [3H] estrone-3-sulfate (E3S) transport in CHO cells and use it to build machine learning models to facilitate a comparison of seven different classification models (Deep learning, Adaboosted decision trees, Bernoulli naïve bayes, k-nearest neighbors (knn), random forest, support vector classifier (SVC), logistic regression (lreg), and XGBoost (xgb)] using ECFP6 fingerprints to perform 5-fold, nested cross validation. In addition, we compared models using 3D pharmacophores, simple chemical descriptors alone or plus ECFP6, as well as ECFP4 and ECFP8 fingerprints. Several machine learning algorithms (SVC, lreg, xgb, and knn) had excellent nested cross validation statistics, particularly for accuracy, AUC, and specificity. An external test set containing 207 unique compounds not in the training set demonstrated that at every threshold SVC outperformed the other algorithms based on a rank normalized score. A prospective validation test set was chosen using prediction scores from the SVC models with ECFP fingerprints and were tested in vitro with 15 of 19 compounds (84% accuracy) predicted as active (≥20% inhibition) showed inhibition. Of these compounds, six (abamectin, asiaticoside, berbamine, doramectin, mobocertinib, and umbralisib) appear to be novel inhibitors of OATP1B1 not previously reported. These validated machine learning models can now be used to make predictions for drug-drug interactions for human OATP1B1 alongside other machine learning models for important drug transporters in our MegaTrans software.

Coordinated Regulation of CB1 Cannabinoid Receptors and Anandamide Metabolism Stabilizes Network Activity during Homeostatic Downscaling.

Neurons express overlapping homeostatic mechanisms to regulate synaptic function and network properties in response to perturbations of neuronal activity. Endocannabinoids (eCBs) are bioactive lipids synthesized in the postsynaptic compartments to regulate synaptic transmission, plasticity, and neuronal excitability primarily through retrograde activation of presynaptic cannabinoid receptor type 1 (CB1). The eCB system is well situated to regulate neuronal network properties and coordinate presynaptic and postsynaptic activity. However, the role of the eCB system in homeostatic adaptations to neuronal hyperactivity is unknown. To address this issue, we used Western blotting and targeted lipidomics to measure adaptations in eCB system to bicuculline (BCC)-induced chronic hyperexcitation in mature cultured rat cortical neurons, and used multielectrode array (MEA) recording and live-cell imaging of glutamate dynamics to test the effects of pharmacological manipulations of eCB on network activities. We show that BCC-induced chronic hyperexcitation triggers homeostatic downscaling and a coordinated adaptation to enhance tonic eCB signaling. Hyperexcitation triggers first the downregulation of fatty acid amide hydrolase (FAAH), the lipase that degrades the eCB anandamide, then an accumulation of anandamide and related metabolites, and finally a delayed upregulation of surface and total CB1. Additionally, we show that BCC-induced downregulation of surface AMPA-type glutamate receptors (AMPARs) and upregulation of CB1 occur through independent mechanisms. Finally, we show that endocannabinoids support baseline network activities before and after downscaling and is engaged to suppress network activity during adaptation to hyperexcitation. We discuss the implications of our findings in the context of downscaling and homeostatic regulation of in vitro oscillatory network activities.

Combining DELs and machine learning for toxicology prediction.

DNA-encoded libraries (DELs) allow starting chemical matter to be identified in drug discovery. The volume of experimental data generated also makes DELs an attractive resource for machine learning (ML). ML allows modeling complex relationships between compounds and numerical endpoints, such as the binding to a target measured by DELs. DELs could also empower other areas of drug discovery. Here, we propose that DELs and ML could be combined to model binding to off-targets, enabling better predictive toxicology. With enough data, ML models can make accurate predictions across a vast chemical space, and they can be reused and expanded across projects. Although there are limitations, more general toxicology models could be applied earlier during drug discovery, illuminating safety liabilities at a lower cost.

MegaSyn: Integrating Generative Molecular Design, Automated Analog Designer, and Synthetic Viability Prediction.

Generative machine learning models have become widely adopted in drug discovery and other fields to produce new molecules and explore molecular space, with the goal of discovering novel compounds with optimized properties. These generative models are frequently combined with transfer learning or scoring of the physicochemical properties to steer generative design, yet often, they are not capable of addressing a wide variety of potential problems, as well as converge into similar molecular space when combined with a scoring function for the desired properties. In addition, these generated compounds may not be synthetically feasible, reducing their capabilities and limiting their usefulness in real-world scenarios. Here, we introduce a suite of automated tools called MegaSyn representing three components: a new hill-climb algorithm, which makes use of SMILES-based recurrent neural network (RNN) generative models, analog generation software, and retrosynthetic analysis coupled with fragment analysis to score molecules for their synthetic feasibility. We show that by deconstructing the targeted molecules and focusing on substructures, combined with an ensemble of generative models, MegaSyn generally performs well for the specific tasks of generating new scaffolds as well as targeted analogs, which are likely synthesizable and druglike. We now describe the development, benchmarking, and testing of this suite of tools and propose how they might be used to optimize molecules or prioritize promising lead compounds using these RNN examples provided by multiple test case examples.

Machine Learning Models for Mycobacterium tuberculosisIn Vitro Activity: Prediction and Target Visualization.

Tuberculosis (TB) is a major global health challenge, with approximately 1.4 million deaths per year. There is still a need to develop novel treatments for patients infected with Mycobacterium tuberculosis (Mtb). There have been many large-scale phenotypic screens that have led to the identification of thousands of new compounds. Yet, there is very limited investment in TB drug discovery which points to the need for new methods to increase the efficiency of drug discovery against Mtb. We have used machine learning approaches to learn from the public Mtb data, resulting in many data sets and models with robust enrichment and hit rates leading to the discovery of new active compounds. Recently, we have curated predominantly small-molecule Mtb data and developed new machine learning classification models with 18 886 molecules at different activity cutoffs. We now describe the further validation of these Bayesian models using a library of over 1000 molecules synthesized as part of EU-funded New Medicines for TB and More Medicines for TB programs. We highlight molecular features which are enriched in these active compounds. In addition, we provide new regression and classification models that can be used for scoring compound libraries or used to design new molecules. We have also visualized these molecules in the context of known molecular targets and identified clusters in chemical property space, which may aid in future target identification efforts. Finally, we are also making these data sets publicly available, representing a significant increase to the available Mtb inhibition data in the public domain.

UV-adVISor: Attention-Based Recurrent Neural Networks to Predict UV-Vis Spectra.

Ultraviolet-visible (UV-Vis) absorption spectra are routinely collected as part of high-performance liquid chromatography (HPLC) analysis systems and can be used to identify chemical reaction products by comparison to the reference spectra. Here, we present UV-adVISor as a new computational tool for predicting the UV-Vis spectra from a molecule's structure alone. UV-Vis prediction was approached as a sequence-to-sequence problem. We utilized Long-Short Term Memory and attention-based neural networks with Extended Connectivity Fingerprint Diameter 6 or molecule SMILES to generate predictive models for the UV spectra. We have produced two spectrum datasets (dataset I, N = 949, and dataset II, N = 2222) using different compound collections and spectrum acquisition methods to train, validate, and test our models. We evaluated the prediction accuracy of the complete spectra by the correspondence of wavelengths of absorbance maxima and with a series of statistical measures (the best test set median model parameters are in parentheses for model II), including RMSE (0.064), R2 (0.71), and dynamic time warping (DTW, 0.194) of the entire spectrum curve. Scrambling molecule structures with the experimental spectra during training resulted in a degraded R2, confirming the utility of the approaches for prediction. UV-adVISor is able to provide fast and accurate predictions for libraries of compounds.

Automated Detection and Analysis of Exocytosis.

Timelapse TIRF microscopy of pH-sensitive GFP (pHluorin) attached to vesicle SNARE proteins is an effective method to visualize single vesicle exocytic events in cell culture. To perform an unbiased, efficient identification and analysis of such events, a computer-vision based approach was developed and implemented in MATLAB. The analysis pipeline consists of a cell segmentation and exocytic-event identification algorithm. The computer-vision approach includes tools for investigating multiple parameters of single events, including the half-life of fluorescence decay and peak ΔF/F, as well as whole-cell analysis of the frequency of exocytosis. These and other parameters of fusion are used in a classification approach to distinguish distinct fusion modes. Here a newly built GUI performs the analysis pipeline from start to finish. Further adaptation of Ripley's K function in R Studio is used to distinguish between clustered, dispersed, or random occurrence of fusion events in both space and time.

Recent advances in drug repurposing using machine learning.

Drug repurposing aims to find new uses for already existing and approved drugs. We now provide a brief overview of recent developments in drug repurposing using machine learning alongside other computational approaches for comparison. We also highlight several applications for cancer using kinase inhibitors, Alzheimer's disease as well as COVID-19.