Evaluation guidelines for machine learning tools in the chemical sciences.

Machine learning (ML) promises to tackle the grand challenges in chemistry and speed up the generation, improvement and/or ordering of research hypotheses. Despite the overarching applicability of ML workflows, one usually finds diverse evaluation study designs. The current heterogeneity in evaluation techniques and metrics leads to difficulty in (or the impossibility of) comparing and assessing the relevance of new algorithms. Ultimately, this may delay the digitalization of chemistry at scale and confuse method developers, experimentalists, reviewers and journal editors. In this Perspective, we critically discuss a set of method development and evaluation guidelines for different types of ML-based publications, emphasizing supervised learning. We provide a diverse collection of examples from various authors and disciplines in chemistry. While taking into account varying accessibility across research groups, our recommendations focus on reporting completeness and standardizing comparisons between tools. We aim to further contribute to improved ML transparency and credibility by suggesting a checklist of retro-/prospective tests and dissecting their importance. We envisage that the wide adoption and continuous update of best practices will encourage an informed use of ML on real-world problems related to the chemical sciences.

Comparing classification models-a practical tutorial.

While machine learning models have become a mainstay in Cheminformatics, the field has yet to agree on standards for model evaluation and comparison. In many cases, authors compare methods by performing multiple folds of cross-validation and reporting the mean value for an evaluation metric such as the area under the receiver operating characteristic. These comparisons of mean values often lack statistical rigor and can lead to inaccurate conclusions. In the interest of encouraging best practices, this tutorial provides an example of how multiple methods can be compared in a statistically rigorous fashion.

Modeling & Informatics at Vertex Pharmaceuticals Incorporated: our philosophy for sustained impact.

Molecular modelers and informaticians have the unique opportunity to integrate cross-functional data using a myriad of tools, methods and visuals to generate information. Using their drug discovery expertise, information is transformed to knowledge that impacts drug discovery. These insights are often times formulated locally and then applied more broadly, which influence the discovery of new medicines. This is particularly true in an organization where the members are exposed to projects throughout an organization, such as in the case of the global Modeling & Informatics group at Vertex Pharmaceuticals. From its inception, Vertex has been a leader in the development and use of computational methods for drug discovery. In this paper, we describe the Modeling & Informatics group at Vertex and the underlying philosophy, which has driven this team to sustain impact on the discovery of first-in-class transformative medicines.

The potency-insolubility conundrum in pharmaceuticals: Mechanism and solution for hepatitis C protease inhibitors.

As compounds are optimized for greater potency during pharmaceutical discovery, their aqueous solubility often decreases, making them less viable as orally-administered drugs. To investigate whether potency and insolubility share a common origin, we examined the structural and thermodynamic properties of telaprevir, a sparingly soluble inhibitor of hepatitis C virus protease. Comparison of the hydrogen bond motifs in crystalline telaprevir with those present in the protease-telaprevir complex revealed striking similarities. Additionally, the thermodynamics of telaprevir dissolution closely resembles those of protein-ligand dissociation. Together, these findings point to a common origin of potency and insolubility rooted in particular amide-amide hydrogen bond patterns. The insolubility of telaprevir is shown by computational analysis to be caused by interactions in the crystal, not unfavorable hydrophobic hydration. Accordingly, competing out the particular amide-amide hydrogen bond motifs in crystalline telaprevir with 4-hydroxybenzoic acid yielded a co-crystalline solid with excellent aqueous dissolution and oral absorption. The analysis suggests a generalizable approach for identifying drug candidate compounds that either can or cannot be rendered orally bioavailable by alteration of their crystalline solid phases, in an approach that provides a pragmatic way to attain substantial enhancements in the success rate of drug discovery and development.