Best practices for constructing, preparing, and evaluating protein-ligand binding affinity benchmarks [Article v0.1].

Free energy calculations are rapidly becoming indispensable in structure-enabled drug discovery programs. As new methods, force fields, and implementations are developed, assessing their expected accuracy on real-world systems (benchmarking) becomes critical to provide users with an assessment of the accuracy expected when these methods are applied within their domain of applicability, and developers with a way to assess the expected impact of new methodologies. These assessments require construction of a benchmark-a set of well-prepared, high quality systems with corresponding experimental measurements designed to ensure the resulting calculations provide a realistic assessment of expected performance when these methods are deployed within their domains of applicability. To date, the community has not yet adopted a common standardized benchmark, and existing benchmark reports suffer from a myriad of issues, including poor data quality, limited statistical power, and statistically deficient analyses, all of which can conspire to produce benchmarks that are poorly predictive of real-world performance. Here, we address these issues by presenting guidelines for (1) curating experimental data to develop meaningful benchmark sets, (2) preparing benchmark inputs according to best practices to facilitate widespread adoption, and (3) analysis of the resulting predictions to enable statistically meaningful comparisons among methods and force fields. We highlight challenges and open questions that remain to be solved in these areas, as well as recommendations for the collection of new datasets that might optimally serve to measure progress as methods become systematically more reliable. Finally, we provide a curated, versioned, open, standardized benchmark set adherent to these standards (PLBenchmarks) and an open source toolkit for implementing standardized best practices assessments (arsenic) for the community to use as a standardized assessment tool. While our main focus is free energy methods based on molecular simulations, these guidelines should prove useful for assessment of the rapidly growing field of machine learning methods for affinity prediction as well.

Fragment Pose Prediction Using Non-equilibrium Candidate Monte Carlo and Molecular Dynamics Simulations.

Part of early stage drug discovery involves determining how molecules may bind to the target protein. Through understanding where and how molecules bind, chemists can begin to build ideas on how to design improvements to increase binding affinities. In this retrospective study, we compare how computational approaches like docking, molecular dynamics (MD) simulations, and a non-equilibrium candidate Monte Carlo (NCMC)-based method (NCMC + MD) perform in predicting binding modes for a set of 12 fragment-like molecules, which bind to soluble epoxide hydrolase. We evaluate each method's effectiveness in identifying the dominant binding mode and finding additional binding modes (if any). Then, we compare our predicted binding modes to experimentally obtained X-ray crystal structures. We dock each of the 12 small molecules into the apo-protein crystal structure and then run simulations up to 1 µs each. Small and fragment-like molecules likely have smaller energy barriers separating different binding modes by virtue of relatively fewer and weaker interactions relative to drug-like molecules and thus likely undergo more rapid binding mode transitions. We expect, thus, to see more rapid transitions between binding modes in our study. Following this, we build Markov State Models to define our stable ligand binding modes. We investigate if adequate sampling of ligand binding modes and transitions between them can occur at the microsecond timescale using traditional MD or a hybrid NCMC+MD simulation approach. Our findings suggest that even with small fragment-like molecules, we fail to sample all the crystallographic binding modes using microsecond MD simulations, but using NCMC+MD, we have better success in sampling the crystal structure while obtaining the correct populations.

Biomolecular Solvation Structure Revealed by Molecular Dynamics Simulations.

To compare ordered water positions from experiment with those from molecular dynamics (MD) simulations, a number of MD models of water structure in crystalline endoglucanase were calculated. The starting MD model was derived from a joint X-ray and neutron diffraction crystal structure, enabling the use of experimentally assigned protonation states. Simulations were performed in the crystalline state, using a periodic 2 × 2 × 2 supercell with explicit solvent. Water X-ray and neutron scattering density maps were computed from MD trajectories using standard macromolecular crystallography methods. In one set of simulations, harmonic restraints were applied to bias the protein structure toward the crystal structure. For these simulations, the recall of crystallographic waters using strong peaks in the MD water electron density was very good, and there also was substantial visual agreement between the boomerang-like wings of the neutron scattering density and the crystalline water hydrogen positions. An unrestrained simulation also was performed. For this simulation, the recall of crystallographic waters was much lower. For both restrained and unrestrained simulations, the strongest water density peaks were associated with crystallographic waters. The results demonstrate that it is now possible to recover crystallographic water structure using restrained MD simulations but that it is not yet reasonable to expect unrestrained MD simulations to do the same. Further development and generalization of MD water models for force-field development, macromolecular crystallography, and medicinal chemistry applications is now warranted. In particular, the combination of room-temperature crystallography, neutron diffraction, and crystalline MD simulations promises to substantially advance modeling of biomolecular solvation.

Improved ligand geometries in crystallographic refinement using AFITT in PHENIX.

Modern crystal structure refinement programs rely on geometry restraints to overcome the challenge of a low data-to-parameter ratio. While the classical Engh and Huber restraints work well for standard amino-acid residues, the chemical complexity of small-molecule ligands presents a particular challenge. Most current approaches either limit ligand restraints to those that can be readily described in the Crystallographic Information File (CIF) format, thus sacrificing chemical flexibility and energetic accuracy, or they employ protocols that substantially lengthen the refinement time, potentially hindering rapid automated refinement workflows. PHENIX-AFITT refinement uses a full molecular-mechanics force field for user-selected small-molecule ligands during refinement, eliminating the potentially difficult problem of finding or generating high-quality geometry restraints. It is fully integrated with a standard refinement protocol and requires practically no additional steps from the user, making it ideal for high-throughput workflows. PHENIX-AFITT refinements also handle multiple ligands in a single model, alternate conformations and covalently bound ligands. Here, the results of combining AFITT and the PHENIX software suite on a data set of 189 protein-ligand PDB structures are presented. Refinements using PHENIX-AFITT significantly reduce ligand conformational energy and lead to improved geometries without detriment to the fit to the experimental data. For the data presented, PHENIX-AFITT refinements result in more chemically accurate models for small-molecule ligands.

Triangulating Syndemic Services and Drug Treatment Policy: Improving Drug Treatment Portal Locations in Baltimore City.

THE PROBLEM: The prevalence of injection drug use (IDU) and incidence of human immunodeficiency virus (HIV) remain high in Baltimore, where IDU is a primary HIV risk factor. Substance use disorders and HIV are related syndemically--their causes and consequences interact synergistically. Baltimore is increasingly considering the syndemic relationship of substance use disorders, IDU, and HIV in making decisions about drug treatment funding and location. PURPOSE OF ARTICLE: Our goal was to empirically identify the optimal location of new drug treatment programs through the development and application of a novel, practical tool. KEY POINTS: Syndemic triangles were constructed to measure and visualize unmet need for drug treatment services. These data were used to determine priority zones for new treatment centers. CONCLUSIONS: The application of this tool helped inform strategies for locating drug treatment services in Baltimore, and its successful use suggests its potential value in other metropolitan areas.

POSIT: Flexible Shape-Guided Docking For Pose Prediction.

We present a new approach to structure-based drug design (POSIT) rigorously built on the simple concept that pose prediction is intimately coupled to the quality and availability of experimental structural data. We demonstrate the feasibility of the approach by performing retrospective analyses on three data sets designed to explore the strengths and weaknesses of POSIT relative to existing methods. We then present results documenting 2.5 years of prospective use of POSIT across a variety of structure-based industrial drug-discovery research projects. We find that POSIT is well-suited to guiding research decision making for structure-based design and, in particular, excels at enabling lead-optimization campaigns. We show that the POSIT framework can drive superior pose-prediction performance and generate results that naturally lend themselves to prospective decision making during lead optimization. We believe the results presented here are (1) the largest prospective validation of a pose prediction method reported to date (71 crystal structures); (2) provide an unprecedented look at the scope of impact of a computational tool; and (3) represent a first-of-its-kind analysis. We hope that this work inspires additional studies that look at the real impact and performance of computational research tools on prospective drug design.

The application of statistical methods to cognate docking: a path forward?

Cognate docking has been used as a test for pose prediction quality in docking engines for decades. In this paper, we report a statistically rigorous analysis of cognate docking performance using tools in the OpenEye docking suite. We address a number of critically important aspects of the cognate docking problem that are often handled poorly: data set quality, methods of comparison of the predicted pose to the experimental pose, and data analysis. The focus of the paper lies in the third problem, extracting maximally predictive knowledge from comparison data. To this end, we present a multistage protocol for data analysis that by combining classical null-hypothesis significance testing with effect size estimation provides crucial information about quantitative differences in performance between methods as well as the probability of finding such differences in future experiments. We suggest that developers of software and users of software have different levels of interest in different parts of this protocol, with users being primarily interested in effect size estimation while developers may be most interested in statistical significance. This protocol is completely general and therefore will provide the basis for method comparisons of many different kinds.

Opioid agonist treatments and heroin overdose deaths in Baltimore, Maryland, 1995-2009.

OBJECTIVES: We examined the association between the expansion of methadone and buprenorphine treatment and the prevalence of heroin overdose deaths in Baltimore, Maryland from 1995 to 2009. METHODS: We conducted a longitudinal time series analysis of archival data using linear regression with the Newey-West method to correct SEs for heteroscedasticity and autocorrelation, adjusting for average heroin purity. RESULTS: Overdose deaths attributed to heroin ranged from a high of 312 in 1999 to a low of 106 in 2008. While mean heroin purity rose sharply (1995-1999), the increasing number of patients treated with methadone was not associated with a change in the number of overdose deaths, but starting in 2000 expansion of opioid agonist treatment was associated with a decline in overdose deaths. Adjusting for heroin purity and the number of methadone patients, there was a statistically significant inverse relationship between heroin overdose deaths and patients treated with buprenorphine (P = .002). CONCLUSIONS: Increased access to opioid agonist treatment was associated with a reduction in heroin overdose deaths. Implementing policies that support evidence-based medication treatment of opiate dependence may decrease heroin overdose deaths.

Essential considerations for using protein-ligand structures in drug discovery.

Protein-ligand structures are the core data required for structure-based drug design (SBDD). Understanding the error present in this data is essential to the successful development of SBDD tools. Methods for assessing protein-ligand structure quality and a new set of identification criteria are presented here. When these criteria were applied to a set of 728 structures previously used to validate molecular docking software, only 17% were found to be acceptable. Structures were re-refined to maintain internal consistency in the comparison and assessment of the quality criteria. This process resulted in Iridium, a highly trustworthy protein-ligand structure database to be used for development and validation of structure-based design tools for drug discovery.

Lepidopteran-active variable-region sequence imparts coleopteran activity in eCry3.1Ab, an engineered Bacillus thuringiensis hybrid insecticidal protein.

A unique, coleopteran-active protein, termed eCry3.1Ab, was generated following variable-region exchange of a Bacillus thuringiensis lepidopteran-active protein, Cry1Ab, with a Cry3A region. Our results support the hypothesis that this variable-region exchange is responsible for imparting strong bioactivity against the larvae of western corn rootworm (WCR) (Diabrotica virgifera virgifera LeConte), a pest species which is not susceptible to either parent protein sequence. This study demonstrates the potential of successfully engineering a portion(s) of a lepidopteran-active B. thuringiensis sequence so that it has activity against coleopterans. Further elucidation of the eCry3.1Ab activity indicated the importance of variable regions 4 to 6 that were derived from Cry1Ab instead of Cry1Ac. There was some flexibility in making domain III of engineered hybrid insecticidal proteins even more Cry1Ab-like and retaining activity, while there was less flexibility in making domain III more Cry3A-like and retaining activity. In vitro binding studies with brush border membrane vesicles demonstrated that there was specific binding of chymotrypsin-processed modified Cry3A (mCry3A), which was not diminished by addition of a 100-fold molar excess of chymotrypsin-processed eCry3.1Ab or unprocessed eCry3.1Ab. In addition, in the converse experiment, specific binding of chymotrypsin-processed eCry3.1Ab was not diminished by the presence of a 75-fold molar excess of chymotrypsin-processed mCry3A. These data support the hypothesis that eCry3.1Ab can interact with different binding sites than the activated form of mCry3A in the WCR brush border and may provide a different mode of action from the standpoint of resistance management.

Conformer generation with OMEGA: algorithm and validation using high quality structures from the Protein Databank and Cambridge Structural Database.

Here, we present the algorithm and validation for OMEGA, a systematic, knowledge-based conformer generator. The algorithm consists of three phases: assembly of an initial 3D structure from a library of fragments; exhaustive enumeration of all rotatable torsions using values drawn from a knowledge-based list of angles, thereby generating a large set of conformations; and sampling of this set by geometric and energy criteria. Validation of conformer generators like OMEGA has often been undertaken by comparing computed conformer sets to experimental molecular conformations from crystallography, usually from the Protein Databank (PDB). Such an approach is fraught with difficulty due to the systematic problems with small molecule structures in the PDB. Methods are presented to identify a diverse set of small molecule structures from cocomplexes in the PDB that has maximal reliability. A challenging set of 197 high quality, carefully selected ligand structures from well-solved models was obtained using these methods. This set will provide a sound basis for comparison and validation of conformer generators in the future. Validation results from this set are compared to the results using structures of a set of druglike molecules extracted from the Cambridge Structural Database (CSD). OMEGA is found to perform very well in reproducing the crystallographic conformations from both these data sets using two complementary metrics of success.

Molecular shape and medicinal chemistry: a perspective.

The eight contributions here provide ample evidence that shape as a volume or as a surface is a vibrant and useful concept when applied to drug discovery. It provides a reliable scaffold for "decoration" with chemical intuition (or bias) for virtual screening and lead optimization but also has its unadorned uses, as in library design, ligand fitting, pose prediction, or active site description. Computing power has facilitated this evolution by allowing shape to be handled precisely without the need to reduce down to point descriptors or approximate metrics, and the diversity of resultant applications argues for this being an important step forward. Certainly, it is encouraging that as computation has enabled our intuition, molecular shape has consistently surprised us in its usefulness and adaptability. The first Aurelius question, "What is the essence of a thing?", seems well answered, however, the third, "What do molecules do?", only partly so. Are the topics covered here exhaustive, or is there more to come? To date, there has been little published on the use of the volumetric definition of shape described here as a QSAR variable, for instance, in the prediction or classification of activity, although other shape definitions have been successful applied, for instance, as embodied in the Compass program described above in "Shape from Surfaces". Crystal packing is a phenomenon much desired to be understood. Although powerful models have been applied to the problem, to what degree is this dominated purely by the shape of a molecule? The shape comparison described here is typically of a global nature, and yet some importance must surely be placed on partial shape matching, just as the substructure matching of chemical graphs has proved useful. The approach of using surfaces, as described here, offers some flavor of this, as does the use of metrics that penalize volume mismatch less than the Tanimoto, e.g., Tversky measures. As yet, there is little to go on as to how useful a paradigm this will be because there is less software and fewer concrete results.Finally, the distance between molecular shapes, or between any shapes defined as volumes or surfaces, is a metric property in the mathematical sense of the word. As yet, there has been little, if any, application of this observation. We cannot know what new application to the design and discovery of pharmaceuticals may yet arise from the simple concept of molecular shape, but it is fair to say that the progress so far is impressive.

Identification of 4-(2-(4-amino-1,2,5-oxadiazol-3-yl)-1-ethyl-7-{[(3S)-3-piperidinylmethyl]oxy}-1H-imidazo[4,5-c]pyridin-4-yl)-2-methyl-3-butyn-2-ol (GSK690693), a novel inhibitor of AKT kinase.

Overexpression of AKT has an antiapoptotic effect in many cell types, and expression of dominant negative AKT blocks the ability of a variety of growth factors to promote survival. Therefore, inhibitors of AKT kinase activity might be useful as monotherapy for the treatment of tumors with activated AKT. Herein, we describe our lead optimization studies culminating in the discovery of compound 3g (GSK690693). Compound 3g is a novel ATP competitive, pan-AKT kinase inhibitor with IC 50 values of 2, 13, and 9 nM against AKT1, 2, and 3, respectively. An X-ray cocrystal structure was solved with 3g and the kinase domain of AKT2, confirming that 3g bound in the ATP binding pocket. Compound 3g potently inhibits intracellular AKT activity as measured by the inhibition of the phosphorylation levels of GSK3beta. Intraperitoneal administration of 3g in immunocompromised mice results in the inhibition of GSK3beta phosphorylation and tumor growth in human breast carcinoma (BT474) xenografts.

From enzyme to zymogen: engineering Vip2, an ADP-ribosyltransferase from Bacillus cereus, for conditional toxicity.

The adenosine diphosphate (ADP)-ribosyltransferase, Vip2 (vegetative insecticidal protein), from Bacillus cereus in combination with another protein from the same organism, Vip1, has insecticidal activity against western corn rootworm larvae. The Vip2 protein exerts its intracellular poisoning effect by modifying actin and preventing actin polymerization. Due to the nature of this toxin, expression of Vip2 in planta is lethal. In this work, we attempted to build an enzyme precursor (proenzyme, zymogen) that would silently reside in one biological system (e.g. plants or yeast) and be activated in the other (insect larvae). Our approach involved engineering a random propeptide library at the C-terminal end of Vip2 and selecting for malfunctional enzyme variants in yeast. A selected proenzyme (proVip2) possesses reduced enzymatic activity as compared with the wild-type Vip2 protein, but remains a potent toxin toward rootworm larvae. In addition, upon analysis of the digestive fate of the engineered enzyme precursor in rootworm larvae, we demonstrated that 'zymogenized' Vip2 can be proteolytically activated by rootworm digestive enzyme machinery. This report represents an example of applying a protein engineering strategy for the creation of a plant-tolerated, zymogen-like form of an otherwise toxic protein. This approach may outline a novel path to address challenges associated with utilizing toxic proteins in certain biotechnological applications.

How to do an evaluation: pitfalls and traps.

The recent literature is replete with papers evaluating computational tools (often those operating on 3D structures) for their performance in a certain set of tasks. Most commonly these papers compare a number of docking tools for their performance in cognate re-docking (pose prediction) and/or virtual screening. Related papers have been published on ligand-based tools: pose prediction by conformer generators and virtual screening using a variety of ligand-based approaches. The reliability of these comparisons is critically affected by a number of factors usually ignored by the authors, including bias in the datasets used in virtual screening, the metrics used to assess performance in virtual screening and pose prediction and errors in crystal structures used.

Pancreatitis in initially occult gastric heterotopic pancreas.

Heterotopic pancreas is an uncommon but important entity in the differential diagnosis of a gastric mass, as it has management and prognostic implications distinct from other gastric tumors. We present a case of heterotopic pancreas in the stomach, which was initially endoscopically occult but developed superimposed pancreatitis leading to its clinical presentation and detection as an apparently new gastric neoplasm.

Phase II trial of thalidomide as maintenance therapy for extensive stage small cell lung cancer after response to chemotherapy.

BACKGROUND: Extensive-stage small cell lung cancer (SCLC) is a highly aggressive malignancy for which little therapeutic progress has been made over the past 20 years. SCLC is a highly angiogenic tumor and targeting angiogenesis is being investigated. The putative mechanism of action of thalidomide is through inhibition of new blood vessel formation. This trial was designed to evaluate thalidomide in ES-SCLC. PATIENTS AND METHODS: Patients who had received first-line chemotherapy without disease progression were eligible. Patients received thalidomide 200 mg daily as maintenance therapy starting 3-6 weeks after completion of chemotherapy. RESULTS: Thirty patients were enrolled. Toxicity was minimal with grade 1 neuropathy in 27% of patients and only one case of grade 3 neuropathy. Median survival from time of initiation of induction chemotherapy was 12.8 months (95% CI: 10.1-15.8 months) and 1-year survival of 51.7% (95% CI: 32.5-67.9%). Median duration on thalidomide was 79 days. CONCLUSION: Thalidomide 200mg daily is well tolerated when given as maintenance therapy for ES-SCLC after induction chemotherapy. Further evaluation of anti-angiogenic agents in SCLC is warranted.

A critical assessment of docking programs and scoring functions.

Docking is a computational technique that samples conformations of small molecules in protein binding sites; scoring functions are used to assess which of these conformations best complements the protein binding site. An evaluation of 10 docking programs and 37 scoring functions was conducted against eight proteins of seven protein types for three tasks: binding mode prediction, virtual screening for lead identification, and rank-ordering by affinity for lead optimization. All of the docking programs were able to generate ligand conformations similar to crystallographically determined protein/ligand complex structures for at least one of the targets. However, scoring functions were less successful at distinguishing the crystallographic conformation from the set of docked poses. Docking programs identified active compounds from a pharmaceutically relevant pool of decoy compounds; however, no single program performed well for all of the targets. For prediction of compound affinity, none of the docking programs or scoring functions made a useful prediction of ligand binding affinity.

Aminoalkoxybenzyl pyrrolidines as novel human urotensin-II receptor antagonists.

High throughput screening of the corporate compound collection led to the discovery of a novel series of substituted aminoalkoxybenzyl pyrrolidines as human urotensin-II receptor antagonists. The synthesis, initial structure-activity relationships, and optimization of the initial hit that led to the identification of a truncated sub-series, represented by SB-436811 (1a), are described.