Chemistry domain of applicability evaluation against existing estrogen receptor high-throughput assay-based activity models.

Introduction: The U. S. Environmental Protection Agency's Endocrine Disruptor Screening Program (EDSP) Tier 1 assays are used to screen for potential endocrine system-disrupting chemicals. A model integrating data from 16 high-throughput screening assays to predict estrogen receptor (ER) agonism has been proposed as an alternative to some low-throughput Tier 1 assays. Later work demonstrated that as few as four assays could replicate the ER agonism predictions from the full model with 98% sensitivity and 92% specificity. The current study utilized chemical clustering to illustrate the coverage of the EDSP Universe of Chemicals (UoC) tested in the existing ER pathway models and to investigate the utility of chemical clustering to evaluate the screening approach using an existing 4-assay model as a test case. Although the full original assay battery is no longer available, the demonstrated contribution of chemical clustering is broadly applicable to assay sets, chemical inventories, and models, and the data analysis used can also be applied to future evaluation of minimal assay models for consideration in screening. Methods: Chemical structures were collected for 6,947 substances via the CompTox Chemicals Dashboard from the over 10,000 UoC and grouped based on structural similarity, generating 826 chemical clusters. Of the 1,812 substances run in the original ER model, 1,730 substances had a single, clearly defined structure. The ER model chemicals with a clearly defined structure that were not present in the EDSP UoC were assigned to chemical clusters using a k-nearest neighbors approach, resulting in 557 EDSP UoC clusters containing at least one ER model chemical. Results and Discussion: Performance of an existing 4-assay model in comparison with the existing full ER agonist model was analyzed as related to chemical clustering. This was a case study, and a similar analysis can be performed with any subset model in which the same chemicals (or subset of chemicals) are screened. Of the 365 clusters containing >1 ER model chemical, 321 did not have any chemicals predicted to be agonists by the full ER agonist model. The best 4-assay subset ER agonist model disagreed with the full ER agonist model by predicting agonist activity for 122 chemicals from 91 of the 321 clusters. There were 44 clusters with at least two chemicals and at least one agonist based upon the full ER agonist model, which allowed accuracy predictions on a per-cluster basis. The accuracy of the best 4-assay subset ER agonist model ranged from 50% to 100% across these 44 clusters, with 32 clusters having accuracy ≥90%. Overall, the best 4-assay subset ER agonist model resulted in 122 false-positive and only 2 false-negative predictions compared with the full ER agonist model. Most false positives (89) were active in only two of the four assays, whereas all but 11 true positive chemicals were active in at least three assays. False positive chemicals also tended to have lower area under the curve (AUC) values, with 110 out of 122 false positives having an AUC value below 0.214, which is lower than 75% of the positives as predicted by the full ER agonist model. Many false positives demonstrated borderline activity. The median AUC value for the 122 false positives from the best 4-assay subset ER agonist model was 0.138, whereas the threshold for an active prediction is 0.1. Conclusion: Our results show that the existing 4-assay model performs well across a range of structurally diverse chemicals. Although this is a descriptive analysis of previous results, several concepts can be applied to any screening model used in the future. First, the clustering of the chemicals provides a means of ensuring that future screening evaluations consider the broad chemical space represented by the EDSP UoC. The clusters can also assist in prioritizing future chemicals for screening in specific clusters based on the activity of known chemicals in those clusters. The clustering approach can be useful in providing a framework to evaluate which portions of the EDSP UoC chemical space are reliably covered by in silico and in vitro approaches and where predictions from either method alone or both methods combined are most reliable. The lessons learned from this case study can be easily applied to future evaluations of model applicability and screening to evaluate future datasets.

Optimizing androgen receptor prioritization using high-throughput assay-based activity models.

Introduction: Computational models using data from high-throughput screening assays have promise for prioritizing and screening chemicals for testing under the U.S. Environmental Protection Agency's Endocrine Disruptor Screening Program (EDSP). The purpose of this work was to demonstrate a data processing method for the determination of optimal minimal assay batteries from a larger comprehensive model, to provide a uniform method of evaluating the performance of future minimal assay batteries compared with the androgen receptor (AR) pathway model, and to incorporate chemical cluster analysis into this evaluation. Although several of the assays in the AR pathway model are no longer available through the original vendor, this approach could be used for future evaluations of minimal assay models for prioritization and screening. Methods: We compared two previously published models and found that an expanded 14-assay model had higher sensitivity for antagonists, whereas the original 11-assay model had slightly higher sensitivity for agonists. We then investigated subsets of assays in the original AR pathway model to optimize overall testing strategies that minimize cost while maintaining sensitivity across a broad chemical space. Results and Discussion: Evaluation of the critical assays across subset models derived from the 14-assay model identified three critical assays for predicting antagonism and two critical assays for predicting agonism. A minimum of nine assays is required for predicting agonism and antagonism with high sensitivity (95%). However, testing workflows guided by chemical structure-based clusters can reduce the average number of assays needed per chemical by basing the assays selected for testing on the likelihood of a chemical being an AR agonist, according to its structure. Our results show that a multi-stage testing workflow can provide 95% sensitivity while requiring only 48% of the resources required for running all assays from the original full models. The resources can be reduced further by incorporating in silico activity predictions. Conclusion: This work illustrates a data-driven approach that incorporates chemical clustering and simultaneous consideration of antagonism and agonism mechanisms to more efficiently screen chemicals. This case study provides a proof of concept for prioritization and screening strategies that can be utilized in future analyses to minimize the overall number of assays needed for predicting AR activity, which will maximize the number of chemicals that can be tested and allow data-driven prioritization of chemicals for further screening under the EDSP.

Linking complex disease and exposure data-insights from an environmental and occupational health study.

The disparate measurement protocols used to collect study data are an intrinsic barrier to combining information from environmental health studies. Using standardized measurement protocols and data standards for environmental exposures addresses this gap by improving data collection quality and consistency. To assess the prevalence of environmental exposures in National Institutes of Health (NIH) public data repositories and resources and to assess the commonality of the data elements, we analyzed clinical measures and exposure assays by comparing the Caribbean Consortium for Research in Environmental and Occupational Health study with selected NIH environmental health resources and studies. Our assessment revealed that (1) environmental assessments are widely collected in these resources, (2) biological assessments are less prevalent, and (3) NIH resources can help identify common data for meta-analysis. We highlight resources to help link environmental exposure data across studies to support data sharing. Including NIH data standards in environmental health research facilitates comparing and combining study data, and the use of NIH resources and adoption of standard measures will allow integration of multiple studies and increase the scientific impact of individual studies.

Mapping Mechanistic Pathways of Acute Oral Systemic Toxicity Using Chemical Structure and Bioactivity Measurements.

Regulatory agencies around the world have committed to reducing or eliminating animal testing for establishing chemical safety. Adverse outcome pathways can facilitate replacement by providing a mechanistic framework for identifying the appropriate non-animal methods and connecting them to apical adverse outcomes. This study separated 11,992 chemicals with curated rat oral acute toxicity information into clusters of structurally similar compounds. Each cluster was then assigned one or more ToxCast/Tox21 assays by looking for the minimum number of assays required to record at least one positive hit call below cytotoxicity for all acutely toxic chemicals in the cluster. When structural information is used to select assays for testing, none of the chemicals required more than four assays and 98% required two assays or less. Both the structure-based clusters and activity from the associated assays were significantly associated with the GHS toxicity classification of the chemicals, which suggests that a combination of bioactivity and structural information could be as reproducible as traditional in vivo studies. Predictivity is improved when the in vitro assay directly corresponds to the mechanism of toxicity, but many indirect assays showed promise as well. Given the lower cost of in vitro testing, a small assay battery including both general cytotoxicity assays and two or more orthogonal assays targeting the toxicological mechanism could be used to improve performance further. This approach illustrates the promise of combining existing in silico approaches, such as the Collaborative Acute Toxicity Modeling Suite (CATMoS), with structure-based bioactivity information as part of an efficient tiered testing strategy that can reduce or eliminate animal testing for acute oral toxicity.

Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks.

The workshop titled "Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks" was co-organized by the Evidence-based Toxicology Collaboration and the European Food Safety Authority (EFSA) and hosted by EFSA at its headquarters in Parma, Italy on October 2 and 3, 2019. The goal was to explore integration of systematic review with mechanistic evidence evaluation. Participants were invited to work on concrete products to advance the exploration of how evidence-based approaches can support the development and application of adverse outcome pathways (AOP) in chemical risk assessment. The workshop discussions were centered around three related themes: 1) assessing certainty in AOPs, 2) literature-based AOP development, and 3) integrating certainty in AOPs and non-animal evidence into decision frameworks. Several challenges, mostly related to methodology, were identified and largely determined the workshop recommendations. The workshop recommendations included the comparison and potential alignment of processes used to develop AOP and systematic review methodology, including the translation of vocabulary of evidence-based methods to AOP and vice versa, the development and improvement of evidence mapping and text mining methods and tools, as well as a call for a fundamental change in chemical risk and uncertainty assessment methodology if to be conducted based on AOPs and new approach methodologies (NAM). The usefulness of evidence-based approaches for mechanism-based chemical risk assessments was stressed, particularly the potential contribution of the rigor and transparency inherent to such approaches in building stakeholders' trust for implementation of NAM evidence and AOPs into chemical risk assessment.

Defining clinical outcome pathways.

Here, we propose a broad concept of 'Clinical Outcome Pathways' (COPs), which are defined as a series of key molecular and cellular events that underlie therapeutic effects of drug molecules. We formalize COPs as a chain of the following events: molecular initiating event (MIE) â†’ intermediate event(s) â†’ clinical outcome. We illustrate the concept with COP examples both for primary and alternative (i.e., drug repurposing) therapeutic applications. We also describe the elucidation of COPs for several drugs of interest using the publicly accessible Reasoning Over Biomedical Objects linked in Knowledge-Oriented Pathways (ROBOKOP) biomedical knowledge graph-mining tool. We propose that broader use of COP uncovered with the help of biomedical knowledge graph mining will likely accelerate drug discovery and repurposing efforts.

Predicting nonlinear relationships between external and internal concentrations with physiologically based pharmacokinetic modeling.

Although external concentrations are more readily quantified and often used as the metric for regulating and mitigating exposures to environmental chemicals, the toxicological response to an environmental chemical is more directly related to its internal concentrations than the external concentration. The processes of absorption, distribution, metabolism, and excretion (ADME) determine the quantitative relationship between the external and internal concentrations, and these processes are often susceptible to saturation at high concentrations, which can lead to nonlinear changes in internal concentrations that deviate from proportionality. Using generic physiologically-based pharmacokinetic (PBPK) models, we explored how saturable absorption or clearance influence the shape of the internal to external concentration (IEC) relationship. We used the models for hypothetical chemicals to show how differences in kinetic parameters can impact the shape of an IEC relationship; and models for styrene and caffeine to explore how exposure route, frequency, and duration impact the IEC relationships in rat and human exposures. We also analyzed available plasma concentration data for 2,4-dichlorophenoxyacetic acid to demonstrate how a PBPK modeling approach can be an alternative to common statistical methods for analyzing dose proportionality. A PBPK modeling approach can be a valuable tool used in the early stages of a chemical safety assessment program to optimize the design of longer-term animal toxicity studies or to interpret study results.

Applying evidence-based methods to the development and use of adverse outcome pathways

The workshop "Application of evidence-based methods to construct mechanistic frameworks for the development and use of non-animal toxicity tests" was organized by the Evidence-based Toxicology Collaboration and hosted by the Grading of Recommendations Assessment, Development and Evaluation Working Group on June 12, 2019. The purpose of the workshop was to bring together international regulatory bodies, risk assessors, academic scientists, and industry to explore how systematic review methods and the adverse outcome pathway framework could be combined to develop and use mechanistic test methods for predicting the toxicity of chemical substances in an evidence-based manner. The meeting covered the history of biological frameworks, the way adverse outcome pathways are currently developed, the basic principles of systematic methodology, including systematic reviews and evidence maps, and assessment of cer­tainty in models, and adverse outcome pathways in particular. Specific topics were discussed via case studies in small break-out groups. The group concluded that adverse outcome pathways provide an important framework to support mechanism-based assessment in environmental health. The process of their development has a few challenges that could be addressed with systematic methods and automation tools. Addressing these challenges will increase the transparency of the evidence behind adverse outcome pathways and the consistency with which they are defined; this in turn will increase their value for supporting public health decisions. It was suggested to explore the details of applying systematic methods to adverse outcome pathway development in a series of case studies and workshops.

Mind the gaps: Prioritizing activities to meet regulatory needs for acute systemic lethality

Efforts are underway to develop and implement nonanimal approaches which can characterize acute systemic lethality. A workshop was held in October 2019 to discuss developments in the prediction of acute oral lethality for chemicals and mixtures, as well as progress and needs in the understanding and modeling of mechanisms of acute lethality. During the workshop, each speaker led the group through a series of charge questions to determine clear next steps to progress the aims of the workshop. Participants concluded that a variety of approaches will be needed and should be applied in a tiered fashion. Non-testing approaches, including waiving tests, computational models for single chemicals, and calculating the acute lethality of mixtures based on the LD50 values of mixture components, could be used for some assessments now, especially in the very toxic or non-toxic classification ranges. Agencies can develop policies indicating contexts under which mathematical approaches for mixtures assessment are acceptable; to expand applicability, poorly predicted mixtures should be examined to understand discrepancies and adapt the approach. Transparency and an understanding of the variability of in vivo approaches are crucial to facilitate regulatory application of new approaches. In a replacement strategy, mechanistically based in vitro or in silico models will be needed to support non-testing approaches especially for highly acutely toxic chemicals. The workshop discussed approaches that can be used in the immediate or near term for some applications and identified remaining actions needed to implement approaches to fully replace the use of animals for acute systemic toxicity testing.

Knowledge Organization Systems for Systematic Chemical Assessments.

BACKGROUND: Although the implementation of systematic review and evidence mapping methods stands to improve the transparency and accuracy of chemical assessments, they also accentuate the challenges that assessors face in ensuring they have located and included all the evidence that is relevant to evaluating the potential health effects an exposure might be causing. This challenge of information retrieval can be characterized in terms of "semantic" and "conceptual" factors that render chemical assessments vulnerable to the streetlight effect. OBJECTIVES: This commentary presents how controlled vocabularies, thesauruses, and ontologies contribute to overcoming the streetlight effect in information retrieval, making up the key components of Knowledge Organization Systems (KOSs) that enable more systematic access to assessment-relevant information than is currently achievable. The concept of Adverse Outcome Pathways is used to illustrate what a general KOS for use in chemical assessment could look like. DISCUSSION: Ontologies are an underexploited element of effective knowledge organization in the environmental health sciences. Agreeing on and implementing ontologies in chemical assessment is a complex but tractable process with four fundamental steps. Successful implementation of ontologies would not only make currently fragmented information about health risks from chemical exposures vastly more accessible, it could ultimately enable computational methods for chemical assessment that can take advantage of the full richness of data described in natural language in primary studies. https://doi.org/10.1289/EHP6994.

In Silico Guidance for In Vitro Androgen and Glucocorticoid Receptor ToxCast Assays.

Molecular initiating events (MIEs) are key events in adverse outcome pathways that link molecular chemistry to target biology. As they are based on chemistry, these interactions are excellent targets for computational chemistry approaches to in silico modeling. In this work, we aim to link ligand chemical structures to MIEs for androgen receptor (AR) and glucocorticoid receptor (GR) binding using ToxCast data. This has been done using an automated computational algorithm to perform maximal common substructure searches on chemical binders for each target from the ToxCast dataset. The models developed show a high level of accuracy, correctly assigning 87.20% of AR binders and 96.81% of GR binders in a 25% test set using holdout cross-validation. The 2D structural alerts developed can be used as in silico models to predict these MIEs and as guidance for in vitro ToxCast assays to confirm hits. These models can target such experimental work, reducing the number of assays to be performed to gain required toxicological insight. Development of these models has also allowed some structural alerts to be identified as predictors for agonist or antagonist behavior at the receptor target. This work represents a first step in using computational methods to guide and target experimental approaches.

An Assessment of Environmental Health Measures in the Deepwater Horizon Research Consortia.

Research consortia play a key role in our understanding of how environmental exposures influence health and wellbeing, especially in the case of catastrophic events such as the Deepwater Horizon oil spill. A common challenge that prevents the optimal use of these data is the difficulty of harmonizing data regarding the environmental exposures and health effects across the studies within and among consortia. A review of the measures used by members of the Deepwater Horizon Research Consortia highlights the challenges associated with balancing timely implementation of a study to support disaster relief with optimizing the long-term value of the data. The inclusion of common, standard measures at the study design phase and a priori discussions regarding harmonization of study-specific measures among consortia members are key to overcoming this challenge. As more resources become available to support the use of standard measures, researchers now have the tools needed to rapidly coordinate their studies without compromising research focus or timely completion of the original study goals.

The Adverse Outcome Pathway: A Multifaceted Framework Supporting 21st Century Toxicology.

The adverse outcome pathway (AOP) framework serves as a knowledge assembly, interpretation, and communication tool designed to support the translation of pathway-specific mechanistic data into responses relevant to assessing and managing risks of chemicals to human health and the environment. As such, AOPs facilitate the use of data streams often not employed by risk assessors, including information from in silico models, in vitro assays and short-term in vivo tests with molecular/biochemical endpoints. This translational capability can increase the capacity and efficiency of safety assessments both for single chemicals and chemical mixtures. Our mini-review describes the conceptual basis of the AOP framework and aspects of its current status relative to use by toxicologists and risk assessors, including four illustrative applications of the framework to diverse assessment scenarios.

Differentiating Pathway-Specific From Nonspecific Effects in High-Throughput Toxicity Data: A Foundation for Prioritizing Adverse Outcome Pathway Development.

The U.S. Environmental Protection Agency's ToxCast program has screened thousands of chemicals for biological activity, primarily using high-throughput in vitro bioassays. Adverse outcome pathways (AOPs) offer a means to link pathway-specific biological activities with potential apical effects relevant to risk assessors. Thus, efforts are underway to develop AOPs relevant to pathway-specific perturbations detected in ToxCast assays. Previous work identified a "cytotoxic burst" (CTB) phenomenon wherein large numbers of the ToxCast assays begin to respond at or near test chemical concentrations that elicit cytotoxicity, and a statistical approach to defining the bounds of the CTB was developed. To focus AOP development on the molecular targets corresponding to ToxCast assays indicating pathway-specific effects, we conducted a meta-analysis to identify which assays most frequently respond at concentrations below the CTB. A preliminary list of potentially important, target-specific assays was determined by ranking assays by the fraction of chemical hits below the CTB compared with the number of chemicals tested. Additional priority assays were identified using a diagnostic-odds-ratio approach which gives greater ranking to assays with high specificity but low responsivity. Combined, the two prioritization methods identified several novel targets (e.g., peripheral benzodiazepine and progesterone receptors) to prioritize for AOP development, and affirmed the importance of a number of existing AOPs aligned with ToxCast targets (e.g., thyroperoxidase, estrogen receptor, aromatase). The prioritization approaches did not appear to be influenced by inter-assay differences in chemical bioavailability. Furthermore, the outcomes were robust based on a variety of different parameters used to define the CTB.

AOP-DB: A database resource for the exploration of Adverse Outcome Pathways through integrated association networks.

The Adverse Outcome Pathway (AOP) framework describes the progression of a toxicity pathway from molecular perturbation to population-level outcome in a series of measurable, mechanistic responses. The controlled, computer-readable vocabulary that defines an AOP has the ability to, automatically and on a large scale, integrate AOP knowledge with publically available sources of biological high-throughput data and its derived associations. To support the discovery and development of putative (existing) and potential AOPs, we introduce the AOP-DB, an exploratory database resource that aggregates association relationships between genes and their related chemicals, diseases, pathways, species orthology information, ontologies, and gene interactions. These associations are mined from publically available annotation databases and are integrated with the AOP information centralized in the AOP-Wiki, allowing for the automatic characterization of both putative and potential AOPs in the context of multiple areas of biological information, referred to here as "biological entities". The AOP-DB acts as a hypothesis-generation tool for the expansion of putative AOPs, as well as the characterization of potential AOPs, through the creation of association networks across these biological entities. Finally, the AOP-DB provides a useful interface between the AOP framework and existing chemical screening and prioritization efforts by the US Environmental Protection Agency.

A Case Study Application of the Aggregate Exposure Pathway (AEP) and Adverse Outcome Pathway (AOP) Frameworks to Facilitate the Integration of Human Health and Ecological End Points for Cumulative Risk Assessment (CRA).

Cumulative risk assessment (CRA) methods promote the use of a conceptual site model (CSM) to apportion exposures and integrate risk from multiple stressors. While CSMs may encompass multiple species, evaluating end points across taxa can be challenging due to data availability and physiological differences among organisms. Adverse outcome pathways (AOPs) describe biological mechanisms leading to adverse outcomes (AOs) by assembling causal pathways with measurable intermediate steps termed key events (KEs), thereby providing a framework for integrating data across species. In this work, we used a case study focused on the perchlorate anion (ClO4-) to highlight the value of the AOP framework for cross-species data integration. Computational models and dose-response data were used to evaluate the effects of ClO4- in 12 species and revealed a dose-response concordance across KEs and taxa. The aggregate exposure pathway (AEP) tracks stressors from sources to the exposures and serves as a complement to the AOP. We discuss how the combined AEP-AOP construct helps to maximize the use of existing data and advances CRA by (1) organizing toxicity and exposure data, (2) providing a mechanistic framework of KEs for integrating data across human health and ecological end points, (3) facilitating cross-species dose-response evaluation, and (4) highlighting data gaps and technical limitations.

A Mechanistic Framework for Integrating Chemical Structure and High-Throughput Screening Results to Improve Toxicity Predictions.

Adverse Outcome Pathways (AOPs) establish a connection between a molecular initiating event (MIE) and an adverse outcome. Detailed understanding of the MIE provides the ideal data for determining chemical properties required to elicit the MIE. This study utilized high-throughput screening data from the ToxCast program, coupled with chemical structural information, to generate chemical clusters using three similarity methods pertaining to nine MIEs within an AOP network for hepatic steatosis. Three case studies demonstrate the utility of the mechanistic information held by the MIE for integrating biological and chemical data. Evaluation of the chemical clusters activating the glucocorticoid receptor identified activity differences in chemicals within a cluster. Comparison of the estrogen receptor results with previous work showed that bioactivity data and structural alerts can be combined to improve predictions in a customizable way where bioactivity data are limited. The aryl hydrocarbon receptor (AHR) highlighted that while structural data can be used to offset limited data for new screening efforts, not all ToxCast targets have sufficient data to define robust chemical clusters. In this context, an alternative to additional receptor assays is proposed where assays for proximal key events downstream of AHR activation could be used to enhance confidence in active calls. These case studies illustrate how the AOP framework can support an iterative process whereby in vitro toxicity testing and chemical structure can be combined to improve toxicity predictions. In vitro assays can inform the development of structural alerts linking chemical structure to toxicity. Consequently, structurally related chemical groups can facilitate identification of assays that would be informative for a specific MIE. Together, these activities form a virtuous cycle where the mechanistic basis for the in vitro results and the breadth of the structural alerts continually improve over time to better predict activity of chemicals for which limited toxicity data exist.

Editor's Highlight: Negative Predictors of Carcinogenicity for Environmental Chemicals.

Recent international efforts have led to proposals for modified carcinogenicity testing paradigms based on data from shorter-term studies. The main goal of the current study was to evaluate the negative predictive value (NPV) of short-term toxicity indicators on carcinogenicity study outcomes and cancer classifications for chemicals previously reviewed by the U.S. Environmental Protection Agency (EPA). Pathology data were analyzed from over 900 acceptable 2-sex guideline subchronic (3-month) and carcinogenicity studies in the U.S. EPA Toxicity Reference Database. Chemical cancer classifications were obtained from annual reports of the U.S. EPA Office of Pesticide Programs. Histopathologic risk signals and evidence of hormonal perturbation in subchronic rat studies provided 56% NPV for any tumor outcome in the rat or mouse and 75% NPV for cancer classifications not requiring quantitative risk assessment (qRA). In comparison, lack of activity in a battery of 35 in vitro cytotoxicity assays from the U.S. EPA ToxCast library provided 49% NPV for any tumor outcome and 80% NPV for cancer classifications not requiring qRA. These findings support the idea that the absence of short-term bioactivity may provide useful information for prioritizing chemicals based on potential carcinogenic risk. Additional data streams are needed to further refine these models.