The Bigger Fish: A Comparison of Meta-Learning QSAR Models on Low-Resourced Aquatic Toxicity Regression Tasks.

Toxicological information as needed for risk assessments of chemical compounds is often sparse. Unfortunately, gathering new toxicological information experimentally often involves animal testing. Simulated alternatives, e.g., quantitative structure-activity relationship (QSAR) models, are preferred to infer the toxicity of new compounds. Aquatic toxicity data collections consist of many related tasks─each predicting the toxicity of new compounds on a given species. Since many of these tasks are inherently low-resource, i.e., involve few associated compounds, this is challenging. Meta-learning is a subfield of artificial intelligence that can lead to more accurate models by enabling the utilization of information across tasks. In our work, we benchmark various state-of-the-art meta-learning techniques for building QSAR models, focusing on knowledge sharing between species. Specifically, we employ and compare transformational machine learning, model-agnostic meta-learning, fine-tuning, and multi-task models. Our experiments show that established knowledge-sharing techniques outperform single-task approaches. We recommend the use of multi-task random forest models for aquatic toxicity modeling, which matched or exceeded the performance of other approaches and robustly produced good results in the low-resource settings we studied. This model functions on a species level, predicting toxicity for multiple species across various phyla, with flexible exposure duration and on a large chemical applicability domain.

Fast and Informative Model Selection Using Learning Curve Cross-Validation.

Common cross-validation (CV) methods like k-fold cross-validation or Monte Carlo cross-validation estimate the predictive performance of a learner by repeatedly training it on a large portion of the given data and testing it on the remaining data. These techniques have two major drawbacks. First, they can be unnecessarily slow on large datasets. Second, beyond an estimation of the final performance, they give almost no insights into the learning process of the validated algorithm. In this article, we present a new approach for validation based on learning curves (LCCV). Instead of creating train-test splits with a large portion of training data, LCCV iteratively increases the number of instances used for training. In the context of model selection, it discards models that are unlikely to become competitive. In a series of experiments on 75 datasets, we could show that in over 90% of the cases using LCCV leads to the same performance as using 5/10-fold CV while substantially reducing the runtime (median runtime reductions of over 50%); the performance using LCCV never deviated from CV by more than 2.5%. We also compare it to a racing-based method and successive halving, a multi-armed bandit method. Additionally, it provides important insights, which for example allows assessing the benefits of acquiring more data.

Predicting Machine Learning Pipeline Runtimes in the Context of Automated Machine Learning.

Automated machine learning (AutoML) seeks to automatically find so-called machine learning pipelines that maximize the prediction performance when being used to train a model on a given dataset. One of the main and yet open challenges in AutoMLis an effective use of computational resources: An AutoML process involves the evaluation of many candidate pipelines, which are costly but often ineffective because they are canceled due to a timeout. In this paper, we present an approach to predict the runtime of two-step machine learning pipelines with up to one pre-processor, which can be used to anticipate whether or not a pipeline will time out. Separate runtime models are trained offline for each algorithm that may be used in a pipeline, and an overall prediction is derived from these models. We empirically show that the approach increases successful evaluations made by an AutoML tool while preserving or even improving on the previously best solutions.

AutoML for Multi-Label Classification: Overview and Empirical Evaluation.

Automated machine learning (AutoML) supports the algorithmic construction and data-specific customization of machine learning pipelines, including the selection, combination, and parametrization of machine learning algorithms as main constituents. Generally speaking, AutoML approaches comprise two major components: a search space model and an optimizer for traversing the space. Recent approaches have shown impressive results in the realm of supervised learning, most notably (single-label) classification (SLC). Moreover, first attempts at extending these approaches towards multi-label classification (MLC) have been made. While the space of candidate pipelines is already huge in SLC, the complexity of the search space is raised to an even higher power in MLC. One may wonder, therefore, whether and to what extent optimizers established for SLC can scale to this increased complexity, and how they compare to each other. This paper makes the following contributions: First, we survey existing approaches to AutoML for MLC. Second, we augment these approaches with optimizers not previously tried for MLC. Third, we propose a benchmarking framework that supports a fair and systematic comparison. Fourth, we conduct an extensive experimental study, evaluating the methods on a suite of MLC problems. We find a grammar-based best-first search to compare favorably to other optimizers.

Cardioprotection by Humoral Factors Released After Remote Ischemic Preconditioning Depends on Anesthetic Regimen.

OBJECTIVES: Remote ischemic preconditioning (RIPC) is a practicable and noninvasive method to protect the heart against ischemia reperfusion injury. Unfortunately results from clinical studies are not convincing. Propofol is suggested to be an inhibiting factor of cardioprotection by RIPC, but the underlying mechanism is still unknown. We investigated whether after RIPC the release of humoral factors and/or the direct cardioprotective effect at the myocardium is inhibited by propofol. DESIGN: Randomized, prospective, blinded laboratory investigation. SETTING: Experimental laboratory. PATIENTS/SUBJECTS: Male Wistar rats. INTERVENTIONS: Repetitive hind limb ischemia in rats-blood plasma transfers to isolated rat heart. MEASUREMENTS AND MAIN RESULTS: In male Wistar rats (six groups, each n = 6/group), RIPC was induced by four cycles of 5 minutes bilateral hind limb ischemia alternately with 5 minutes of reperfusion. Blood samples were taken with (RIPC) and without RIPC (Con). Rats received continuous anesthesia with pentobarbital (Pento, 40 mg/kg body weight/hr) or propofol (Prop, 12 mg/kg body weight/hr), respectively. Cardioprotective properties of the blood plasma was investigated in the rat heart in vitro (six groups, each n = 6/group) perfused with Krebs-Henseleit buffer alone or with propofol (10 µM). Plasma was administered over 10 minutes before myocardial ischemia. All hearts underwent 33 minutes of global ischemia followed by 1 hour of reperfusion. At the end of the experiments, infarct size was determined by triphenyl-tetrazolium-chloride staining. RIPC plasma from pentobarbital anesthetized rats (Pento-RIPC) reduced infarct size from 64% (62-71%) (Pento-Con) to 34% (30-39%) (p < 0.0001). Infarct size with control plasma from propofol anesthetized rats was 59% (58-64%) (Prop-Con). RIPC plasma could not induce cardioprotection (Prop-RIPC: 63% [56-70%] ns vs Prop-Con). In contrast, RIPC plasma from pentobarbital anesthetized rats induced a significant infarct size reduction under propofol perfusion (Pento-RIPC: 34% [30-42%] vs Pento-Con: 54% [53-63%]; p < 0.0001). CONCLUSIONS: Loss of cardioprotection by RIPC during propofol anesthesia depends on inhibition of release of humoral factors.