Learning active subspaces and discovering important features with Gaussian radial basis functions neural networks.

Providing a model that achieves a strong predictive performance and is simultaneously interpretable by humans is one of the most difficult challenges in machine learning research due to the conflicting nature of these two objectives. To address this challenge, we propose a modification of the radial basis function neural network model by equipping its Gaussian kernel with a learnable precision matrix. We show that precious information is contained in the spectrum of the precision matrix that can be extracted once the training of the model is completed. In particular, the eigenvectors explain the directions of maximum sensitivity of the model revealing the active subspace and suggesting potential applications for supervised dimensionality reduction. At the same time, the eigenvectors highlight the relationship in terms of absolute variation between the input and the latent variables, thereby allowing us to extract a ranking of the input variables based on their importance to the prediction task enhancing the model interpretability. We conducted numerical experiments for regression, classification, and feature selection tasks, comparing our model against popular machine learning models, the state-of-the-art deep learning-based embedding feature selection techniques, and a transformer model for tabular data. Our results demonstrate that the proposed model does not only yield an attractive prediction performance compared to the competitors but also provides meaningful and interpretable results that potentially could assist the decision-making process in real-world applications. A PyTorch implementation of the model is available on GitHub at the following link.1.

Comparison of parallel optimization algorithms on computationally expensive groundwater remediation designs.

Contaminated groundwater resources threaten human health and destroy ecosystems in many areas worldwide. Groundwater remediation is crucial for environmental recovery; however, it can be cost prohibitive. Planning a cost-effective remediation design can take a long time, as it may involve the evaluation of many management decisions, and the corresponding simulation models are computationally demanding. Parallel optimization can facilitate much faster management decisions for cost-effective groundwater remediation design using complex pollutant transport models. However, the efficiency of different parallel optimization algorithms varies depending on both the search strategy and parallelism. In this paper, we show the performance of a parallel surrogate-based optimization algorithm called parallel stochastic radial basis function (p-SRBF), which has not been previously used on contaminant remediation problems. The two case problems involve two superfund sites (i.e., the Umatilla Aquifer and Blaine Aquifer), and one objective evaluation takes 5 and 30 min for the two problems, respectively. Exceptional speedup (superlinear) is achieved with 4 to 16 cores, and excellent speedup is achieved using up to 64 cores, obtaining a good solution at 80 % efficiency. We compare our p-SRBF with three different parallel derivative-free optimization algorithms, including genetic algorithm, mesh adaptive direct search, and asynchronous parallel pattern search optimization, in terms of objective quality, computational reduction and robust behavior across multiple trials. p-SRBF outperforms the other algorithms, as it finds the best solution in both the Umatilla and Blaine cases and reduces the computational budget by at least 50 % in both problems. Additionally, statistical comparisons show that the p-SRBF results are better than those of the alternative algorithms at the 5 % significant level. This study enriches theoretical real-world groundwater remediation methods. The results demonstrate that p-SRBF is promising for environmental management problems that involve computationally expensive models.

Efficient, parallelized global optimization of groundwater pumping in a regional aquifer with land subsidence constraints.

The design of groundwater exploitation schedules with constraints on pumping-induced land subsidence is a computationally intensive task. Physical process-based groundwater flow and land subsidence simulations are high-dimensional, nonlinear, dynamic and computationally demanding, as they require solving large systems of partial differential equations (PDEs). This work is the first application of a parallelized surrogate-based global optimization algorithm to mitigate land subsidence issues by controlling the pumping schedule of multiple groundwater wellfields over space and time. The application was demonstrated in a 6500 km2 region in China, involving a large-scale coupled groundwater flow-land subsidence model that is computationally expensive in terms of computational resources, including runtime and CPU memory for one single evaluation. In addition, the optimization problem contains 50 decision variables and up to 13 constraints, which adds to the computational effort, thus an efficient optimization is required. The results show that parallel DYSOC (dynamic search with surrogate-based constrained optimization) can achieve an approximately 100% parallel efficiency when upscaling computing resources. Compared with two other widely used optimization algorithms, DYSOC is 2-6 times faster, achieving computational cost savings of at least 50%. The findings demonstrate that the integration of surrogate constraints and dynamic search process can aid in the exploration and exploitation of the search space and accelerate the search for optimal solutions to complicated problems.

A combined system of microwave-functionalized rice husk and poly-aluminium chloride for trace cadmium-contaminated source water purification: Exploration of removal efficiency and mechanism.

Cadmium is highly poisonous to mammals and related water pollution incidents are increasing world-widely. Here, the clean-up of trace Cd(II) by a combined process of microwave-functionalized rice husk (RHMW-M) and poly aluminium chloride (PAC) was investigated for the first time, with the exploration of removal mechanism and efficacy. Microwave irradiation was found to be a new approach to achieve the functionalized procedure, which could decrease the processing time from 2.5 h to 390 s with the Cd(II) uptake of the outcome product soaring from 137.16 mg/g to 191.32 mg/g. The ultra-rapidly prepared RHMW-M exhibited a fast adsorption equilibrium within 30 min over a wide pH range of 5.0-8.0, and the FT-IR and XPS studies confirmed that both ion exchange and chelation were functioned in the Cd(II) uptake process. Controlled by the turbidity threshold of drinking water treatment plant, the feasible dosage of RHMW-M in the absence and presence of 30 mg/L PAC increased from 30 to 760 mg/L, which could effectively deal with the trace Cd(II) at the concentration from 33 µg/L up to 0.933 mg/L, exhibiting much better performance than traditional alkali precipitation. Predictably, this simple and scalable RHMW-M/PAC system could afford a promising end-of-pipe solution for heavy-metal contamination.

Dynamic Optimization with Particle Swarms (DOPS): a meta-heuristic for parameter estimation in biochemical models.

BACKGROUND: Mathematical modeling is a powerful tool to analyze, and ultimately design biochemical networks. However, the estimation of the parameters that appear in biochemical models is a significant challenge. Parameter estimation typically involves expensive function evaluations and noisy data, making it difficult to quickly obtain optimal solutions. Further, biochemical models often have many local extrema which further complicates parameter estimation. Toward these challenges, we developed Dynamic Optimization with Particle Swarms (DOPS), a novel hybrid meta-heuristic that combined multi-swarm particle swarm optimization with dynamically dimensioned search (DDS). DOPS uses a multi-swarm particle swarm optimization technique to generate candidate solution vectors, the best of which is then greedily updated using dynamically dimensioned search. RESULTS: We tested DOPS using classic optimization test functions, biochemical benchmark problems and real-world biochemical models. We performed [Formula: see text] = 25 trials with [Formula: see text] = 4000 function evaluations per trial, and compared the performance of DOPS with other commonly used meta-heuristics such as differential evolution (DE), simulated annealing (SA) and dynamically dimensioned search (DDS). On average, DOPS outperformed other common meta-heuristics on the optimization test functions, benchmark problems and a real-world model of the human coagulation cascade. CONCLUSIONS: DOPS is a promising meta-heuristic approach for the estimation of biochemical model parameters in relatively few function evaluations. DOPS source code is available for download under a MIT license at http://www.varnerlab.org .

Efficacy and immunogenicity of single-dose AdVAV intranasal anthrax vaccine compared to anthrax vaccine absorbed in an aerosolized spore rabbit challenge model.

AdVAV is a replication-deficient adenovirus type 5-vectored vaccine expressing the 83-kDa protective antigen (PA83) from Bacillus anthracis that is being developed for the prevention of disease caused by inhalation of aerosolized B. anthracis spores. A noninferiority study comparing the efficacy of AdVAV to the currently licensed Anthrax Vaccine Absorbed (AVA; BioThrax) was performed in New Zealand White rabbits using postchallenge survival as the study endpoint (20% noninferiority margin for survival). Three groups of 32 rabbits were vaccinated with a single intranasal dose of AdVAV (7.5 × 10(7), 1.5 × 10(9), or 3.5 × 10(10) viral particles). Three additional groups of 32 animals received two doses of either intranasal AdVAV (3.5 × 10(10) viral particles) or intramuscular AVA (diluted 1:16 or 1:64) 28 days apart. The placebo group of 16 rabbits received a single intranasal dose of AdVAV formulation buffer. All animals were challenged via the inhalation route with a targeted dose of 200 times the 50% lethal dose (LD50) of aerosolized B. anthracis Ames spores 70 days after the initial vaccination and were followed for 3 weeks. PA83 immunogenicity was evaluated by validated toxin neutralizing antibody and serum anti-PA83 IgG enzyme-linked immunosorbent assays (ELISAs). All animals in the placebo cohort died from the challenge. Three of the four AdVAV dose cohorts tested, including two single-dose cohorts, achieved statistical noninferiority relative to the AVA comparator group, with survival rates between 97% and 100%. Vaccination with AdVAV also produced antibody titers with earlier onset and greater persistence than vaccination with AVA.

Local Derivative-Free Approximation of Computationally Expensive Posterior Densities.

Bayesian inference using Markov chain Monte Carlo (MCMC) is computationally prohibitive when the posterior density of interest, π, is computationally expensive to evaluate. We develop a derivative-free algorithm GRIMA to accurately approximate π by interpolation over its high-probability density (HPD) region, which is initially unknown. Our local approach reduces the waste of computational budget on approximation of π in the low-probability region, which is inherent in global experimental designs. However, estimation of the HPD region is nontrivial when derivatives of π are not available or are not informative about the shape of the HPD region. Without relying on derivatives, GRIMA iterates (a) sequential knot selection over the estimated HPD region of π to refine the surrogate posterior and (b) re-estimation of the HPD region using an MCMC sample from the updated surrogate density, which is inexpensive to obtain. GRIMA is applicable to approximation of general unnormalized posterior densities. To determine the range of tractable problem dimensions, we conduct simulation experiments on test densities with linear and nonlinear component-wise dependence, skewness, kurtosis and multimodality. Subsequently, we use GRIMA in a case study to calibrate a computationally intensive nonlinear regression model to real data from the Town Brook watershed. Supplemental materials for this article are available online.

Uncertainty Analysis for Computationally Expensive Models with Multiple Outputs.

Bayesian MCMC calibration and uncertainty analysis for computationally expensive models is implemented using the SOARS (Statistical and Optimization Analysis using Response Surfaces) methodology. SOARS uses a radial basis function interpolator as a surrogate, also known as an emulator or meta-model, for the logarithm of the posterior density. To prevent wasteful evaluations of the expensive model, the emulator is built only on a high posterior density region (HPDR), which is located by a global optimization algorithm. The set of points in the HPDR where the expensive model is evaluated is determined sequentially by the GRIMA algorithm described in detail in another paper but outlined here. Enhancements of the GRIMA algorithm were introduced to improve efficiency. A case study uses an eight-parameter SWAT2005 (Soil and Water Assessment Tool) model where daily stream flows and phosphorus concentrations are modeled for the Town Brook watershed which is part of the New York City water supply. A Supplemental Material file available online contains additional technical details and additional analysis of the Town Brook application.

Efficient Interpolation of Computationally Expensive Posterior Densities With Variable Parameter Costs.

Markov chain Monte Carlo (MCMC) is nowadays a standard approach to numerical computation of integrals of the posterior density π of the parameter vector η. Unfortunately, Bayesian inference using MCMC is computationally intractable when the posterior density π is expensive to evaluate. In many such problems, it is possible to identify a minimal subvector ß of η responsible for the expensive computation in the evaluation of π. We propose two approaches, DOSKA and INDA, that approximate π by interpolation in ways that exploit this computational structure to mitigate the curse of dimensionality. DOSKA interpolates π directly while INDA interpolates π indirectly by interpolating functions, for example, a regression function, upon which π depends. Our primary contribution is derivation of a Gaussian processes interpolant that provably improves over some of the existing approaches by reducing the effective dimension of the interpolation problem from dim(η) to dim(ß). This allows a dramatic reduction of the number of expensive evaluations necessary to construct an accurate approximation of π when dim(η) is high but dim(ß) is low. We illustrate the proposed approaches in a case study for a spatio-temporal linear model for air pollution data in the greater Boston area. Supplemental materials include proofs, details, and software implementation of the proposed procedures.

Optimal groundwater contamination monitoring using pumping wells.

This study presents a new method for selecting monitoring wells for optimal evaluation of groundwater quality. The basic approach of this work is motivated by difficulties in interpolating groundwater quality from information collected for only few sampled wells. The well selection relies on other existing data relevant to contaminant distribution in the sampling domain, e.g. predictions of models which rely on past measurements. The objective of this study is to develop a method of selecting the optimal wells, from which measurements could best serve some external model, e.g. a kriging system for characterizing the entire plume distribution, a flow-and-transport model for predicting a future distribution, or an inverse model for locating contaminant sources or estimating aquifer parameters. The decision variable at each sampling round determines the specific wells to be sampled. The study objective is accomplished through a spatially-continuous utility density function (UDF) which describes the utility of sampling at every point. The entire methodology which utilizes the UDF in conjunction with a sampling algorithm is entitled the UDF method. By applying calculations in steady and unsteady state sampling domains the effectiveness of the UDF method is demonstrated.

Implementing a method of screening one-well hydraulic barrier design alternatives.

This article provides details of applying the method developed by the authors (Rubin et al. 2008b) for screening one-well hydraulic barrier design alternatives. The present article with its supporting information (manual and electronic spreadsheets with a case history example) provides the reader complete details and examples of solving the set of nonlinear equations developed by Rubin et al. (2008b). It allows proper use of the analytical solutions and also depicting the various charts given by Rubin et al. (2008b). The final outputs of the calculations are the required position and the discharge of the pumping well. If the contaminant source is nonaqueous phase liquid (NAPL) entrapped within the aquifer, then the method provides an estimate of the aquifer remediation progress (which is a by-product) due to operating the hydraulic barrier.

Screening of one-well hydraulic barrier design alternatives.

Abstract This study develops a robust method for screening one-well hydraulic barrier design alternatives that can be easily computed without a numerical simulation model. The paper outlines the general method and shows its implementation with hydraulic barriers using a single pumping well. For such barriers, the method is easily computable with spreadsheets and/or charts depicted within the paper and posted online. The method applies the potential flow theory, which leads to using a curvilinear coordinate system for all types of calculations. For contaminant transport calculations, the method applies the boundary layer theory. For calculations of aquifer remediation, the method refers to bulk characteristics of the domain. As an example, the method has been applied to calculate the possible containment of a wide part of the coastal plain aquifer in Israel, which is contaminated by entrapped kerosene (a light nonaqueous phase liquid).

Evaluating Models for Spruce Budworm-Forest Management: Comparing Output with Regional Field Data.

The evaluation of models used in the management of populations can be complicated by the number of component parts and by the large temporal and spatial scales often required. This is particularly true of models developed for the analysis of management policy in forest pest situations. In this study, two large-scale spruce budworm-forest simulation models were evaluated by comparing their output with data collected annually by the Maine Forest Service survey at 1000 sites from 1975 to 1980. In practice, model evaluation typically involves a comparison of observations, independent of those used to construct the model, with overall model output. We did this, and in addition, separate tests were performed on major components of each large-scale budworm model. These components represent Maine's forest protection policy, the budworm-forest dynamics, and pest control efficacy. Both models produced output that was in some way inconsistent with the Maine survey data. Inconsistencies were most prevalent at low budworm densities, especially after pesticide spraying, when model output predicts budworm populations increase more slowly than the survey data suggest. These inconsistencies pointed to inaccuracies in the models' representation of Maine's forest protection policy, of budworm population growth at low densities, and of the effectiveness of spraying (especially at low budworm densities). Problems translating the results of studies of nonlinear population dynamics from small experimental plots to the larger spatial scale used in the models are implicated. Our results suggest that the optimal threshold density of budworm for insecticide application is probably higher than the upper threshold of 20 egg masses/m2 inferred from the models.