Applications of nature-inspired metaheuristic algorithms for tackling optimization problems across disciplines.

Nature-inspired metaheuristic algorithms are important components of artificial intelligence, and are increasingly used across disciplines to tackle various types of challenging optimization problems. This paper demonstrates the usefulness of such algorithms for solving a variety of challenging optimization problems in statistics using a nature-inspired metaheuristic algorithm called competitive swarm optimizer with mutated agents (CSO-MA). This algorithm was proposed by one of the authors and its superior performance relative to many of its competitors had been demonstrated in earlier work and again in this paper. The main goal of this paper is to show a typical nature-inspired metaheuristic algorithmi, like CSO-MA, is efficient for tackling many different types of optimization problems in statistics. Our applications are new and include finding maximum likelihood estimates of parameters in a single cell generalized trend model to study pseudotime in bioinformatics, estimating parameters in the commonly used Rasch model in education research, finding M-estimates for a Cox regression in a Markov renewal model, performing matrix completion tasks to impute missing data for a two compartment model, and selecting variables optimally in an ecology problem in China. To further demonstrate the flexibility of metaheuristics, we also find an optimal design for a car refueling experiment in the auto industry using a logistic model with multiple interacting factors. In addition, we show that metaheuristics can sometimes outperform optimization algorithms commonly used in statistics.

Mathematical programming tools for randomization purposes in small two-arm clinical trials: A case study with real data.

Modern randomization methods in clinical trials are invariably adaptive, meaning that the assignment of the next subject to a treatment group uses the accumulated information in the trial. Some of the recent adaptive randomization methods use mathematical programming to construct attractive clinical trials that balance the group features, such as their sizes and covariate distributions of their subjects. We review some of these methods and compare their performance with common covariate-adaptive randomization methods for small clinical trials. We introduce an energy distance measure that compares the discrepancy between the two groups using the joint distribution of the subjects' covariates. This metric is more appealing than evaluating the discrepancy between the groups using their marginal covariate distributions. Using numerical experiments, we demonstrate the advantages of the mathematical programming methods under the new measure. In the supplementary material, we provide R codes to reproduce our study results and facilitate comparisons of different randomization procedures.

Optimizing the design of a pharmacokinetic trial to evaluate the dosing scheme of a novel tuberculosis drug in children living with or without HIV.

Pharmacokinetic (PK) studies in children are usually small and have ethical constraints due to the medical complexities of drawing blood in this special population. Often, population PK models for the drug(s) of interest are available in adults, and these models can be extended to incorporate the expected deviations seen in children. As a consequence, there is increasing interest in the use of optimal design methodology to design PK sampling schemes in children that maximize information using a small sample size and limited number of sampling times per dosing period. As a case study, we use the novel tuberculosis drug delamanid, and show how applications of optimal design methodology can result in highly efficient and model-robust designs in children for estimating PK parameters using a limited number of sampling measurements. Using developed population PK models based on available data from adults living with and without HIV, and limited data on children without HIV, competing designs for children living with HIV were derived and assessed based on robustness to model uncertainty.

An interactive tool for designing efficient toxicology experiments.

The design of dose-response experiments is an important part of toxicology research. Efficient design of these experiments requires choosing optimal doses and assigning the correct number of subjects to those doses under a given criterion. Optimal design theory provides the tools to find the most efficient experimental designs in terms of cost and statistical efficiency. However, the mathematical details can be distracting and make these designs inaccessible to many toxicologists. To facilitate use of these designs, we present an easy to use web-app for finding two types of optimal designs for models commonly used in toxicology. We include tools for checking the optimality of a given design and for assessing efficiency of any user-supplied design. Using state-of-the-art nature-inspired metaheuristic algorithms, the web-app allows the user to quickly find optimal designs for estimating model parameters or the benchmark dose.

An Overview of Adaptive Designs and Some of Their Challenges, Benefits, and Innovative Applications.

Adaptive designs are increasingly developed and used to improve all phases of clinical trials and in biomedical studies in various ways to address different statistical issues. We first present an overview of adaptive designs and note their numerous advantages over traditional clinical trials. In particular, we provide a concrete demonstration that shows how recent adaptive design strategies can further improve an adaptive trial implemented 13 years ago. Despite their usefulness, adaptive designs are still not widely implemented in clinical trials. We offer a few possible reasons and propose some ways to use them more broadly in practice, which include greater availability of software tools and interactive websites to generate optimal adaptive trials freely and effectively, including the use of metaheuristics to facilitate the search for an efficient trial design. To this end, we present several web-based tools for finding various adaptive and nonadaptive optimal designs and discuss nature-inspired metaheuristics. Metaheuristics are assumptions-free general purpose optimization algorithms widely used in computer science and engineering to tackle all kinds of challenging optimization problems, and their use in designing clinical trials is just emerging. We describe a few recent such applications and some of their capabilities for designing various complex trials. Particle swarm optimization is an exemplary nature-inspired algorithm, and similar to others, it has a simple definition but many moving parts, making it hard to study its properties analytically. We investigated one of its hitherto unstudied issues on how to bring back out-of-range candidates during the search for the optimum of the search domain and show that different strategies can impact the success and time of the search. We conclude with a few caveats on the use of metaheuristics for a successful search.

Optimal Designs for Nonlinear Mixed-effects Models Using Competitive Swarm Optimizer with Mutated Agents.

Nature-inspired meta-heuristic algorithms are increasingly used in many disciplines to tackle challenging optimization problems. Our focus is to apply a newly proposed nature-inspired meta-heuristics algorithm called CSO-MA to solve challenging design problems in biosciences and demonstrate its flexibility to find various types of optimal approximate or exact designs for nonlinear mixed models with one or several interacting factors and with or without random effects. We show that CSO-MA is efficient and can frequently outperform other algorithms either in terms of speed or accuracy. The algorithm, like other meta-heuristic algorithms, is free of technical assumptions and flexible in that it can incorporate cost structure or multiple user-specified constraints, such as, a fixed number of measurements per subject in a longitudinal study. When possible, we confirm some of the CSO-MA generated designs are optimal with theory by developing theory-based innovative plots. Our applications include searching optimal designs to estimate (i) parameters in mixed nonlinear models with correlated random effects, (ii) a function of parameters for a count model in a dose combination study, and (iii) parameters in a HIV dynamic model. In each case, we show the advantages of using a meta-heuristic approach to solve the optimization problem, and the added benefits of the generated designs.

The Future Glioblastoma Clinical Trials Landscape: Early Phase 0, Window of Opportunity, and Adaptive Phase I-III Studies.

PURPOSE OF REVIEW: Innovative clinical trial designs for glioblastoma (GBM) are needed to expedite drug discovery. Phase 0, window of opportunity, and adaptive designs have been proposed, but their advanced methodologies and underlying biostatistics are not widely known. This review summarizes phase 0, window of opportunity, and adaptive phase I-III clinical trial designs in GBM tailored to physicians. RECENT FINDINGS: Phase 0, window of opportunity, and adaptive trials are now being implemented for GBM. These trials can remove ineffective therapies earlier during drug development and improve trial efficiency. There are two ongoing adaptive platform trials: GBM Adaptive Global Innovative Learning Environment (GBM AGILE) and the INdividualized Screening trial of Innovative GBM Therapy (INSIGhT). The future clinical trials landscape in GBM will increasingly involve phase 0, window of opportunity, and adaptive phase I-III studies. Continued collaboration between physicians and biostatisticians will be critical for implementing these trial designs.

Statistical tests for homogeneity of variance for clinical trials and recommendations.

In most clinical trials, the main interest is to test whether there are differences in the mean outcomes among the treatment groups. When the outcome is continuous, a common statistical test is a usual t-test for a two-group comparison. For more than 2 groups, an ANOVA setup is used and the test for equality for all groups is based on the F-distribution. A key assumption for these parametric tests is that data are normally, independently distributed and the response variances are equal. The robustness of these tests to the first two assumptions is quite well investigated, but the issues arising from heteroscedasticity are less studied. This paper reviews different methods for ascertaining homogeneity of variance across groups and investigates the consequences of heteroscedasticity on the tests. Simulations based on normal, heavy-tailed, and skewed normal data demonstrate that some of the less known methods, such as the Jackknife or Cochran's test, are quite effective in detecting differences in the variances.

Appropriate noise addition to metaheuristic algorithms can enhance their performance.

Nature-inspired swarm-based algorithms are increasingly applied to tackle high-dimensional and complex optimization problems across disciplines. They are general purpose optimization algorithms, easy to implement and assumption-free. Some common drawbacks of these algorithms are their premature convergence and the solution found may not be a global optimum. We propose a general, simple and effective strategy, called heterogeneous Perturbation-Projection (HPP), to enhance an algorithm's exploration capability so that our sufficient convergence conditions are guaranteed to hold and the algorithm converges almost surely to a global optimum. In summary, HPP applies stochastic perturbation on half of the swarm agents and then project all agents onto the set of feasible solutions. We illustrate this approach using three widely used nature-inspired swarm-based optimization algorithms: particle swarm optimization (PSO), bat algorithm (BAT) and Ant Colony Optimization for continuous domains (ACO). Extensive numerical experiments show that the three algorithms with the HPP strategy outperform the original versions with 60-80% the times with significant margins.

Multi-scale, domain knowledge-guided attention + random forest: a two-stage deep learning-based multi-scale guided attention models to diagnose idiopathic pulmonary fibrosis from computed tomography images.

BACKGROUND: Idiopathic pulmonary fibrosis (IPF) is a progressive, irreversible, and usually fatal lung disease of unknown reasons, generally affecting the elderly population. Early diagnosis of IPF is crucial for triaging patients' treatment planning into anti-fibrotic treatment or treatments for other causes of pulmonary fibrosis. However, current IPF diagnosis workflow is complicated and time-consuming, which involves collaborative efforts from radiologists, pathologists, and clinicians and it is largely subject to inter-observer variability. PURPOSE: The purpose of this work is to develop a deep learning-based automated system that can diagnose subjects with IPF among subjects with interstitial lung disease (ILD) using an axial chest computed tomography (CT) scan. This work can potentially enable timely diagnosis decisions and reduce inter-observer variability. METHODS: Our dataset contains CT scans from 349 IPF patients and 529 non-IPF ILD patients. We used 80% of the dataset for training and validation purposes and 20% as the holdout test set. We proposed a two-stage model: at stage one, we built a multi-scale, domain knowledge-guided attention model (MSGA) that encouraged the model to focus on specific areas of interest to enhance model explainability, including both high- and medium-resolution attentions; at stage two, we collected the output from MSGA and constructed a random forest (RF) classifier for patient-level diagnosis, to further boost model accuracy. RF classifier is utilized as a final decision stage since it is interpretable, computationally fast, and can handle correlated variables. Model utility was examined by (1) accuracy, represented by the area under the receiver operating characteristic curve (AUC) with standard deviation (SD), and (2) explainability, illustrated by the visual examination of the estimated attention maps which showed the important areas for model diagnostics. RESULTS: During the training and validation stage, we observe that when we provide no guidance from domain knowledge, the IPF diagnosis model reaches acceptable performance (AUC±SD = 0.93±0.07), but lacks explainability; when including only guided high- or medium-resolution attention, the learned attention maps are not satisfactory; when including both high- and medium-resolution attention, under certain hyperparameter settings, the model reaches the highest AUC among all experiments (AUC±SD = 0.99±0.01) and the estimated attention maps concentrate on the regions of interests for this task. Three best-performing hyperparameter selections according to MSGA were applied to the holdout test set and reached comparable model performance to that of the validation set. CONCLUSIONS: Our results suggest that, for a task with only scan-level labels available, MSGA+RF can utilize the population-level domain knowledge to guide the training of the network, which increases both model accuracy and explainability.

Optimal designs for health risk assessments using fractional polynomial models.

Fractional polynomials (FP) have been shown to be more flexible than polynomial models for fitting data from an univariate regression model with a continuous outcome but design issues for FP models have lagged. We focus on FPs with a single variable and construct D-optimal designs for estimating model parameters and I-optimal designs for prediction over a user-specified region of the design space. Some analytic results are given, along with a discussion on model uncertainty. In addition, we provide an applet to facilitate users find tailor made optimal designs for their problems. As applications, we construct optimal designs for three studies that used FPs to model risk assessments of (a) testosterone levels from magnesium accumulation in certain areas of the brains in songbirds, (b) rats subject to exposure of different chemicals, and (c) hormetic effects due to small toxic exposure. In each case, we elaborate the benefits of having an optimal design in terms of cost and quality of the statistical inference.

Single-cell generalized trend model (scGTM): a flexible and interpretable model of gene expression trend along cell pseudotime.

MOTIVATION: Modeling single-cell gene expression trends along cell pseudotime is a crucial analysis for exploring biological processes. Most existing methods rely on nonparametric regression models for their flexibility; however, nonparametric models often provide trends too complex to interpret. Other existing methods use interpretable but restrictive models. Since model interpretability and flexibility are both indispensable for understanding biological processes, the single-cell field needs a model that improves the interpretability and largely maintains the flexibility of nonparametric regression models. RESULTS: Here, we propose the single-cell generalized trend model (scGTM) for capturing a gene's expression trend, which may be monotone, hill-shaped or valley-shaped, along cell pseudotime. The scGTM has three advantages: (i) it can capture non-monotonic trends that are easy to interpret, (ii) its parameters are biologically interpretable and trend informative, and (iii) it can flexibly accommodate common distributions for modeling gene expression counts. To tackle the complex optimization problems, we use the particle swarm optimization algorithm to find the constrained maximum likelihood estimates for the scGTM parameters. As an application, we analyze several single-cell gene expression datasets using the scGTM and show that scGTM can capture interpretable gene expression trends along cell pseudotime and reveal molecular insights underlying biological processes. AVAILABILITY AND IMPLEMENTATION: The Python package scGTM is open-access and available at https://github.com/ElvisCuiHan/scGTM. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Orthogonal array composite designs for drug combination experiments with applications for tuberculosis.

The aim of this article is to provide an overview of the orthogonal array composite design (OACD) methodology, illustrate the various advantages, and provide a real-world application. An OACD combines a two-level factorial design with a three-level orthogonal array and it can be used as an alternative to existing composite designs for building response surface models. We compare the D$$ D $$ -efficiencies of OACDs relative to the commonly used central composite design (CCD) when there are a few missing observations and demonstrate that OACDs are more robust to missing observations for two scenarios. The first scenario assumes one missing observation either from one factorial point or one additional point. The second scenario assumes two missing observations either from two factorial points or from two additional points, or from one factorial point and one additional point. Furthermore, we compare OACDs and CCDs in terms of I$$ I $$ -optimality for precise predictions. Lastly, a real-world application of an OACD for a tuberculosis drug combination study is provided.

A model-based approach to designing developmental toxicology experiments using sea urchin embryos.

The key aim of this paper is to suggest a more quantitative approach to designing a dose-response experiment, and more specifically, a concentration-response experiment. The work proposes a departure from the traditional experimental design to determine a dose-response relationship in a developmental toxicology study. It is proposed that a model-based approach to determine a dose-response relationship can provide the most accurate statistical inference for the underlying parameters of interest, which may be estimating one or more model parameters or pre-specified functions of the model parameters, such as lethal dose, at maximal efficiency. When the design criterion or criteria can be determined at the onset, there are demonstrated efficiency gains using a more carefully selected model-based optimal design as opposed to an ad-hoc empirical design. As an illustration, a model-based approach was theoretically used to construct efficient designs for inference in a developmental toxicity study of sea urchin embryos exposed to trimethoprim. This study compares and contrasts the results obtained using model-based optimal designs versus an ad-hoc empirical design.

Spatial Two-stage Designs for Phase II Clinical Trials.

A common endpoint in a single-arm phase II study is tumor response as a binary variable. Two widely used designs for such a study are Simon's two-stage minimax and optimal designs. The minimax design minimizes the maximal sample size and the optimal design minimizes the expected sample size under the null hypothesis. The optimal design generally has the larger total sample size than the minimax design, but its first stage's sample size is smaller than that of the minimax design. The difference in the total sample size between two types of designs can be large and so both designs can be unappealing to investigators. We develop novel designs that compromise on the two optimality criteria and avoid such occurrences using the spatial information on the first stage's required sample size and the total required sample size. We study properties of these spatial designs and show our proposed designs have advantages over Simon's designs and one of its extensions by Lin and Shih. As applications, we construct spatial designs for real-life studies on patients with Hodgkin disease and another study on effect of head and neck cancer on apnea.

Early palliative care: the surprise question and the palliative care screening tool-better together.

OBJECTIVE: The 'surprise question' (SQ) and the palliative care screening tool (PCST) are the common assessment tools in the early identification of patients requiring palliative care. However, the comparison of their prognostic accuracies has not been extensively studied. This study aimed to compare the prognostic accuracy of SQ and PCST in terms of recognising patients nearing end of life (EOL) and those appropriate for palliative care. METHODS: This prospective study used both the SQ and PCST to predict patients' 12-month mortality and identified those appropriate for palliative care. All adult patients admitted to Taipei City Hospital in 2015 were included in this cohort study. The c-statistic value was calculated to indicate the predictive accuracies of the SQ and PCST. RESULTS: Out of 21 109 patients, with a mean age of 62.8 years, 12.4% and 11.1% had a SQ response of 'no' and a PCST score of ≥4, respectively. After controlling for other covariates, an SQ response of 'no' and a PCST score of ≥4 were the independent predictors of 12-month mortality. The c-statistic values of the SQ and PCST at recognising patients in their last year of life were 0.680 and 0.689, respectively. When using a combination of both SQ and PCST in predicting patients' 12-month mortality risk, the predictive value of the c-statistic increased to 0.739 and was significantly higher than either one in isolation (p<0.001). CONCLUSION: A combination of the SQ with PCST has better prognostic accuracy than either one in isolation.

ICAOD: An R Package for Finding Optimal designs for Nonlinear Statistical Models by Imperialist Competitive Algorithm.

Optimal design ideas are increasingly used in different disciplines to rein in experimental costs. Given a nonlinear statistical model and a design criterion, optimal designs determine the number of experimental points to observe the responses, the design points and the number of replications at each design point. Currently, there are very few free and effective computing tools for finding different types of optimal designs for a general nonlinear model, especially when the criterion is not differentiable. We introduce an R package ICAOD to find various types of optimal designs and they include locally, minimax and Bayesian optimal designs for different nonlinear statistical models. Our main computational tool is a novel metaheuristic algorithm called imperialist competitive algorithm (ICA) and inspired by socio-political behavior of humans and colonialism. We demonstrate its capability and effectiveness using several applications. The package also includes several theory-based tools to assess optimality of a generated design when the criterion is a convex function of the design.

G-optimal designs for hierarchical linear models: an equivalence theorem and a nature-inspired meta-heuristic algorithm.

Hierarchical linear models are widely used in many research disciplines and estimation issues for such models are generally well addressed. Design issues are relatively much less discussed for hierarchical linear models but there is an increasing interest as these models grow in popularity. This paper discusses the G-optimality for predicting individual parameters in such models and establishes an equivalence theorem for confirming the G-optimality of an approximate design. Because the criterion is non-differentiable and requires solving multiple nested optimization problems, it is much harder to find and study G-optimal designs analytically. We propose a nature-inspired meta-heuristic algorithm called competitive swarm optimizer (CSO) to generate G-optimal designs for linear mixed models with different means and covariance structures. We further demonstrate that CSO is flexible and generally effective for finding the widely used locally D-optimal designs for nonlinear models with multiple interacting factors and some of the random effects are correlated. Our numerical results for a few examples suggest that G and D-optimal designs may be equivalent and we establish that D and G-optimal designs for hierarchical linear models are equivalent when the models have only a random intercept only. The challenging mathematical question of whether their equivalence applies more generally to other hierarchical models remains elusive.

Metaheuristics for pharmacometrics.

Metaheuristics is a powerful optimization tool that is increasingly used across disciplines to tackle general purpose optimization problems. Nature-inspired metaheuristic algorithms is a subclass of metaheuristic algorithms and have been shown to be particularly flexible and useful in solving complicated optimization problems in computer science and engineering. A common practice with metaheuristics is to hybridize it with another suitably chosen algorithm for enhanced performance. This paper reviews metaheuristic algorithms and demonstrates some of its utility in tackling pharmacometric problems. Specifically, we provide three applications using one of its most celebrated members, particle swarm optimization (PSO), and show that PSO can effectively estimate parameters in complicated nonlinear mixed-effects models and to gain insights into statistical identifiability issues in a complex compartment model. In the third application, we demonstrate how to hybridize PSO with sparse grid, which is an often-used technique to evaluate high dimensional integrals, to search for D -efficient designs for estimating parameters in nonlinear mixed-effects models with a count outcome. We also show the proposed hybrid algorithm outperforms its competitors when sparse grid is replaced by its competitor, adaptive gaussian quadrature to approximate the integral, or when PSO is replaced by three notable nature-inspired metaheuristic algorithms.

Constructing robust and efficient experimental designs in groundwater modeling using a Galerkin method, proper orthogonal decomposition, and metaheuristic algorithms.

Estimating parameters accurately in groundwater models for aquifers is challenging because the models are non-explicit solutions of complex partial differential equations. Modern research methods, such as Monte Carlo methods and metaheuristic algorithms, for searching an efficient design to estimate model parameters require hundreds, if not thousands of model calls, making the computational cost prohibitive. One method to circumvent the problem and gain valuable insight on the behavior of groundwater is to first apply a Galerkin method and convert the system of partial differential equations governing the flow to a discrete problem and then use a Proper Orthogonal Decomposition to project the high-dimensional model space of the original groundwater model to create a reduced groundwater model with much lower dimensions. The reduced model can be solved several orders of magnitude faster than the full model and able to provide an accurate estimate of the full model. The task is still challenging because the optimization problem is non-convex, non-differentiable and there are continuous variables and integer-valued variables to optimize. Following convention, heuristic algorithms and a combination is used search to find efficient designs for the reduced groundwater model using various optimality criteria. The main goals are to introduce new design criteria and the concept of design efficiency for experimental design research in hydrology. The two criteria have good utility but interestingly, do not seem to have been implemented in hydrology. In addition, design efficiency is introduced. Design efficiency is a method to assess how robust a design is under a change of criteria. The latter is an important issue because the design criterion may be subjectively selected and it is well known that an optimal design can perform poorly under another criterion. It is thus desirable that the implemented design has relatively high efficiencies under a few criteria. As applications, two heuristic algorithms are used to find optimal designs for a small synthetic aquifer design problem and a design problem for a large-scale groundwater model and assess their robustness properties to other optimality criteria. The results show the proof of concept is workable for finding a more informed and efficient model-based design for a water resource study.