Optimizing Graphical Procedures for Multiplicity Control in a Confirmatory Clinical Trial via Deep Learning.

In confirmatory clinical trials, it has been proposed to use a simple iterative graphical approach to construct and perform intersection hypotheses tests with a weighted Bonferroni-type procedure to control Type I errors in the strong sense. Given Phase II study results or other prior knowledge, it is usually of main interest to find the optimal graph that maximizes a certain objective function in a future Phase III study. In this article, we evaluate the performance of two existing derivative-free constrained methods, and further propose a deep learning enhanced optimization framework. Our method numerically approximates the objective function via feedforward neural networks (FNNs) and then performs optimization with available gradient information. It can be constrained so that some features of the testing procedure are held fixed while optimizing over other features. Simulation studies show that our FNN-based approach has a better balance between robustness and time efficiency than some existing derivative-free constrained optimization algorithms. Compared to the traditional stochastic search method, our optimizer has moderate multiplicity adjusted power gain when the number of hypotheses is relatively large. We further apply it to a case study to illustrate how to optimize a multiple testing procedure with respect to a specific study objective.

Applications of general multistage gatekeeping and graphical multiple testing strategies in a clinical trial setting.

Multiplicity adjustment plays a critical role for testing multiple endpoints and/or multiple doses in clinical trials. Under the clinical trial setting, multiple hypotheses are usually grouped into primary and secondary families and hierarchically ordered between and within families. The determination of the order and grouping of the hypotheses depends on the objectives of the trial. In such scenarios, strong control of the family-wise error rate (FWER) can be achieved via either gatekeeping procedures or the graphical approach to sequential testing procedures. The aforementioned two types of procedures are related but are not completely overlapping. Both approaches are assessed in this manuscript, with a focus on the comparison and relationship between the two. In addition, the performance of various constructions of gatekeeping and graphical multiple comparison procedures (MCPs) under a typical clinical trial setting with multiple doses and multiple endpoints is studied.

A Bayesian credible subgroups approach to identifying patient subgroups with positive treatment effects.

Many new experimental treatments benefit only a subset of the population. Identifying the baseline covariate profiles of patients who benefit from such a treatment, rather than determining whether or not the treatment has a population-level effect, can substantially lessen the risk in undertaking a clinical trial and expose fewer patients to treatments that do not benefit them. The standard analyses for identifying patient subgroups that benefit from an experimental treatment either do not account for multiplicity, or focus on testing for the presence of treatment-covariate interactions rather than the resulting individualized treatment effects. We propose a Bayesian credible subgroups method to identify two bounding subgroups for the benefiting subgroup: one for which it is likely that all members simultaneously have a treatment effect exceeding a specified threshold, and another for which it is likely that no members do. We examine frequentist properties of the credible subgroups method via simulations and illustrate the approach using data from an Alzheimer's disease treatment trial. We conclude with a discussion of the advantages and limitations of this approach to identifying patients for whom the treatment is beneficial.

Overview of multiple testing methodology and recent development in clinical trials.

Multiplicity control is an important statistical issue in clinical trials where strong control of the type I error rate is required. Many multiple testing methods have been proposed and applied to address multiplicity issues in clinical trials. This paper provides an application oriented and comprehensive overview of commonly used multiple testing procedures and recent developments in statistical methodology in multiple testing in clinical trials. Commonly used multiple testing procedures are applied to test non-hierarchical hypotheses and gatekeeping procedures can be used to test hierarchically ordered hypotheses while controlling the overall type I error rate. The recently developed graphical approach has the flexibility to integrate hierarchical and non-hierarchical procedures into one framework. A graphical multiple testing procedure with "no-dead-end" provides an opportunity to fully recycle α across hypothesis families. Two hypothetical clinical trial examples are used to illustrate applications of these procedures. The advantages and disadvantages of the different procedures are briefly discussed.

A new classification approach for comparing two active treatments when there is no prior projection on which one is better.

We developed a new classification approach in this paper to compare two active treatments. This approach is especially useful when there is no prior judgment on which treatment is better and the traditional hypothesis testing approach is thus not applicable. Our method classifies all the possible outcomes into categories and draws conclusions on the difference in the outcome measurement between two treatment arms according to the location of the confidence interval for the treatment difference in the response variable. This method controls the misclassification rate regardless of the true difference in the response between the two treatment arms. The method was applied to a diabetes clinical trial.

Challenge of multiple co-primary endpoints: a new approach.

There are many disorders where regulatory agencies have required a new treatment to demonstrate efficacy on multiple co-primary endpoints, all significant at the one-sided 2.5 per cent level, before accepting the treatment's effect for the disorder. This requirement, rooted in the intersection-union (IU) test, has led many researchers to increase the study sample size to make up for the reduction in the statistical power at the study level. Unfortunately, the increase in sample size could be substantial when the endpoints are minimally correlated and the treatment effects on the multiple endpoints are comparable. In this paper, we demonstrate that the frequentist concept of controlling the maximum false positive rate, even when applied to a restricted null space, has only limited success in keeping the sample size increase at a reasonable level. We therefore propose an approach that is based on the notion of controlling an average type I error rate. By employing an upper bound for the average type I error rate, the new approach provides an adjustment to the significance level that depends only on the correlation among the endpoints. For the most common case of two or three co-primary endpoints, the adjusted significance level is at most 5 per cent (one-sided) when the endpoints are moderately correlated. We show how sample size could be calculated under the proposed approach and contrast the needed sample size with that required under the IU test. We provide additional comments and discuss why the new approach is consistent with the principle requiring evidence of significance in the drug development and approval process.

Gatekeeping strategies for clinical trials that do not require all primary effects to be significant.

In this paper we describe methods for addressing multiplicity issues arising in the analysis of clinical trials with multiple endpoints and/or multiple dose levels. Efficient 'gatekeeping strategies' for multiplicity problems of this kind are developed. One family of hypotheses (comprising the primary objectives) is treated as a 'gatekeeper', and the other family or families (comprising secondary and tertiary objectives) are tested only if one or more gatekeeper hypotheses have been rejected. We discuss methods for constructing gatekeeping testing procedures using weighted Bonferroni tests, weighted Simes tests, and weighted resampling-based tests, all within the closed testing framework. The new strategies are illustrated using an example from a clinical trial with co-primary endpoints, and using an example from a dose-finding study with multiple endpoints. Power comparisons with competing methods show the gatekeeping methods are more powerful when the primary objective of the trial must be met.