Testing strategy in phase 3 trials with multiple doses.

In this paper, we consider multiplicity testing approaches mainly for phase 3 trials with two doses. We review a few available approaches and propose some new ones. The doses selected for phase 3 usually have the same or a similar efficacy profile, so they have some degree of consistency in efficacy. We review the Hochberg procedure, the Bonferroni procedure, and a few consistency-adjusted procedures, and suggest new ones by applying the available procedures to the pooled dose and the high dose, the dose that is thought to be more efficacious between two doses. The reason behind the idea is that the pooled dose and the high dose are more consistent than the original two doses if the high dose is more efficacious than the low dose. We compare all approaches via simulations and recommend using a procedure combining 4A and the pooling approach. We also discuss briefly the testing strategy for trials with more than two doses.

Changes are still needed on multiple co-primary endpoints.

The Food and Drug Administration in the United States issued a much-awaited draft guidance on 'Multiple Endpoints in Clinical Trials' in January 2017. The draft guidance is well written and contains consistent message on the technical implementation of the principles laid out in the guidance. In this commentary, we raise a question on applying the principles to studies designed from a safety perspective. We then direct our attention to issues related to multiple co-primary endpoints. In a paper published in the Drug Information Journal in 2007, Offen et al. give examples of disorders where multiple co-primary endpoints are required by regulators. The standard test for multiple co-primary endpoints is the min test which tests each endpoint individually, at the one-sided 2.5% level, for a confirmatory trial. This approach leads to a substantial loss of power when the number of co-primary endpoints exceeds 2, a fact acknowledged in the draft guidance. We review approaches that have been proposed to tackle the problem of power loss and propose a new one. Using recommendations by Chen et al. for the assessment of drugs for vulvar and vaginal atrophy published in the Drug Information Journal in 2010, we argue the need for more changes and urge a path forward that uses different levels of claims to reflect the effectiveness of a product on multiple endpoints that are equally important and mostly unrelated. Copyright © 2017 John Wiley & Sons, Ltd.

Testing each hypothesis marginally at alpha while still controlling FWER: how and when.

This paper proposes a multiple testing procedure that allows one to reject each individual hypothesis at a prespecified level α, while still controlling the familywise error rate at α in the strong sense. Typically, rejecting a hypothesis when its marginal p-value is ≤ α in a multiple hypothesis testing setting will lead to an inflation of familywise error rate. However, this inflation can be avoided if a particular consistency criterion is prespecified and incorporated in the testing algorithm. The criterion is equivalent to requiring that all p-values be smaller than or equal to a particular threshold in the one-sided hypothesis testing setting. Extensions to the two-sided hypothesis testing setting and extensions to situations where the criterion can be chosen per user's preference are also presented.

Gatekeeping testing via adaptive alpha allocation.

In a typical clinical trial, there are one or two primary endpoints, and a few secondary endpoints. When at least one primary endpoint achieves statistical significance, there is considerable interest in using results for the secondary endpoints to enhance characterization of the treatment effect. Because multiple endpoints are involved, regulators may require that the familywise type I error rate be controlled at a pre-set level. This requirement can be achieved by using "gatekeeping" methods. However, existing methods suffer from logical oddities such as allowing results for secondary endpoint(s) to impact the likelihood of success for the primary endpoint(s). We propose a novel and easy-to-implement gatekeeping procedure that is devoid of such deficiencies. A real data example and simulation results are used to illustrate efficiency gains of our method relative to existing methods.