Twenty-five years of confirmatory adaptive designs: opportunities and pitfalls.

'Multistage testing with adaptive designs' was the title of an article by Peter Bauer that appeared 1989 in the German journal Biometrie und Informatik in Medizin und Biologie. The journal does not exist anymore but the methodology found widespread interest in the scientific community over the past 25 years. The use of such multistage adaptive designs raised many controversial discussions from the beginning on, especially after the publication by Bauer and Köhne 1994 in Biometrics: Broad enthusiasm about potential applications of such designs faced critical positions regarding their statistical efficiency. Despite, or possibly because of, this controversy, the methodology and its areas of applications grew steadily over the years, with significant contributions from statisticians working in academia, industry and agencies around the world. In the meantime, such type of adaptive designs have become the subject of two major regulatory guidance documents in the US and Europe and the field is still evolving. Developments are particularly noteworthy in the most important applications of adaptive designs, including sample size reassessment, treatment selection procedures, and population enrichment designs. In this article, we summarize the developments over the past 25 years from different perspectives. We provide a historical overview of the early days, review the key methodological concepts and summarize regulatory and industry perspectives on such designs. Then, we illustrate the application of adaptive designs with three case studies, including unblinded sample size reassessment, adaptive treatment selection, and adaptive endpoint selection. We also discuss the availability of software for evaluating and performing such designs. We conclude with a critical review of how expectations from the beginning were fulfilled, and - if not - discuss potential reasons why this did not happen.

Continuous event monitoring via a Bayesian predictive approach.

In clinical trials, continuous monitoring of event incidence rate plays a critical role in making timely decisions affecting trial outcome. For example, continuous monitoring of adverse events protects the safety of trial participants, while continuous monitoring of efficacy events helps identify early signals of efficacy or futility. Because the endpoint of interest is often the event incidence associated with a given length of treatment duration (e.g., incidence proportion of an adverse event with 2 years of dosing), assessing the event proportion before reaching the intended treatment duration becomes challenging, especially when the event onset profile evolves over time with accumulated exposure. In particular, in the earlier part of the study, ignoring censored subjects may result in significant bias in estimating the cumulative event incidence rate. Such a problem is addressed using a predictive approach in the Bayesian framework. In the proposed approach, experts' prior knowledge about both the frequency and timing of the event occurrence is combined with observed data. More specifically, during any interim look, each event-free subject will be counted with a probability that is derived using prior knowledge. The proposed approach is particularly useful in early stage studies for signal detection based on limited information. But it can also be used as a tool for safety monitoring (e.g., data monitoring committee) during later stage trials. Application of the approach is illustrated using a case study where the incidence rate of an adverse event is continuously monitored during an Alzheimer's disease clinical trial. The performance of the proposed approach is also assessed and compared with other Bayesian and frequentist methods via simulation.

Adaptive Clinical Trials: Overview of Phase III Designs and Challenges.

Adaptive designs use accruing data to make changes in an ongoing trial according to a prespecified plan and potentially offer great efficiencies for clinical development. There are many types of adaptive designs and many trial aspects that could in theory be adapted. However, the scope of adaptive designs with relevance in confirmatory trials is narrower, and in addition, extensive pre-planning is needed and various types of challenges need to be addressed in order to use these designs in this stage of development. Nevertheless, with careful planning, there are opportunities for these designs to offer important benefits even in the confirmatory stage of development. We provide an overview of adaptive designs that have relevance for confirmatory trials and discuss considerations that may affect whether they should or should not be used in particular trials or programs as well as the challenges that need to be addressed.

Adaptive Design Applied to Identification of the Minimum Effective Dose in Schizophrenia: Simulations of Scientific and Commercial Value.

The International Society for CNS Clinical Trials and Methodology (ISCTM) Adaptive Design Working Group (IADWG) designed a case study simulation exercise to compare the value of traditional versus adaptive design approaches to phase II clinical trial design in schizophrenia in statistical and economic terms. Operational characteristics of both designs were compared across 7 likely dose-response curves. Based on IADWG members' recent research experience in schizophrenia, estimates of expected net present value (eNPV) for the molecule were compared for the traditional and adaptive designs. Across dose-response curve scenarios with a minimum effective dose (MED), the adaptive design was more likely to show proof of concept and correctly identify the MED than was the traditional design. Even with a conservative weighting of possible dose-response curves, using an adaptive design in phase II resulted in higher eNPV. This simulation supports the statistical and economic value for decision makers exploring the use of adaptive approaches to phase II research in schizophrenia.

Confidence intervals for confirmatory adaptive two-stage designs with treatment selection.

The construction of adequate confidence intervals for adaptive two-stage designs remains an area of ongoing research. We propose a conditional likelihood-based approach to construct a Wald confidence interval and two confidence intervals based on inverting the likelihood ratio test, one of them using first-order inference methods and the second one using higher order inference methods. The coverage probabilities of these confidence intervals, and also the average bias and mean square error of the corresponding point estimates, compare favorably with other available techniques. A small simulation study is used to evaluate the performance of the new methods. We investigate other extensions of practical interest for normal endpoints and illustrate them using real data, including the selection of more than one treatment for the second stage, selection rules based on both efficacy and safety endpoints, and the inclusion of a control/placebo arm. The new method also allows adjustment for covariates, and has been extended to deal with binomial data and other distributions from the exponential family. Although conceptually simple, the new methods have a much wider scope than the methods currently available.

Views on Emerging Issues Pertaining to Data Monitoring Committees for Adaptive Trials.

In this paper, the authors express their views on a range of topics related to data monitoring committees (DMCs) for adaptive trials that have emerged recently. The topics pertain to DMC roles and responsibilities, membership, training, and communication. DMCs have been monitoring trials using the group sequential design (GSD) for over 30 years. While decisions may be more complicated with novel adaptive designs, the fundamental roles and responsibilities of a DMC will remain the same, namely, to protect patient safety and ensure the scientific integrity of the trial. It will be the DMC's responsibility to recommend changes to the trial within the scope of a prespecified adaptation plan or decision criteria and not to otherwise recommend changes to the study design except for serious safety-related concerns. Nevertheless, compared with traditional data monitoring, some additional considerations are necessary when convening DMCs for novel adaptive designs. They include the need to identify DMC members who are familiar with adaptive design and to consider possible sponsor involvement in unique situations. The need for additional expertise in DMC members has prompted some researchers to propose alternative DMC models or alternative governance model. These various options and authors' views on them are expressed in this article.

A bivariate Bayesian dose-finding procedure applied to a seamless phase I/II trial in rheumatoid arthritis.

We describe a dose escalation procedure for a combined phase I/II clinical trial. The procedure is based on a Bayesian model for the joint distribution of the occurrence of a dose limiting event and of some indicator of efficacy (both considered binary variables), making no assumptions other than monotonicity. Thus, the chances of each outcome are assumed to be non-decreasing in dose level. We applied the procedure to the design of a placebo-controlled, sequential trial in rheumatoid arthritis, in each stage of which patients were randomized between placebo and all dose levels that currently appeared safe and non-futile. On the basis of data from a pilot study, we constructed five different scenarios for the dose-response relationships under which we simulated the trial and assessed the performance of the procedure. The new design appears to have satisfactory operating characteristics and can be adapted to the requirements of a range of trial situations.

A bayesian dose-finding design adapting to efficacy and tolerability response.

We propose a new adaptive Bayesian design, explicitly modeling the trade-off between efficacy and tolerability in dose-finding studies. This design incorporates a continuous efficacy variable and a dichotomous tolerability variable. This adaptive design was developed in the context of a drug under development for treatment of major depression, but is easily extended to any setting with a continuous efficacy and a dichotomous tolerability or safety variable. The goal is to identify a target dose that was most efficacious while still being safe. Via simulations under various scenarios we show that our design performs extremely efficiently. Our design incorporates stopping rules, adaptive allocation, and dose-response estimation (for both efficacy and tolerability), among other features. We present various metrics from our simulation study, and conclude that this is an extremely efficient way of characterizing the risk-benefit profile of a drug during clinical development.

How Adaptive Trial Designs can Increase Efficiency in Psychiatric Drug Development: A Case Study.

This paper uses a recently completed study to illustrate how adaptive trial designs can increase efficiency of psychiatric drug development. The design employed allowed a continuous reassessment of the estimated dose-response such that patients were randomized in a double-blind fashion to one of seven doses of the investigational drug, placebo, or active comparator. The study design also permitted early detection of futility allowing for early study termination. By using the adaptive trial design approach, only 202 patients were needed to make the determination of futility. In contrast, a conventional design would have required enrollment of 450 patients and considerably more time and expense to reach the same conclusion. Adaptive trial designs are important at this time when many pharmaceutical companies are abandoning the development of psychiatric medications because of the inefficiency of conventional approaches.

An introduction to adaptive designs and adaptation in CNS trials.

Adaptive designs learn from accumulating trial data in real time and apply this knowledge to optimize subsequent study execution. A set of design rules define a priori which modifications may be incorporated into the trial design. Judicious use of adaptive designs may increase the information value per resource unit invested by avoiding allocation of patients to non-efficacious/unsafe therapies and allowing stopping decisions to be made at the earliest possible time point. Ultimately this may accelerate the development of promising therapies.

Viewpoints on the FDA draft adaptive designs guidance from the PhRMA working group.

The US Food and Drug Administration has recently released a draft guidance document on adaptive clinical trials. We comment on the document from the particular perspective of the authors as members of a PhRMA working group on this topic, which has interacted with FDA personnel on adaptive trial issue during recent years. We describe the activities and prior work of our working group, and use this as a basis to discuss the content of the guidance document as it relates to several issues of current relevance, such as data monitoring processes, adaptive dose finding, so-called seamless trial designs, and sample size reestimation.

Adaptive Dc-optimal designs for dose finding based on a continuous efficacy endpoint.

We introduce a new optimal design for dose finding with a continuous efficacy endpoint. This design is studied in the context of a flexible model for the mean of the dose-response. The design incorporates aspects of both D- and c-optimality and can be used when the study goals under consideration include dose-response estimation, followed by identification of the target dose. Different optimality criteria are considered. Simulations are shown with results comparing our adaptive design to the fixed allocation (without adaptations). We show that both the estimation of dose-response and identification of the minimum effective dose are improved using our design.

Likelihood inference for a two-stage design with treatment selection.

A conditional likelihood-based approach is proposed to construct confidence intervals for the parameters of interest in a two-stage design with treatment selection after the first stage. Both a Wald confidence interval and a confidence interval based on inverting the likelihood ratio test are proposed. The operating characteristics of these confidence intervals: the coverage probabilities and average confidence interval lengths, as well as the average bias and mean-square error of the corresponding point estimates, compare favorably with other available techniques. Possible extensions and an alternative unconditional approach based on the likelihood with missing at random mechanism are also briefly described.

Two-stage design for dose-finding that accounts for both efficacy and safety.

We introduce a two-stage design for dose-finding in the context of Phase I/II studies, where two binary correlated endpoints are available, for instance, one for efficacy and one for toxicity. The bivariate probit model is used as a working model for the dose-response relationship. Given a 'desirable point' for the marginal probabilities of efficacy and toxicity, the goal is to find the target dose that is 'closest' to the desirable point. The criterion of optimality (objective function) is the variance of the estimator for that dose. Optimal experimental design methodology is used to construct efficient dose allocation procedures that treat patients in the study at doses that are both safe and efficacious.

Adaptive designs in clinical drug development: opportunities, challenges, and scope reflections following PhRMA's November 2006 workshop.

This paper provides reflections on the opportunities, scope and challenges of adaptive design as discussed at PhRMA's workshop held in November 2006. We also provide a status report of workstreams within PhRMA's working group on adaptive designs, which were triggered by the November workshop. Rather than providing a comprehensive review of the presentations given, we limit ourselves to a selection of key statements. The authors reflect the position of PhRMA's working group on adaptive designs.

Adaptive designs for dose-finding studies based on sigmoid Emax model.

We propose an adaptive procedure for dose-finding in clinical trials when the primary efficacy endpoint is continuous. We model the mean of the efficacy endpoint, given the dose, as a four-parameter logistic function. The efficacy endpoint at each dose is distributed according to either a normal or a gamma distribution. We consider the cases of fixed variance and fixed coefficient of variation assuming them to be both known and unknown. The analytic formulae for the Fisher information matrix are obtained, which are used to build the locally and adaptive D-optimal designs.

Implementation of an adaptive group sequential design in a bioequivalence study.

The study design was a multi-center, multiple-dose, randomized, open-label, 2 x 2 crossover study in patients with advanced solid tumors. Each patient was randomized to receive the test formulation or the reference formulation of the drug. The primary objective of the study was to demonstrate the bioequivalence of the test formulation T relative to the reference formulation R. The primary pharmacokinetic endpoints were AUC and Cmax. Since there were different bioequivalence criteria, different endpoints, with different and highly variable coefficients of variation, an adaptive design with a stopping rule for early establishing the bioequivalence as well as early stopping for futility with a flexible information-based monitoring based on error spending approach was implemented to manage uncertainty in assumptions of variability and expected slow enrollment rates.

Adaptive designs in clinical drug development--an Executive Summary of the PhRMA Working Group.

A PhRMA Working Group on adaptive clinical trial designs has been formed to investigate and facilitate opportunities for wider acceptance and usage of adaptive designs and related methodologies. A White Paper summarizing the findings of the group is in preparation; this article is an Executive Summary for that full White Paper, and summarizes the findings and recommendations of the group. Logistic, operational, procedural, and statistical challenges associated with adaptive designs are addressed. Three particular areas where it is felt that adaptive designs can be utilized beneficially are discussed: dose finding, seamless Phase II/III trials designs, and sample size reestimation.