Incorporating external real-world data (RWD) in confirmatory adaptive design.

Adaptive designs, such as group sequential designs (and the ones with additional adaptive features) or adaptive platform trials, have been quintessential efficient design strategies in trials of unmet medical needs, especially for generating evidence from global regions. Such designs allow interim decision making and making adjustment to study design when necessary, meanwhile maintaining study integrity and operating characteristics. However, driven by the heightened competitive landscape and the desire to bring effective treatment to patients faster, innovation in the already functional designs is still germane to further propel drug development to a more efficient path. One way to achieve this is by leveraging external real-world data (RWD) in the adaptive designs to support interim or final decision making. In this paper, we propose a novel framework of incorporating external RWD in adaptive design to improve interim and/or final analysis decision making. Within this framework, researchers can prespecify the decision process and choose the timing and amount of borrowing while maintaining objectivity and controlling of type I error. Simulation studies in various scenarios are provided to describe power, type I error, and other performance metrics for interim/final decision making. A case study in non-small cell lung cancer is used for illustration on proposed design framework.

Platform trials to evaluate the benefit-risk of COVID-19 therapeutics: Successes, learnings, and recommendations for future pandemics.

BACKGROUND: In response to the COVID-19 global pandemic, multiple platform trials were initiated to accelerate evidence generation of potential therapeutic interventions. Given a rapidly evolving and dynamic pandemic, platform trials have a key advantage over traditional randomized trials: multiple interventions can be investigated under a master protocol sharing a common infrastructure. METHODS: This paper focuses on nine platform trials that were instrumental in advancing care in COVID-19 in the hospital and community setting. A semi-structured qualitative interview was conducted with the principal investigators and lead statisticians of these trials. Information from the interviews and public sources were tabulated and summarized across trials, and recommendations for best practice for the next health crisis are provided. RESULTS: Based on the information gathered takeaways were identified as 1) the existence of some aspect of trial design or conduct (e.g., existing network of investigators or colleagues, infrastructure for data capture and relevant statistical expertise) was a key success factor; 2) the choice of treatments (e.g., repurposed drugs) had major impact on the trials as did the choice of primary endpoint; and 3) the lack of coordination across trials was flagged as an area for improvement. CONCLUSION: These trials deployed during the COVID-19 pandemic demonstrate how to achieve both speed and quality of evidence generation regarding clinical benefit (or not) of existing therapies to treat new pathogens in a pandemic setting. As a group, these trials identified treatments that worked, and many that did not, in a matter of months.

Extent of safety database in pediatric drug development: types of assessment, analytical precision, and pathway for extrapolation through on-target effects.

Pediatric patients should have access to medicines that have been appropriately evaluated for safety and efficacy through revised labelling. Given this goal, the adequacy of the pediatric clinical development plan and resulting safety database are critical factors to inform a favorable benefit-risk assessment for the intended use of the medicinal product. While extrapolation from adults can be used to support efficacy of drugs in children, there may be a reluctance to use the same approach in safety assessments, wiping out potential gains in trial efficiency through a reduction of sample size. To address this issue, we explore safety review in pediatric trials, including specific types of safety assessments and precision on the estimation of event rates for specific adverse events (AEs) that can be achieved. In addition, we discuss the assessments which can provide a benchmark for the use of extrapolation of safety that focuses on on-target effects. Finally, we explore a unified approach for understanding precision using Bayesian approaches as the most appropriate methodology to describe or ascertain risk in probabilistic terms for the estimate of the event rate of specific AEs.

Composite Likelihoods with Bounded Weights in Extrapolation of Data.

Among many efforts to facilitate timely access to safe and effective medicines to children, increased attention has been given to extrapolation. Loosely, it is the leveraging of conclusions or available data from adults or older age groups to draw conclusions for the target pediatric population when it can be assumed that the course of the disease and the expected response to a medicinal product would be sufficiently similar in the pediatric and the reference population. Extrapolation then can be characterized as a statistical mapping of information from the reference (adults or older age groups) to the target pediatric population. The translation, or loosely mapping of information, can be through a composite likelihood approach where the likelihood of the reference population is weighted by exponentiation and that this exponent is related to the value of the mapped information in the target population. The weight is bounded above and below recognizing the fact that similarity (of the disease and the expected response) is still valid despite variability of response between the cohorts. Maximum likelihood approaches are then used for estimation of parameters, and asymptotic theory is used to derive distributions of estimates for use in inference. Hence, the estimation of effects in the target population borrows information from the reference population. In addition, this manuscript also talks about how this method is related to the Bayesian statistical paradigm.

Matching within a hybrid RCT/RWD: framework on associated causal estimands.

As the regulatory environment becomes progressively receptive toward utilizing real-world evidence, a spectrum of real-world data incorporation techniques in trial conduct and analysis has seen increasing interest and adoption in different stages of drug development. Of particular interest is leveraging external control data to augment the control arm in a concurrent randomized controlled trial, where patients are enrolled in both investigational treatment arm and the control arm. Yet despite the emerging literature in external data borrowing in a hybrid trial setting, very little discussion focuses on delineating what should be matched and what is actually being estimated, especially when a variety of matching schemes can be considered. In general, external control can be matched in four different ways: (1) matching with the intersection between investigational treatment and concurrent control, (2) matching with the union of concurrent investigational treatment and concurrent control, (3) matching with concurrent control alone, and (4) matching with investigational treatment alone. In this article, the formulation of estimands for different matching schemes are detailed to describe what these matching methods facilitate to answer. Simulation studies are also conducted to evaluate the performance characteristics under different matching schemes, estimation methods, effect size assumptions, and missingness of confounders.

Statistical methods of indirect comparison with real-world data for survival endpoint under non-proportional hazards.

In clinical studies that utilize real-world data, time-to-event outcomes are often germane to scientific questions of interest. Two main obstacles are the presence of non-proportional hazards and confounding bias. Existing methods that could adjust for NPH or confounding bias, but no previous work delineated the complexity of simultaneous adjustments for both. In this paper, a propensity score stratified MaxCombo and weighted Cox model is proposed. This model can adjust for confounding bias and NPH and can be pre-specified when NPH pattern is unknown in advance. The method has robust performance as demonstrated in simulation studies and in a case study.

Dynamic incorporation of real world evidence within the framework of adaptive design.

For the clinical studies in rare diseases or small patient populations, having an adequately powered randomized controlled trial is further complicated by variability. As such, sample size re-estimation can be a useful tool if at an interim look the trial sample size needs to be increased to achieve adequate power to reject the null hypothesis. Meanwhile, borrowing or extrapolating information from real-world data or real-world evidence has gained increasing use in trial design and analysis since 2014. Combining these two strategies, high-quality real-world data, if leveraged properly, has the potential to generate real-world evidence that can assist interim decision-making, lower enrollment burden, and reduce study timeline and costs. With proper borrowing from historical control, some of the challenges in these high unmet medical need studies could be resolved considerably. We examine the incorporation of real-world evidence within the framework of adaptive design strategy in pediatric type II diabetes trials where recruitment has been challenging and the completion is hardly on time. Simulations under various scenarios are conducted to assess the borrowing strategy, i.e., the matching method in combination of sample size re-estimation. Comparisons of performance metrics are presented to showcase the advantages of proposed method.

Optimal sample size determination for single-arm trials in pediatric and rare populations with Bayesian borrowing.

In many therapeutic areas with unmet medical needs, such as pediatric oncology and rare diseases, one of the deterrent factors for clinical trial interpretability is the limited sample size with less-than-ideal operating characteristics. Single arm is usually the only viable design due to feasibility and ethical concerns. For the trial results to be more interpretable and conclusive, the evaluation of operating characteristics, such as type I error rate and power, and the appropriate utilization of prior information for study design, shall be prespecified and fully investigated during the trial planning phase. So far, very few existing literature addressed optimal sample size determination issues for the planning of pediatric and rare population trials, with majority of research focusing on analysis perspective with focus on Bayesian borrowing. In practice, when a single-arm trial is designed for rare population, it is not uncommon that the only information available is from an earlier trial and/or a few clinical publications based on observational studies, often constituting mixed or uncertain conclusions. In light of this, an optimal Bayesian sample size determination method for single-arm trial with binary or continuous endpoint is proposed, where conflicting prior beliefs can be readily incorporated. Prior effective sample size can be calculated to assess the robustness as well as the prior information borrowed. Moreover, due to the lack of closed-form posterior distributions in general, an alternative approach for calculating Bayesian power is described. Simulation studies are provided to demonstrate the utility of the proposed methods. In addition, a case study in pediatric patients with leukemia is included to illustrate the proposed method with the existing approaches.

Incorporating propensity scores for evidence synthesis under bayesian framework: review and recommendations for clinical studies.

The amount of real-world data (RWD) available from sources other than randomized-controlled trials (RCTs) has grown ultra-rapidly in recent years. It provides the impetus for generating substantial evidence of effectiveness and safety from both RCTs and RWD to accelerate medical product development. Especially in the areas of unmet needs, the conduct of fully powered RCTs is generally infeasible because of their sizes, duration, cost, or ethical constraints. The unique challenges in such areas include a small patient population, heterogeneity in disease presentation, and a lack of established endpoints. However, merging information from disparate sources is an intricate task. The value of the Bayesian framework has gained more recognition due to its flexibility in calibrating uncertainty and handling data heterogeneity, and its inherent updating process ideal for synthesizing information. Meanwhile, propensity score, as a powerful tool in causal inference, can be used in various ways to adjust for confounders. As a newly emerging data borrowing strategy in a regulatory setting, integrating propensity scores in a Bayesian setting not only utilizes the strengths from Bayesian models but also minimizes bias from external data borrowing. These methods potentially allow information sharing among data sources, provide more reliable estimates when the sample size is small, and improve the efficiency of treatment effect estimation. In this paper, we will review the recent development of methods incorporating propensity score for evidence synthesis under the Bayesian framework, and discuss different examples of incorporating external data with or without RCTs, as well as the recommendations for reporting in clinical studies.

Ensuring exchangeability in data-based priors for a Bayesian analysis of clinical trials.

In many orphan diseases and pediatric indications, the randomized controlled trials may be infeasible because of their size, duration, and cost. Leveraging information on the control through a prior can potentially reduce sample size. However, unless an objective prior is used to impose complete ignorance for the parameter being estimated, it results in biased estimates and inflated type-I error. Hence, it is essential to assess both the confirmatory and supplementary knowledge available during the construction of the prior to avoid "cherry-picking" advantageous information. For this purpose, propensity score methods are employed to minimize selection bias by weighting supplemental control subjects according to their similarity in terms of pretreatment characteristics to the subjects in the current trial. The latter can be operationalized through a proposed measure of overlap in propensity-score distributions. In this paper, we consider single experimental arm in the current trial and the control arm is completely borrowed from the supplemental data. The simulation experiments show that the proposed method reduces prior and data conflict and improves the precision of the of the average treatment effect.

Propensity-score-based meta-analytic predictive prior for incorporating real-world and historical data.

As the availability of real-world data sources (eg, EHRs, claims data, registries) and historical data has rapidly surged in recent years, there is an increasing interest and need from investigators and health authorities to leverage all available information to reduce patient burden and accelerate both drug development and regulatory decision making. Bayesian meta-analytic approaches are a popular historical borrowing method that has been developed to leverage such data using robust hierarchical models. The model structure accounts for various degrees of between-trial heterogeneity, resulting in adaptively discounting the external information in the case of data conflict. In this article, we propose to integrate the propensity score method and Bayesian meta-analytic-predictive (MAP) prior to leverage external real-world and historical data. The propensity score methodology is applied to select a subset of patients from external data that are similar to those in the current study with regards to key baseline covariates and to stratify the selected patients together with those in the current study into more homogeneous strata. The MAP prior approach is used to obtain stratum-specific MAP prior and derive the overall propensity score integrated meta-analytic predictive (PS-MAP) prior. Additionally, we allow for tuning the prior effective sample size for the proposed PS-MAP prior, which quantifies the amount of information borrowed from external data. We evaluate the performance of the proposed PS-MAP prior by comparing it to the existing propensity score-integrated power prior approach in a simulation study and illustrate its implementation with an example of a single-arm phase II trial.

Flexible Bayesian subgroup analysis in early and confirmatory trials.

Subgroup analysis is one of the most important issues in clinical trials. In confirmatory trials, it is critical to investigate consistency of the treatment effect across subgroups, which could potentially result in incorrect scientific conclusion or regulatory decision. There are many challenges and methodological complications of interpreting subgroup results beyond the regulatory setting. For the early phase or proof of concept trials, particularly in basket trials, it is also important to have reliable estimation of subgroup treatment effect in order to guide the next phase go/no-go decision making when large biases can be introduced due to small sample size or random variability. In this paper, we review several recent methods that have been proposed for subgroup analysis in the Bayesian framework to correct for bias. We present simulation results from applying various novel Bayesian hierarchical models for subgroup analysis to a phase II basket trial. For different scenarios considered, we compare the average total sample size, and frequentist-like operating characteristics of power and familywise type I error rate. We compare the precision of the model estimates of the treatment effect by assessing average relative bias and the width of the 95% credible interval for the bias. We also demonstrate flexible Bayesian hierarchical models in a case study of a phase III oncology trial for subgroup treatment effect estimation to help with regulatory decision making. Finally, we conclude our findings in the discussion section and give recommendations on how these methods could be implemented in confirmatory and early phase clinical trials.

Real-World Evidence Utilization in Clinical Development Reflected by US Product Labeling: Statistical Review.

The US Food and Drug Administration (FDA) has shown scientific discretion in interpreting the substantial evidence requirement for the approval of new drugs with its considerations on the use of single controlled or uncontrolled trials (Federal Food, Drug, and Cosmetic Act § 505(d), 21 USC 355(d), 1962). With the passage of the 21st Centuries Cures Act (21st Century Cures-patients. House, Energy and Commerce Committee, Washington, DC, 2019 available at: https://energycommerce.house.gov/sites/republicans.energycommerce.house.gov/files/analysis/21stCenturyCures/20140516PatientsWhitePaper.pdf ), the FDA is mandated to expand the role of real-world evidence (RWE) in support of drug approval. This mandate further broadens the scope of scientific discretion to include data collected outside clinical trials. We summarize the agency's past acceptance of real-world data (RWD) sources for supporting drug approval in new indications which have been reflected in US labels. In our summary, we focus on the type of RWD and statistical methodologies presented in these labels. Furthermore, two labels were selected for in-depth assessment of the RWE presented in these labels. Through these examples, we demonstrate the issues that can be raised in data collection that could affect interpretation. In addition, a brief discussion of statistical methods that can be used to incorporate RWE to clinical development is presented.

Propensity-score-based priors for Bayesian augmented control design.

Drug developers are required to demonstrate substantial evidence of effectiveness through the conduct of adequate and well-controlled (A&WC) studies to obtain marketing approval of their medicine. What constitutes A&WC is interpreted as the conduct of randomized controlled trials (RCTs). However, these trials are sometimes unfeasible because of their size, duration, and cost. One way to reduce sample size is to leverage information on the control through a prior. One consideration when forming data-driven prior is the consistency of the external and the current data. It is essential to make this process less susceptible to choosing information that only helps improve the chances toward making an effectiveness claim. For this purpose, propensity score methods are employed for two reasons: (1) it gives the probability of a patient to be in the trial, and (2) it minimizes selection bias by pairing together treatment and control within the trial and control subjects in the external data that are similar in terms of their pretreatment characteristics. Two matching schemes based on propensity scores, estimated through generalized boosted methods, are applied to a real example with the objective of using external data to perform Bayesian augmented control in a trial where the allocation is disproportionate. The simulation results show that the data augmentation process prevents prior and data conflict and improves the precision of the estimator of the average treatment effect.

Propensity score matched augmented controls in randomized clinical trials: A case study.

Existing statutes in the United States and Europe require manufacturers to demonstrate evidence of effectiveness through the conduct of adequate and well-controlled studies to obtain marketing approval of a therapeutic product. What constitutes adequate and well-controlled studies is usually interpreted as randomized controlled trials (RCTs). However, these trials are sometimes unfeasible because of their size, duration, cost, patient preference, or in some cases, ethical concerns. For example, RCTs may not be fully powered in rare diseases or in infections caused by multidrug resistant pathogens because of the low number of enrollable patients. In this case, data available from external controls (including historical controls and observational studies or data registries) can complement information provided by RCT. Propensity score matching methods can be used to select or "borrow" additional patients from the external controls, for maintaining a one-to-one randomization between the treatment arm and active control, by matching the new treatment and control units based on a set of measured covariates, ie, model-based pairing of treatment and control units that are similar in terms of their observable pretreatment characteristics. To this end, 2 matching schemes based on propensity scores are explored and applied to a real clinical data example with the objective of using historical or external observations to augment data in a trial where the randomization is disproportionate or asymmetric.

Non-inferiority and networks: inferring efficacy from a web of data.

In the absence of placebo-controlled trials, the efficacy of a test treatment can be alternatively examined by showing its non-inferiority to an active control; that is, the test treatment is not worse than the active control by a pre-specified margin. The margin is based on the effect of the active control over placebo in historical studies. In other words, the non-inferiority setup involves a network of direct and indirect comparisons between test treatment, active controls, and placebo. Given this framework, we consider a Bayesian network meta-analysis that models the uncertainty and heterogeneity of the historical trials into the non-inferiority trial in a data-driven manner through the use of the Dirichlet process and power priors. Depending on whether placebo was present in the historical trials, two cases of non-inferiority testing are discussed that are analogs of the synthesis and fixed-margin approach. In each of these cases, the model provides a more reliable estimate of the control given its effect in other trials in the network, and, in the case where placebo was only present in the historical trials, the model can predict the effect of the test treatment over placebo as if placebo had been present in the non-inferiority trial. It can further answer other questions of interest, such as comparative effectiveness of the test treatment among its comparators. More importantly, the model provides an opportunity for disproportionate randomization or the use of small sample sizes by allowing borrowing of information from a network of trials to draw explicit conclusions on non-inferiority.