A more powerful test for three-arm non-inferiority via risk difference: Frequentist and Bayesian approaches.

Necessity for finding improved intervention in many legacy therapeutic areas are of high priority. This has the potential to decrease the expense of medical care and poor outcomes for many patients. Typically, clinical efficacy is the primary evaluating criteria to measure any beneficial effect of a treatment. Albeit, there could be situations when several other factors (e.g. side-effects, cost-burden, less debilitating, less intensive, etc.) which can permit some slightly less efficacious treatment options favorable to a subgroup of patients. This often leads to non-inferiority (NI) testing. NI trials may or may not include a placebo arm due to ethical reasons. However, when included, the resulting three-arm trial is more prudent since it requires less stringent assumptions compared to a two-arm placebo-free trial. In this article, we consider both Frequentist and Bayesian procedures for testing NI in the three-arm trial with binary outcomes when the functional of interest is risk difference. An improved Frequentist approach is proposed first, which is then followed by a Bayesian counterpart. Bayesian methods have a natural advantage in many active-control trials, including NI trial, as it can seamlessly integrate substantial prior information. In addition, we discuss sample size calculation and draw an interesting connection between the two paradigms.

A novel approach to augment single-arm clinical studies with real-world data.

In this paper, we develop a methodology for leveraging real-world data into single-arm clinical trial studies. In recent years, the idea of augmenting randomized clinical trials data with real-world data has emerged as a particularly attractive technique for health organizations and drug developers to accelerate the drug development process. Major regulatory authorities such as the Food and Drug Administration and European Medicines Agency have recognized the potential of utilizing real-world data and are advancing toward making regulatory decisions based on real-world evidence. Several statistical methods have been developed in recent years for borrowing data from real-world sources such as electronic health records, product and disease registries, as well as claims and billing data. We propose a novel approach to augment single-arm clinical trials with the real-world data derived from single or multiple data sources. Furthermore, we illustrate the proposed method in the presence of missing data and conduct simulation studies to evaluate its performance in diverse settings.

New approaches for testing non-inferiority for three-arm trials with Poisson distributed outcomes.

With the availability of limited resources, innovation for improved statistical method for the design and analysis of randomized controlled trials (RCTs) is of paramount importance for newer and better treatment discovery for any therapeutic area. Although clinical efficacy is almost always the primary evaluating criteria to measure any beneficial effect of a treatment, there are several important other factors (e.g., side effects, cost burden, less debilitating, less intensive, etc.), which can permit some less efficacious treatment options favorable to a subgroup of patients. This leads to non-inferiority (NI) testing. The objective of NI trial is to show that an experimental treatment is not worse than an active reference treatment by more than a pre-specified margin. Traditional NI trials do not include a placebo arm for ethical reason; however, this necessitates stringent and often unverifiable assumptions. On the other hand, three-arm NI trials consisting of placebo, reference, and experimental treatment, can simultaneously test the superiority of the reference over placebo and NI of experimental treatment over the reference. In this article, we proposed both novel Frequentist and Bayesian procedures for testing NI in the three-arm trial with Poisson distributed count outcome. RCTs with count data as the primary outcome are quite common in various disease areas such as lesion count in cancer trials, relapses in multiple sclerosis, dermatology, neurology, cardiovascular research, adverse event count, etc. We first propose an improved Frequentist approach, which is then followed by it's Bayesian version. Bayesian methods have natural advantage in any active-control trials, including NI trial when substantial historical information is available for placebo and established reference treatment. In addition, we discuss sample size calculation and draw an interesting connection between the two paradigms.

Non-inferiority Testing for Risk Ratio, Odds Ratio and Number Needed to Treat in Three-arm Trial.

Three-arm non-inferiority (NI) trial including the experimental treatment, an active reference treatment, and a placebo where the outcome of interest is binary are considered. While the risk difference (RD) is the most common and well explored functional form for testing efficacy (or effectiveness), however, recent FDA guideline suggested measures such as relative risk (RR), odds ratio (OR), number needed to treat (NNT) among others, on the basis of which NI can be claimed for binary outcome. Albeit, developing test based on these different functions of binary outcome are challenging. This is because the construction and interpretation of NI margin for such functions are non-trivial extensions of RD based approach. A Frequentist test based on traditional fraction margin approach for RR, OR and NNT are proposed first. Furthermore a conditional testing approach is developed by incorporating assay sensitivity (AS) condition directly into NI testing. A detailed discussion of sample size/power calculation are also put forward which could be readily used while designing such trials in practice. A clinical trial data is reanalyzed to demonstrate the presented approach.

Bayesian Approach for Assessing Non-inferiority in Three-arm Trials for Risk Ratio and Odds Ratio.

In this paper we consider three-arm non-inferiority (NI) trial that includes an experimental, a reference, and a placebo arm. While for binary outcomes the risk difference (RD) is the most common and well explored functional form for testing efficacy (or effectiveness), recent FDA guideline suggested other measures such as relative risk (RR) and odds ratio (OR) on the basis of which NI of an experimental treatment can be claimed. However, developing test based on these different functions of binary outcomes are challenging since the construction and interpretation of NI margin for such functions are not trivial extensions of RD based approach. Recently, we have proposed Frequentist approaches for testing NI for these functionals. In this article we further develop Bayesian approaches for testing NI based on effect retention approach for RR and OR. Bayesian paradigm provides a natural path to integrate historical trials' information, as well as it allows the usage of patients'/clinicians' opinions as prior information via sequential learning. In addition we discuss, in detail, the sample size/power calculation which could be readily used while designing such trials in practice.

Approaches for testing noninferiority in two-arm trials for risk ratio and odds ratio.

For an existing established drug regimen, active control trials are defacto standard due to ethical reason as well as for clinical equipoise. However, when superiority claim of a new drug against the active control is unlikely to be successful, researchers often address the issue in terms of noninferiority (NI), provided the experimental drug demonstrates the evidence of other benefits beyond efficacy. Such trials aim to demonstrate that an experimental treatment is non-inferior to an existing comparator by not more than a pre-specified margin. The issue of choosing such a margin is complex. In this article, two-arm NI trials with binary outcomes are considered when margin is defined in terms of relative risk or odds ratio. A Frequentist test based on proposed NI margin is developed first. Since two-arm NI trials without placebo arm are dependent upon historical information, in order to make accurate and meaningful interpretation of their results, a Bayesian approach is developed next. Bayesian approach is flexible to incorporate the available information from the historical trial. The operating characteristics of the proposed methods are studied in terms of power and sample size for varying design factors. A clinical trial data is reanalyzed to study the properties of the proposed approach.

Hierarchical Bayes approach for subgroup analysis.

In clinical data analysis, both treatment effect estimation and consistency assessment are important for a better understanding of the drug efficacy for the benefit of subjects in individual subgroups. The linear mixed-effects model has been used for subgroup analysis to describe treatment differences among subgroups with great flexibility. The hierarchical Bayes approach has been applied to linear mixed-effects model to derive the posterior distributions of overall and subgroup treatment effects. In this article, we discuss the prior selection for variance components in hierarchical Bayes, estimation and decision making of the overall treatment effect, as well as consistency assessment of the treatment effects across the subgroups based on the posterior predictive p-value. Decision procedures are suggested using either the posterior probability or the Bayes factor. These decision procedures and their properties are illustrated using a simulated example with normally distributed response and repeated measurements.

Bayesian approach for assessing noninferiority in a three-arm trial with binary endpoint.

With the recent advancement in many therapeutic areas, quest for better and enhanced treatment options is ever increasing. While the "efficacy" metric plays the most important role in this development, emphasis on other important clinical factors such as less intensive side effects, lower toxicity, ease of delivery, and other less debilitating factors may result in the selection of treatment options, which may not beat current established treatment option in terms efficacy, yet prove to be desirable for subgroups of patients. The resultant clinical trial by means of which one establishes such slightly less efficacious treatment is known as noninferiority (NI) trial. Noninferiority trials often involve an active established comparator arm, along with a placebo and an experimental treatment arm, resulting into a 3-arm trial. Most of the past developments in a 3-arm NI trial consider defining a prespecified fraction of unknown effect size of reference drug, i.e., without directly specifying a fixed NI margin. However, in some recent developments, more direct approach is being considered with prespecified fixed margin, albeit in the frequentist setup. In this article, we consider Bayesian implementation of such trial when primary outcome of interest is binary. Bayesian paradigm is important, as it provides a path to integrate historical trials and current trial information via sequential learning. We use several approximation-based and 2 exact fully Bayesian methods to evaluate the feasibility of the proposed approach. Finally, a clinical trial example is reanalyzed to demonstrate the benefit of the proposed approach.

Extended likelihood ratio test-based methods for signal detection in a drug class with application to FDA's adverse event reporting system database.

A likelihood ratio test, recently developed for the detection of signals of adverse events for a drug of interest in the FDA Adverse Events Reporting System database, is extended to detect signals of adverse events simultaneously for all the drugs in a drug class. The extended likelihood ratio test methods, based on Poisson model (Ext-LRT) and zero-inflated Poisson model (Ext-ZIP-LRT), are discussed and are analytically shown, like the likelihood ratio test method, to control the type-I error and false discovery rate. Simulation studies are performed to evaluate the performance characteristics of Ext-LRT and Ext-ZIP-LRT. The proposed methods are applied to the Gadolinium drug class in FAERS database. An in-house likelihood ratio test tool, incorporating the Ext-LRT methodology, is being developed in the Food and Drug Administration.

Bayesian approach for assessing non-inferiority in a three-arm trial with pre-specified margin.

Non-inferiority trials are becoming increasingly popular for comparative effectiveness research. However, inclusion of the placebo arm, whenever possible, gives rise to a three-arm trial which has lesser burdensome assumptions than a standard two-arm non-inferiority trial. Most of the past developments in a three-arm trial consider defining a pre-specified fraction of unknown effect size of reference drug, that is, without directly specifying a fixed non-inferiority margin. However, in some recent developments, a more direct approach is being considered with pre-specified fixed margin albeit in the frequentist setup. Bayesian paradigm provides a natural path to integrate historical and current trials' information via sequential learning. In this paper, we propose a Bayesian approach for simultaneous testing of non-inferiority and assay sensitivity in a three-arm trial with normal responses. For the experimental arm, in absence of historical information, non-informative priors are assumed under two situations, namely when (i) variance is known and (ii) variance is unknown. A Bayesian decision criteria is derived and compared with the frequentist method using simulation studies. Finally, several published clinical trial examples are reanalyzed to demonstrate the benefit of the proposed procedure.

Bayesian approach to non-inferiority trials for normal means.

Regulatory framework recommends that novel statistical methodology for analyzing trial results parallels the frequentist strategy, e.g. the new method must protect type-I error and arrive at a similar conclusion. Keeping these in mind, we construct a Bayesian approach for non-inferiority trials with normal response. A non-informative prior is assumed for the mean response of the experimental treatment and Jeffrey's prior for its corresponding variance when it is unknown. The posteriors of the mean response and variance of the treatment in historical trials are then assumed as priors for its corresponding parameters in the current trial, where that treatment serves as the active control. From these priors, a Bayesian decision criterion is derived to determine whether the experimental treatment is non-inferior to the active control. This criterion is evaluated and compared with the frequentist method using simulation studies. Results show that both Bayesian and frequentist approaches perform alike, but the Bayesian approach has a higher power when the variances are unknown. Both methods also arrive at the same conclusion of non-inferiority when applied on two real datasets. A major advantage of the proposed Bayesian approach lies in its ability to provide posterior probabilities for varying effect sizes of the experimental treatment over the active control.

Meta-analysis using Dirichlet process.

This article develops a Bayesian approach for meta-analysis using the Dirichlet process. The key aspect of the Dirichlet process in meta-analysis is the ability to assess evidence of statistical heterogeneity or variation in the underlying effects across study while relaxing the distributional assumptions. We assume that the study effects are generated from a Dirichlet process. Under a Dirichlet process model, the study effects parameters have support on a discrete space and enable borrowing of information across studies while facilitating clustering among studies. We illustrate the proposed method by applying it to a dataset on the Program for International Student Assessment on 30 countries. Results from the data analysis, simulation studies, and the log pseudo-marginal likelihood model selection procedure indicate that the Dirichlet process model performs better than conventional alternative methods.

Signal detection in FDA AERS database using Dirichlet process.

In the recent two decades, data mining methods for signal detection have been developed for drug safety surveillance, using large post-market safety data. Several of these methods assume that the number of reports for each drug-adverse event combination is a Poisson random variable with mean proportional to the unknown reporting rate of the drug-adverse event pair. Here, a Bayesian method based on the Poisson-Dirichlet process (DP) model is proposed for signal detection from large databases, such as the Food and Drug Administration's Adverse Event Reporting System (AERS) database. Instead of using a parametric distribution as a common prior for the reporting rates, as is the case with existing Bayesian or empirical Bayesian methods, a nonparametric prior, namely, the DP, is used. The precision parameter and the baseline distribution of the DP, which characterize the process, are modeled hierarchically. The performance of the Poisson-DP model is compared with some other models, through an intensive simulation study using a Bayesian model selection and frequentist performance characteristics such as type-I error, false discovery rate, sensitivity, and power. For illustration, the proposed model and its extension to address a large amount of zero counts are used to analyze statin drugs for signals using the 2006-2011 AERS data.

Bayesian approach to the design and analysis of non-inferiority trials for anti-infective products.

In the absence of placebo-controlled trials, determining the non-inferiority (NI) margin for comparing an experimental treatment with an active comparator is based on carefully selected well-controlled historical clinical trials. With this approach, information on the effect of the active comparator from other sources including observational studies and early phase trials is usually ignored because of the need to maintain active comparator effect across trials. This may lead to conservative estimates of the margin that translate into larger sample-size requirements for the design and subsequent frequentist analysis, longer trial durations, and higher drug development costs. In this article, we provide methodological approaches to determine NI margins that can utilize all relevant historical data through a novel power adjusted Bayesian meta-analysis, with Dirichlet process priors, that puts ordered weights on the amount of information a set of data contributes. We also provide a Bayesian decision rule for the non-inferiority analysis that is based on a broader use of available prior information and a sample-size determination that is based on this Bayesian decision rule. Finally, the methodology is illustrated through several examples.

A Review of Statistical Methods for Safety Surveillance.

The data-mining statistical methods used for disproportionality analysis of drug-adverse event combinations from large drug safety databases such as the FDA's Adverse Event Reporting System (FAERS), consisting of spontaneous reports on adverse events for postmarket drugs, are called passive surveillance methods. However, the statistical signal detection methods for longitudinal data, as the data accrue in time, are called active surveillance methods. A review of the most commonly used passive surveillance statistical methods and the relationships among them is presented with unified notations. These methods are applied to the 2006-2012 FAERS data; the number of drug signals of disproportionate rates (SDRs) detected by each of these methods with the common SDRs from all of these methods, for the adverse event myocardial infarction, are given. Finally, there is a brief discussion on the recently developed active surveillance methods.

Bayesian approach for clinical trial safety data using an Ising prior.

In drug safety, development of statistical methods for multiplicity adjustments has exploited potential relationships among adverse events (AEs) according to underlying medical features. Due to the coarseness of the biological features used to group AEs together, which serves as the basis for the adjustment, it is possible that a single adverse event can be simultaneously described by multiple biological features. However, existing methods are limited in that they are not structurally flexible enough to accurately exploit this multi-dimensional characteristic of an adverse event. In order to preserve the complex dependencies present in clinical safety data, a Bayesian approach for modeling the risk differentials of the AEs between the treatment and comparator arms is proposed which provides a more appropriate clinical description of the drug's safety profile. The proposed procedure uses an Ising prior to unite medically related AEs. The proposed method and an existing Bayesian method are applied to a clinical dataset, and the signals from the two methods are presented. Results from a small simulation study are also presented.

Likelihood ratio test-based method for signal detection in drug classes using FDA's AERS database.

In 1968 the Food and Drug Administration (FDA) established the Adverse Event Reporting System (AERS) database containing data on adverse events (AEs) reported by patients, health care providers, and other sources through a spontaneous reporting system. FDA uses AERS for monitoring the safety of the drugs on the market after approval. Most statistical methods that are available in the literature to analyze large postmarket drug safety data for identifying drug-event combinations with disproportionately high frequencies are designed to explore signals of a single drug-AE combination, but not signals including a drug class or a group of AEs simultaneously. Those methods are also not designed to control type I error and are subject to high false discovery rates. In this paper, we first briefly review a recently developed method, known as the likelihood ratio test (LRT)-based method, which has been demonstrated to control the family-wise type I error and false discovery rates. By introducing a concept of weight matrix for the drugs (or for AEs), we then extend the LRT method for detecting signals including a class of drugs (or AEs) in addition to detecting signals of single drug (or AE). A simplified Bayesian method is also proposed and compared with LRT method. The proposed methods are applied to study the signal patterns of drug classes, namely, the gadolinium drug class for magnetic resonance imaging (MRI) and statins for hypercholesterolemia, over different time periods, using the datasets with only suspect drugs and with both suspect and concomitant drugs from the AERS database. The signals detected by the statistical methods can be confirmed by signals detected across different databases, existing medical evidence from research or regulatory resources, prospective biological studies, and also through simulation as illustrated in the application.

A generalized p-value approach for assessing noninferiority in a three-arm trial.

The existing generalized p-value approach, from statistical literature, is applied to assess noninferiority of an experimental treatment in a three-arm clinical trial including a placebo. Two generalized test functions (GTFs) are constructed and Monte Carlo simulations are used to compute the p-value. The GTFs perform well in terms of maintaining the Type-I error probabilities, and the power of the tests are shown to increase to 1 as both the sample size and the parameter denoting the fraction of the effect of the reference drug with respect to placebo increase. The generalized confidence intervals are shown to retain the coverage probabilities. A published dataset is re-analysed using the proposed test and the results are in agreement with earlier findings.

Statistical approaches to group sequential monitoring of postmarket safety surveillance data: current state of the art for use in the Mini-Sentinel pilot.

PURPOSE: This manuscript describes the current statistical methodology available for active postmarket surveillance of pre-specified safety outcomes using a prospective incident user concurrent control cohort design with existing electronic healthcare data. METHODS: Motivation of the active postmarket surveillance setting is provided using the Food and Drug Administration's Mini-Sentinel Pilot as an example. Four sequential monitoring statistical methods are presented including the Lan-Demets error spending approach, a matched likelihood ratio test statistic approach with the binomial MaxSPRT as a special case, the conditional sequential sampling procedure with stratification, and a generalized estimating equation regression approach using permutation. Information on the assumptions, limitations, and advantages of each approach is provided, including how each method defines sequential monitoring boundaries, what test statistic is used, and how robust it is to settings of rare events or frequent testing. RESULTS: A hypothetical example of how the approaches could be applied to data comparing a medical product of interest, drug A, to a concurrent control drug, drug B, is presented including providing the type of information one would have available for monitoring such drugs.

Bayesian approach to noninferiority trials for proportions.

Noninferiority trials are unique because they are dependent upon historical information in order to make meaningful interpretation of their results. Hence, a direct application of the Bayesian paradigm in sequential learning becomes apparently useful in the analysis. This paper describes a Bayesian procedure for testing noninferiority in two-arm studies with a binary primary endpoint that allows the incorporation of historical data on an active control via the use of informative priors. In particular, the posteriors of the response in historical trials are assumed as priors for its corresponding parameters in the current trial, where that treatment serves as the active control. The Bayesian procedure includes a fully Bayesian method and two normal approximation methods on the prior and/or on the posterior distributions. Then a common Bayesian decision criterion is used but with two prespecified cutoff levels, one for the approximation methods and the other for the fully Bayesian method, to determine whether the experimental treatment is noninferior to the active control. This criterion is evaluated and compared with the frequentist method using simulation studies in keeping with regulatory framework that new methods must protect type I error and arrive at a similar conclusion with existing standard strategies. Results show that both methods arrive at comparable conclusions of noninferiority when applied to a modified real data set. The advantage of the proposed Bayesian approach lies in its ability to provide posterior probabilities for effect sizes of the experimental treatment over the active control.