Suspected serial killers and unsuspected statistical blunders.

A whole branch of theoretical statistics devotes itself to the analysis of clusters, the aim being to distinguish an apparent cluster arising randomly from one that is more likely to have been produced as a result of some systematic influence. There are many examples in medicine and some that involve both medicine and the legal field; criminal law in particular. Observed clusters or a series of cases in a given setting can set off alarm bells, the recent conviction of Lucy Letby in England being an example. It was an observed cluster, a series of deaths among neonates, that prompted the investigation of Letby. There have been other similar cases in the past and there will be similar cases in the future. Our purpose is not to reconsider any particular trial but, rather, to work with similar, indeed more extreme numbers of cases as a way to underline the statistical mistakes that can be made when attempting to make sense of the data. These notions are illustrated via a made-up case of 10 incidents where the anticipated count was only 2. The most common statistical analysis would associate a probability of less than 0.00005 with this outcome: A very rare event. However, a more careful analysis that avoids common pitfalls results in a probability close to 0.5, indicating that, given the circumstances, we were as likely to see 10 or more as we were to see less than 10.

Making sense of breast cancer risk estimates.

Individual probabilistic assessments on the risk of cancer, primary or secondary, will not be understood by most patients. That is the essence of our arguments in this paper. Greater understanding can be achieved by extensive, intensive, and detailed counseling. But since probability itself is a concept that easily escapes our everyday intuition-consider the famous Monte Hall paradox-then it would also be wise to advise patients and potential patients, to not put undue weight on any probabilistic assessment. Such assessments can be of value to the epidemiologist in the investigation of different potential etiologies describing cancer evolution or to the clinical trialist as a way to maximize design efficiency. But to an ordinary individual we cannot anticipate that these assessments will be correctly interpreted.

Controlled amplification in oncology dose-finding trials.

In oncology clinical trials the guiding principle of model-based dose-finding designs for cytotoxic agents is to progress as fast as possible towards, and identify, the dose level most likely to be the MTD. Recent developments with non-cytotoxic agents have broadened the scope of early phase trials to include multiple objectives. The ultimate goal of dose-finding designs in our modern era is to collect the relevant information in the study for final RP2D determination. While some information is collected on dose levels below and in the vicinity of the MTD during the escalation (using conventional tools such as the Continual Reassessment Method for example), designs that include expansion cohorts or backfill patients effectively amplify the information collected on the lower dose levels. This is achieved by allocating patients to dose levels slightly differently during the study in order to take into account the possibility that "less (dose) might be more". The objective of this paper is to study the concept of amplification. Under the heading of controlled amplification we can include dose expansion cohorts and backfill patients among others. We make some general observations by defining these concepts more precisely and study a specific design that exploits the concept of controlled amplification.

Deconstructing the Kaplan-Meier curve: Quantification of treatment effect using the treatment effect process.

In studies of survival and its association with treatment and other prognostic variables, elapsed time alone will often show itself to be among the strongest, if not the strongest, of the predictor variables. Kaplan-Meier curves will show the overall survival of each group and the general differences between groups due to treatment. However, the time-dependent nature of treatment effects is not always immediately transparent from these curves. More sophisticated tools are needed to spotlight the treatment effects. An important tool in this context is the treatment effect process. This tool can be potent in revealing the complex myriad of ways in which treatment can affect survival time. We look at a recently published study in which the outcome was relapse-free survival, and we illustrate how the use of the treatment effect process can provide a much deeper understanding of the relationship between time and treatment in this trial.

Testing for Differences in Survival When Treatment Effects Are Persistent, Decaying, or Delayed.

A statistical test for the presence of treatment effects on survival will be based on a null hypothesis (absence of effects) and an alternative (presence of effects). The null is very simply expressed. The most common alternative, also simply expressed, is that of proportional hazards. For this situation, not only do we have a very powerful test in the log-rank test but also the outcome is readily interpreted. However, many modern treatments fall outside this relatively straightforward paradigm and, as such, have attracted attention from statisticians eager to do their best to avoid losing power as well as to maintain interpretability when the alternative hypothesis is less simple. Examples include trials where the treatment effect decays with time, immunotherapy trials where treatment effects may be slow to manifest themselves as well as the so-called crossing hazards problem. We review some of the solutions that have been proposed to deal with these issues. We pay particular attention to the integrated log-rank test and how it can be combined with the log-rank test itself to obtain powerful tests for these more complex situations.

Stopping rules for phase I clinical trials with dose expansion cohorts.

Many clinical trials incorporate stopping rules to terminate early if the clinical question under study can be answered with a high degree of confidence. While common in later-stage trials, these rules are rarely implemented in dose escalation studies, due in part to the relatively smaller sample size of these designs. However, even with a small sample size, this paper shows that easily implementable stopping rules can terminate dose-escalation early with minimal loss to the accuracy of maximum tolerated dose estimation. These stopping rules are developed when the goal is to identify one or two dose levels, as the maximum tolerated dose and co-maximum tolerated dose. In oncology, this latter goal is frequently considered when the study includes dose-expansion cohorts, which are used to further estimate and compare the safety and efficacy of one or two dose levels. As study protocols do not typically halt accrual between escalation and expansion, early termination is of clinical importance as it either allows for additional patients to be treated as part of the dose expansion cohort to obtain more precise estimates of the study endpoints or allows for an overall reduction in the total sample size.

SEMIPARAMETRIC DOSE FINDING METHODS FOR PARTIALLY ORDERED DRUG COMBINATIONS.

We investigate a statistical framework for Phase I clinical trials that test the safety of two or more agents in combination. For such studies, the traditional assumption of a simple monotonic relation between dose and the probability of an adverse event no longer holds. Nonetheless, the dose toxicity (adverse event) relationship will obey an assumption of partial ordering in that there will be pairs of combinations for which the ordering of the toxicity probabilities is known. Some authors have considered how to best estimate the maximum tolerated dose (a dose providing a rate of toxicity as close as possible to some target rate) in this setting. A related, and equally interesting, problem is to partition the 2-dimensional dose space into two sub-regions: doses with probabilities of toxicity lower and greater than the target. We carry out a detailed investigation of this problem. The theoretical framework for this is the recently presented semiparametric dose finding method. This results in a number of proposals one of which can be viewed as an extension of the Product of Independent beta Priors Escalation method (PIPE). We derive useful asymptotic properties which also apply to the PIPE method when seen as a special case of the more general method given here. Simulation studies provide added confidence concerning the good behaviour of the operating characteristics.

Controlled backfill in oncology dose-finding trials.

The use of backfill in early phase dose-finding trials is a relatively recent practice. It consists of assigning patients to dose levels below the level where the study is at. The main reason for backfilling is to collect additional pharmacokinetic, pharmacodynamic and response data, in order to assess whether a plateau may exist on the dose-efficacy curve. This is a possibility in oncology with molecularly targeted agents or immunotherapy. Recommending for further study a dose level lower than the maximum tolerated dose could be supported in such situations. How to best allocate backfill patients to dose levels is not yet established. In this paper we propose to randomise backfill patients below the dose level where the study is at. A refinement of this would be to stop backfilling to lower dose levels when these show insufficient efficacy compared to higher levels, starting at dose level 1 and repeating this process sequentially. At study completion, data from all patients (both backfill patients and dose-finding patients) is used to estimate the dose-response curve. The fit from a change point model is compared to the fit of a monotonic model to identify a potential plateau. Using simulations, we show that this approach can identify the plateau on the dose-response curve when such a plateau exists, allowing the recommendation of a dose level lower than the maximum tolerated dose for future studies. This contribution provides a methodological framework for backfilling, from the perspective of both design and analysis in early phase oncology trials.

Randomised Phase 1 clinical trials in oncology.

The aims of Phase 1 trials in oncology have broadened considerably from simply demonstrating that the agent/regimen of interest is well tolerated in a relatively heterogeneous patient population to addressing multiple objectives under the heading of early-phase trials and, if possible, obtaining reliable evidence regarding clinical activity to lead to drug approvals via the Accelerated Approval approach or Breakthrough Therapy designation in cases where the tumours are rare, prognosis is poor or where there might be an unmet therapeutic need. Constructing a Phase 1 design that can address multiple objectives within the context of a single trial is not simple. Randomisation can play an important role, but carrying out such randomisation according to the principles of equipoise is a significant challenge in the Phase 1 setting. If the emerging data are not sufficient to definitively address the aims early on, then a proper design can reduce biases, enhance interpretability, and maximise information so that the Phase 1 data can be more compelling. This article outlines objectives and design considerations that need to be adhered to in order to respect ethical and scientific principles required for research in human subjects in early phase clinical trials.

Phase I clinical trials in adoptive T-cell therapies.

We develop three approaches to phase I dose finding designs for engineered T cells in oncology. Our goal is to address a very particular difficulty in this clinical setting: an inability to fully administer the dose allocated to some patients. Current designs can be biased as a result of this incomplete information being ignored or discarded from the analysis. The performance of the three proposed solutions is largely similar, and all offer an advantage over the currently used design. One of the three methods is supported by theoretical study, and we provide some new results on this approach.

Early phase dose-finding trials in virology.

Little has been published in terms of dose-finding methodology in virology. Aside from a few papers focusing on HIV, the considerable progress in dose-finding methodology of the last 25 years has focused almost entirely on oncology. While adverse reactions to cytotoxic drugs may be life threatening, for anti-viral agents we anticipate something different: side effects that provoke the cessation of treatment. This would correspond to treatment failure. On the other hand, success would not be yes/no but would correspond to a range of responses, from small, no more than say 20% reduction in viral load to the complete elimination of the virus. Less than total success matters since this may allow the patient to achieve immune-mediated clearance. The motivation for this article is an upcoming dose-finding trial in chronic norovirus infection. We propose a novel methodology whose goal is twofold: first, to identify the dose that provides the most favorable distribution of treatment outcomes, and, second, to do this in a way that maximizes the treatment benefit for the patients included in the study.

Coherence principles in interval-based dose finding.

This paper studies the notion of coherence in interval-based dose-finding methods. An incoherent decision is either (a) a recommendation to escalate the dose following an observed dose-limiting toxicity or (b) a recommendation to deescalate the dose following a non-dose-limiting toxicity. In a simulated example, we illustrate that the Bayesian optimal interval method and the Keyboard method are not coherent. We generated dose-limiting toxicity outcomes under an assumed set of true probabilities for a trial of n=36 patients in cohorts of size 1, and we counted the number of incoherent dosing decisions that were made throughout this simulated trial. Each of the methods studied resulted in 13/36 (36%) incoherent decisions in the simulated trial. Additionally, for two different target dose-limiting toxicity rates, 20% and 30%, and a sample size of n=30 patients, we randomly generated 100 dose-toxicity curves and tabulated the number of incoherent decisions made by each method in 1000 simulated trials under each curve. For each method studied, the probability of incurring at least one incoherent decision during the conduct of a single trial is greater than 75%. Coherency is an important principle in the conduct of dose-finding trials. Interval-based methods violate this principle for cohorts of size 1 and require additional modifications to overcome this shortcoming. Researchers need to take a closer look at the dose assignment behavior of interval-based methods when using them to plan dose-finding studies.

Consequences of Performing Parallel Dose Finding Trials in Heterogeneous Groups of Patients.

Patient heterogeneity, in which patients can be grouped by risk of toxicity, is a design challenge in early phase dose finding trials. Carrying out independent trials for each group is a readily available approach for dose finding. However, this often leads to dose recommendations that violate the known order of toxicity risk by group, or reversals in dose recommendation. In this manuscript, trials for partially ordered groups are simulated using four approaches: independent parallel trials using the continual reassessment method (CRM), Bayesian optimal interval design, and 3 + 3 methods, as well as CRM for partially ordered groups. Multiple group order structures are considered, allowing for varying amounts of group frailty order information. These simulations find that parallel trials in the presence of partially ordered groups display a high frequency of trials resulting in reversals. Reversals occur when dose recommendations do not follow known order of toxicity risk by group, such as recommending a higher dose level in a group of patients known to have a higher risk of toxicity. CRM for partially ordered groups eliminates the issue of reversals, and simulation results indicate improved frequency of maximum tolerated dose selection as well as treating a greater proportion of trial patients at this dose compared with parallel trials. When information is available on differences in toxicity risk by patient subgroup, methods designed to account for known group ordering should be considered to avoid reversals in dose recommendations and improve operating characteristics.

Phase I Designs that Allow for Uncertainty in the Attribution of Adverse Events.

In determining dose limiting toxicities in Phase I studies, it is necessary to attribute adverse events (AE) to being drug related or not. Such determination is subjective and may introduce bias. In this paper, we develop methods for removing or at least diminishing the impact of this bias on the estimation of the maximum tolerated dose (MTD). The approach we suggest takes into account the subjectivity in the attribution of AE by using model-based dose escalation designs. The results show that gains can be achieved in terms of accuracy by recovering information lost to biases. These biases are a result of ignoring the errors in toxicity attribution.

Faulty BRCA1, BRCA2 genes: how poor is the prognosis?

We take a critical look at the meaning behind the number 87% given to 25-year-old Sophie, a BRCA1 and BRCA2 carrier. Sophie has been told she has an 87% chance of getting breast cancer. She is contemplating a preventive double mastectomy after genetic counseling and her physician's advice. Some 92% of British general practitioners are in favor of prophylactic mastectomy as a treatment option for women similar to Sophie. The treatment decision results, to a very large extent, from the size of the number (87%) alone. The central argument of this study is that physicians, their patients, and the public need a much better understanding on what is meant by probability estimates of 0.87. The figure on its own does not tell us much, and we need to be very cautious in its interpretation. It is important to know that the very same genetic and statistical models, and observed data, resulting in a verdict of an 87% lifetime chance of getting breast cancer, based on BRCA1, BRCA2, and familial information, simultaneously show Sophie to have a greater than 99% chance of surviving beyond the next 5 years cancer free. If she succeeds-the chances are overwhelmingly in her favor-then, given that fact, her chances of surviving a further 5 years are once again greater than 98%. Her chances of not dying due to breast cancer over the next 20 years are greater than 97%, a percentage that changes little if instead of 20 we write the number 30. In a word, although the diagnosis of a faulty BRAC gene may be a disappointment, there is no immediate peril and no need for undue alarm. Sophie, and her primary care providers, can carefully consider her options without feeling that they are under any kind of acute pressure. Whatever the threat, it is not an imminent one.

Dose Transition Pathways: The Missing Link Between Complex Dose-Finding Designs and Simple Decision-Making.

The ever-increasing pace of development of novel therapies mandates efficient methodologies for assessment of their tolerability and activity. Evidence increasingly support the merits of model-based dose-finding designs in identifying the recommended phase II dose compared with conventional rule-based designs such as the 3 + 3 but despite this, their use remains limited. Here, we propose a useful tool, dose transition pathways (DTP), which helps overcome several commonly faced practical and methodologic challenges in the implementation of model-based designs. DTP projects in advance the doses recommended by a model-based design for subsequent patients (stay, escalate, de-escalate, or stop early), using all the accumulated information. After specifying a model with favorable statistical properties, we utilize the DTP to fine-tune the model to tailor it to the trial's specific requirements that reflect important clinical judgments. In particular, it can help to determine how stringent the stopping rules should be if the investigated therapy is too toxic. Its use to design and implement a modified continual reassessment method is illustrated in an acute myeloid leukemia trial. DTP removes the fears of model-based designs as unknown, complex systems and can serve as a handbook, guiding decision-making for each dose update. In the illustrated trial, the seamless, clear transition for each dose recommendation aided the investigators' understanding of the design and facilitated decision-making to enable finer calibration of a tailored model. We advocate the use of the DTP as an integral procedure in the co-development and successful implementation of practical model-based designs by statisticians and investigators. Clin Cancer Res; 23(24); 7440-7. ©2017 AACR.

Accuracy of predictive ability measures for survival models.

One aspect of an analysis of survival data based on the proportional hazards model that has been receiving increasing attention is that of the predictive ability or explained variation of the model. A number of contending measures have been suggested, including one measure, R2 (ß), which has been proposed given its several desirable properties, including its capacity to accommodate time-dependent covariates, a major feature of the model and one that gives rise to great generality. A thorough study of the properties of available measures, including the aforementioned measure, has been carried out recently. In that work, the authors used bootstrap techniques, particularly complex in the setting of censored data, in order to obtain estimates of precision. The motivation of this work is to provide analytical expressions of precision, in particular confidence interval estimates for R2 (ß). We use Taylor series approximations with and without local linearizing transforms. We also consider a very simple expression based on the Fisher's transformation. This latter approach has two great advantages. It is very easy and quick to calculate, and secondly, it can be obtained for any of the methods given in the recent review. A large simulation study is carried out to investigate the properties of the different methods. Finally, three well-known datasets in breast cancer, lymphoma and lung cancer research are given as illustrations. Copyright © 2017 John Wiley & Sons, Ltd.

Sequential monitoring of Phase I dose expansion cohorts.

A relatively recent development in the design of Phase I dose-finding studies is the inclusion of expansion cohort(s), that is, the inclusion of several more patients at a level considered to be the maximum tolerated dose established at the conclusion of the 'pure' Phase I part. Little attention has been given to the additional statistical analysis, including design considerations, that we might wish to consider for this more involved design. For instance, how can we best make use of new information that may confirm or may tend to contradict the estimate of the maximum tolerated dose based on the dose escalation phase. Those patients included during the dose expansion phase may possess different eligibility criteria. During the expansion phase, we will also wish to have an eye on any evidence of efficacy, an aspect that clearly distinguishes such studies from the classical Phase I study. Here, we present a methodology that enables us to continue the monitoring of safety in the dose expansion cohort while simultaneously trying to assess efficacy and, in particular, which disease types may be the most promising to take forward for further study. The most elementary problem is where we only wish to take account of further toxicity information obtained during the dose expansion cohort, and where the initial design was model based or the standard 3+3. More complex set-ups also involve efficacy and the presence of subgroups. Copyright © 2016 John Wiley & Sons, Ltd.