The analysis of continuous data from n-of-1 trials using paired cycles: a simple tutorial.

N-of-1 trials are defined and the popular paired cycle design is introduced, together with an explanation as to how suitable sequences may be constructed.Various approaches to analysing such trials are explained and illustrated using a simulated data set. It is explained how choosing an appropriate analysis depends on the question one wishes to answer. It is also shown that for a given question, various equivalent approaches to analysis can be found, a fact which may be exploited to expand the possible software routines that may be used.Sets of N-of-1 trials are analogous to sets of parallel group trials. This means that software for carrying out meta-analysis can be used to combine results from N-of-1 trials. In doing so, it is necessary to make one important change, however. Because degrees of freedom for estimating variances for individual subjects will be scarce, it is advisable to estimate local standard errors using pooled variances. How this may be done is explained and fixed and random effect approaches to combining results are illustrated.

Statistics, philosophy, and health: the SMAC 2021 webconference.

SMAC 2021 was a webconference organized in June 2021. The aim of this conference was to bring together data scientists, (bio)statisticians, philosophers, and any person interested in the questions of causality and Bayesian statistics, ranging from technical to philosophical aspects. This webconference consisted of keynote speakers and contributed speakers, and closed with a round-table organized in an unusual fashion. Indeed, organisers asked world renowned scientists to prepare two videos: a short video presenting a question of interest to them and a longer one presenting their point of view on the question. The first video served as a "teaser" for the conference and the second were presented during the conference as an introduction to the round-table. These videos and this round-table generated original scientific insights and discussion worthy of being shared with the community which we do by means of this paper.

Student and the Lanarkshire milk experiment.

A detailed examination of the 1930 Lanarkshire Milk Experiment (LME) by the famous statistician William Sealy Gossett ("Student"), which appeared in Biometrika in 1931, is re-examined from a more modern perspective. The LME had a complicated design whereby 67 schools in Lanarkshire were allocated to receive either raw or pasteurised milk but pupils within the schools were allocated to either receive milk or to act as controls. Student's criticisms are considered in detail and examined in terms of subsequent developments on the design and analysis of experiments, in particular as regards appropriate estimation of standard errors of treatment estimates when an incomplete blocks structure has been used. An analogy with a more modern trial in osteoarthritis is made. Suggestions are made as to how analysis might proceed if the original data were available. Some lessons for observational studies in epidemiology are drawn and it is speculated that hidden clustering structures might be an explanation as to why results may vary from observational study to observational study by more than conventionally calculated standard errors might suggest.

On the relevance of prognostic information for clinical trials: A theoretical quantification.

The question of how individual patient data from cohort studies or historical clinical trials can be leveraged for designing more powerful, or smaller yet equally powerful, clinical trials becomes increasingly important in the era of digitalization. Today, the traditional statistical analyses approaches may seem questionable to practitioners in light of ubiquitous historical prognostic information. Several methodological developments aim at incorporating historical information in the design and analysis of future clinical trials, most importantly Bayesian information borrowing, propensity score methods, stratification, and covariate adjustment. Adjusting the analysis with respect to a prognostic score, which was obtained from some model applied to historical data, received renewed interest from a machine learning perspective, and we study the potential of this approach for randomized clinical trials. In an idealized situation of a normal outcome in a two-arm trial with 1:1 allocation, we derive a simple sample size reduction formula as a function of two criteria characterizing the prognostic score: (1) the coefficient of determination R2 on historical data and (2) the correlation ρ between the estimated and the true unknown prognostic scores. While maintaining the same power, the original total sample size n planned for the unadjusted analysis reduces to ( 1 - R 2 ρ 2 ) × n $(1 - R^2 \rho ^2) \times n$ in an adjusted analysis. Robustness in less ideal situations was assessed empirically. We conclude that there is potential for substantially more powerful or smaller trials, but only when prognostic scores can be accurately estimated.

Viewpoint: Do not resurrect the two-stage procedure.

In 1989, Peter Freeman published a paper that challenged a commonly accepted approach for analyzing cross-over trials, the so-called two-stage procedure. Freeman himself recommended using the Bayesian approach of Andy Grieve. The flaws Freeman exposed were serious and led many statisticians to conclude that the procedure was unacceptable. Unfortunately, more than 30 years later, one still encounters its use. This note explains, using a simple simulation, why the two-stage procedure is, indeed, as Freeman showed unacceptable and should not be used.

The design and analysis of vaccine trials for COVID-19 for the purpose of estimating efficacy.

After a preliminary explanation as to how I came to know Andy Grieve and some remarks about his career and mine and how they have intersected, I consider the design and analysis of trials of vaccines for COVID-19 for the purpose of estimating efficacy. Five large trials, run by the sponsors Pfizer/BioNTech, AstraZeneca/Oxford University, Moderna, Novavax and J&J Janssen are considered briefly. Frequentist approaches to analysis were used for four of the trials but Pfizer/BioNTech nominated a Bayesian approach. The design and analysis of this trial is considered in some detail, in particular as regards the choice of prior distribution. I conclude by drawing some general lessons.

Giving and taking: ethical treatment assignment in controlled trials.

The current version of the Declaration of Helsinki states that 'the benefits, risks, burdens and effectiveness of a new intervention must be tested against those of the best current proven intervention(s) … '. This wording implies that it is acceptable for patients to be assigned to receive an unproven new intervention and to be denied a best current proven intervention. We assert that patients being invited to participate in controlled trials cannot, ethically, be expected to forego proven beneficial forms of care. Patients being treated in controlled trials should not knowingly be disadvantaged compared with similar patients being treated in usual clinical care, where they have access to beneficial care. In this article, we have tried to separate for discussion 'the withholding of effective care from trial participants', 'informed consent to treatment', 'blinding' and 'use of placebos'.

The statistical properties of RCTs and a proposal for shrinkage.

We abstract the concept of a randomized controlled trial as a triple (ß,b,s) , where ß is the primary efficacy parameter, b the estimate, and s the standard error ( s>0 ). If the parameter ß is either a difference of means, a log odds ratio or a log hazard ratio, then it is reasonable to assume that b is unbiased and normally distributed. This then allows us to estimate the joint distribution of the z-value z=b/s and the signal-to-noise ratio SNR=ß/s from a sample of pairs (bi,si) . We have collected 23 551 such pairs from the Cochrane database. We note that there are many statistical quantities that depend on (ß,b,s) only through the pair (z,SNR) . We start by determining the estimated distribution of the achieved power. In particular, we estimate the median achieved power to be only 13%. We also consider the exaggeration ratio which is the factor by which the magnitude of ß is overestimated. We find that if the estimate is just significant at the 5% level, we would expect it to overestimate the true effect by a factor of 1.7. This exaggeration is sometimes referred to as the winner's curse and it is undoubtedly to a considerable extent responsible for disappointing replication results. For this reason, we believe it is important to shrink the unbiased estimator, and we propose a method for doing so. We show that our shrinkage estimator successfully addresses the exaggeration. As an example, we re-analyze the ANDROMEDA-SHOCK trial.

Implementing Historical Controls in Oncology Trials.

Drug development in oncology has broadened from mainly considering randomized clinical trials to also including single-arm trials tailored for very specific subtypes of cancer. They often use historical controls, and this article discusses benefits and risks of this paradigm and provide various regulatory and statistical considerations. While leveraging the information brought by historical controls could potentially shorten development time and reduce the number of patients enrolled, a careful selection of the past studies, a prespecified statistical analysis accounting for the heterogeneity between studies, and early engagement with regulators will be key to success. Although both the European Medicines Agency and the U.S. Food and Drug Administration have already approved medicines based on nonrandomized experiments, the evidentiary package can be perceived as less comprehensive than randomized experiments. Use of historical controls, therefore, is better suited for cases of high unmet clinical need, where the disease course is well characterized and the primary endpoint is objective. IMPLICATIONS FOR PRACTICE: Incorporating historical data in single-arm oncology trials has the potential to accelerate drug development and to reduce the number of patients enrolled, compared with standard randomized controlled clinical trials. Given the lack of blinding and randomization, such an approach is better suited for cases of high unmet clinical need and/or difficult experimental situations, in which the trajectory of the disease is well characterized and the endpoint can be measured objectively. Careful pre-specification and selection of the historical data, matching of the patient characteristics with the concurrent trial data, and innovative statistical methodologies accounting for between-study variation will be needed. Early engagement with regulators (e.g., via Scientific Advice) is highly recommended.

Clustered allocation as a way of understanding historical controls: Components of variation and regulatory considerations.

There has been increasing interest in recent years in the possibility of increasing the efficiency of clinical trials by using historical controls. There has been a general recognition that in replacing concurrent by historical controls, the potential for bias is serious and requires some down-weighting to the apparent amount of historical information available. However, such approaches have generally assumed that what is required is some modification to the standard inferential model offered by the parallel group trial. In our opinion, the correct starting point that requires modification is a trial in which treatments are allocated to clusters. This immediately shows that the amount of information available is governed not just by the number of historical patients but also by the number of centres and of historical studies. Furthermore, once one accepts that external patients may be used as controls, this raises the issue as to which patients should be used. Thus, abandoning concurrent control has implications for many aspects of design and analysis of trials, including (a) identification, pre-specification and agreement on a suitable historical dataset; (b) an agreed, enforceable and checkable plan for recruiting the experimental arm; (c) a finalised analysis plan prior to beginning the trial and (d) use of a hierarchical model with sufficient complexity. We discuss these issues and suggest approaches to design and analysis making extensive reference to the partially randomised Therapeutic Arthritis Research and Gastrointestinal Event Trial study. We also compare some Bayesian and frequentist approaches and provide some important regulatory considerations. We conclude that effective use of historical data will require considerable circumspection and discipline.

Demonstrating Heterogeneity of Treatment Effects Among Patients: An Overlooked but Important Step Toward Precision Medicine.

Although heterogeneity in the observed outcomes in clinical trials is often assumed to reflect a true heterogeneous response, it could actually be due to random variability. This retrospective analysis of four randomized, double-blind, placebo-controlled multiperiod (i.e., episode) crossover trials of fentanyl for breakthrough cancer pain illustrates the use of multiperiod crossover trials to examine heterogeneity of treatment response. A mixed-effects model, including fixed effects for treatment and episode and random effects for patient and treatment-by-patient interaction, was used to assess the heterogeneity in patients' responses to treatment during each episode. A significant treatment-by-patient interaction was found for three of four trials (P < 0.05), suggesting heterogeneity of the effect of fentanyl among different patients in each trial. Similar analyses in other therapeutic areas could identify conditions and therapies that should be investigated further for predictors of treatment response in efforts to maximize the efficiency of developing precision medicine strategies.

Random main effects of treatment: A case study with a network meta-analysis.

If the number of treatments in a network meta-analysis is large, it may be possible and useful to model the main effect of treatment as random, that is to say as random realizations from a normal distribution of possible treatment effects. This then constitutes a third sort of random effect that may be considered in connection with such analyses. The first and most common models treatment-by-trial interaction as being random and the second, rather rarer, models the main effects of trial as being random and thus permits the recovery of intertrial information. Taking the example of a network meta-analysis of 44 similar treatments in 10 trials, we illustrate how a hierarchical approach to modeling a random main effect of treatment can be used to produce shrunk (toward the overall mean) estimates of effects for individual treatments. As a related problem, we also consider the issue of using a random-effect model for the within-trial variances from trial to trial. We provide a number of possible graphical representations of the results and discuss the advantages and disadvantages of such an approach.