Adjusting for bias in the mean for primary and secondary outcomes when trials are in sequence.

When designing a clinical trial, one key aspect of the design is the sample size calculation. The sample size calculation tends to rely on a target or expected difference. The expected difference can be based on the observed data from previous studies, which results in bias. It has been reported that large treatment effects observed in trials are often not replicated in subsequent trials. If these values are used to design subsequent studies, the sample sizes may be biased which results in an unethical study. Regression to the mean (RTM) is one explanation for this. If only health technologies which meet a particular continuation criterion (such as p<0.05 in the first study) are progressed to a second confirmatory trial, it is highly likely that the observed effect in the second trial will be lower than that observed in the first trial. It will be shown how when moving from one trial to the next, a truncated normal distribution is inherently imposed on the first study. This results in a lower observed effect size in the second trial. A simple adjustment method is proposed based on the mathematical properties of the truncated normal distribution. This adjustment method was confirmed using simulations in R and compared with other previous adjustments. The method can be applied to the observed effect in a trial, which is being used in the design of a second confirmatory trial, resulting in a more stable estimate for the 'true' treatment effect. The adjustment accounts for the bias in the primary and secondary endpoints in the first trial with the bias being affected by the power of that study. Tables of results have been provided to aid implementation, along with a worked example. In summary, there is a bias introduced when the point estimate from one trial is used to assist the design of a second trial. It is recommended that any observed point estimates be used with caution and the adjustment method developed in this article be implemented to significantly reduce this bias.

Practical help for specifying the target difference in sample size calculations for RCTs: the DELTA2 five-stage study, including a workshop.

BACKGROUND: The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the 'target difference' and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups. OBJECTIVE: The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives. METHODS: The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document. RESULTS: Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered. CONCLUSIONS: Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items. FUNDING: Funded by the Medical Research Council (MRC) UK and the National Institute for Health Research as part of the MRC-National Institute for Health Research Methodology Research programme.

This Difference ELicitation in TriAls² (DELTA²) advice and recommendations document aims to help researchers choose the 'target difference' in a type of research study called a randomised controlled trial. The number of people needed to be involved in a study ­ the sample size ­ is usually based on a calculation aimed to ensure that the difference in benefit between treatments is likely to be detected. The calculation also accounts for the risk of a false-positive finding. No more patients than necessary should be involved. Choosing a 'target difference' is an important step in calculating the sample size. The target difference is defined as the amount of difference in the participants' response to the treatments that we wish to detect. It is probably the most important piece of information used in the sample size calculation. How we decide what the target difference should be depends on various factors. One key decision to make is how we should measure the benefits that treatments offer. For example, if we are evaluating a treatment for high blood pressure, the obvious thing to focus on would be blood pressure. We could then proceed to consider what an important difference in blood pressure between treatments would be, based on experts' views or evidence from previous research studies. This document seeks to provide assistance to researchers on how to choose the target difference when designing a trial. It also provides advice to help them clearly present what was done and why, when writing up the study proposal or reporting the study's findings. The document is also intended to be read by those who decide whether or not a proposed study should be funded. Clarifying a study's aim and getting a sensible sample size is important. It can affect not only those involved in the study, but also future patients who will receive treatment.

Correction to: Choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial - the development of the DELTA2 guidance.

Following publication of the original article [1], we have been notified of a few mistakes.

DELTA2 guidance on choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial.

BACKGROUND: A key step in the design of a RCT is the estimation of the number of participants needed in the study. The most common approach is to specify a target difference between the treatments for the primary outcome and then calculate the required sample size. The sample size is chosen to ensure that the trial will have a high probability (adequate statistical power) of detecting a target difference between the treatments should one exist. The sample size has many implications for the conduct and interpretation of the study. Despite the critical role that the target difference has in the design of a RCT, the way in which it is determined has received little attention. In this article, we summarise the key considerations and messages from new guidance for researchers and funders on specifying the target difference, and undertaking and reporting a RCT sample size calculation. This article on choosing the target difference for a randomised controlled trial (RCT) and undertaking and reporting the sample size calculation has been dual published in the BMJ and BMC Trials journals METHODS: The DELTA2 (Difference ELicitation in TriAls) project comprised five major components: systematic literature reviews of recent methodological developments (stage 1) and existing funder guidance (stage 2); a Delphi study (stage 3); a two-day consensus meeting bringing together researchers, funders and patient representatives (stage 4); and the preparation and dissemination of a guidance document (stage 5). RESULTS AND DISCUSSION: The key messages from the DELTA2 guidance on determining the target difference and sample size calculation for a randomised caontrolled trial are presented. Recommendations for the subsequent reporting of the sample size calculation are also provided.

A study of target effect sizes in randomised controlled trials published in the Health Technology Assessment journal.

BACKGROUND: When designing a randomised controlled trial (RCT), an important consideration is the sample size required. This is calculated from several components; one of which is the target difference. This study aims to review the currently reported methods of elicitation of the target difference as well as to quantify the target differences used in Health Technology Assessment (HTA)-funded trials. METHODS: Trials were identified from the National Institute of Health Research Health Technology Assessment journal. A total of 177 RCTs published between 2006 and 2016 were assessed for eligibility. Eligibility was established by the design of the trial and the quality of data available. The trial designs were parallel-group, superiority RCTs with a continuous primary endpoint. Data were extracted and the standardised anticipated and observed effect size estimates were calculated. Exclusion criteria was based on trials not providing enough detail in the sample size calculation and results, and trials not being of parallel-group, superiority design. RESULTS: A total of 107 RCTs were included in the study from 102 reports. The most commonly reported method for effect size derivation was a review of evidence and use of previous research (52.3%). This was common across all clinical areas. The median standardised target effect size was 0.30 (interquartile range: 0.20-0.38), with the median standardised observed effect size 0.11 (IQR 0.05-0.29). The maximum anticipated and observed effect sizes were 0.76 and 1.18, respectively. Only two trials had anticipated target values above 0.60. CONCLUSION: The most commonly reported method of elicitation of the target effect size is previous published research. The average target effect size was 0.3. A clear distinction between the target difference and the minimum clinically important difference is recommended when designing a trial. Transparent explanation of target difference elicitation is advised, with multiple methods including a review of evidence and opinion-seeking advised as the more optimal methods for effect size quantification.

Choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial - the development of the DELTA2 guidance.

BACKGROUND: A key step in the design of a randomised controlled trial is the estimation of the number of participants needed. The most common approach is to specify a target difference in the primary outcome between the randomised groups and then estimate the corresponding sample size. The sample size is chosen to provide reassurance that the trial will have high statistical power to detect the target difference at the planned statistical significance level. Alternative approaches are also available, though most still require specification of a target difference. The sample size has many implications for the conduct of the study, as well as incurring scientific and ethical aspects. Despite the critical role of the target difference for the primary outcome in the design of a randomised controlled trial (RCT), the manner in which it is determined has received little attention. This article reports the development of the DELTA2 guidance on the specification and reporting of the target difference for the primary outcome in a sample size calculation for a RCT. METHODS: The DELTA2 (Difference ELicitation in TriAls) project has five components comprising systematic literature reviews of recent methodological developments (stage 1) and existing funder guidance (stage 2), a Delphi study (stage 3), a 2-day consensus meeting bringing together researchers, funders and patient representatives (stage 4), and the preparation and dissemination of a guidance document (stage 5). RESULTS: The project started in April 2016. The literature search identified 28 articles of methodological developments relevant to a method for specifying a target difference. A Delphi study involving 69 participants, along with a 2-day consensus meeting were conducted. In addition, further engagement sessions were held at two international conferences. The main guidance text was finalised on April 18, 2018, after revision informed by feedback gathered from stages 2 and 3 and from funder representatives. DISCUSSION: The DELTA2 Delphi study identified a number of areas (such as practical recommendations and examples, greater coverage of different trial designs and statistical approaches) of particular interest amongst stakeholders which new guidance was desired to meet. New relevant references were identified by the review. Such findings influenced the scope, drafting and revision of the guidance. While not all suggestions could be accommodated, it is hoped that the process has led to a more useful and practical document.

Choosing the target difference ('effect size') for a randomised controlled trial - DELTA2 guidance protocol.

BACKGROUND: A key step in the design of a randomised controlled trial (RCT) is the estimation of the number of participants needed. By far the most common approach is to specify a target difference and then estimate the corresponding sample size; this sample size is chosen to provide reassurance that the trial will have high statistical power to detect such a difference between the randomised groups (at the planned statistical significance level). The sample size has many implications for the conduct of the study, as well as carrying scientific and ethical aspects to its choice. Despite the critical role of the target difference for the primary outcome in the design of an RCT, the manner in which it is determined has received little attention. This article reports the protocol of the Difference ELicitation in TriAls (DELTA2) project, which will produce guidance on the specification and reporting of the target difference for the primary outcome in a sample size calculation for RCTs. METHODS/DESIGN: The DELTA2 project has five components: systematic literature reviews of recent methodological developments (stage 1) and existing funder guidance (stage 2); a Delphi study (stage 3); a 2-day consensus meeting bringing together researchers, funders and patient representatives, as well as one-off engagement sessions at relevant stakeholder meetings (stage 4); and the preparation and dissemination of a guidance document (stage 5). DISCUSSION: Specification of the target difference for the primary outcome is a key component of the design of an RCT. There is a need for better guidance for researchers and funders regarding specification and reporting of this aspect of trial design. The aim of this project is to produce consensus based guidance for researchers and funders.