Applying the Estimand Framework to Non­Inferiority Trials.

Most published applications of the estimand framework have focused on superiority trials. However, non-inferiority trials present specific challenges compared to superiority trials. The International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use notes in their addendum on estimands and sensitivity analysis in clinical trials that there may be special considerations to the implementation of estimands in clinical trials with a non-inferiority objective yet provides little guidance. This paper discusses considerations that trial teams should make when defining estimands for a clinical trial with a non-inferiority objective. We discuss how the pre-addendum way of establishing non-inferiority can be embraced by the estimand framework including a discussion of the role of the Per Protocol analysis set. We examine what clinical questions of interest can be formulated in the context of non-inferiority trials and outline why we do not think it is sensible to describe an estimand as 'conservative'. The impact of the estimand framework on key considerations in non-inferiority trials such as whether trials should have more than one primary estimand, the choice of non-inferiority margin, assay sensitivity, switching from non-inferiority to superiority and estimation are discussed. We conclude by providing a list of recommendations, and important considerations for defining estimands for trials with a non-inferiority objective.

Adherence, per-protocol effects, and the estimands framework.

In the statistical literature, treatment effects in clinical trials are frequently described as either ITT or per-protocol effects. The estimand given for the per-protocol effect is the effect in adherers, where adherers are typically defined as adhering to the intervention as specified in the trial protocol. This dichotomy of treatment effects is unhelpful when there are in reality multiple treatment effects that can be of clinical interest and relevance. The terms "per-protocol" and "adherence" are confusing to non-statisticians. Protocols always allow for discontinuation of randomized treatment so participants discontinuing have actually followed the protocol. When rescue or additional medication is available, the effect in adherers could mean the effect regardless of use of these medications or the effect in a counterfactual world where the participant did not take the medication. Adherence can mean continuing to be prescribed a treatment or some arbitrary level of compliance with a medication that has been prescribed. The ICH E9 (R1) estimands framework provides an improved alternative for the description of treatment effects in clinical trials. Identification of important intercurrent events and the strategy used to handle these events is key to determining the treatment effect. When designing a trial, estimands should be properly defined according to this framework. It is time the statistical literature abandoned describing treatment effects as the effect in adherers or the per-protocol effect.

Why estimands are needed to define treatment effects in clinical trials.

BACKGROUND: The estimand for a clinical trial is a precise definition of the treatment effect to be estimated. Traditionally, estimates of treatment effects are based on either an ITT analysis or a per-protocol analysis. However, there are important clinical questions which are not addressed by either of these analyses. For example, consider a trial where patients take a rescue medication. The ITT analysis includes data after use of rescue, while the per-protocol analysis excludes these patients altogether. Neither of these analyses addresses the important question of what the treatment effect would have been if patients did not take rescue medication. MAIN TEXT: Trial estimands provide a broader perspective compared to the limitations of ITT and per-protocol analysis. Trial treatment effects depend on how events occurring after treatment initiation such as use of alternative medication or discontinuation of the intervention are included in the definition. These events can be accounted for in different ways, depending on the clinical question of interest. CONCLUSION: The estimand framework is an important step forward in improving the clarity and transparency of clinical trials. The centrality of estimands to clinical trials is currently not reflected in methods recommended by the Cochrane group or the CONSORT statement, the current standard for reporting clinical trials in medical journals. We encourage revisions to these guidelines.

Estimation of a treatment policy estimand for time to event data using data collected post discontinuation of randomised treatment.

Discontinuation from randomised treatment is a common intercurrent event in clinical trials. When the target estimand uses a treatment policy strategy to deal with this intercurrent event, data after cessation of treatment is relevant to estimate the estimand and all efforts should be made to collect such data. Missing data may nevertheless occur due to participants withdrawing from the study and assumptions regarding the values for data that are missing are required for estimation. A missing-at-random assumption is commonly made in this setting, but it may not always be viewed as appropriate. Another potential approach is to assume missing values are similar to data collected after treatment discontinuation. This idea has been previously proposed in the context of recurrent event data. Here we extend this approach to time-to-event outcomes using the hazard function. We propose imputation models that allow for different hazard rates before and after treatment discontinuation and use the posttreatment discontinuation hazard to impute events for participants with missing follow-up periods due to study withdrawal. The imputation models are fitted as Andersen-Gill models. We illustrate the proposed methods with an example of a clinical trial in patients with chronic obstructive pulmonary disease.

From DREAM to REALITI-A and beyond: Mepolizumab for the treatment of eosinophil-driven diseases.

Effective treatment of inflammatory diseases is often challenging owing to their heterogeneous pathophysiology. Understanding of the underlying disease mechanisms is improving and it is now clear that eosinophils play a complex pathophysiological role in a broad range of type 2 inflammatory diseases. Standard of care for these conditions often still includes oral corticosteroids (OCS) and/or cytotoxic immune therapies, which are associated with debilitating side effects. Selective, biological eosinophil-reducing agents provide treatment options that improve clinical symptoms associated with eosinophilic inflammation and reduce OCS use. Mepolizumab is a humanized monoclonal antibody that binds to and neutralizes interleukin-5, the major cytokine involved in eosinophil proliferation, activation, and survival. Mepolizumab is approved for the treatment of severe eosinophilic asthma, eosinophilic granulomatosis with polyangiitis and hypereosinophilic syndrome. Additionally, the efficacy of add-on mepolizumab has been observed in patients with severe chronic rhinosinusitis with nasal polyposis and chronic obstructive pulmonary disease with an eosinophilic phenotype. Here, we review the development, approval, and real-world effectiveness of mepolizumab for the treatment of patients with severe eosinophilic asthma, from the DREAM to REALITI-A studies, and describe how knowledge from this journey extended to the use of mepolizumab and other biologics across a broad spectrum of eosinophilic diseases.

Why ITT analysis is not always the answer for estimating treatment effects in clinical trials.

For many years there has been a consensus among the Clinical Research community that ITT analysis represents the correct approach for the vast majority of trials. Recent worldwide regulatory guidance for pharmaceutical industry trials has allowed discussion of alternatives to the ITT approach to analysis; different treatment effects can be considered which may be more clinically meaningful and more relevant to patients and prescribers. The key concept is of a trial "estimand", a precise description of the estimated treatment effect. The strategy chosen to account for patients who discontinue treatment or take alternative medications which are not part of the randomised treatment regimen are important determinants of this treatment effect. One strategy to account for these events is treatment policy, which corresponds to an ITT approach. Alternative equally valid strategies address what the treatment effect is if the patient actually takes the treatment or does not use specific alternative medication. There is no single right answer to which strategy is most appropriate, the solution depends on the key clinical question of interest. The estimands framework discussed in the new guidance has been particularly useful in the context of the current COVID-19 pandemic and has clarified what choices are available to account for the impact of COVID-19 on clinical trials. Specifically, an ITT approach addresses a treatment effect that may not be generalisable beyond the current pandemic.

Assessing efficacy in important subgroups in confirmatory trials: An example using Bayesian dynamic borrowing.

Assessment of efficacy in important subgroups - such as those defined by sex, age, race and region - in confirmatory trials is typically performed using separate analysis of the specific subgroup. This ignores relevant information from the complementary subgroup. Bayesian dynamic borrowing uses an informative prior based on analysis of the complementary subgroup and a weak prior distribution centred on a mean of zero to construct a robust mixture prior. This combination of priors allows for dynamic borrowing of prior information; the analysis learns how much of the complementary subgroup prior information to borrow based on the consistency between the subgroup of interest and the complementary subgroup. A tipping point analysis can be carried out to identify how much prior weight needs to be placed on the complementary subgroup component of the robust mixture prior to establish efficacy in the subgroup of interest. An attractive feature of the tipping point analysis is that it enables the evidence from the source subgroup, the evidence from the target subgroup, and the combined evidence to be displayed alongside each other. This method is illustrated with an example trial in severe asthma where efficacy in the adolescent subgroup was assessed using a mixture prior combining an informative prior from the adult data in the same trial with a non-informative prior.

Clinical Development of Mepolizumab for the Treatment of Severe Eosinophilic Asthma: On the Path to Personalized Medicine.

The development of mepolizumab, an anti-IL-5 monoclonal antibody for the treatment of severe eosinophilic asthma, is an example of a clinical development program that evolved over time based on sound, basic scientific principles. Initial clinical data on the effects of mepolizumab on lung function in a general asthmatic population were disappointing. However, it became clear that mepolizumab may be more effective against other clinical endpoints, particularly asthma exacerbations, in patients with more severe disease. Furthermore, a developing understanding of asthma disease pathobiology led to the identification of an appropriate target population and predictive biomarker for mepolizumab treatment: patients with severe eosinophilic asthma and blood eosinophil count. Mepolizumab use provides clinically meaningful benefits in this target population, fulfilling an unmet need. This Clinical Commentary Review describes the clinical development of mepolizumab and details of how this program informed the development of other biologic therapies in patients with severe asthma. This account highlights how a personalized approach toward treatment of patients with severe eosinophilic asthma, supported by a large body of scientific evidence, ultimately led to new and effective treatments and improved patient outcomes.

What matters most? Different stakeholder perspectives on estimands for an invented case study in COPD.

In drug development, we ask ourselves which population, endpoint and treatment comparison should be investigated. In this context, we also debate what matters most to the different stakeholders that are involved in clinical drug development, for example, patients, physicians, regulators and payers. With the publication of draft ICH E9 addendum on estimands in 2017, we now have a common framework and language to discuss such questions in an informed and transparent way. This has led to the estimand discussion being a key element in study development, including design, analysis and interpretation of a treatment effect. At an invited session at the 2018 PSI annual conference, PSI hosted a role-play debate where the aim of the session was to mimic a regulatory and payer scientific advice discussion for a COPD drug. Including role-play views from an industry sponsor, a patient, a regulator and a payer. This paper presents the invented COPD case-study design and considerations relating to appropriate estimands are discussed by each of the stakeholders from their differing viewpoints with the additional inclusion of a technical (academic) perspective. The rationale for each perspective on approaches for handling intercurrent events is presented, with a key emphasis on the application of while-on-treatment and treatment policy estimands in this context. It is increasingly recognised that the treatment effect estimated by the treatment policy approach may not always be of primary clinical interest and may not appropriately communicate to patients the efficacy they can expect if they take the treatment as directed.

Efficacy of add-on mepolizumab in adolescents with severe eosinophilic asthma.

Adolescents (12-17 years of age) with severe eosinophilic asthma experience frequent exacerbations and reduced lung function leading to poor health-related quality of life. Mepolizumab is approved for add-on maintenance therapy in patients with severe eosinophilic asthma ≥ 6 years of age in the EU and ≥ 12 years of age in other regions (including the USA), based on a Phase II/III program demonstrating reduced exacerbation rates with 4-weekly treatment. A total of 34 adolescent patients were recruited across the Phase III mepolizumab trials. Consistent with outcomes in the overall population, there was a reduction in the annual rate of clinically significant exacerbations, along with a reduction in blood eosinophil counts in response to mepolizumab in adolescent patients. The safety profile in adolescent patients was consistent with that seen in the overall population. Data from the Phase III clinical development program provide evidence for comparable efficacy and safety of mepolizumab between adolescents with severe eosinophilic asthma and the overall population. Clinical trial registration DREAM, NCT01000506 [MEA112997]; MENSA, NCT01691521 [MEA115588]; SIRIUS, NCT01691508 [MEA115575]; MUSCA, NCT02281318 [200862]; COSMOS, NCT01842607 [MEA115661].

Disease burden and efficacy of mepolizumab in patients with severe asthma and blood eosinophil counts of ≥150-300 cells/µL.

BACKGROUND: In patients with severe eosinophilic asthma, treatment decisions can be determined by blood eosinophil counts; however, a specific blood eosinophil threshold has not been defined for starting mepolizumab treatment. METHODS: We summarized the disease burden and efficacy of mepolizumab in patients with severe eosinophilic asthma and baseline blood eosinophil counts of ≥150-300 cells/µL and ≥300 cells/µL using data from the mepolizumab clinical development program (DREAM [NCT01000506], MENSA [NCT01691521], SIRIUS [NCT01691508] and MUSCA [NCT02281318]). RESULTS: The morbidity of asthma in patients with baseline blood eosinophil counts ≥150-300 cells/µL was similar to that in patients with blood eosinophil counts ≥300 cells/µL, with similar rates of exacerbations (2.8-3.5 events/year versus 2.8-3.8 events/year, respectively), asthma related emergency room visits, intubations and near fatal events. Use of maintenance oral corticosteroids (OCS) was similar across blood eosinophil count subgroups. Reductions in the rates of clinically significant exacerbations with mepolizumab versus placebo ranged from 27 to 49% in patients with blood eosinophil counts of ≥150-300 cells/µL for DREAM, MENSA and MUSCA. The odds of achieving a reduction in OCS in SIRIUS was 2.03 (95% CI: 0.53, 7.75) versus 1.79 (95% CI: 0.71, 4.52) in patients with blood eosinophil counts ≥150-300 cells/µL and ≥300 cells/µL, respectively. CONCLUSION: There is a high unmet clinical need in patients with blood eosinophil counts ≥150-300 cells/µL, and a clinically meaningful benefit is seen with mepolizumab in this subgroup. Mepolizumab is an efficacious treatment option for patients with blood eosinophil counts ≥150 cells/µL.

Strategies for composite estimands in confirmatory clinical trials: Examples from trials in nasal polyps and steroid reduction.

The draft addendum to the ICH E9 regulatory guideline asks for explicit definition of the treatment effect to be estimated in clinical trials. The draft guideline also introduces the concept of intercurrent events to describe events that occur post-randomisation that may affect efficacy assessment. Composite estimands allow incorporation of intercurrent events in the definition of the endpoint. A common example of an intercurrent event is discontinuation of randomised treatment and use of a composite strategy would assess treatment effect based on a variable that combines the outcome variable of interest with discontinuation of randomised treatment. Use of a composite estimand may avoid the need for imputation which would be required by a treatment policy estimand. The draft guideline gives the example of a binary approach for specifying a composite estimand. When the variable is measured on a non-binary scale, other options are available where the intercurrent event is given an extreme unfavourable value, for example comparison of median values or analysis based on categories of response. This paper reviews approaches to deriving a composite estimand and contrasts the use of this estimand to the treatment policy estimand. The benefits of using each strategy are discussed and examples of the use of the different approaches are given for a clinical trial in nasal polyposis and a steroid reduction trial in severe asthma.

Treatment policy estimands for recurrent event data using data collected after cessation of randomised treatment.

In the past, many clinical trials have withdrawn subjects from the study when they prematurely stopped their randomised treatment and have therefore only collected 'on-treatment' data. Thus, analyses addressing a treatment policy estimand have been restricted to imputing missing data under assumptions drawn from these data only. Many confirmatory trials are now continuing to collect data from subjects in a study even after they have prematurely discontinued study treatment as this event is irrelevant for the purposes of a treatment policy estimand. However, despite efforts to keep subjects in a trial, some will still choose to withdraw. Recent publications for sensitivity analyses of recurrent event data have focused on the reference-based imputation methods commonly applied to continuous outcomes, where imputation for the missing data for one treatment arm is based on the observed outcomes in another arm. However, the existence of data from subjects who have prematurely discontinued treatment but remained in the study has now raised the opportunity to use this 'off-treatment' data to impute the missing data for subjects who withdraw, potentially allowing more plausible assumptions for the missing post-study-withdrawal data than reference-based approaches. In this paper, we introduce a new imputation method for recurrent event data in which the missing post-study-withdrawal event rate for a particular subject is assumed to reflect that observed from subjects during the off-treatment period. The method is illustrated in a trial in chronic obstructive pulmonary disease (COPD) where the primary endpoint was the rate of exacerbations, analysed using a negative binomial model.

Asthma Exacerbations Associated with Lung Function Decline in Patients with Severe Eosinophilic Asthma.

BACKGROUND: Limited data describe the association between the frequency of asthma exacerbations and the decline in lung function in severe asthma. OBJECTIVE: To determine whether asthma exacerbations are associated with enhanced decline in lung function. METHODS: Changes in lung function were analyzed retrospectively using data from the DREAM and MENSA studies of mepolizumab intervention in patients with severe asthma. Patients were either nonsmokers or former smokers. A linear regression model was used to analyze the relationship between the number of exacerbations and decline in FEV1 across treatment groups. RESULTS: In a combined post hoc analysis, 57% (n = 572) of patients had no exacerbations and experienced an improvement in postbronchodilator FEV1 of 143 mL. In contrast, in patients who experienced 3 or more exacerbations, there was a decrease in postbronchodilator FEV1 of 77 mL in the combined analysis. The linear modeling analysis estimated that for each exacerbation seen during the observational period, there was a decrease of 50 mL in FEV1 (P < .001). CONCLUSIONS: A direct relationship between the number of exacerbations in patients with severe eosinophilic asthma and decline in lung function was observed. Repeated exacerbations may be associated with accelerated loss of lung function.

Evaluation of Potential Continuation Rules for Mepolizumab Treatment of Severe Eosinophilic Asthma.

BACKGROUND: Mepolizumab significantly reduces exacerbations in patients with severe eosinophilic asthma. The early identification of patients likely to receive long-term benefit from treatment could ensure effective resource allocation. OBJECTIVE: To assess potential continuation rules for mepolizumab in addition to initiation criteria defined as 2 or more exacerbations in the previous year and blood eosinophil counts of 150 cells/µL or more at initiation or 300 cells/µL or more in the previous year. METHODS: This post hoc analysis included data from 2 randomized, double-blind, placebo-controlled studies (NCT01000506 and NCT01691521) of mepolizumab in patients with severe eosinophilic asthma (N = 1,192). Rules based on blood eosinophils, physician-rated response to treatment, FEV1, Asthma Control Questionnaire (ACQ-5) score, and exacerbation reduction were assessed at week 16. To assess these rules, 2 key metrics accounting for the effects observed in the placebo arm were developed. RESULTS: Patients not meeting continuation rules based on physician-rated response, FEV1, and the ACQ-5 score still derived long-term benefit from mepolizumab. Nearly all patients failing to reduce blood eosinophils had counts of 150 cells/µL or less at baseline. For exacerbations, assessment after 16 weeks was potentially premature for predicting future exacerbations. CONCLUSION: There was no evidence of a reliable physician-rated response, ACQ-5 score, or lung function-based continuation rule. The added value of changes in blood eosinophils at week 16 over baseline was marginal. Initiation criteria for mepolizumab treatment provide the best method for assessing patient benefit from mepolizumab treatment, and treatment continuation should be reviewed on the basis of a predefined reduction in long-term exacerbation frequency and/or oral corticosteroid dose.

Biomarkers for severe eosinophilic asthma.

The last decade has seen the approval of several new biologics for the treatment of severe asthma-targeting specific endotypes and phenotypes. This review will examine how evidence generated from the mepolizumab clinical development program showed that blood eosinophil counts, rather than sputum or tissue eosinophil counts, evolved as a pharmacodynamic and predictive biomarker for the efficacy of treatment with mepolizumab in patients with severe eosinophilic asthma. Based on the available evidence and combined with clinical judgement, a baseline blood eosinophil threshold of 150 cells/µL or greater or a historical blood eosinophil threshold of 300 cells/µL or greater will allow selection of patients with severe eosinophilic asthma who are most likely to achieve clinically significant reductions in the rate of exacerbations with mepolizumab treatment.

Estimands: discussion points from the PSI estimands and sensitivity expert group.

ICH E9 Statistical Principles for Clinical Trials was issued in 1998. In October 2014, an addendum to ICH E9 was proposed relating to estimands and sensitivity analyses. In preparation for the release of the addendum, Statisticians in the Pharmaceutical Industry held a 1-day expert group meeting in February 2015. Topics debated included definition, development, implementation, education and communication challenges associated with estimands and sensitivity analyses. The topic of estimands is an important and relatively new one in clinical development. A clear message from the meeting was that estimands bridge the gap between study objectives and statistical methods. When defining estimands, an iterative process linking trial objectives, estimands, trial design, statistical and sensitivity analysis needs to be established. Each objective should have at least one distinct estimand, supported by sensitivity analyses. Because clinical trials are multi-faceted and expensive, it is unrealistic to restrict a study to a single objective and associated estimand. The actual set of estimands and sensitivity analyses for a study will depend on the study objectives, the disease setting and the needs of the various stakeholders. Copyright © 2016 John Wiley & Sons, Ltd.