Covariate adjustment in randomized controlled trials: General concepts and practical considerations.

There has been a growing interest in covariate adjustment in the analysis of randomized controlled trials in past years. For instance, the US Food and Drug Administration recently issued guidance that emphasizes the importance of distinguishing between conditional and marginal treatment effects. Although these effects may sometimes coincide in the context of linear models, this is not typically the case in other settings, and this distinction is often overlooked in clinical trial practice. Considering these developments, this article provides a review of when and how to use covariate adjustment to enhance precision in randomized controlled trials. We describe the differences between conditional and marginal estimands and stress the necessity of aligning statistical analysis methods with the chosen estimand. In addition, we highlight the potential misalignment of commonly used methods in estimating marginal treatment effects. We hereby advocate for the use of the standardization approach, as it can improve efficiency by leveraging the information contained in baseline covariates while remaining robust to model misspecification. Finally, we present practical considerations that have arisen in our respective consultations to further clarify the advantages and limitations of covariate adjustment.

Decentralized Clinical Trials: Scientific Considerations Through the Lens of the Estimand Framework.

While industry and regulators' interest in decentralized clinical trials (DCTs) is long-standing, the Covid-19 pandemic accelerated and broadened the adoption and experience with these trials. The key idea in decentralization is bringing the clinical trial design, typically on-site, closer to the patient's experience (on-site or off-site). Thus, potential benefits of DCTs include reducing the burden of participation in trials, broadening access to a more diverse population, or using innovative endpoints collected off-site. This paper helps researchers to carefully evaluate the added value and the implications of DCTs beyond the operational aspects of their implementation. The proposed approach is to use the ICH E9(R1) estimand framework to guide the strategic decisions around each decentralization component. Furthermore, the framework can guide the process for clinical trialists to systematically consider the implications of decentralization, in turn, for each attribute of the estimand. We illustrate the use of this approach with a fully DCT case study and show that the proposed systematic process can uncover the scientific opportunities, assumptions, and potential risks associated with a possible use of decentralization components in the design of a trial. This process can also highlight the benefits of specifying estimand attributes in a granular way. Thus, we demonstrate that bringing a decentralization component into the design will not only impact estimators and estimation but can also correspond to addressing more granular questions, thereby uncovering new target estimands.

Distribution-free hyperrectangular tolerance regions for setting multivariate reference regions in laboratory medicine.

Reference regions are important in laboratory medicine to interpret the test results of patients, and usually given by tolerance regions. Tolerance regions of p ( ≥ 2 ) $$ p\;\left(\ge 2\right) $$ dimensions are highly desirable when the test results contains p $$ p $$ outcome measures. Nonparametric hyperrectangular tolerance regions are attractive in real problems due to their robustness with respect to the underlying distribution of the measurements and ease of intepretation, and methods to construct them have been recently provided by Young and Mathew [Stat Methods Med Res. 2020;29:3569-3585]. However, their validity is supported by a simulation study only. In this paper, nonparametric hyperrectangular tolerance regions are constructed by using Tukey's [Ann Math Stat. 1947;18:529-539; Ann Math Stat. 1948;19:30-39] elegant results of equivalence blocks. The validity of these new tolerance regions is proven mathematically in [Ann Math Stat. 1947;18:529-539; Ann Math Stat. 1948;19:30-39] under the only assumption that the underlying distribution of the measurements is continuous. The methodology is applied to analyze the kidney function problem considered in Young and Mathew [Stat Methods Med Res. 2020;29:3569-3585].

Confidence sets for a level set in linear regression.

Regression modeling is the workhorse of statistics and there is a vast literature on estimation of the regression function. It has been realized in recent years that in regression analysis the ultimate aim may be the estimation of a level set of the regression function, ie, the set of covariate values for which the regression function exceeds a predefined level, instead of the estimation of the regression function itself. The published work on estimation of the level set has thus far focused mainly on nonparametric regression, especially on point estimation. In this article, the construction of confidence sets for the level set of linear regression is considered. In particular, 1 - α $$ 1-\alpha $$ level upper, lower and two-sided confidence sets are constructed for the normal-error linear regression. It is shown that these confidence sets can be easily constructed from the corresponding 1 - α $$ 1-\alpha $$ level simultaneous confidence bands. It is also pointed out that the construction method is readily applicable to other parametric regression models where the mean response depends on a linear predictor through a monotonic link function, which include generalized linear models, linear mixed models and generalized linear mixed models. Therefore, the method proposed in this article is widely applicable. Simulation studies with both linear and generalized linear models are conducted to assess the method and real examples are used to illustrate the method.

Optimal test procedures for multiple hypotheses controlling the familywise expected loss.

We consider the problem of testing multiple null hypotheses, where a decision to reject or retain must be made for each one and embedding incorrect decisions into a real-life context may inflict different losses. We argue that traditional methods controlling the Type I error rate may be too restrictive in this situation and that the standard familywise error rate may not be appropriate. Using a decision-theoretic approach, we define suitable loss functions for a given decision rule, where incorrect decisions can be treated unequally by assigning different loss values. Taking expectation with respect to the sampling distribution of the data allows us to control the familywise expected loss instead of the conventional familywise error rate. Different loss functions can be adopted, and we search for decision rules that satisfy certain optimality criteria within a broad class of decision rules for which the expected loss is bounded by a fixed threshold under any parameter configuration. We illustrate the methods with the problem of establishing efficacy of a new medicinal treatment in non-overlapping subgroups of patients.

Generating the Right Evidence at the Right Time: Principles of a New Class of Flexible Augmented Clinical Trial Designs.

To support informed decision making, clear descriptions of the beneficial and harmful effects of a treatment are needed by various stakeholders. The current paradigm is to generate evidence sequentially through different experiments. However, data generated later, perhaps through observational studies, can be difficult to compare with earlier randomized trial data, resulting in confusion in understanding and interpretation of treatment effects. Moreover, the scientific questions these later experiments can serve to answer often remain vague. We propose Flexible Augmented Clinical Trial for Improved eVidence gEneration (FACTIVE), a new class of study designs enabling flexible augmentation of confirmatory randomized controlled trials with concurrent and close-to-real-world elements. Our starting point is to use clearly defined objectives for evidence generation, which are formulated through early discussion with health technology assessment (HTA) bodies and are additional to regulatory requirements for authorization of a new treatment. These enabling designs facilitate estimation of certain well-defined treatment effects in the confirmatory part and other complementary treatment effects in a concurrent real-world part. Each stakeholder should use the evidence that is relevant within their own decision-making framework. High quality data are generated under one single protocol and the use of randomization ensures rigorous statistical inference and interpretation within and between the different parts of the experiment. Evidence for the decision making of HTA bodies could be available earlier than is currently the case.

Missing data imputation using utility-based regression and sampling approaches.

Data are often missing not at random (MNAR) in scientific experiments. We treat the MNAR problem as an imbalanced learning task. Standard predictive error measures of regression (e.g., mean squared error) are not suitable for imbalanced learning problems, such as in clinical trials where extreme values tend to be MNAR. We investigate hybrid imbalanced learning approaches that combine utility-based regression (UBR) with synthetic minority oversampling technique for regression (SMOTER) in cross-sectional trial settings. UBR optimizes the product of the conditional probability density (estimated by quantile regression forests) and a utility function which takes the relevance of the target variable value and the prediction error into account. SMOTER oversamples the relevant rare cases. Simulations show that the proposed method provides plausible predictions and reduces the bias for realistic missing data scenarios when compared with standard approaches like random forests and multiple imputation (systematic bias is observed in those methods, i.e., a tendency to underestimate the mean and standard deviation given the presence of MNAR in the area of high values of the target variable). The proposed method is implemented in a real dataset from an antidepressant clinical trial, and similar pattern of the systematic bias from commonly used methods is observed in the real data compare to the proposed method. Therefore, we encourage the integration of utility-based learning strategies for handling of missing data in the analysis of clinical trials.

Rationale for the update algorithm of the graphical approach to sequentially rejective multiple test procedures.

The graphical approach by Bretz et al. is a convenient tool to construct, visualize and perform multiple test procedures that are tailored to structured families of hypotheses while controlling the familywise error rate. A critical step is to update the transition weights following a pre-specified algorithm. In their original publication, however, the authors did not provide a detailed rationale for the update formula. This paper closes the gap and provides three alternative arguments for the update of the transition weights of the graphical approach. It is a legacy of the first author, based on an unpublished technical report from 2014, and after his untimely death reconstructed by the other two authors as a tribute to Willi Maurer's collaboration with Andy Grieve and contributions to biostatistics over many years.

Combined statistical decision limits based on two GH-2000 scores for the detection of growth hormone misuse.

The growth hormone-2000 biomarker method, based on the measurements of insulin-like growth factor-I and the amino-terminal pro-peptide of type III collagen, has been developed as a powerful technique for the detection of growth hormone misuse by athletes. Insulin-like growth factor-I and amino-terminal pro-peptide of type III collagen are combined in gender-specific formulas to create the growth hormone-2000 score, which is used to determine whether growth hormone has been administered. To comply with World Anti-Doping Agency regulations, each analyte must be measured by two methods. Insulin-like growth factor-I and amino-terminal pro-peptide of type III collagen can be measured by a number of approved methods, each leading to its own growth hormone-2000 score. Single decision limits for each growth hormone-2000 score have been introduced and developed by Bassett, Erotokritou-Mulligan, Holt, Böhning and their co-authors in a series of papers. These have been incorporated into the guidelines of the World Anti-Doping Agency. A joint decision limit was constructed based on the sample correlation between the two growth hormone-2000 scores generated from an available sample to increase the sensitivity of the biomarker method. This paper takes this idea further into a fully developed statistical approach. It constructs combined decision limits when two growth hormone-2000 scores from different assay combinations are used to decide whether an athlete has been misusing growth hormone. The combined decision limits are directly related to tolerance regions and constructed using a Bayesian approach. It is also shown to have highly satisfactory frequentist properties. The new approach meets the required false-positive rate with a pre-specified level of certainty.

[Multiple Endpoints in Oncology Clinical Trials].

Like most complex(or multifactorial)diseases, cancer results not from a single factor, but rather from the interaction of multiple genes and environmental factors. Thus patients can experience different signs and symptoms that reflect more than one consequence of suffering the disease. When evaluating the effects of new treatments in cancer clinical trials, the multidimensional assessment using multiple outcomes to measure improvements in the patients' signs and symptoms associated with treatments would be preferred. Most cancer clinical trials use more than one clinical outcome as multiple primary, or primary and(key)secondary endpoints, such as overall survival, endpoints based on tumor assessments(e.g., disease-free survival, event-free survival, objective response rate, time to progression, progression-free survival), and endpoints involving symptom assessment. Utilizing multiple endpoints may provide the opportunity for characterizing the intervention's multidimensional effects, but also creates challenges, specifically controlling the Type Ⅰ and/or Type Ⅱ errors in hypotheses testing and trial designs associated with multiple endpoints. In this article, we review issues in design, monitoring, analysis and reporting of clinical trials with multiple endpoints, with illustrating examples in oncology disease settings. We outline several methods for controlling the Type Ⅰ error associated multiple tests, which have been commonly used in clinical trials. We also briefly discuss issues in interim analyses and group sequential designs for clinical trials with multiple endpoints.

[What Is an Estimand?-The Most Important Clinical Question in Oncology Trials].

Patients can experience different disease journeys and clinical trials that investigate the benefit of oncology treatments need to account for this diversity. When defining the treatment effect of interest in a trial, researchers thus have to account for events occurring after treatment initiation, such as the start of a new therapy, before observing the variable of interest. We review the estimand framework recently introduced by the International Council for Harmonisation(ICH, 2019)to structure discussions on the relationship between patient journeys and the treatment effect of interest in oncology trials. This framework is expected to improve coherence between trial objectives, design, analysis, reporting and interpretation, as illustrated in this article by examples in oncology disease settings.

Missing data imputation in clinical trials using recurrent neural network facilitated by clustering and oversampling.

In clinical practice, the composition of missing data may be complex, for example, a mixture of missing at random (MAR) and missing not at random (MNAR) assumptions. Many methods under the assumption of MAR are available. Under the assumption of MNAR, likelihood-based methods require specification of the joint distribution of the data, and the missingness mechanism has been introduced as sensitivity analysis. These classic models heavily rely on the underlying assumption, and, in many realistic scenarios, they can produce unreliable estimates. In this paper, we develop a machine learning based missing data prediction framework with the aim of handling more realistic missing data scenarios. We use an imbalanced learning technique (i.e., oversampling of minority class) to handle the MNAR data. To implement oversampling in longitudinal continuous variable, we first perform clustering via k$k$ -mean trajectories. And use the recurrent neural network (RNN) to model the longitudinal data. Further, we apply bootstrap aggregating to improve the accuracy of prediction and also to consider the uncertainty of a single prediction. We evaluate the proposed method using simulated data. The prediction result is evaluated at the individual patient level and the overall population level. We demonstrate the powerful predictive capability of RNN for longitudinal data and its flexibility for nonlinear modeling. Overall, the proposed method provides an accurate individual prediction for both MAR and MNAR data and reduce the bias of missing data in treatment effect estimation when compared to standard methods and classic models. Finally, we implement the proposed method in a real dataset from an antidepressant clinical trial. In summary, this paper offers an opportunity to encourage the integration of machine learning strategies for handling of missing data in the analysis of randomized clinical trials.

Estimands and Complex Innovative Designs.

Since the release of the ICH E9(R1) (International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use Addendum on Estimands and Sensitivity Analysis in Clinical Trials to the Guideline on Statistical Principles for Clinical Trials) document in 2019, the estimand framework has become a fundamental part of clinical trial protocols. In parallel, complex innovative designs have gained increased popularity in drug development, in particular in early development phases or in difficult experimental situations. While the estimand framework is relevant to any study in which a treatment effect is estimated, experience is lacking as regards its application to these designs. In a basket trial for example, should a different estimand be specified for each subpopulation of interest, defined, for example, by cancer site? Or can a single estimand focusing on the general population (defined, for example, by the positivity to a certain biomarker) be used? In the case of platform trials, should a different estimand be proposed for each drug investigated? In this work we discuss possible ways of implementing the estimand framework for different types of complex innovative designs. We consider trials that allow adding or selecting experimental treatment arms, modifying the control arm or the standard of care, and selecting or pooling populations. We also address the potentially data-driven, adaptive selection of estimands in an ongoing trial and disentangle certain statistical issues that pertain to estimation rather than to estimands, such as the borrowing of nonconcurrent information. We hope this discussion will facilitate the implementation of the estimand framework and its description in the study protocol when the objectives of the trial require complex innovative designs.

Testing for similarity of binary efficacy-toxicity responses.

Clinical trials often aim to compare two groups of patients for efficacy and/or toxicity depending on covariates such as dose. Examples include the comparison of populations from different geographic regions or age classes or, alternatively, of different treatment groups. Similarity of these groups can be claimed if the difference in average outcome is below a certain margin over the entire covariate range. In this article, we consider the problem of testing for similarity in the case that efficacy and toxicity are measured as binary outcome variables. We develop a new test for the assessment of similarity of two groups for a single binary endpoint. Our approach is based on estimating the maximal deviation between the curves describing the responses of the two groups, followed by a parametric bootstrap test. Further, using a two-dimensional Gumbel-type model we develop methodology to establish similarity for (correlated) binary efficacy-toxicity outcomes. We investigate the operating characteristics of the proposed methodology by means of a simulation study and present a case study as an illustration.

Simultaneous confidence tubes for comparing several multivariate linear regression models.

Much of the research on multiple comparison and simultaneous inference in the past 60 years or so has been for the comparisons of several population means. Spurrier seems to have been the first to investigate multiple comparisons of several simple linear regression lines using simultaneous confidence bands. In this paper, we extend the work of Liu et al. for finite comparisons of several univariate linear regression models using simultaneous confidence bands to finite comparisons of several multivariate linear regression models using simultaneous confidence tubes. We show how simultaneous confidence tubes can be constructed to allow more informative inferences for the comparison of several multivariate linear regression models than the current approach of hypotheses testing. The methods are illustrated with examples.