Impact of methodological choices in comparative effectiveness studies: application in natalizumab versus fingolimod comparison among patients with multiple sclerosis.

BACKGROUND: Natalizumab and fingolimod are used as high-efficacy treatments in relapsing-remitting multiple sclerosis. Several observational studies comparing these two drugs have shown variable results, using different methods to control treatment indication bias and manage censoring. The objective of this empirical study was to elucidate the impact of methods of causal inference on the results of comparative effectiveness studies. METHODS: Data from three observational multiple sclerosis registries (MSBase, the Danish MS Registry and French OFSEP registry) were combined. Four clinical outcomes were studied. Propensity scores were used to match or weigh the compared groups, allowing for estimating average treatment effect for treated or average treatment effect for the entire population. Analyses were conducted both in intention-to-treat and per-protocol frameworks. The impact of the positivity assumption was also assessed. RESULTS: Overall, 5,148 relapsing-remitting multiple sclerosis patients were included. In this well-powered sample, the 95% confidence intervals of the estimates overlapped widely. Propensity scores weighting and propensity scores matching procedures led to consistent results. Some differences were observed between average treatment effect for the entire population and average treatment effect for treated estimates. Intention-to-treat analyses were more conservative than per-protocol analyses. The most pronounced irregularities in outcomes and propensity scores were introduced by violation of the positivity assumption. CONCLUSIONS: This applied study elucidates the influence of methodological decisions on the results of comparative effectiveness studies of treatments for multiple sclerosis. According to our results, there are no material differences between conclusions obtained with propensity scores matching or propensity scores weighting given that a study is sufficiently powered, models are correctly specified and positivity assumption is fulfilled.

More robust estimation of average treatment effects using kernel optimal matching in an observational study of spine surgical interventions.

Inverse probability of treatment weighting (IPTW), which has been used to estimate average treatment effects (ATE) using observational data, tenuously relies on the positivity assumption and the correct specification of the treatment assignment model, both of which are problematic assumptions in many observational studies. Various methods have been proposed to overcome these challenges, including truncation, covariate-balancing propensity scores, and stable balancing weights. Motivated by an observational study in spine surgery, in which positivity is violated and the true treatment assignment model is unknown, we present the use of optimal balancing by kernel optimal matching (KOM) to estimate ATE. By uniformly controlling the conditional mean squared error of a weighted estimator over a class of models, KOM simultaneously mitigates issues of possible misspecification of the treatment assignment model and is able to handle practical violations of the positivity assumption, as shown in our simulation study. Using data from a clinical registry, we apply KOM to compare two spine surgical interventions and demonstrate how the result matches the conclusions of clinical trials that IPTW estimates spuriously refute.

A tutorial on dealing with time-varying eligibility for treatment: Comparing the risk of major bleeding with direct-acting oral anticoagulants vs warfarin.

In this tutorial, we focus on the problem of how to define and estimate treatment effects when some patients develop a contraindication and are thus ineligible to receive a treatment of interest during follow-up. We first describe the concept of positivity, which is the requirement that all subjects in an analysis be eligible for all treatments of interest conditional on their baseline covariates, and the extension of this concept in the longitudinal treatment setting. We demonstrate using simulated datasets and regression analysis that under violations of longitudinal positivity, typical associational estimates between treatment over time and the outcome of interest may be misleading depending on the data-generating structure. Finally, we explain how one may define "treatment strategies," such as "treat with medication unless contraindicated," to overcome the problems linked to time-varying eligibility. Finally, we show how contrasts between the expected potential outcomes under these strategies may be consistently estimated with inverse probability weighting methods. We provide R code for all the analyses described.

Inverse probability weighting methods for Cox regression with right-truncated data.

Right-truncated data arise when observations are ascertained retrospectively, and only subjects who experience the event of interest by the time of sampling are selected. Such a selection scheme, without adjustment, leads to biased estimation of covariate effects in the Cox proportional hazards model. The existing methods for fitting the Cox model to right-truncated data, which are based on the maximization of the likelihood or solving estimating equations with respect to both the baseline hazard function and the covariate effects, are numerically challenging. We consider two alternative simple methods based on inverse probability weighting (IPW) estimating equations, which allow consistent estimation of covariate effects under a positivity assumption and avoid estimation of baseline hazards. We discuss problems of identifiability and consistency that arise when positivity does not hold and show that although the partial tests for null effects based on these IPW methods can be used in some settings even in the absence of positivity, they are not valid in general. We propose adjusted estimating equations that incorporate the probability of observation when it is known from external sources, which results in consistent estimation. We compare the methods in simulations and apply them to the analyses of human immunodeficiency virus latency.

Optimal probability weights for estimating causal effects of time-varying treatments with marginal structural Cox models.

Marginal structural Cox models have been used to estimate the causal effect of a time-varying treatment on a survival outcome in the presence of time-dependent confounders. These methods rely on the positivity assumption, which states that the propensity scores are bounded away from zero and one. Practical violations of this assumption are common in longitudinal studies, resulting in extreme weights that may yield erroneous inferences. Truncation, which consists of replacing outlying weights with less extreme ones, is the most common approach to control for extreme weights to date. While truncation reduces the variability in the weights and the consequent sampling variability of the estimator, it can also introduce bias. Instead of truncated weights, we propose using optimal probability weights, defined as those that have a specified variance and the smallest Euclidean distance from the original, untruncated weights. The set of optimal weights is obtained by solving a constrained quadratic optimization problem. The proposed weights are evaluated in a simulation study and applied to the assessment of the effect of treatment on time to death among people in Sweden who live with human immunodeficiency virus and inject drugs.

Practice of causal inference with the propensity of being zero or one: assessing the effect of arbitrary cutoffs of propensity scores.

Causal inference methodologies have been developed for the past decade to estimate the unconfounded effect of an exposure under several key assumptions. These assumptions include, but are not limited to, the stable unit treatment value assumption, the strong ignorability of treatment assignment assumption, and the assumption that propensity scores be bounded away from zero and one (the positivity assumption). Of these assumptions, the first two have received much attention in the literature. Yet the positivity assumption has been recently discussed in only a few papers. Propensity scores of zero or one are indicative of deterministic exposure so that causal effects cannot be defined for these subjects. Therefore, these subjects need to be removed because no comparable comparison groups can be found for such subjects. In this paper, using currently available causal inference methods, we evaluate the effect of arbitrary cutoffs in the distribution of propensity scores and the impact of those decisions on bias and efficiency. We propose a tree-based method that performs well in terms of bias reduction when the definition of positivity is based on a single confounder. This tree-based method can be easily implemented using the statistical software program, R. R code for the studies is available online.

Propensity score estimators for the average treatment effect and the average treatment effect on the treated may yield very different estimates.

OBJECTIVE: Propensity score matching is typically used to estimate the average treatment effect for the treated while inverse probability of treatment weighting aims at estimating the population average treatment effect. We illustrate how different estimands can result in very different conclusions. STUDY DESIGN: We applied the two propensity score methods to assess the effect of continuous positive airway pressure on mortality in patients hospitalized for acute heart failure. We used Monte Carlo simulations to investigate the important differences in the two estimates. RESULTS: Continuous positive airway pressure application increased hospital mortality overall, but no continuous positive airway pressure effect was found on the treated. Potential reasons were (1) violation of the positivity assumption; (2) treatment effect was not uniform across the distribution of the propensity score. From simulations, we concluded that positivity bias was of limited magnitude and did not explain the large differences in the point estimates. However, when treatment effect varies according to the propensity score (E[Y(1)-Y(0)|g(X)] is not constant, Y being the outcome and g(X) the propensity score), propensity score matching ATT estimate could strongly differ from the inverse probability of treatment weighting-average treatment effect estimate. We show that this empirical result is supported by theory. CONCLUSION: Although both approaches are recommended as valid methods for causal inference, propensity score-matching for ATT and inverse probability of treatment weighting for average treatment effect yield substantially different estimates of treatment effect. The choice of the estimand should drive the choice of the method.