Nonparametric Bayesian Q-learning for optimization of dynamic treatment regimes in the presence of partial compliance.

Existing methods for estimation of dynamic treatment regimes are mostly limited to intention-to-treat analyses-which estimate the effect of randomization to a particular treatment regime without considering the compliance behavior of patients. In this article, we propose a novel nonparametric Bayesian Q-learning approach to construct optimal sequential treatment regimes that adjust for partial compliance. We consider the popular potential compliance framework, where some potential compliances are latent and need to be imputed. The key challenge is learning the joint distribution of the potential compliances, which we accomplish using a Dirichlet process mixture model. Our approach provides two kinds of treatment regimes: (1) conditional regimes that depend on the potential compliance values; and (2) marginal regimes where the potential compliances are marginalized. Extensive simulation studies highlight the usefulness of our method compared to intention-to-treat analyses. We apply our method to the Adaptive Treatment for Alcohol and Cocaine Dependence (ENGAGE) study , where the goal is to construct optimal treatment regimes to engage patients in therapy.

A non-parametric Bayesian approach for adjusting partial compliance in sequential decision making.

Existing methods for estimating the mean outcome under a given sequential treatment rule often rely on intention-to-treat analyses, which estimate the effect of following a certain treatment rule regardless of compliance behavior of patients. There are two major concerns with intention-to-treat analyses: (1) the estimated effects are often biased toward the null effect; (2) the results are not generalizable and reproducible due to the potentially differential compliance behavior. These are particularly problematic in settings with a high level of non-compliance, such as substance use disorder studies. Our work is motivated by the Adaptive Treatment for Alcohol and Cocaine Dependence study (ENGAGE), which is a multi-stage trial that aimed to construct optimal treatment strategies to engage patients in therapy. Due to the relatively low level of compliance in this trial, intention-to-treat analyses essentially estimate the effect of being randomized to a certain treatment, instead of the actual effect of the treatment. We obviate this challenge by defining the target parameter as the mean outcome under a dynamic treatment regime conditional on a potential compliance stratum. We propose a flexible non-parametric Bayesian approach based on principal stratification, which consists of a Gaussian copula model for the joint distribution of the potential compliances, and a Dirichlet process mixture model for the treatment sequence specific outcomes. We conduct extensive simulation studies which highlight the utility of our approach in the context of multi-stage randomized trials. We show robustness of our estimator to non-linear and non-Gaussian settings as well.