Evaluating tests for cluster-randomized trials with few clusters under generalized linear mixed models with covariate adjustment: A simulation study.

Generalized linear mixed models (GLMM) are commonly used to analyze clustered data, but when the number of clusters is small to moderate, standard statistical tests may produce elevated type I error rates. Small-sample corrections have been proposed for continuous or binary outcomes without covariate adjustment. However, appropriate tests to use for count outcomes or under covariate-adjusted models remains unknown. An important setting in which this issue arises is in cluster-randomized trials (CRTs). Because many CRTs have just a few clusters (eg, clinics or health systems), covariate adjustment is particularly critical to address potential chance imbalance and/or low power (eg, adjustment following stratified randomization or for the baseline value of the outcome). We conducted simulations to evaluate GLMM-based tests of the treatment effect that account for the small (10) or moderate (20) number of clusters under a parallel-group CRT setting across scenarios of covariate adjustment (including adjustment for one or more person-level or cluster-level covariates) for both binary and count outcomes. We find that when the intraclass correlation is non-negligible ( ≥ $$ \ge $$ 0.01) and the number of covariates is small ( ≤ $$ \le $$ 2), likelihood ratio tests with a between-within denominator degree of freedom have type I error rates close to the nominal level. When the number of covariates is moderate ( ≥ $$ \ge $$ 5), across our simulation scenarios, the relative performance of the tests varied considerably and no method performed uniformly well. Therefore, we recommend adjusting for no more than a few covariates and using likelihood ratio tests with a between-within denominator degree of freedom.

Nurse Care Management for Opioid Use Disorder Treatment: The PROUD Cluster Randomized Clinical Trial.

Importance: Few primary care (PC) practices treat patients with medications for opioid use disorder (OUD) despite availability of effective treatments. Objective: To assess whether implementation of the Massachusetts model of nurse care management for OUD in PC increases OUD treatment with buprenorphine or extended-release injectable naltrexone and secondarily decreases acute care utilization. Design, Setting, and Participants: The Primary Care Opioid Use Disorders Treatment (PROUD) trial was a mixed-methods, implementation-effectiveness cluster randomized clinical trial conducted in 6 diverse health systems across 5 US states (New York, Florida, Michigan, Texas, and Washington). Two PC clinics in each system were randomized to intervention or usual care (UC) stratified by system (5 systems were notified on February 28, 2018, and 1 system with delayed data use agreement on August 31, 2018). Data were obtained from electronic health records and insurance claims. An implementation monitoring team collected qualitative data. Primary care patients were included if they were 16 to 90 years old and visited a participating clinic from up to 3 years before a system's randomization date through 2 years after. Intervention: The PROUD intervention included 3 components: (1) salary for a full-time OUD nurse care manager; (2) training and technical assistance for nurse care managers; and (3) 3 or more PC clinicians agreeing to prescribe buprenorphine. Main Outcomes and Measures: The primary outcome was a clinic-level measure of patient-years of OUD treatment (buprenorphine or extended-release injectable naltrexone) per 10 000 PC patients during the 2 years postrandomization (follow-up). The secondary outcome, among patients with OUD prerandomization, was a patient-level measure of the number of days of acute care utilization during follow-up. Results: During the baseline period, a total of 130 623 patients were seen in intervention clinics (mean [SD] age, 48.6 [17.7] years; 59.7% female), and 159 459 patients were seen in UC clinics (mean [SD] age, 47.2 [17.5] years; 63.0% female). Intervention clinics provided 8.2 (95% CI, 5.4-∞) more patient-years of OUD treatment per 10 000 PC patients compared with UC clinics (P = .002). Most of the benefit accrued in 2 health systems and in patients new to clinics (5.8 [95% CI, 1.3-∞] more patient-years) or newly treated for OUD postrandomization (8.3 [95% CI, 4.3-∞] more patient-years). Qualitative data indicated that keys to successful implementation included broad commitment to treat OUD in PC from system leaders and PC teams, full financial coverage for OUD treatment, and straightforward pathways for patients to access nurse care managers. Acute care utilization did not differ between intervention and UC clinics (relative rate, 1.16; 95% CI, 0.47-2.92; P = .70). Conclusions and Relevance: The PROUD cluster randomized clinical trial intervention meaningfully increased PC OUD treatment, albeit unevenly across health systems; however, it did not decrease acute care utilization among patients with OUD. Trial Registration: ClinicalTrials.gov Identifier: NCT03407638.

Adversarial meta-learning of Gamma-minimax estimators that leverage prior knowledge.

Bayes estimators are well known to provide a means to incorporate prior knowledge that can be expressed in terms of a single prior distribution. However, when this knowledge is too vague to express with a single prior, an alternative approach is needed. Gamma-minimax estimators provide such an approach. These estimators minimize the worst-case Bayes risk over a set Γ of prior distributions that are compatible with the available knowledge. Traditionally, Gamma-minimaxity is defined for parametric models. In this work, we define Gamma-minimax estimators for general models and propose adversarial meta-learning algorithms to compute them when the set of prior distributions is constrained by generalized moments. Accompanying convergence guarantees are also provided. We also introduce a neural network class that provides a rich, but finite-dimensional, class of estimators from which a Gamma-minimax estimator can be selected. We illustrate our method in two settings, namely entropy estimation and a prediction problem that arises in biodiversity studies.

Prediction sets adaptive to unknown covariate shift.

Predicting sets of outcomes-instead of unique outcomes-is a promising solution to uncertainty quantification in statistical learning. Despite a rich literature on constructing prediction sets with statistical guarantees, adapting to unknown covariate shift-a prevalent issue in practice-poses a serious unsolved challenge. In this article, we show that prediction sets with finite-sample coverage guarantee are uninformative and propose a novel flexible distribution-free method, PredSet-1Step, to efficiently construct prediction sets with an asymptotic coverage guarantee under unknown covariate shift. We formally show that our method is asymptotically probably approximately correct, having well-calibrated coverage error with high confidence for large samples. We illustrate that it achieves nominal coverage in a number of experiments and a data set concerning HIV risk prediction in a South African cohort study. Our theory hinges on a new bound for the convergence rate of the coverage of Wald confidence intervals based on general asymptotically linear estimators.

Baseline representativeness of patients in clinics enrolled in the PRimary care Opioid Use Disorders treatment (PROUD) trial: comparison of trial and non-trial clinics in the same health systems.

BACKGROUND: Pragmatic primary care trials aim to test interventions in "real world" health care settings, but clinics willing and able to participate in trials may not be representative of typical clinics. This analysis compared patients in participating and non-participating clinics from the same health systems at baseline in the PRimary care Opioid Use Disorders treatment (PROUD) trial. METHODS: This observational analysis relied on secondary electronic health record and administrative claims data in 5 of 6 health systems in the PROUD trial. The sample included patients 16-90 years at an eligible primary care visit in the 3 years before randomization. Each system contributed 2 randomized PROUD trial clinics and 4 similarly sized non-trial clinics. We summarized patient characteristics in trial and non-trial clinics in the 2 years before randomization ("baseline"). Using mixed-effect regression models, we compared trial and non-trial clinics on a baseline measure of the primary trial outcome (clinic-level patient-years of opioid use disorder (OUD) treatment, scaled per 10,000 primary care patients seen) and a baseline measure of the secondary trial outcome (patient-level days of acute care utilization among patients with OUD). RESULTS: Patients were generally similar between the 10 trial clinics (n = 248,436) and 20 non-trial clinics (n = 341,130), although trial clinics' patients were slightly younger, more likely to be Hispanic/Latinx, less likely to be white, more likely to have Medicaid/subsidized insurance, and lived in less wealthy neighborhoods. Baseline outcomes did not differ between trial and non-trial clinics: trial clinics had 1.0 more patient-year of OUD treatment per 10,000 patients (95% CI: - 2.9, 5.0) and a 4% higher rate of days of acute care utilization than non-trial clinics (rate ratio: 1.04; 95% CI: 0.76, 1.42). CONCLUSIONS: trial clinics and non-trial clinics were similar regarding most measured patient characteristics, and no differences were observed in baseline measures of trial primary and secondary outcomes. These findings suggest trial clinics were representative of comparably sized clinics within the same health systems. Although results do not reflect generalizability more broadly, this study illustrates an approach to assess representativeness of clinics in future pragmatic primary care trials.

Individualized treatment rules under stochastic treatment cost constraints.

Estimation and evaluation of individualized treatment rules have been studied extensively, but real-world treatment resource constraints have received limited attention in existing methods. We investigate a setting in which treatment is intervened upon based on covariates to optimize the mean counterfactual outcome under treatment cost constraints when the treatment cost is random. In a particularly interesting special case, an instrumental variable corresponding to encouragement to treatment is intervened upon with constraints on the proportion receiving treatment. For such settings, we first develop a method to estimate optimal individualized treatment rules. We further construct an asymptotically efficient plug-in estimator of the corresponding average treatment effect relative to a given reference rule.

Universal sieve-based strategies for efficient estimation using machine learning tools.

Suppose that we wish to estimate a finite-dimensional summary of one or more function-valued features of an underlying data-generating mechanism under a nonparametric model. One approach to estimation is by plugging in flexible estimates of these features. Unfortunately, in general, such estimators may not be asymptotically efficient, which often makes these estimators difficult to use as a basis for inference. Though there are several existing methods to construct asymptotically efficient plug-in estimators, each such method either can only be derived using knowledge of efficiency theory or is only valid under stringent smoothness assumptions. Among existing methods, sieve estimators stand out as particularly convenient because efficiency theory is not required in their construction, their tuning parameters can be selected data adaptively, and they are universal in the sense that the same fits lead to efficient plug-in estimators for a rich class of estimands. Inspired by these desirable properties, we propose two novel universal approaches for estimating function-valued features that can be analyzed using sieve estimation theory. Compared to traditional sieve estimators, these approaches are valid under more general conditions on the smoothness of the function-valued features by utilizing flexible estimates that can be obtained, for example, using machine learning.

Optimal individualized decision rules using instrumental variable methods.

There is an extensive literature on the estimation and evaluation of optimal individualized treatment rules in settings where all confounders of the effect of treatment on outcome are observed. We study the development of individualized decision rules in settings where some of these confounders may not have been measured but a valid binary instrument is available for a binary treatment. We first consider individualized treatment rules, which will naturally be most interesting in settings where it is feasible to intervene directly on treatment. We then consider a setting where intervening on treatment is infeasible, but intervening to encourage treatment is feasible. In both of these settings, we also handle the case that the treatment is a limited resource so that optimal interventions focus the available resources on those individuals who will benefit most from treatment. Given a reference rule, we evaluate an optimal individualized rule by its average causal effect relative to a prespecified reference rule. We develop methods to estimate optimal individualized rules and construct asymptotically efficient plug-in estimators of the corresponding average causal effect relative to a prespecified reference rule.

Correction to: "Optimal Individualized Decision Rules Using Instrumental Variable Methods".

[This corrects the article PMC7959164.].

Addressing identification bias in the design and analysis of cluster-randomized pragmatic trials: a case study.

BACKGROUND: Pragmatic trials provide the opportunity to study the effectiveness of health interventions to improve care in real-world settings. However, use of open-cohort designs with patients becoming eligible after randomization and reliance on electronic health records (EHRs) to identify participants may lead to a form of selection bias referred to as identification bias. This bias can occur when individuals identified as a result of the treatment group assignment are included in analyses. METHODS: To demonstrate the importance of identification bias and how it can be addressed, we consider a motivating case study, the PRimary care Opioid Use Disorders treatment (PROUD) Trial. PROUD is an ongoing pragmatic, cluster-randomized implementation trial in six health systems to evaluate a program for increasing medication treatment of opioid use disorders (OUDs). A main study objective is to evaluate whether the PROUD intervention decreases acute care utilization among patients with OUD (effectiveness aim). Identification bias is a particular concern, because OUD is underdiagnosed in the EHR at baseline, and because the intervention is expected to increase OUD diagnosis among current patients and attract new patients with OUD to the intervention site. We propose a framework for addressing this source of bias in the statistical design and analysis. RESULTS: The statistical design sought to balance the competing goals of fully capturing intervention effects and mitigating identification bias, while maximizing power. For the primary analysis of the effectiveness aim, identification bias was avoided by defining the study sample using pre-randomization data (pre-trial modeling demonstrated that the optimal approach was to use individuals with a prior OUD diagnosis). To expand generalizability of study findings, secondary analyses were planned that also included patients newly diagnosed post-randomization, with analytic methods to account for identification bias. CONCLUSION: As more studies seek to leverage existing data sources, such as EHRs, to make clinical trials more affordable and generalizable and to apply novel open-cohort study designs, the potential for identification bias is likely to become increasingly common. This case study highlights how this bias can be addressed in the statistical study design and analysis. TRIAL REGISTRATION: ClinicalTrials.gov, NCT03407638. Registered on 23 January 2018.

Impact of poorly controlled epilepsy in the Republic of Guinea.

PURPOSE: To characterize people with epilepsy (PWE) presenting to a free neurology consultation and antiepileptic drug (AED) service in the Republic of Guinea. METHODS: Guinea is a low-income country in West Africa that recently experienced an Ebola Virus Disease epidemic. Community-dwelling PWE were seen at a public referral hospital in Conakry, the capital city. During two visits in 2017, an African-U.S. team performed structured interviews and electroencephalograms and provided AEDs. RESULTS: Of 257 participants (143 children, 122 female), 25% had untreated epilepsy and 72% met our criteria for poorly controlled epilepsy. 59% had >100 lifetime seizures, and 58% reported a history consistent with status epilepticus. 38 school-aged children were not in school and 26 adults were unemployed. 115 were not currently taking an AED, including 50 participants who had previously taken an AED and stopped. Commonly cited reasons for AED discontinuation were perceived side effects, unaffordability, and unavailability of AEDs. Traditional medicine use was more frequent among children versus adults (92/143 vs. 60/114, p = 0.048). 57 participants had head injuries, 29 had burns, and 18 had fractures. In a multivariable regression analysis, >100 lifetime seizure count was strongly associated with seizure-related injury (p < 0.001). Burns were more likely to occur among females (p = 0.02). CONCLUSIONS: There is an urgent need to improve the standard of care for PWE in Guinea. Several missed opportunities were identified, including low use of AEDs and high use of traditional medicines, particularly in children. Targeted programs should be developed to prevent unintentional injury and improve seizure control.

Modeling Continuous Admixture Using Admixture-Induced Linkage Disequilibrium.

Recent migrations and inter-ethnic mating of long isolated populations have resulted in genetically admixed populations. To understand the complex population admixture process, which is critical to both evolutionary and medical studies, here we used admixture induced linkage disequilibrium (LD) to infer continuous admixture events, which is common for most existing admixed populations. Unlike previous studies, we expanded the typical continuous admixture model to a more general scenario with isolation after a certain duration of continuous gene flow. Based on the new models, we developed a method, CAMer, to infer the admixture history considering continuous and complex demographic process of gene flow between populations. We evaluated the performance of CAMer by computer simulation and further applied our method to real data analysis of a few well-known admixed populations.