Case study of semaglutide and cardiovascular outcomes: An application of the Causal Roadmap to a hybrid design for augmenting an RCT control arm with real-world data.

Introduction: Increasing interest in real-world evidence has fueled the development of study designs incorporating real-world data (RWD). Using the Causal Roadmap, we specify three designs to evaluate the difference in risk of major adverse cardiovascular events (MACE) with oral semaglutide versus standard-of-care: (1) the actual sequence of non-inferiority and superiority randomized controlled trials (RCTs), (2) a single RCT, and (3) a hybrid randomized-external data study. Methods: The hybrid design considers integration of the PIONEER 6 RCT with RWD controls using the experiment-selector cross-validated targeted maximum likelihood estimator. We evaluate 95% confidence interval coverage, power, and average patient time during which participants would be precluded from receiving a glucagon-like peptide-1 receptor agonist (GLP1-RA) for each design using simulations. Finally, we estimate the effect of oral semaglutide on MACE for the hybrid PIONEER 6-RWD analysis. Results: In simulations, Designs 1 and 2 performed similarly. The tradeoff between decreased coverage and patient time without the possibility of a GLP1-RA for Designs 1 and 3 depended on the simulated bias. In real data analysis using Design 3, external controls were integrated in 84% of cross-validation folds, resulting in an estimated risk difference of -1.53%-points (95% CI -2.75%-points to -0.30%-points). Conclusions: The Causal Roadmap helps investigators to minimize potential bias in studies using RWD and to quantify tradeoffs between study designs. The simulation results help to interpret the level of evidence provided by the real data analysis in support of the superiority of oral semaglutide versus standard-of-care for cardiovascular risk reduction.

An application of the Causal Roadmap in two safety monitoring case studies: Causal inference and outcome prediction using electronic health record data.

Background: Real-world data, such as administrative claims and electronic health records, are increasingly used for safety monitoring and to help guide regulatory decision-making. In these settings, it is important to document analytic decisions transparently and objectively to assess and ensure that analyses meet their intended goals. Methods: The Causal Roadmap is an established framework that can guide and document analytic decisions through each step of the analytic pipeline, which will help investigators generate high-quality real-world evidence. Results: In this paper, we illustrate the utility of the Causal Roadmap using two case studies previously led by workgroups sponsored by the Sentinel Initiative - a program for actively monitoring the safety of regulated medical products. Each case example focuses on different aspects of the analytic pipeline for drug safety monitoring. The first case study shows how the Causal Roadmap encourages transparency, reproducibility, and objective decision-making for causal analyses. The second case study highlights how this framework can guide analytic decisions beyond inference on causal parameters, improving outcome ascertainment in clinical phenotyping. Conclusion: These examples provide a structured framework for implementing the Causal Roadmap in safety surveillance and guide transparent, reproducible, and objective analysis.

A causal roadmap for generating high-quality real-world evidence.

Increasing emphasis on the use of real-world evidence (RWE) to support clinical policy and regulatory decision-making has led to a proliferation of guidance, advice, and frameworks from regulatory agencies, academia, professional societies, and industry. A broad spectrum of studies use real-world data (RWD) to produce RWE, ranging from randomized trials with outcomes assessed using RWD to fully observational studies. Yet, many proposals for generating RWE lack sufficient detail, and many analyses of RWD suffer from implausible assumptions, other methodological flaws, or inappropriate interpretations. The Causal Roadmap is an explicit, itemized, iterative process that guides investigators to prespecify study design and analysis plans; it addresses a wide range of guidance within a single framework. By supporting the transparent evaluation of causal assumptions and facilitating objective comparisons of design and analysis choices based on prespecified criteria, the Roadmap can help investigators to evaluate the quality of evidence that a given study is likely to produce, specify a study to generate high-quality RWE, and communicate effectively with regulatory agencies and other stakeholders. This paper aims to disseminate and extend the Causal Roadmap framework for use by clinical and translational researchers; three companion papers demonstrate applications of the Causal Roadmap for specific use cases.

Applicability of the Vascular Quality Initiative mortality prediction model for infrainguinal revascularization in a tertiary limb preservation center population.

OBJECTIVE: Patients undergoing revascularization for chronic limb-threatening ischemia (CLTI) are at elevated risk for both mortality and limb loss. To facilitate therapeutic decision-making, a mortality prediction model derived from the Vascular Quality Initiative (VQI) database has stratified patients into low, medium, and high risk, defined by 30-day mortality estimates of ≤3%, 3%-5%, or >5% and 2-year mortality estimates of ≤30%, 30%-50%, or ≥50%, respectively. The purpose of this study was to compare expected mortality risk derived from this model with observed outcomes in a tertiary center. METHODS: Consecutive patients treated at a single center between 2016 and 2019 were analyzed. Baseline demographics, approach, and mortality events were reviewed. Observed mortality was obtained using life-table methods and compared using a log-rank test with the expected mortality risk that was calculated using the VQI model. RESULTS: This study cohort consisted of 195 revascularization procedures in 169 unique patients stratified into 128 (66%) low-, 50 (26%) medium-, and 17 (8%) high-risk cases based on the VQI model. Ninety percent of revascularizations were performed for tissue loss. Compared with the VQI population, comorbidities were prevalent and included unstable angina or myocardial infarction within 6 months (6% vs 2.4% in VQI; P < .001), congestive heart failure (30% vs 23%; P < .001), and dialysis dependence (14% vs 0.9%; P < .001). Patients were also older (31% vs 21% ≥80 years old; P < .001) and more likely to be frail (45% vs 64% independent; P < .001). High-risk patients were more prevalent in the endovascular group (11% of 132 endovascular interventions vs 3% of 63 bypasses; P = .056). Thirty-day observed mortality exceeded expected VQI prediction model mortality in all groups, although was not statistically significant. The VQI model adequately stratified the studied population into risk groups (P < .001). Low-risk patients with CLTI (65% of the overall cohort) experienced 2-year mortality of 18.9%. However, observed mortality rates for medium- and high-risk VQI strata were similar. After a median follow-up of 28 months, medium-risk patients incurred a significantly higher mortality than predicted (53.5% ± 2.1% vs 36.8% ± 1.1%; P = .016). CONCLUSIONS: The VQI mortality prediction model discriminates mortality risk after limb revascularization in CLTI, accurately identifying a majority subgroup of patients who are suitable for either open or endovascular intervention. However, it may underestimate mortality in a tertiary referral population with high comorbidity burden and was not well calibrated for the medium-risk group. It may be more appropriate to dichotomize patients with CLTI who are candidates for limb salvage into an average-risk and high-risk group.