Derivation of an Annualized Claims-Based Major Adverse Cardiovascular Event Estimator in Type 2 Diabetes.

Background: Major adverse cardiovascular events (MACE) are a leading cause of morbidity and mortality among adults with type 2 diabetes. Currently, available MACE prediction models have important limitations, including reliance on data that may not be routinely available, narrow focus on primary prevention, limited patient populations, and longtime horizons for risk prediction. Objectives: The purpose of this study was to derive and internally validate a claims-based prediction model for 1-year risk of MACE in type 2 diabetes. Methods: Using medical and pharmacy claims for adults with type 2 diabetes enrolled in commercial, Medicare Advantage, and Medicare fee-for-service plans between 2014 and 2021, we derived and internally validated the annualized claims-based MACE estimator (ACME) model to predict the risk of MACE (nonfatal acute myocardial infarction, nonfatal stroke, and all-cause mortality). The Cox proportional hazards model was composed of 30 covariates, including patient age, sex, comorbidities, and medications. Results: The study cohort comprised 6,623,526 adults with type 2 diabetes, mean age 68.1 ± 10.6 years, 49.8% women, and 73.0% Non-Hispanic White. ACME had a concordance index of 0.74 (validation index range: 0.739-0.741). The predicted 1-year risk of the study cohort ranged from 0.4% to 99.9%, with a median risk of 3.4% (IQR: 2.3%-6.5%). Conclusions: ACME was derived in a large usual care population, relies on routinely available data, and estimates short-term MACE risk. It can support population risk stratification at the health system and payer levels, participant identification for decentralized clinical trials of cardiovascular disease, and risk-stratified observational studies using real-world data.

Effectiveness of glucose-lowering medications on cardiovascular outcomes in patients with type 2 diabetes at moderate cardiovascular risk.

Cardiovascular disease (CVD) is the leading cause of death among people with type 2 diabetes1-5, most of whom are at moderate CVD risk6, yet there is limited evidence on the preferred choice of glucose-lowering medication for CVD risk reduction in this population. Here, we report the results of a retrospective cohort study where data for US adults with type 2 diabetes and moderate risk for CVD are used to compare the risks of experiencing a major adverse cardiovascular event with initiation of glucagon-like peptide-1 receptor agonists (GLP-1RA; n = 44,188), sodium-glucose cotransporter 2 inhibitors (SGLT2i; n = 47,094), dipeptidyl peptidase-4 inhibitors (DPP4i; n = 84,315) and sulfonylureas (n = 210,679). Compared to DPP4i, GLP-1RA (hazard ratio (HR) 0.87; 95% confidence interval (CI) 0.82-0.93) and SGLT2i (HR 0.85; 95% CI 0.81-0.90) were associated with a lower risk of a major adverse cardiovascular event, whereas sulfonylureas were associated with a higher risk (HR 1.19; 95% CI 1.16-1.22). Thus, GLP-1RA and SGLT2i may be the preferred glucose-lowering agents for cardiovascular risk reduction in patients at moderate baseline risk for CVD. ClinicalTrials.gov registration: NCT05214573.

American clusters: using machine learning to understand health and health care disparities in the United States.

Health and health care access in the United States are plagued by high inequality. While machine learning (ML) is increasingly used in clinical settings to inform health care delivery decisions and predict health care utilization, using ML as a research tool to understand health care disparities in the United States and how these are connected to health outcomes, access to health care, and health system organization is less common. We utilized over 650 variables from 24 different databases aggregated by the Agency for Healthcare Research and Quality in their Social Determinants of Health (SDOH) database. We used k-means-a non-hierarchical ML clustering method-to cluster county-level data. Principal factor analysis created county-level index values for each SDOH domain and 2 health care domains: health care infrastructure and health care access. Logistic regression classification was used to identify the primary drivers of cluster classification. The most efficient cluster classification consists of 3 distinct clusters in the United States; the cluster having the highest life expectancy comprised only 10% of counties. The most efficient ML clusters do not identify the clusters with the widest health care disparities. ML clustering, using county-level data, shows that health care infrastructure and access are the primary drivers of cluster composition.

Assessing the impact of the president's emergency plan for AIDS relief on all-cause mortality.

This study estimated the impacts of PEPFAR on all-cause mortality (ACM) rates (deaths per 1,000 population) across PEPFAR recipient countries from 2004-2018. As PEPFAR moves into its 3rd decade, this study supplements the existing literature on PEPFAR 's overall effectiveness in saving lives by focusing impact estimates on the important subgroups of countries that received different intensities of aid, and provides estimates of impact for different phases of this 15-year period study. The study uses a country-level panel data set of 157 low- and middle-income countries (LMICs) from 1990-2018, including 90 PEPFAR recipient countries receiving bilateral aid from the U.S. government, employing difference-in-differences (DID) econometric models with several model specifications, including models with differing baseline covariates, and models with yearly covariates including other donor spending and domestic health spending. Using five different model specifications, a 10-21% decline in ACM rates from 2004 to 2018 is attributed to PEPFAR presence in the group of 90 recipient countries. Declines are somewhat larger (15-25%) in those countries that are subject to PEPFAR's country operational planning (COP) process, and where PEPFAR per capita aid amounts are largest (17-27%). Across the 90 recipient countries we study, the average impact across models is estimated to be a 7.6% reduction in ACM in the first 5-year period (2004-2008), somewhat smaller in the second 5-year period (5.5%) and in the third 5-year period (4.7%). In COP countries the impacts show decreases in ACM of 7.4% in the first period attributed to PEPFAR, 7.7% reductions in the second, and 6.6% reductions in the third. PEPFAR presence is correlated with large declines in the ACM rate, and the overall life-saving results persisted over time. The effects of PEFAR on ACM have been large, suggesting the possibility of spillover life-saving impacts of PEPFAR programming beyond HIV disease alone.

Analysis of economic and educational spillover effects in PEPFAR countries.

The United States President's Emergency Plan for AIDS Relief (PEPFAR) has been credited with saving millions lives and helping to change the trajectory of the global human immunodeficiency virus (HIV) epidemic. This study assesses whether PEPFAR has had impacts beyond health by examining changes in five economic and educational outcomes in PEPFAR countries: the gross domestic product (GDP) per capita growth rate; the share of girls and share of boys, respectively, who are out of school; and female and male employment rates. We constructed a panel data set for 157 low- and middle-income countries between 1990 and 2018 to estimate the macroeconomic impacts of PEPFAR. Our PEPFAR group included 90 countries that had received PEPFAR support over the period. Our comparison group included 67 low- and middle-income countries that had not received any PEPFAR support or had received minimal PEPFAR support (<$1M or <$.05 per capita) between 2004 and 2018. We used differences in differences (DID) methods to estimate the program impacts on the five economic and educational outcome measures. This study finds that PEPFAR is associated with increases in the GDP per capita growth rate and educational outcomes. In some models, we find that PEPFAR is associated with reductions in male and female employment. However, these effects appear to be due to trends in the comparison group countries rather than programmatic impacts of PEPFAR. We show that these impacts are most pronounced in COP countries receiving the highest levels of PEPFAR investment.

Equity of the HIV epidemic response in 13 African countries.

For over 25 years, new programs to attempt to stem the HIV epidemic have been developed in Africa by country governments as well as external donors. These programs and activities have built and operated facilities, trained clinicians, financed drugs and commodities, supported and helped finance government health planning and operations, and contributed in other ways. Who has benefited from this massive mobilization? While some single country and narrowly focused studies have been done, the issue of equity of HIV programs for vulnerable populations has not been examined in a large set of countries. Using Population-based HIV Impact Assessment (PHIA) data, we examine equity of the HIV programs in 13 African countries to determine if vulnerable groups (such as those with low wealth, rural populations, young adults, and females) have achieved comparable levels of access to HIV program services. In contrast, we also compare the equity of the HIV response to rural and low-wealth populations with the equity of corresponding domestic health systems using Demographic and Health Survey data.This study found that in over half of the countries, the HIV response indicators were equitable for vulnerable population segments including the low-wealth population (in seven countries) and rural population segment (in nine countries). In no country was the domestic health system equitable for these two groups. However, HIV programming does show some clear patterns of inequity for low-wealth and rural populations in some countries. For gender and young adults, the HIV response indicators show that in all 13 countries men and young adults are consistently underserved relative to their counterparts.

Analysis of maternal and child health spillover effects in PEPFAR countries.

OBJECTIVES: This study examined whether the US President's Emergency Plan for AIDS Relief (PEPFAR) funding had effects beyond HIV, specifically on several measures of maternal and child health in low-income and middle-income countries (LMICs). The results of previous research on the question of PEPFAR health spillovers have been inconsistent. This study, using a large, multicountry panel data set of 157 LMICs including 90 recipient countries, adds to the literature. DESIGN: Seven indicators including child and maternal mortality, several child vaccination rates and anaemia among childbearing-age women are important population health indicators. Panel data and difference-in-differences estimators (DID) were used to estimate the impact of the PEPFAR programme from inception in 2004 to 2018 using a comparison group of 67 LMICs. Several different models of baseline (2004) covariates were used to help balance the comparison and treatment groups. Staggered DID was used to estimate impacts since all countries did not start receiving aid at PEPFAR's inception. SETTING: All 157 LMICs from 1990 to 2018. PARTICIPANTS: 90 LMICs receiving PEPFAR aid and cohorts of those countries, including those required to submit annual country operational plans (COP), other recipient countries (non-COP), and three groupings of countries based on cumulative amount of per capita aid received (high, medium, low). INTERVENTIONS: PEPFAR aid to combat the HIV epidemic. PRIMARY OUTCOME MEASURES: Maternal mortality and child mortality rates, vaccination rates to protect children for diphtheria, whooping cough and tetanus, measles, HepB3, and tetanus, and prevalence of anaemia in women of childbearing age. RESULTS: Across PEPFAR recipient countries, large, favourable PEPFAR health effects were found for rates of childhood immunisation, child mortality and maternal mortality. These beneficial health effects were large and significant in all segments of PEPFAR recipient countries studied. We also found significant and favourable programme effects on the prevalence of anaemia in women of childbearing age in PEPFAR recipient countries receiving the most intensive financial support from the PEPFAR programme. Other recipient countries did not demonstrate significant effects on anaemia. CONCLUSIONS: This study demonstrated that important health indicators, beyond HIV, have been consistently and favourably influenced by PEPFAR presence. Child and maternal mortality have been substantially reduced, and childhood immunisation rates increased. We also found no evidence of 'crowding out' or negative spillovers in these resource-poor countries. These findings add to the body of evidence that PEPFAR has had favourable health effects beyond HIV. The implications of these findings are that foreign aid for health in one area may have favourable health effects in other areas in recipient countries. More research is needed on the influence of the mechanisms at work that create these spillover health effects of PEPFAR.

Assessing the use of observational methods and real-world data to emulate ongoing randomized controlled trials.

BACKGROUND/AIMS: There has been growing interest in better understanding the potential of observational research methods in medical product evaluation and regulatory decision-making. Previously, we used linked claims and electronic health record data to emulate two ongoing randomized controlled trials, characterizing the populations and results of each randomized controlled trial prior to publication of its results. Here, our objective was to compare the populations and results from the emulated trials with those of the now-published randomized controlled trials. METHODS: This study compared participants' demographic and clinical characteristics and study results between the emulated trials, which used structured data from OptumLabs Data Warehouse, and the published PRONOUNCE and GRADE trials. First, we examined the feasibility of implementing the baseline participant characteristics included in the published PRONOUNCE and GRADE trials' using real-world data and classified each variable as ascertainable, partially ascertainable, or not ascertainable. Second, we compared the emulated trials and published randomized controlled trials for baseline patient characteristics (concordance determined using standardized mean differences <0.20) and results of the primary and secondary endpoints (concordance determined by direction of effect estimates and statistical significance). RESULTS: The PRONOUNCE trial enrolled 544 participants, and the emulated trial included 2226 propensity score-matched participants. In the PRONOUNCE trial publication, one of the 32 baseline participant characteristics was listed as an exclusion criterion on ClinicalTrials.gov but was ultimately not used. Among the remaining 31 characteristics, 9 (29.0%) were ascertainable, 11 (35.5%) were partially ascertainable, and 10 (32.2%) were not ascertainable using structured data from OptumLabs. For one additional variable, the PRONOUNCE trial did not provide sufficient detail to allow its ascertainment. Of the nine variables that were ascertainable, values in the emulated trial and published randomized controlled trial were discordant for 6 (66.7%). The primary endpoint of time from randomization to the first major adverse cardiovascular event and secondary endpoints of nonfatal myocardial infarction and stroke were concordant between the emulated trial and published randomized controlled trial. The GRADE trial enrolled 5047 participants, and the emulated trial included 7540 participants. In the GRADE trial publication, 8 of 34 (23.5%) baseline participant characteristics were ascertainable, 14 (41.2%) were partially ascertainable, and 11 (32.4%) were not ascertainable using structured data from OptumLabs. For one variable, the GRADE trial did not provide sufficient detail to allow for ascertainment. Of the eight variables that were ascertainable, values in the emulated trial and published randomized controlled trial were discordant for 4 (50.0%). The primary endpoint of time to hemoglobin A1c ≥7.0% was mostly concordant between the emulated trial and the published randomized controlled trial. CONCLUSION: Despite challenges, observational methods and real-world data can be leveraged in certain important situations for a more timely evaluation of drug effectiveness and safety in more diverse and representative patient populations.

Quality of life burden on United States infants and caregivers due to lower respiratory tract infection and adjusting for selective testing: Pilot prospective observational study.

Background and Aims: Policymakers need data about the burden of respiratory syncytial virus (RSV) lower respiratory tract infections (LRTI) among infants. This study estimates quality of life (QoL) for otherwise healthy term US infants with RSV-LRTI and their caregivers, previously limited to premature and hospitalized infants, and corrects for selective testing. Methods: The study enrolled infants <1 year with a clinically diagnosed LRTI encounter between January and May 2021. Using an established 0-100 scale, the 36 infants' and caregivers' QoL at enrollment and quality-adjusted life year losses per 1000 LRTI episodes (quality-adjusted life years [QALYs]/1000) were validated and analyzed. Regression analyses examined predictors of RSV-testing and RSV-positivity, creating modeled positives. Results: Mean QoL at enrollment in outpatient (n = 11) LRTI-tested infants (66.4) was lower than that in not-tested LRTI infants (79.6, p = 0.096). For outpatient LRTI infants (n = 23), median QALYs/1000 losses were 9.8 and 0.25 for their caregivers. RSV-positive outpatient LRTI infants (n = 6) had significantly milder QALYs/1000 losses (7.0) than other LRTI-tested infants (n = 5)(21.8, p = 0.030). Visits earlier in the year were more likely to be RSV-positive than later visits (p = 0.023). Modeled RSV-positivity (51.9%) was lower than the observed rate (55.0%). Infants' and caregivers' QALYs/1000 loss were positively correlated (rho = 0.34, p = 0.046), indicating that infants perceived as sicker imposed greater burdens on caregivers. Conclusions: The overall median QALYs/1000 losses for LRTI (9.0) and RSV-LRTI (5.6) in US infants are substantial, with additional losses for their caregivers (0.25 and 0.20, respectively). These losses extend equally to outpatient episodes. This study is the first reporting QALY losses for infants with LRTI born at term or presenting in nonhospitalized settings, and their caregivers.

Can Observational Analyses of Routinely Collected Data Emulate Randomized Trials? Design and Feasibility of the Observational Patient Evidence for Regulatory Approval Science and Understanding Disease Project.

OBJECTIVES: The Observational Patient Evidence for Regulatory Approval Science and Understanding Disease (OPERAND) project examines whether real-world data (RWD) can be used to inform regulatory decision making. METHODS: OPERAND evaluates whether observational analyses using RWD to emulate index trials can produce effect estimates similar to those of the trials and examines the impact of relaxing the eligibility criteria of the observational analyses to obtain samples that more closely match the real-world populations receiving the treatments. In OPERAND, 2 research teams independently attempt to emulate the ROCKET Atrial Fibrillation and LEAD-2 trials using OptumLabs data. This article describes the design of the project, summarizes the approaches of the 2 research teams, and presents feasibility results for 2 emulations using new-user designs. RESULTS: There were differences in the teams' conceptualizations of the emulation, design decisions for cohort identification, and resulting RWD cohorts. These differences occurred even though both teams were guided by the same index trials and had access to the same source of RWD. CONCLUSIONS: Reasonable alternative design and analysis approaches may be taken to answer the same research question, even when attempting to emulate the same index trial. Researcher decision making is an understudied and potentially important source of variability across RWD analyses.

HARmonized Protocol Template to Enhance Reproducibility of hypothesis evaluating real-world evidence studies on treatment effects: A good practices report of a joint ISPE/ISPOR task force.

PROBLEM: Ambiguity in communication of key study parameters limits the utility of real-world evidence (RWE) studies in healthcare decision-making. Clear communication about data provenance, design, analysis, and implementation is needed. This would facilitate reproducibility, replication in independent data, and assessment of potential sources of bias. WHAT WE DID: The International Society for Pharmacoepidemiology (ISPE) and ISPOR-The Professional Society for Health Economics and Outcomes Research (ISPOR) convened a joint task force, including representation from key international stakeholders, to create a harmonized protocol template for RWE studies that evaluate a treatment effect and are intended to inform decision-making. The template builds on existing efforts to improve transparency and incorporates recent insights regarding the level of detail needed to enable RWE study reproducibility. The overarching principle was to reach for sufficient clarity regarding data, design, analysis, and implementation to achieve 3 main goals. One, to help investigators thoroughly consider, then document their choices and rationale for key study parameters that define the causal question (e.g., target estimand), two, to facilitate decision-making by enabling reviewers to readily assess potential for biases related to these choices, and three, to facilitate reproducibility. STRATEGIES TO DISSEMINATE AND FACILITATE USE: Recognizing that the impact of this harmonized template relies on uptake, we have outlined a plan to introduce and pilot the template with key international stakeholders over the next 2 years. CONCLUSION: The HARmonized Protocol Template to Enhance Reproducibility (HARPER) helps to create a shared understanding of intended scientific decisions through a common text, tabular and visual structure. The template provides a set of core recommendations for clear and reproducible RWE study protocols and is intended to be used as a backbone throughout the research process from developing a valid study protocol, to registration, through implementation and reporting on those implementation decisions.

Procedure for Organizing a Post-FDA-approval Evaluation of Antidepressants.

PURPOSE: The study reports the construction of a cohort used to study the effectiveness of antidepressants. METHODS: The cohort includes experiences of 3,678,082 patients with depression in the United States on antidepressants between January 1, 2001, and December 31, 2018. A total of 10,221,145 antidepressant treatment episodes were analyzed. Patients who had no utilization of health services for at least two years, or who had died, were excluded from the analysis. Follow-up was passive, automatic, and collated from fragmented clinical services of diverse providers. RESULTS: The average follow-up was 2.93 years, resulting in 15,096,055 person-years of data. The mean age of the cohort was 46.54 years (standard deviation of 17.48) at first prescription of antidepressant, which was also the enrollment event (16.92% were over 65 years), and most were female (69.36%). In 10,221,145 episodes, within the first 100 days of start of the episode, 4,729,372 (46.3%) continued their treatment, 1,306,338 (12.8%) switched to another medication, 3,586,156 (35.1%) discontinued their medication, and 599,279 (5.9%) augmented their treatment. CONCLUSIONS: We present a procedure for constructing a cohort using claims data. A surrogate measure for self-reported symptom remission based on the patterns of use of antidepressants has been proposed to address the absence of outcomes in claims. Future studies can use the procedures described here to organize studies of the comparative effectiveness of antidepressants.

HARmonized Protocol Template to Enhance Reproducibility of Hypothesis Evaluating Real-World Evidence Studies on Treatment Effects: A Good Practices Report of a Joint ISPE/ISPOR Task Force.

OBJECTIVES: Ambiguity in communication of key study parameters limits the utility of real-world evidence (RWE) studies in healthcare decision-making. Clear communication about data provenance, design, analysis, and implementation is needed. This would facilitate reproducibility, replication in independent data, and assessment of potential sources of bias. METHODS: The International Society for Pharmacoepidemiology (ISPE) and ISPOR-The Professional Society for Health Economics and Outcomes Research (ISPOR) convened a joint task force, including representation from key international stakeholders, to create a harmonized protocol template for RWE studies that evaluate a treatment effect and are intended to inform decision-making. The template builds on existing efforts to improve transparency and incorporates recent insights regarding the level of detail needed to enable RWE study reproducibility. The over-arching principle was to reach for sufficient clarity regarding data, design, analysis, and implementation to achieve 3 main goals. One, to help investigators thoroughly consider, then document their choices and rationale for key study parameters that define the causal question (e.g., target estimand), two, to facilitate decision-making by enabling reviewers to readily assess potential for biases related to these choices, and three, to facilitate reproducibility. STRATEGIES TO DISSEMINATE AND FACILITATE USE: Recognizing that the impact of this harmonized template relies on uptake, we have outlined a plan to introduce and pilot the template with key international stakeholders over the next 2 years. CONCLUSION: The HARmonized Protocol Template to Enhance Reproducibility (HARPER) helps to create a shared understanding of intended scientific decisions through a common text, tabular and visual structure. The template provides a set of core recommendations for clear and reproducible RWE study protocols and is intended to be used as a backbone throughout the research process from developing a valid study protocol, to registration, through implementation and reporting on those implementation decisions.

Emulating the GRADE trial using real world data: retrospective comparative effectiveness study.

OBJECTIVE: To emulate the GRADE (Glycemia Reduction Approaches in Diabetes: A Comparative Effectiveness Study) trial using real world data before its publication. GRADE directly compared second line glucose lowering drugs for their ability to lower glycated hemoglobin A1c (HbA1c). DESIGN: Observational study. SETTING: OptumLabs® Data Warehouse (OLDW), a nationwide claims database in the US, 25 January 2010 to 30 June 2019. PARTICIPANTS: Adults with type 2 diabetes and HbA1c 6.8-8.5% while using metformin monotherapy, identified according to the GRADE trial specifications, who also used glimepiride, liraglutide, sitagliptin, or insulin glargine. MAIN OUTCOME MEASURES: The primary outcome was time to HbA1c ≥7.0%. Secondary outcomes were time to HbA1c >7.5%, incident microvascular complications, incident macrovascular complications, adverse events, all cause hospital admissions, and all cause mortality. Propensity scores were estimated using the gradient boosting machine method, and inverse propensity score weighting was used to emulate randomization of the treatment groups, which were then compared using Cox proportional hazards regression. RESULTS: 8252 people were identified (19.7% of adults starting the study drugs in OLDW) who met eligibility criteria for the GRADE trial (glimepiride arm=4318, liraglutide arm=690, sitagliptin arm=2993, glargine arm=251). The glargine arm was excluded from analyses owing to small sample size. Median times to HbA1c ≥7.0% were 442 days (95% confidence interval 394 to 480 days) for glimepiride, 764 (741 to not calculable) days for liraglutide, and 427 (380 to 483) days for sitagliptin. Liraglutide was associated with lower risk of reaching HbA1c ≥7.0% compared with glimepiride (hazard ratio 0.57, 95% confidence interval 0.43 to 0.75) and sitagliptin (0.55, 0.41 to 0.73). Results were consistent for the secondary outcome of time to HbA1c >7.5%. No significant differences were observed among treatment groups for the remaining secondary outcomes. CONCLUSIONS: In this emulation of the GRADE trial, liraglutide was statistically significantly more effective at maintaining glycemic control than glimepiride or sitagliptin when added to metformin monotherapy. Generating timely evidence on medical treatments using real world data as a complement to prospective trials is of value.

Machine Learning Methods in Health Economics and Outcomes Research-The PALISADE Checklist: A Good Practices Report of an ISPOR Task Force.

Advances in machine learning (ML) and artificial intelligence offer tremendous potential benefits to patients. Predictive analytics using ML are already widely used in healthcare operations and care delivery, but how can ML be used for health economics and outcomes research (HEOR)? To answer this question, ISPOR established an emerging good practices task force for the application of ML in HEOR. The task force identified 5 methodological areas where ML could enhance HEOR: (1) cohort selection, identifying samples with greater specificity with respect to inclusion criteria; (2) identification of independent predictors and covariates of health outcomes; (3) predictive analytics of health outcomes, including those that are high cost or life threatening; (4) causal inference through methods, such as targeted maximum likelihood estimation or double-debiased estimation-helping to produce reliable evidence more quickly; and (5) application of ML to the development of economic models to reduce structural, parameter, and sampling uncertainty in cost-effectiveness analysis. Overall, ML facilitates HEOR through the meaningful and efficient analysis of big data. Nevertheless, a lack of transparency on how ML methods deliver solutions to feature selection and predictive analytics, especially in unsupervised circumstances, increases risk to providers and other decision makers in using ML results. To examine whether ML offers a useful and transparent solution to healthcare analytics, the task force developed the PALISADE Checklist. It is a guide for balancing the many potential applications of ML with the need for transparency in methods development and findings.

Comparative effectiveness and safety of pharmaceuticals assessed in observational studies compared with randomized controlled trials.

BACKGROUND: There have been ongoing efforts to understand when and how data from observational studies can be applied to clinical and regulatory decision making. The objective of this review was to assess the comparability of relative treatment effects of pharmaceuticals from observational studies and randomized controlled trials (RCTs). METHODS: We searched PubMed and Embase for systematic literature reviews published between January 1, 1990, and January 31, 2020, that reported relative treatment effects of pharmaceuticals from both observational studies and RCTs. We extracted pooled relative effect estimates from observational studies and RCTs for each outcome, intervention-comparator, or indication assessed in the reviews. We calculated the ratio of the relative effect estimate from observational studies over that from RCTs, along with the corresponding 95% confidence interval (CI) for each pair of pooled RCT and observational study estimates, and we evaluated the consistency in relative treatment effects. RESULTS: Thirty systematic reviews across 7 therapeutic areas were identified from the literature. We analyzed 74 pairs of pooled relative effect estimates from RCTs and observational studies from 29 reviews. There was no statistically significant difference (based on the 95% CI) in relative effect estimates between RCTs and observational studies in 79.7% of pairs. There was an extreme difference (ratio < 0.7 or > 1.43) in 43.2% of pairs, and, in 17.6% of pairs, there was a significant difference and the estimates pointed in opposite directions. CONCLUSIONS: Overall, our review shows that while there is no significant difference in the relative risk ratios between the majority of RCTs and observational studies compared, there is significant variation in about 20% of comparisons. The source of this variation should be the subject of further inquiry to elucidate how much of the variation is due to differences in patient populations versus biased estimates arising from issues with study design or analytical/statistical methods.

Real-world Cardiovascular Outcomes Associated With Degarelix vs Leuprolide for Prostate Cancer Treatment.

Importance: With a growing interest in the use of real-world evidence for regulatory decision-making, it is important to understand whether real-world data can be used to emulate the results of randomized clinical trials. Objective: To use electronic health record and administrative claims data to emulate the ongoing PRONOUNCE trial (A Trial Comparing Cardiovascular Safety of Degarelix Versus Leuprolide in Patients With Advanced Prostate Cancer and Cardiovascular Disease). Design, Setting, and Participants: This retrospective, propensity-matched cohort study included adult men with a diagnosis of prostate cancer and cardiovascular disease who initiated either degarelix or leuprolide between December 24, 2008, and June 30, 2019. Participants were commercially insured individuals and Medicare Advantage beneficiaries included in a large US administrative claims database. Exposures: Degarelix or leuprolide. Main Outcomes and Measures: The primary end point was time to first occurrence of a major adverse cardiovascular event (MACE), defined as death due to any cause, myocardial infarction, or stroke, analogous to the PRONOUNCE trial. Secondary end points were time to death due to any cause, myocardial infarction, stroke, and angina. Cox proportional hazards regression was used to evaluate primary and secondary end points. Results: A total of 32 172 men initiated degarelix or leuprolide for prostate cancer; of them, 9490 (29.5%) had cardiovascular disease, and 7800 (24.2%) met the PRONOUNCE trial eligibility criteria and were included in this study. Overall, 165 participants (2.1%) were Asian, 1390 (17.8%) were Black, 663 (8.5%) were Hispanic, and 5258 (67.4%) were White. The mean (SD) age was 74.4 (7.4) years. Among 2226 propensity score-matched patients, no significant difference was observed in the risk of MACE for patients taking degarelix vs those taking leuprolide (10.18 vs 8.60 events per 100 person-years; hazard ratio [HR], 1.18; 95% CI, 0.86-1.61). Degarelix was associated with a higher risk of death from any cause (HR, 1.48; 95% CI, 1.01-2.18) but not of myocardial infarction (HR, 1.16; 95% CI, 0.60-2.25), stroke (HR, 0.92; 95% CI, 0.45-1.85), or angina (HR, 1.36; 95% CI, 0.43-4.27). Conclusions and Relevance: In this emulation of a clinical trial of men with cardiovascular disease undergoing treatment for prostate cancer, degarelix was not associated with a lower risk of cardiovascular events than leuprolide. Comparison of these data with PRONOUNCE trial results, when published, will help enhance our understanding of the appropriate role of using real-world data to emulate clinical trials.

Decomposition analysis as a framework for understanding heterogeneity of treatment effects in non-randomized health care studies.

This paper uses the decomposition framework from the economics literature to examine the statistical structure of treatment effects estimated with observational data compared to those estimated from randomized studies. It begins with the estimation of treatment effects using a dummy variable in regression models and then presents the decomposition method from economics which estimates separate regression models for the comparison groups and recovers the treatment effect using bootstrapping methods. This method shows that the overall treatment effect is a weighted average of structural relationships of patient features with outcomes within each treatment arm and differences in the distributions of these features across the arms. In large randomized trials, it is assumed that the distribution of features across arms is very similar. Importantly, randomization not only balances observed features but also unobserved. Applying high dimensional balancing methods such as propensity score matching to the observational data causes the distributional terms of the decomposition model to be eliminated but unobserved features may still not be balanced in the observational data. Finally, a correction for non-random selection into the treatment groups is introduced via a switching regime model. Theoretically, the treatment effect estimates obtained from this model should be the same as those from a randomized trial. However, there are significant challenges in identifying instrumental variables that are necessary for estimating such models. At a minimum, decomposition models are useful tools for understanding the relationship between treatment effects estimated from observational versus randomized data.