Noninterventional studies in the COVID-19 era: methodological considerations for study design and analysis.

The global COVID-19 pandemic has generated enormous morbidity and mortality, as well as large health system disruptions including changes in use of prescription medications, outpatient encounters, emergency department admissions, and hospitalizations. These pandemic-related disruptions are reflected in real-world data derived from electronic medical records, administrative claims, disease or medication registries, and mobile devices. We discuss how pandemic-related disruptions in healthcare utilization may impact the conduct of noninterventional studies designed to characterize the utilization and estimate the effects of medical interventions on health-related outcomes. Using hypothetical studies, we highlight consequences that the pandemic may have on study design elements including participant selection and ascertainment of exposures, outcomes, and covariates. We discuss the implications of these pandemic-related disruptions on possible threats to external validity (participant selection) and internal validity (for example, confounding, selection bias, missing data bias). These concerns may be amplified in populations disproportionately impacted by COVID-19, such as racial/ethnic minorities, rural residents, or people experiencing poverty. We propose a general framework for researchers to carefully consider during the design and analysis of noninterventional studies that use real-world data from the COVID-19 era.

Demonstrating Benefit-Risk Profiles of Novel Therapeutic Strategies in Kidney Transplantation: Opportunities and Challenges of Real-World Evidence.

While great progress has been made in transplantation medicine, long-term graft failure and serious side effects still pose a challenge in kidney transplantation. Effective and safe long-term treatments are needed. Therefore, evidence of the lasting benefit-risk of novel therapies is required. Demonstrating superiority of novel therapies is unlikely via conventional randomized controlled trials, as long-term follow-up in large sample sizes pose statistical and operational challenges. Furthermore, endpoints generally accepted in short-term clinical trials need to be translated to real-world (RW) care settings, enabling robust assessments of novel treatments. Hence, there is an evidence gap that calls for innovative clinical trial designs, with RW evidence (RWE) providing an opportunity to facilitate longitudinal transplant research with timely translation to clinical practice. Nonetheless, the current RWE landscape shows considerable heterogeneity, with few registries capturing detailed data to support the establishment of new endpoints. The main recommendations by leading scientists in the field are increased collaboration between registries for data harmonization and leveraging the development of technology innovations for data sharing under high privacy standards. This will aid the development of clinically meaningful endpoints and data models, enabling future long-term research and ultimately establish optimal long-term outcomes for transplant patients.

Real-world Overall Survival Using Oncology Electronic Health Record Data: Friends of Cancer Research Pilot.

In prior work, Friends of Cancer Research convened multiple data partners to establish standardized definitions for oncology real-world end points derived from electronic health records (EHRs) and claims data. Here, we assessed the performance of real-world overall survival (rwOS) from data sets sourced from EHRs by evaluating the ability of the end point to reflect expected differences from a previous randomized controlled trial across five data sources, after applying inclusion/exclusion criteria. The KEYNOTE-189 clinical trial protocol of platinum doublet chemotherapy (chemotherapy) vs. programmed cell death protein 1 (PD-1) in combination with platinum doublet chemotherapy (PD-1 combination) in first-line nonsquamous metastatic non-small cell lung cancer guided retrospective cohort selection. The Kaplan-Meier product limit estimator was used to calculate 12-month rwOS with 95% confidence intervals (CIs) in each data source. Cox proportional hazards models estimated hazard ratios (HRs) and associated 95% CIs, controlled for prognostic factors. Once the inclusion/exclusion criteria were applied, the five resulting data sets included 155 to 1,501 patients in the chemotherapy cohort and 36 to 405 patients in the PD-1 combination cohort. Twelve-month rwOS ranged from 45% to 58% in the chemotherapy cohort and 44% to 68% in the PD-1 combination cohort. The adjusted HR for death ranged from 0.80 (95% CI: 0.69, 0.93) to 1.15 (95% CI: 0.71, 1.85), controlling for age, gender, performance status, and smoking status. This study yielded insights regarding data capture, including ability of real-world data to precisely identify patient populations and the impact of criteria on end points. Sensitivity analyses could elucidate data set-specific factors that drive results.

The Friends of Cancer Research Real-World Data Collaboration Pilot 2.0: Methodological Recommendations from Oncology Case Studies.

The purpose of this study was to evaluate the potential collective opportunities and challenges of transforming real-world data (RWD) to real-world evidence for clinical effectiveness by focusing on aligning analytic definitions of oncology end points. Patients treated with a qualifying therapy for advanced non-small cell lung cancer in the frontline setting meeting broad eligibility criteria were included to reflect the real-world population. Although a trend toward improved outcomes in patients receiving PD-(L)1 therapy over standard chemotherapy was observed in RWD analyses, the magnitude and consistency of treatment effect was more heterogeneous than previously observed in controlled clinical trials. The study design and analysis process highlighted the identification of pertinent methodological issues and potential innovative approaches that could inform the development of high-quality RWD studies.

A Framework for Extension Studies Using Real-World Data to Examine Long-Term Safety and Effectiveness.

Understanding the long-term benefits and risks of treatments, devices, and vaccines is critically important for individual- and population-level healthcare decision-making. Extension studies, or 'roll-over studies,' are studies that allow for patients participating in a parent clinical trial to 'roll-over' into a subsequent related study to continue to observe and measure long-term safety, tolerability, and/or effectiveness. These designs are not new and are often used as an approach to satisfy regulatory post-approval safety requirements. However, designs using traditional clinical trial infrastructure can be expensive and burdensome to conduct, particularly, when following patients for many years post trial completion. Given the increasing availability and access of real-world data (RWD) sources, direct-to-patient technologies, and novel real-world study designs, there are more cost-efficient approaches to conducting extension studies while assessing important long-term outcomes. Here, we describe various fit-for-purpose design options for extension studies, discuss related methodological considerations, and provide scientific and operational guidance on practices when planning to conduct an extension study using RWD. This manuscript is endorsed by the International Society for Pharmacoepidemiology (ISPE).

Comparing Findings From a Friends of Cancer Research Exploratory Analysis of Real-World End Points With the Cancer Analysis System in England.

PURPOSE: This study compared real-world end points extracted from the Cancer Analysis System (CAS), a national cancer registry with linkage to national mortality and other health care databases in England, with those from diverse US oncology data sources, including electronic health care records, insurance claims, unstructured medical charts, or a combination, that participated in the Friends of Cancer Research Real-World Evidence Pilot Project 1.0. Consistency between data sets and between real-world overall survival (rwOS) was assessed in patients with immunotherapy-treated advanced non-small-cell lung cancer (aNSCLC). PATIENTS AND METHODS: Patients with aNSCLC, diagnosed between January 2013 and December 2017, who initiated treatment with approved programmed death ligand-1 (PD-[L]1) inhibitors until March 2018 were included. Real-world end points, including rwOS and real-world time to treatment discontinuation (rwTTD), were assessed using Kaplan-Meier analysis. A synthetic data set, Simulacrum, on the basis of conditional random sampling of the CAS data was used to develop and refine analysis scripts while protecting patient privacy. RESULTS: Characteristics (age, sex, and histology) of the 2,035 patients with immunotherapy-treated aNSCLC included in the CAS study were broadly comparable with US data sets. In CAS, a higher proportion (46.7%) of patients received a PD-(L)1 inhibitor in the first line than in US data sets (18%-30%). Median rwOS (11.4 months; 95% CI, 10.4 to 12.7) and rwTTD (4.9 months; 95% CI, 4.7 to 5.1) were within the range of US-based data sets (rwOS, 8.6-13.5 months; rwTTD, 3.2-7.0 months). CONCLUSION: The CAS findings were consistent with those from US-based oncology data sets. Such consistency is important for regulatory decision making. Differences observed between data sets may be explained by variation in health care settings, such as the timing of PD-(L)1 approval and reimbursement, and data capture.

Masking in Pragmatic Trials: Who, What, and When to Blind.

Masking (or blinding) of treatment assignment is routinely implemented in classical randomized clinical trials (RCTs) to isolate the effect of the intervention itself and to minimize the potential for bias that could occur with traditional trials. Such biases could be introduced with the conduct, assessment of endpoints, management of conditions, analysis, and reporting when the treatment assignments are known. However, masking of treatments is not only complex but it hinders how generalizable the findings are to the "real world" clinical setting. Pragmatic RCTs (pRCTs) are intended to evaluate the effects of interventions within routine medical care, and as such, do not typically mask treatment groups; moreover, pRCTs assess comparators that are available in routine medical practice, not masked placebos. Whether pRCTs should be masked if intended for regulatory or other purposes has recently been questioned. The literature on pRCTs, while extensive, does not address how much actual benefit is gained from masking outcomes and how masking may affect the "real world" nature of a study. Here, we propose an approach to evaluate sources of bias, describe stakeholders in the conduct of pRCTs who are most likely affected, and offer a framework for considering how masking may be implemented effectively while maintaining generalizability.

An Exploratory Analysis of Real-World End Points for Assessing Outcomes Among Immunotherapy-Treated Patients With Advanced Non-Small-Cell Lung Cancer.

PURPOSE: This pilot study examined the ability to operationalize the collection of real-world data to explore the potential use of real-world end points extracted from data from diverse health care data organizations and to assess how these relate to similar end points in clinical trials for immunotherapy-treated advanced non-small-cell lung cancer. PATIENTS AND METHODS: Researchers from six organizations followed a common protocol using data from administrative claims and electronic health records to assess real-world end points, including overall survival (rwOS), time to next treatment, time to treatment discontinuation (rwTTD), time to progression, and progression-free survival, among patients with advanced non-small-cell lung cancer treated with programmed death 1/programmed death-ligand 1 inhibitors in real-world settings. Data sets included from 269 to 6,924 patients who were treated between January 2011 and October 2017. Results from contributors were anonymized. RESULTS: Correlations between real-world intermediate end points (rwTTD and time to next treatment) and rwOS were moderate to high (range, 0.6 to 0.9). rwTTD was the most consistent end points as treatment detail was available in all data sets. rwOS at 1 year post-programmed death-ligand 1 initiation ranged from 40% to 57%. In addition, rwOS as assessed via electronic health records and claims data fell within the range of median OS values observed in relevant clinical trials. Data sources had been used extensively for research with ongoing data curation to assure accuracy and practical completeness before the initiation of this research. CONCLUSION: These findings demonstrate that real-world end points are generally consistent with each other and with outcomes observed in randomized clinical trials, which substantiates the potential validity of real-world data to support regulatory and payer decision making. Differences observed likely reflect true differences between real-world and protocol-driven practices.

Incretin-mimetic therapies and pancreatic disease: a review of observational data.

BACKGROUND/OBJECTIVE: Signals from the FDA Adverse Event Reporting System (AERS) and pre-clinical and human pancreata obtained from organ donors have suggested that incretin-based therapies used to treat type 2 diabetes mellitus, such as glucagon-like peptide 1 (GLP-1) receptor agonists and dipeptidyl peptidase-4 (DPP-4) inhibitors, may increase the risk of acute pancreatitis (AP) and pancreatic cancer (PC). However, data from observational studies and randomized trials have been conflicting. We conducted a literature review to identify and summarize all observational data published assessing the pancreatic safety of incretins. METHODS: Searches were conducted in MEDLINE via PubMed and Embase using the key terms for the time period of 1 January 2005, to 12 February 2014. A total of 180 articles were screened in abstract form and 49 were subsequently reviewed in full text for inclusion. Data from 12 articles are included in this report. FINDINGS: Data from the FDA AERS database suggest increased risk of AP and PC with GLP-1 receptor agonist and DPP-4 inhibitor use. These findings are not supported by population-based observational studies for either AP or PC; however, studies assessing the relationship between PC and incretin-based therapies are limited. CONCLUSIONS: Current evidence is conflicting and inadequate to conclude whether use of incretin-based therapies increases the risk of AP and PC. Further studies, with the ability to provide long term follow-up, are needed.

Determining triglyceride reductions needed for clinical impact in severe hypertriglyceridemia.

BACKGROUND: Patients with severe hypertriglyceridemia have an increased risk of cardiovascular disease and pancreatitis. Target triglyceride levels associated with clinical benefit for patients with severe hypertriglyceridemia are not currently known. This study evaluates the association between lower follow-up triglyceride levels and incidence of clinical events for patients with severe hypertriglyceridemia. METHODS: By using claims data from 2 large US healthcare databases, we conducted a retrospective cohort study and identified 41,210 adults with severe hypertriglyceridemia (triglycerides ≥ 500 mg/dL) between June 2001 and September 2010. The date of the first severe hypertriglyceridemia laboratory result was the index date. Patients were categorized into 1 of 5 triglyceride ranges (<200 mg/dL, 200-299 mg/dL, 300-399 mg/dL, 400-499 mg/dL, and ≥ 500 mg/dL) based on a follow-up triglyceride level assessed 6 to 24 weeks after initial triglyceride levels were measured. Adjusted Cox regression models were developed to evaluate the impact of follow-up triglyceride levels on rates of pancreatitis episodes and cardiovascular events. RESULTS: The mean age of patients was 50 years, 72% were male, and the mean follow-up was 825 days. Patients with severe hypertriglyceridemia with follow-up triglyceride levels <200 mg/dL experienced a lower rate of pancreatitis episodes (adjusted incidence rate ratio, 0.45; 95% confidence interval, 0.34-0.60) and cardiovascular events (adjusted incidence rate ratio, 0.71; 95% confidence interval, 0.64-0.78) with some clinical benefit in adults with severe hypertriglyceridemia with follow-up triglyceride levels 200 to 299 mg/dL and 300 to 399 mg/dL (P < .001 for trend). CONCLUSIONS: We observed the greatest impact on clinical events among patients with severe hypertriglyceridemia with the lowest follow-up triglyceride levels.

Right answers, wrong questions in clinical research.

To ensure that clinical research arrives at the "right" answers to the right questions for patients, studies should be designed to more closely approximate real-world use of therapeutics and devices.

The economic burden of progressive chronic kidney disease among patients with type 2 diabetes.

AIMS: To estimate the rate of progression of chronic kidney disease (CKD) among patients with type 2 diabetes (T2D) and calculate medical costs associated with progression. METHODS: We conducted a retrospective cohort study of 25,576 members at Kaiser Permanente who had T2D and at least one serum creatinine measurement in 2005. Using estimated glomerular filtration rate (eGFR), we assigned patients to baseline stages of kidney function (stage 0-2, >60ml/min/1.73m(2), n=21,008; stage 3, 30-59, n=3,885; stage 4, 15-29, n=683). We examined all subsequent eGFRs through 2010 to assess progression of kidney disease. Medical costs at baseline and incremental costs during follow-up were assessed. RESULTS: Mean age of patients was 60.6years, 51% were men, and mean diabetes duration was 5.3years. At baseline, 17.9% of patients with T2D also had stage 3 or 4 CKD. Incremental adjusted costs that occurred over follow-up (from baseline) was on average $4569, $12,617, and $33,162 per patient per year higher among patients who progressed from baseline stage 0-2, stage 3, and stage 4 CKD, respectively, compared to those who did not progress. Across all stages of CKD, those who progressed to a higher stage of CKD from baseline had follow-up costs that ranged from 2 to 4 times higher than those who did not progress. CONCLUSIONS: Progression of CKD in T2D drives substantial medical care costs. Interventions designed to minimize decline in progressive kidney function, particularly among patients with stage 3 or 4 CKD, may reduce the economic burden of CKD in T2D.

Longitudinal changes in high-density lipoprotein cholesterol and cardiovascular events in older adults.

OBJECTIVE: While low high-density lipoprotein cholesterol (HDL-C) is associated with increased risk of cardiovascular (CV) events, there are limited data evaluating the association of longitudinal change in HDL-C with CV event risk in older populations. The aim of this study was to examine the association between within-subject changes in HDL-C levels and CV events in an older population. DESIGN: Observational cohort study. PATIENTS: 1293 men and 1422 women age ≥50 years, with ≥2 consecutive HDL measurements, and no prior CVD as part of Framingham Offspring Study. MEASUREMENTS: A clinical CV event was defined as the first occurrence of any of the following: coronary heart disease (coronary death, myocardial infarction, coronary insufficiency and angina), cerebrovascular event, peripheral artery disease or heart failure. RESULTS: Median total follow-up time across subjects was 9·6 years. Change in HDL-C was evaluated as between-exam (approximately 3·5 years) percentage change in HDL-C, categorized as ≥10% decrease, <10% change (stable) and ≥10% increase. Crude and adjusted sex-specific Cox hazards regression models with change in HDL-C as a time-dependent covariate quantified the association with CV events. Mean baseline age of the analysis sample was 53 years. There were 233 and 111 CV events among men and women, respectively. Change in HDL-C was not significantly associated with CVD incidence in men or women, without or with adjustment for confounders including baseline HDL-C or use of relevant medications. CONCLUSION: In conclusion, relatively short-term (3·5 years) changes in HDL-C levels do not affect CV events in men and women.

HDL-C Response Variability to Niacin ER in US Adults.

Background. Niacin is the most effective treatment currently available for raising HDL-C levels. Objective. To evaluate if gender and baseline lipid levels have an effect on the HDL-C response of niacin ER and to identify factors that predict response to niacin ER at the 500 mg dose. Material and Methods. The change in HDL-C effect between baseline and follow-up levels was quantified in absolute change as well as dichotomized into high versus low response (high response was defined as an HDL-C effect of >15% increase and low response was HDL-C <5%) in a sample of 834 individuals. Results. Both males and females with low HDL-C levels at baseline exhibited a response to treatment in the multivariate model (males, HDL-C <40 mg/dL: OR = 5.18, 95% CI: 2.36-11.39; females, HDL-C <50 mg/dL: OR = 5.40, 95% CI: 1.84-15.79). There was also a significant difference in the mean HDL-C effect between baseline and follow-up HDL-C levels in the 500 mg niacin ER dose group for both males (mean HDL-C effect = 0.08, P < 0.001) and females (mean HDL-C effect = 0.10, P = 0.019). Conclusion. Baseline HDL-C levels are the biggest predictor of response to niacin ER treatment for both males and females among the factors evaluated.

Cardiac imaging approaches to evaluate drug-induced myocardial dysfunction.

The ability to make informed benefit-risk assessments for potentially cardiotoxic new compounds is of considerable interest and importance at the public health, drug development, and individual patient levels. Cardiac imaging approaches in the evaluation of drug-induced myocardial dysfunction will likely play an increasing role. However, the optimal choice of myocardial imaging modality and the recommended frequency of monitoring are undefined. These decisions are complicated by the array of imaging techniques, which have varying sensitivities, specificities, availabilities, local expertise, safety, and costs, and by the variable time-course of tissue damage, functional myocardial depression, or recovery of function. This White Paper summarizes scientific discussions of members of the Cardiac Safety Research Consortium on the main factors to consider when selecting nonclinical and clinical cardiac function imaging techniques in drug development. We focus on 3 commonly used imaging modalities in the evaluation of cardiac function: echocardiography, magnetic resonance imaging, and radionuclide (nuclear) imaging and highlight areas for future research.

Clinical and economic benefits observed when follow-up triglyceride levels are less than 500 mg/dL in patients with severe hypertriglyceridemia.

BACKGROUND: Increased levels of triglycerides are associated with an increased risk of cardiovascular disease and pancreatitis. In this study we investigated the association between patients with severely increased triglycerides whose follow-up triglyceride levels were <500 mg/dL and reduction of important clinical events and associated health care costs. METHODS: By using two large U.S. health care claims databases, we identified an initial cohort of 41,210 patients with severe hypertriglyceridemia between June 2001 and September 2010 who had a follow-up laboratory test result 6 to <24 weeks after the initial severe hypertriglyceridemia laboratory value. Of these, 8493 patients' follow-up triglyceride levels remained elevated (≥500 mg/dL) whereas 32,717 were <500 mg/dL. After their qualifying follow-up triglyceride level, patients' cardiovascular events, diabetes-related events, pancreatitis episodes, kidney disease, and related costs were identified. Adjusted incidence rate ratios with the use of Cox proportional hazards models were developed for each outcome. RESULTS: Patients whose triglycerides remained ≥500 mg/dL had a greater rate of pancreatitis episodes (hazard ratio [HR]1.79; 95% confidence interval [CI] 1.47-2.18), cardiovascular events (HR1.19; 95% CI 1.10-1.28), diabetes-related events (HR1.42; 95% CI 1.27-1.59), and kidney disease (HR1.13; 95% CI 1.04-1.22) compared with patients whose follow-up triglycerides were <500 mg/dL, after we adjusted for important confounders. Adjusted all-cause total and cardiovascular-related costs were significantly lower in the first 3 years in patients whose follow-up triglyceride levels were <500 mg/dL compared with those whose triglyceride levels remained increased. CONCLUSION: When follow-up triglyceride levels were <500 mg/dL, we observed an associated reduction in the risk of clinical events and decrease in health care resource use and costs.

Prevalence, clinical characteristics and treatment patterns of low high-density lipoprotein cholesterol in the US population: National Health and Nutrition Examination Survey 2005-2008.

AIMS: The aims of this study were to estimate the prevalence of low high-density lipoprotein cholesterol (HDL-C) in US adults, assess the association between low HDL-C levels and clinical characteristics, and quantify the utilization of dyslipidemic agents as it relates to the distribution of HDL-C. METHODS: We analyzed a sample of 4129 adults (>20 years) who underwent fasting blood evaluations in the National Health and Nutrition Examination Survey (NHANES) 2005-2008. Sex-specific crude and adjusted logistic models were developed to evaluate the association between individual characteristics and low HDL-C, in which low HDL-C was defined as less than 40  mg/dl for men and less than 50  mg/dl for women. RESULTS: Approximately 24% of men and 27% of women had low HDL-C levels. Factors most strongly associated with low HDL-C levels for men included being obese [odds ratio (OR) = 3.27, 95% confidence interval (CI): 1.98-5.40], having elevated triglyceride levels (>200  mg/dl: OR = 8.17, 95% CI: 5.54-12.03) and having apolipoprotein B levels more than 117  mg/dl (OR = 5.99, 95% CI: 2.74-13.13). The same factors were associated with low HDL-C levels among women: being obese (OR = 2.89, 95% CI: 1.78-4.71), having elevated triglyceride levels (>200  mg/dl: OR = 13.35, 95% CI: 7.49-23.77) and having apolipoprotein B levels more than 117  mg/dl (OR = 5.88, 95% CI: 2.29-15.11). Approximately 82% of men and 79% of women with low HDL-C levels reported not using any dyslipidemic medication. CONCLUSION: Although having low HDL-C was common among US adults, few reported taking a dyslipidemic agent. Our study also confirmed some of the known risk factors associated with low HDL-C levels in the general US population.

Prevalence, characteristics, and risk factors of elevated triglyceride levels in US children.

Limited information is available on the epidemiology of hypertriglyceridemia (HTG; 150-499 mg/dL) and severe HTG (SHTG; >500 mg/dL) in children. This study estimates the prevalence of HTG and SHTG, evaluates factors that may be associated with these conditions, and describes the use of dyslipidemic agents in children. The sample included children 12 to 19 years old who participated in National Health and Nutrition Examination Survey (NHANES) 2001-2008 (n = 3248) and children 5 to 19 years of age who were part of a large managed-care claims database in the United States (n = 65 258). Results from NHANES confirm the rarity of SHTG in the US pediatric population (ie, 0.2%). Factors statistically significantly associated with having HTG or SHTG in the claims database were being male, 12 to 19 years old, having high low-density lipoprotein (LDL), having low high-density lipoprotein (HDL), diabetes, and psychological disorders. Fibrates were the most commonly prescribed triglyceride-lowering agent among children with SHTG, followed by statins and Lovaza.