Use of Instrumental Variable Analyses for Evaluating Comparative Effectiveness in Empirical Applications of Oncology: A Systematic Review.

PURPOSE: This systematic review aims to characterize the use and trends of instrumental variables (IVs) in oncology research, assess the quality and completeness of IV reporting, and evaluate the agreement and interpretation of IV results in comparison with other techniques used for determining comparative effectiveness in observational research. METHODS: We performed a systematic search of observational empirical oncology papers evaluating the comparative effectiveness of cancer treatments using IV methods. EMBASE and MEDLINE (through June 2021) were used for a keyword search; Scopus and Web of Science were used for a citation search. Publication details and characteristics of IV analysis and reporting were extracted from each study to examine the uptake and quality of IV applications. RESULTS: Sixty-five empirical papers were identified from February 2001 through June 2021. Geographic variation (50.8%) was the most common type of IV used, and the majority of IV applications constructed binary instruments (53.8%). Concurrent analyses using another non-IV method to adjust for confounding were conducted in 56 (86.2%) studies, 17 (30.4%) of which produced results divergent from IV approaches. We observed a modest uptake of IV methods between 2011 and 2021 together with its dissemination, which remained fairly limited to the United States (76.9%). The quality and completeness of IV reporting varied greatly. The underlying assumptions required for a valid IV analysis were only accounted for in full by 20 (30.8%) studies. CONCLUSION: There are limited use and variable quality of IV analyses in oncology. Future research should look to establish standards to better facilitate the quality, transparency, and completeness of IV reporting in this setting.

Validating Restricted Mean Survival Time Estimates From Reconstructed Kaplan-Meier Data Against Original Trial Individual Patient Data From Trials Conducted by the Canadian Cancer Trials Group.

OBJECTIVES: The development of novel cancer therapies, including immuno-oncology agents, has increased interest in reconstructed individual patient data (IPD) based restricted mean survival time (RMST) analyses. Additionally, reconstructed IPD-based RMST is recommended in cost-effectiveness analyses when original trial IPD are not available. Nevertheless, recently concerns regarding potential bias of reconstructed-IPD RMST have been presented, because reconstructed-IPD RMSTs have not been validated and previous validation endpoints may not capture the entire Kaplan-Meier (KM) curve, especially the "tail." Our study aims to validate the recommended method of IPD reconstruction by comparing reconstructed IPD- and original trial IPD-based RMST. METHODS: Canadian Cancer Trials Group trials from 1990 to 2017 were included. Overall survival and progression-free survival IPD were reconstructed based on published KM curves using the Guyot method. Analysts were blinded to original trial IPD. RMST was calculated at 1 year and over the entire KM curve. Reconstructed-IPD and original trial-IPD (gold-standard) RMSTs were compared for accuracy and predictive error via mean deviation, mean absolute error (MAE), mean percentage bias, and Bland-Altman plots and across KM curve quality (vector traced or bitmapped). RESULTS: We identified 39 trials. The mean deviation, MAE, and mean percentage bias of RMST between the reconstructed IPD and original trial IPD were small. In particular, the mean deviation was -0.01 months and -0.04 months, MAE was 0.19 months and 0.24 months, and mean percentage bias was 0.82% and 0.84% in overall survival KM curves in control and experimental arms, respectively. Accuracy was generally not associated with KM curve quality. CONCLUSIONS: RMST derived from reconstructed IPD displayed excellent accuracy and predictive error compared with the gold standard. Reconstructed IPD could be used to calculate RMST in lieu of original trial IPD, to facilitate decision making for clinicians, researchers, and policy makers.

Parametric Survival Extrapolation of Early Survival Data in Economic Analyses: A Comparison of Projected Versus Observed Updated Survival.

OBJECTIVES: To establish the value of cancer drugs by cost-effectiveness analysis, lifetime parametric survival extrapolations are often fitted to early data. Recent literature suggests that the benefit of cancer agents in primary publications is often different compared with updated data. This study aimed to examine the projected survival based on parametric extrapolations compared with observed survival based on updated data. METHODS: US Food and Drug Administration oncology approvals from January 2006 to December 2015 were reviewed to identify randomized controlled trials, with updated overall survival (OS) or progression-free survival (PFS) data within 5 years. Individual patient data were reconstructed using established methods on initial and updated publications. Projected survival was calculated as the best-fit parametric restricted mean survival time (RMST) based on extrapolated initial Kaplan-Meier curves whereas observed survival was calculated as observed RMST based on updated Kaplan-Meier curves. Mean deviations, mean absolute error (MAE), mean absolute percentage error, and linear regressions were conducted to examine the relationship between projected and observed survival. RESULTS: In total, 32 randomized controlled trials were included. The MAE between the projected RMST and observed RMST was 3.18 months (OS) and 2.84 months (PFS) and absolute percentage error of 100% (OS) and 23% (PFS), suggesting substantial imprecision of the projected RMST in predicting the updated RMST. The linear regression indicated MAE increased as time extrapolated and as the percentage of censored patients increased. CONCLUSIONS: This study demonstrated substantial difference in projected survival between initial and updated publications. Health technology assessment committees need to be aware of the potential uncertainty of incremental effectiveness and resultant value-for-money assessment when making reimbursement decisions based on initial publications with immature survival data.

Mapping Canadian Data Assets to Generate Real-World Evidence: Lessons Learned from Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration's RWE Data Working Group.

Canadian provinces routinely collect patient-level data for administrative purposes. These real-world data (RWD) can be used to generate real-world evidence (RWE) to inform clinical care and healthcare policy. The CanREValue Collaboration is developing a framework for the use of RWE in cancer drug funding decisions. A Data Working Group (WG) was established to identify data assets across Canada for generating RWE of oncology drugs. The mapping exercise was conducted using an iterative scan with informant surveys and teleconference. Data experts from ten provinces convened for a total of three teleconferences and two in-person meetings from March 2018 to September 2019. Following each meeting, surveys were developed and shared with the data experts which focused on identifying databases and data elements, as well as a feasibility assessment of conducting RWE studies using existing data elements and resources. Survey responses were compiled into an interim data report, which was used for public stakeholder consultation. The feedback from the public consultation was used to update the interim data report. We found that databases required to conduct real-world studies are often held by multiple different data custodians. Ninety-seven databases were identified across Canada. Provinces held on average 9 distinct databases (range: 8-11). An Essential RWD Table was compiled that contains data elements that are necessary, at a minimal, to conduct an RWE study. An Expanded RWD Table that contains a more comprehensive list of potentially relevant data elements was also compiled and the availabilities of these data elements were mapped. While most provinces have data on patient demographics (e.g., age, sex) and cancer-related variables (e.g., morphology, topography), the availability and linkability of data on cancer treatment, clinical characteristics (e.g., morphology and topography), and drug costs vary among provinces. Based on current resources, data availability, and access processes, data experts in most provinces noted that more than 12 months would be required to complete an RWE study. The CanREValue Collaboration's Data WG identified key data holdings, access considerations, as well as gaps in oncology treatment-specific data. This data catalogue can be used to facilitate future oncology-specific RWE analyses across Canada.