The impact of feedback training on prediction of cancer clinical trial results.

INTRODUCTION: Funders must make difficult decisions about which squared treatments to prioritize for randomized trials. Earlier research suggests that experts have no ability to predict which treatments will vindicate their promise. We tested whether a brief training module could improve experts' trial predictions. METHODS: We randomized a sample of breast cancer and hematology-oncology experts to the presence or absence of a feedback training module where experts predicted outcomes for five recently completed randomized controlled trials and received feedback on accuracy. Experts then predicted primary outcome attainment for a sample of ongoing randomized controlled trials. Prediction skill was assessed by Brier scores, which measure the average deviation between their predictions and actual outcomes. Secondary outcomes were discrimination (ability to distinguish between positive and non-positive trials) and calibration (higher predictions reflecting higher probability of trials being positive). RESULTS: A total of 148 experts (46 for breast cancer, 54 for leukemia, and 48 for lymphoma) were randomized between May and December 2017 and included in the analysis (1217 forecasts for 25 trials). Feedback did not improve prediction skill (mean Brier score for control: 0.22, 95% confidence interval = 0.20-0.24 vs feedback arm: 0.21, 95% confidence interval = 0.20-0.23; p = 0.51). Control and feedback arms showed similar discrimination (area under the curve = 0.70 vs 0.73, p = 0.24) and calibration (calibration index = 0.01 vs 0.01, p = 0.81). However, experts in both arms offered predictions that were significantly more accurate than uninformative forecasts of 50% (Brier score = 0.25). DISCUSSION: A short training module did not improve predictions for cancer trial results. However, expert communities showed unexpected ability to anticipate positive trials.Pre-registration record: https://aspredicted.org/4ka6r.pdf.

The proportion of North American cancer trials that evaluate novel targets.

Major advances in cancer care often emerge from the development of novel targets. We randomly sampled 10% of cancer trials on clinicaltrials.gov with start dates 2013-2016 to determine the proportion of trials and research subjects directed at evaluating novel targets. We found that 87 of 378 trials (23.0%) enrolling 9225 of 44,525 patients (20.7%) tested interventions that are directed towards novel targets. 146 of 378 trials (38.6%) enrolling 19,132 of 44,525 patients (43.0%) investigated treatments that were not FDA approved but utilized a previously studied target for treating cancer. Combined, 233 of 378 trials (61.6%) enrolling 28,357 of 44,525 patients (63.9%) investigated treatments that were not FDA approved. Furthermore, 36 of 378 trials (9.5%) enrolling 6592 of 44,525 patients (14.8%) investigated FDA approved anticancer drugs in their approved indication and combination while 109 of 378 trials (28.8%) enrolling 9576 of 44,525 patients (21.5%) investigated FDA approved anticancer drugs outside of their approved indication or combination. Logistic regression found that phase 1 trials were significantly more likely to test novel target interventions than phase 2 and 3 trials (p value = 0.00197 and 0.00130 respectively). Industry sponsored trials were also significantly more likely to involve novel target interventions than non-industry trials (p value <0.001). In conclusion, most cancer trials involve unapproved treatments, but a majority of these treatments are well-characterized or involve a previously studied target to treat cancer.

Risks and benefits of unapproved disease-modifying treatments for neurodegenerative disease.

OBJECTIVE: To determine whether patients randomized to unapproved, disease-modifying interventions in neurodegenerative disease trials have better outcomes than patients randomized to placebo by performing a systematic review and meta-analysis of risk and benefit experienced by patients in randomized placebo-controlled trials testing investigational treatments for Alzheimer disease, Parkinson disease, Huntington disease, or amyotrophic lateral sclerosis (ALS). METHODS: We searched MEDLINE, Embase, and ClinicalTrials.gov for results of randomized trials testing non-Food and Drug Administration-approved, putatively disease-modifying interventions from January 2005 to May 2018. Trial characteristics were double-extracted. Coprimary endpoints were the treatment advantage over placebo on efficacy (standardized mean difference in outcomes) and safety (risk ratios of serious adverse events and withdrawals due to adverse events), calculated with random effects meta-analyses. The study was registered on PROSPERO (CRD42018103798). RESULTS: We included 113 trials (n = 39,875 patients). There was no significant efficacy advantage associated with assignment to putatively disease-modifying interventions compared to placebo for Alzheimer disease (standardized mean difference [SMD] -0.03, 95% confidence interval [CI] -0.07 to 0.01), Parkinson disease (SMD -0.09, 95% CI -0.32 to 0.15), ALS (SMD 0.02, 95% CI -0.25 to 0.30), or Huntington disease (0.02, 95% CI -0.27 to 0.31). Patients with Alzheimer disease assigned to active treatment were at higher risk of experiencing serious adverse events (risk ratio [RR] 1.15, 95% CI 1.04-1.27) and withdrawals due to adverse events (RR 1.44, 95% CI 1.21-1.70). CONCLUSIONS: Assignment to active treatment was not beneficial for any of the indications examined and may have been slightly disadvantageous for patients with Alzheimer disease. Our findings suggest that patients with neurodegenerative diseases are not, on the whole, harmed by assignment to placebo when participating in trials.