Evidence for generalizability of edaravone efficacy using a novel machine learning risk-based subgroup analysis tool.

Introduction: The edaravone development program for amyotrophic lateral sclerosis (ALS) included trials MCI186-16 (Study 16) and MCI186-19 (Study 19). A cohort enrichment strategy was based on a Study 16 post hoc analysis and applied to Study 19 to elucidate a treatment effect in that study. To determine whether the Study 19 results could be generalized to a broader ALS population, we used a machine learning (ML) model to create a novel risk-based subgroup analysis tool. Methods: A validated ML model was used to rank order all Study 16 participants by predicted time to 50% expected vital capacity. Subjects were stratified into nearest-neighbor risk-based subgroups that were systematically expanded to include the entire Study 16 population. For each subgroup, a statistical analysis generated heat maps that revealed statistically significant effect sizes. Results: A broad region of the Study 16 heat map with significant effect sizes was identified, including up to 70% of the trial population. Incorporating participants identified in the cohort enrichment strategy yielded a broad group comprising 76% of the original participants with a statistically significant treatment effect. This broad group spanned the full range of the functional score progression observed in Study 16. Conclusions: This analysis, applying predictions derived using an ML model to a novel methodology for subgroup identification, ascertained a statistically significant edaravone treatment effect in a cohort of participants with broader disease characteristics than the Study 19 inclusion criteria. This novel methodology may assist clinical interpretation of study results and potentially inform efficient future clinical trial design strategies.

Improved stratification of ALS clinical trials using predicted survival.

INTRODUCTION: In small trials, randomization can fail, leading to differences in patient characteristics across treatment arms, a risk that can be reduced by stratifying using key confounders. In ALS trials, riluzole use (RU) and bulbar onset (BO) have been used for stratification. We hypothesized that randomization could be improved by using a multifactorial prognostic score of predicted survival as a single stratifier. METHODS: We defined a randomization failure as a significant difference between treatment arms on a characteristic. We compared randomization failure rates when stratifying for RU and BO ("traditional stratification") to failure rates when stratifying for predicted survival using a predictive algorithm. We simulated virtual trials using the PRO-ACT database without application of a treatment effect to assess balance between cohorts. We performed 100 randomizations using each stratification method - traditional and algorithmic. We applied these stratification schemes to a randomization simulation with a treatment effect using survival as the endpoint and evaluated sample size and power. RESULTS: Stratification by predicted survival met with fewer failures than traditional stratification. Stratifying predicted survival into tertiles performed best. Stratification by predicted survival was validated with an external dataset, the placebo arm from the BENEFIT-ALS trial. Importantly, we demonstrated a substantial decrease in sample size required to reach statistical power. CONCLUSIONS: Stratifying randomization based on predicted survival using a machine learning algorithm is more likely to maintain balance between trial arms than traditional stratification methods. The methodology described here can translate to smaller, more efficient clinical trials for numerous neurological diseases.

Longitudinal modeling to predict vital capacity in amyotrophic lateral sclerosis.

OBJECTIVES: Death in amyotrophic lateral sclerosis (ALS) patients is related to respiratory failure, which is assessed in clinical settings by measuring vital capacity. We developed ALS-VC, a modeling tool for longitudinal prediction of vital capacity in ALS patients. METHODS: A gradient boosting machine (GBM) model was trained using the PRO-ACT (Pooled Resource Open-access ALS Clinical Trials) database of over 10,000 ALS patient records. We hypothesized that a reliable vital capacity predictive model could be developed using PRO-ACT. RESULTS: The model was used to compare FVC predictions with a 30-day run-in period to predictions made from just baseline. The internal root mean square deviations (RMSD) of the run-in and baseline models were 0.534 and 0.539, respectively, across the 7L FVC range captured in PRO-ACT. The RMSDs of the run-in and baseline models using an unrelated, contemporary external validation dataset (0.553 and 0.538, respectively) were comparable to the internal validation. The model was shown to have similar accuracy for predicting SVC (RMSD = 0.562). The most important features for both run-in and baseline models were "Baseline forced vital capacity" and "Days since baseline." CONCLUSIONS: We developed ALS-VC, a GBM model trained with the PRO-ACT ALS dataset that provides vital capacity predictions generalizable to external datasets. The ALS-VC model could be helpful in advising and counseling patients, and, in clinical trials, it could be used to generate virtual control arms against which observed outcomes could be compared, or used to stratify patients into slowly, average, and rapidly progressing subgroups.

Predicting disease progression in amyotrophic lateral sclerosis.

OBJECTIVE: It is essential to develop predictive algorithms for Amyotrophic Lateral Sclerosis (ALS) disease progression to allow for efficient clinical trials and patient care. The best existing predictive models rely on several months of baseline data and have only been validated in clinical trial research datasets. We asked whether a model developed using clinical research patient data could be applied to the broader ALS population typically seen at a tertiary care ALS clinic. METHODS: Based on the PRO-ACT ALS database, we developed random forest (RF), pre-slope, and generalized linear (GLM) models to test whether accurate, unbiased models could be created using only baseline data. Secondly, we tested whether a model could be validated with a clinical patient dataset to demonstrate broader applicability. RESULTS: We found that a random forest model using only baseline data could accurately predict disease progression for a clinical trial research dataset as well as a population of patients being treated at a tertiary care clinic. The RF Model outperformed a pre-slope model and was similar to a GLM model in terms of root mean square deviation at early time points. At later time points, the RF Model was far superior to either model. Finally, we found that only the RF Model was unbiased and was less subject to overfitting than either of the other two models when applied to a clinic population. INTERPRETATION: We conclude that the RF Model delivers superior predictions of ALS disease progression.