Predicting 30-day risk from benzodiazepine/Z-drug dispensations in older adults using administrative data: A prognostic machine learning approach.

OBJECTIVE: To develop a machine-learning (ML) model using administrative data to estimate risk of adverse outcomes within 30-days of a benzodiazepine (BZRA) dispensation in older adults for use by health departments/regulators. DESIGN, SETTING AND PARTICIPANTS: This study was conducted in Alberta, Canada during 2018-2019 in Albertans 65 years of age and older. Those with any history of malignancy or palliative care were excluded. EXPOSURE: Each BZRA dispensation from a community pharmacy served as the unit of analysis. MAIN OUTCOMES AND MEASURES: ML algorithms were developed on 2018 administrative data to predict risk of any-cause hospitalization, emergency department visit or death within 30-days of a BZRA dispensation. Validation on 2019 administrative data was done using XGBoost to evaluate discrimination, calibration and other relevant metrics on ranked predictions. Daily and quarterly predictions were simulated on 2019 data. RESULTS: 65,063 study participants were included which represented 633,333 BZRA dispensation during 2018-2019. The validation set had 314,615 dispensations linked to 55,928 all-cause outcomes representing a pre-test probability of 17.8%. C-statistic for the XGBoost model was 0.75. Measuring risk at the end of 2019, the top 0.1 percentile of predicted risk had a LR + of 40.31 translating to a post-test probability of 90%. Daily and quarterly classification simulations resulted in uninformative predictions with positive likelihood ratios less than 10 in all risk prediction categories. Previous history of admissions was ranked highest in variable importance. CONCLUSION: Developing ML models using only administrative health data may not provide health regulators with sufficient informative predictions to use as decision aids for potential interventions, especially if considering daily or quarterly classifications of BZRA risks in older adults. ML models may be informative for this context if yearly classifications are preferred. Health regulators should have access to other types of data to improve ML prediction.

Predicting falls-related admissions in older adults in Alberta, Canada: a machine-learning falls prevention tool developed using population administrative health data.

OBJECTIVE: To construct a machine-learning (ML) model for health systems with organised falls prevention programmes to identify older adults at risk for fall-related admissions. DESIGN: This prognostic study used population-level administrative health data to develop an ML prediction model. SETTING: This study took place in Alberta, Canada during 2018-2019. PARTICIPANTS: Albertans aged 65 and older with at least one prior admission. Those with palliative conditions or emigrated out of Alberta were excluded. EXPOSURE: Unit of analysis was the individual person. MAIN OUTCOMES/MEASURES: We identified fall-related admissions. A CatBoost model was developed on 2018 data to predict risk of fall-related emergency department visits or hospitalisations. Temporal validation was done using 2019 data to evaluate model performance. We reported discrimination, calibration and other relevant metrics measured at the end of 2019 on both ranked predictions and predicted probability thresholds. A cost-savings simulation was performed using 2019 data. RESULTS: Final number of study participants was 224 445. The validation set had 203 584 participants with 19 389 fall-related events (9.5% pretest probability) and an ML model c-statistic of 0.70. The highest ranked predictions had post-test probabilities ranging from 40% to 50%. Net benefit analysis presented mixed results with some net benefit using the ML model in the 6%-30% range. The top 50 percentile of predicted risks represented nearly $C60 million in health system costs related to falls. Intervening on the top 25 or 50 percentiles of predicted risk could realise substantial (up to $C16 million) savings. CONCLUSION: ML prediction models based on population-level administrative data can assist health systems with fall prevention programmes identify older adults at risk of fall-related admissions and reduce costs. ML predictions based on ranked predictions or probability thresholds could guide subsequent interventions to mitigate fall risks. Increased access to diverse forms of data could improve ML performance and further reduce costs.