Development and validation of machine-learning algorithms predicting retention, overdoses, and all-cause mortality among US military veterans treated with buprenorphine for opioid use disorder.

BACKGROUND: Buprenorphine for opioid use disorder (B-MOUD) is essential to improving patient outcomes; however, retention is essential. OBJECTIVE: To develop and validate machine-learning algorithms predicting retention, overdoses, and all-cause mortality among US military veterans initiating B-MOUD. METHODS: Veterans initiating B-MOUD from fiscal years 2006-2020 were identified. Veterans' B-MOUD episodes were randomly divided into training (80%;n = 45,238) and testing samples (20%;n = 11,309). Candidate algorithms [multiple logistic regression, least absolute shrinkage and selection operator regression, random forest (RF), gradient boosting machine (GBM), and deep neural network (DNN)] were used to build and validate classification models to predict six binary outcomes: 1) B-MOUD retention, 2) any overdose, 3) opioid-related overdose, 4) overdose death, 5) opioid overdose death, and 6) all-cause mortality. Model performance was assessed using standard classification statistics [e.g., area under the receiver operating characteristic curve (AUC-ROC)]. RESULTS: Episodes in the training sample were 93.0% male, 78.0% White, 72.3% unemployed, and 48.3% had a concurrent drug use disorder. The GBM model slightly outperformed others in predicting B-MOUD retention (AUC-ROC = 0.72). RF models outperformed others in predicting any overdose (AUC-ROC = 0.77) and opioid overdose (AUC-ROC = 0.77). RF and GBM outperformed other models for overdose death (AUC-ROC = 0.74 for both), and RF and DNN outperformed other models for opioid overdose death (RF AUC-ROC = 0.79; DNN AUC-ROC = 0.78). RF and GBM also outperformed other models for all-cause mortality (AUC-ROC = 0.76 for both). No single predictor accounted for >3% of the model's variance. CONCLUSIONS: Machine-learning algorithms can accurately predict OUD-related outcomes with moderate predictive performance; however, prediction of these outcomes is driven by many characteristics.

Legal Services for Veterans (LSV): Protocol for evaluating the grant-based LSV initiative supporting community organizations' delivery of legal services to veterans.

BACKGROUND: 1.8 million Veterans are estimated to need legal services, such as for housing eviction prevention, discharge upgrades, and state and federal Veterans benefits. While having one's legal needs met is known to improve one's health and its social determinants, many Veterans' legal needs remain unmet. Public Law 116-315 enacted in 2021 authorizes VA to fund legal services for Veterans (LSV) by awarding grants to legal service providers including nonprofit organizations and law schools' legal assistance programs. This congressionally mandated LSV initiative will award grants to about 75 competitively selected entities providing legal services. This paper describes the protocol for evaluating the initiative. The evaluation will fulfill congressional reporting requirements, and inform continued implementation and sustainment of LSV over time. METHODS: Our protocol calls for a prospective, mixed-methods observational study with a repeated measures design, aligning to the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) and Integrated Promoting Action on Research Implementation in Health Services (i-PARIHS) frameworks. In 2023, competitively selected legal services-providing organizations will be awarded grants to implement LSV. The primary outcome will be the number of Veterans served by LSV in the 12 months after the awarding of the grant. The evaluation has three Aims. Aim 1 will focus on measuring primary and secondary LSV implementation outcomes aligned to RE-AIM. Aim 2 will apply the mixed quantitative-qualitative Matrixed Multiple Case Study method to identify patterns in implementation barriers, enablers, and other i-PARIHS-aligned factors that relate to observed outcomes. Aim 3 involves a mixed-methods economic evaluation to understand the costs and benefits of LSV implementation. DISCUSSION: The LSV initiative is a new program that VA is implementing to help Veterans who need legal assistance. To optimize ongoing and future implementation of this program, it is important to rigorously evaluate LSV's outcomes, barriers and enablers, and costs and benefits. We have outlined the protocol for such an evaluation, which will lead to recommending strategies and resource allocation for VA's LSV implementation.

Engaging stakeholders to retrospectively discern implementation strategies to support program evaluation: Proposed method and case study.

BACKGROUND: Availability of evidence-based practices (EBPs) is critical for improving health care outcomes, but diffusion can be challenging. Implementation activities increase the adoption of EBPs and support sustainability. However, when implementation activities are a part of quality improvement processes, evaluation of the time and cost associated with these activities is challenged by the need for a correct classification of these activities to a known taxonomy of implementation strategies by implementation actors. DESIGN: Observational study of a four-stage, stakeholder-engaged process for identifying implementation activities and estimating the associated costs. RESULTS: A national initiative in the Veterans Health Administration (VHA) to improve Advance Care Planning (ACP) via Group Visits (ACP-GV) for rural veterans identified 49 potential implementation activities. Evaluators translated and reduced these to 14 strategies used across three groups with the aid of implementation actors. Data were collected to determine the total implementation effort and applied cost estimates to estimate the budget impact of implementation for VHA. LIMITATIONS: Recall bias may influence the identification of potential implementation activities. CONCLUSIONS: This process improved understanding of the implementation effort and allowed estimation of ACP-GV 's budget impact. IMPLICATIONS: A four-stage, stakeholder-engaged methodology can be applied to other initiatives when a pragmatic evaluation of implementation efforts is needed.

Cost comparison of in-person and telehealth modalities for a suicide safety planning group intervention: interim results from the "Project Life Force" randomized clinical trial.

Suicide prevention is a clinical priority for the US Veterans Health Administration. Evidence-based interventions, including developing a suicide safety plan, are recommended practices and are becoming more widespread. Adaptations to further augment safety planning include a manualized group intervention (Project Life Force, PLF) that combines safety planning with the teaching of skills to maximize use of the plan. A multi-year randomized controlled trial to test efficacy of PLF compared to treatment as usual is currently in progress. However, approximately a year into the study, in-person groups were converted to telehealth groups due to the COVID-19 pandemic. This study compares the per-veteran cost of PLF when delivered in-person versus by telehealth using preliminary trial data from the first 2.5 years of the trial. Cost to deliver PLF was obtained from the Veterans Health Administration's Managerial Cost Accounting data, which relies on activity-based costing. We found no significant differences in the average number of sessions or average group size between in-person and telehealth. However, the cost per group session was lower for the telehealth modality and this led to significant overall per-veteran savings. While efficacy data comparing from the two arms is still underway and we await the ongoing RCT results, our interim cost analysis highlights potential savings with the telehealth modality.

Economic analyses of behavioral health intervention implementation: Perspective on stakeholder engagement.

To provide full potential benefits to patients, behavioral health interventions often require comprehensive and systematic implementation efforts. The costs of these efforts should therefore be included when organizations decide to fund or adopt a new intervention. However, existing guidelines for conducting economic analyses like cost-effectiveness analyses and budget impact analyses are not well-suited to the complexity of the behavioral healthcare pathway and its many stakeholders. Stakeholder engagement, when used effectively with recent innovations in economic analysis, advance more equitable access to interventions for individuals living with behavioral health conditions. But early and ongoing stakeholder engagement has not yet been incorporated into best-practice guidelines for economic evaluation. We discuss our perspective, as researchers and clinicians in a large integrated health system, on how the integration of stakeholder engagement with existing economic analysis methods could improve decision-making about implementation of behavioral health interventions.