Implementation of Prediction Models in the Emergency Department from an Implementation Science Perspective-Determinants, Outcomes, and Real-World Impact: A Scoping Review.

STUDY OBJECTIVE: Prediction models offer a promising form of clinical decision support in the complex and fast-paced environment of the emergency department (ED). Despite significant advancements in model development and validation, implementation of such models in routine clinical practice remains elusive. This scoping review aims to survey the current state of prediction model implementation in the ED and to provide insights on contributing factors and outcomes from an implementation science perspective. METHODS: We searched 4 databases from their inception to May 20, 2022: MEDLINE (through PubMed), Embase, Scopus, and CINAHL. Articles that reported implementation outcomes and/or contextual determinants under the Reach, Effectiveness, Adoption, Implementation Maintenance (RE-AIM)/Practical, Robust, Implementation, and Sustainability Model (PRISM) framework were included. Characteristics of studies, models, and results of the RE-AIM/PRISM domains were summarized narratively. RESULTS: Thirty-six reports on 31 implementations were included. The most common prediction models implemented were early warning scores. The most common implementation strategies used were training stakeholders, infrastructural changes, and using evaluative or iterative strategies. Only one report examined ED patients' perspectives, whereas the rest were focused on the experience of health care workers or organizational stakeholders. Key determinants of successful implementation include strong stakeholder engagement, codevelopment of workflows and implementation strategies, education, and usability. CONCLUSION: Examining ED prediction models from an implementation science perspective can provide valuable insights and help guide future implementations.

The Role of Drones in Out-of-Hospital Cardiac Arrest: A Scoping Review.

Drones may be able to deliver automated external defibrillators (AEDs) directly to bystanders of out-of-hospital cardiac arrest (OHCA) events, improving survival outcomes by facilitating early defibrillation. We aimed to provide an overview of the available literature on the role and impact of drones in AED delivery in OHCA. We conducted this scoping review using the PRISMA-ScR and Arksey and O'Malley framework, and systematically searched five bibliographical databases (Medline, EMBASE, Cochrane CENTRAL, PsychInfo and Scopus) from inception until 28 February 2022. After excluding duplicate articles, title/abstract screening followed by full text review was conducted by three independent authors. Data from the included articles were abstracted and analysed, with a focus on potential time savings of drone networks in delivering AEDs in OHCA, and factors that influence its implementation. Out of the 26 included studies, 23 conducted simulations or physical trials to optimise drone network configuration and evaluate time savings from drone delivery of AEDs, compared to the current emergency medical services (EMS), along with 1 prospective trial conducted in Sweden and 2 qualitative studies. Improvements in response times varied across the studies, with greater time savings in rural areas. However, emergency call to AED attachment time was not reduced in the sole prospective study and a South Korean study that accounted for weather and topography. With growing interest in drones and their potential use in AED delivery spurring new research in the field, our included studies demonstrate the potential advantages of unmanned aerial vehicle (UAV) network implementation in controlled environments to deliver AEDs faster than current EMS. However, for these time savings to translate to reduced times to defibrillation and improvement in OHCA outcomes, careful evaluation and addressing of real-world delays, challenges, and barriers to drone use in AED delivery is required.

Benchmarking emergency department prediction models with machine learning and public electronic health records.

The demand for emergency department (ED) services is increasing across the globe, particularly during the current COVID-19 pandemic. Clinical triage and risk assessment have become increasingly challenging due to the shortage of medical resources and the strain on hospital infrastructure caused by the pandemic. As a result of the widespread use of electronic health records (EHRs), we now have access to a vast amount of clinical data, which allows us to develop prediction models and decision support systems to address these challenges. To date, there is no widely accepted clinical prediction benchmark related to the ED based on large-scale public EHRs. An open-source benchmark data platform would streamline research workflows by eliminating cumbersome data preprocessing, and facilitate comparisons among different studies and methodologies. Based on the Medical Information Mart for Intensive Care IV Emergency Department (MIMIC-IV-ED) database, we created a benchmark dataset and proposed three clinical prediction benchmarks. This study provides future researchers with insights, suggestions, and protocols for managing data and developing predictive tools for emergency care.

Development and validation of an interpretable prehospital return of spontaneous circulation (P-ROSC) score for patients with out-of-hospital cardiac arrest using machine learning: A retrospective study.

Background: Return of spontaneous circulation (ROSC) before arrival at the emergency department is an early indicator of successful resuscitation in out-of-hospital cardiac arrest (OHCA). Several ROSC prediction scores have been developed with European cohorts, with unclear applicability in Asian settings. We aimed to develop an interpretable prehospital ROSC (P-ROSC) score for ROSC prediction based on patients with OHCA in Asia. Methods: This retrospective study examined patients who suffered from OHCA between Jan 1, 2009 and Jun 17, 2018 using data recorded in the Pan-Asian Resuscitation Outcomes Study (PAROS) registry. AutoScore, an interpretable machine learning framework, was used to develop P-ROSC. On the same cohort, the P-ROSC was compared with two clinical scores, the RACA and the UB-ROSC. The predictive power was evaluated using the area under the curve (AUC) in the receiver operating characteristic analysis. Findings: 170,678 cases were included, of which 14,104 (8.26%) attained prehospital ROSC. The P-ROSC score identified a new variable, prehospital drug administration, which was not included in the RACA score or the UB-ROSC score. Using only five variables, the P-ROSC score achieved an AUC of 0.806 (95% confidence interval [CI] 0.799-0.814), outperforming both RACA and UB-ROSC with AUCs of 0.773 (95% CI 0.765-0.782) and 0.728 (95% CI 0.718-0.738), respectively. Interpretation: The P-ROSC score is a practical and easily interpreted tool for predicting the probability of prehospital ROSC. Funding: This research received funding from SingHealth Duke-NUS ACP Programme Funding (15/FY2020/P2/06-A79).

Implementation of prediction models in the emergency department from an implementation science perspective-Determinants, outcomes and real-world impact: A scoping review protocol.

The number of prediction models developed for use in emergency departments (EDs) have been increasing in recent years to complement traditional triage systems. However, most of these models have only reached the development or validation phase, and few have been implemented in clinical practice. There is a gap in knowledge on the real-world performance of prediction models in the ED and how they can be implemented successfully into routine practice. Existing reviews of prediction models in the ED have also mainly focused on model development and validation. The aim of this scoping review is to summarize the current landscape and understanding of implementation of predictions models in the ED. This scoping review follows the Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist. We will include studies that report implementation outcomes and/or contextual determinants according to the RE-AIM/PRISM framework for prediction models used in EDs. We will include outcomes or contextual determinants studied at any point of time in the implementation process except for effectiveness, where only post-implementation results will be included. Conference abstracts, theses and dissertations, letters to editors, commentaries, non-research documents and non-English full-text articles will be excluded. Four databases (MEDLINE (through PubMed), Embase, Scopus and CINAHL) will be searched from their inception using a combination of search terms related to the population, intervention and outcomes. Two reviewers will independently screen articles for inclusion and any discrepancy resolved with a third reviewer. Results from included studies will be summarized narratively according to the RE-AIM/PRISM outcomes and domains. Where appropriate, a simple descriptive summary of quantitative outcomes may be performed.

Validation of the CaRdiac Arrest Survival Score (CRASS) for predicting good neurological outcome after out-of-hospital cardiac arrest in an Asian emergency medical service system.

BACKGROUND: Survival with favorable neurological outcomes is an important indicator of successful resuscitation in out-of-hospital cardiac arrest (OHCA). We sought to validate the CaRdiac Arrest Survival Score (CRASS), derived using data from the German Resuscitation Registry, in predicting the likelihood of good neurological outcomes after OHCA in Singapore. METHODS: We conducted a retrospective population-based validation study among EMS-attended OHCA patients (≥18 years) in Singapore, using data from the prospective Pan-Asian Resuscitation Outcomes Study registry. Good neurological outcome was defined as a cerebral performance category of 1 or 2. To evaluate the CRASS score in light of the difference in patient characteristics, we used the default constant coefficient (0.8) and the adjusted coefficient (0.2) to calculate the probability of good neurological outcomes. RESULTS: Out of 11,404 analyzed patients recruited between April 2010 and December 2018, 260 had good and 11,144 had poor neurological function. The CRASS score demonstrated good discrimination, with an area under the curve of 0.963 (95% confidence interval: 0.952-0.974). Using the default constant coefficient of 0.8, the CRASS score consistently overestimated the predicted probability of a good outcome. Following adjustment of the coefficient to 0.2, the CRASS score showed improved calibration. CONCLUSION: CRASS demonstrated good discrimination and moderate calibration in predicting favorable neurological outcomes in the validation Singapore cohort. Our study established a good foundation for future large-scale, cross-country validations of the CRASS score in diverse sociocultural, geographical, and clinical settings.