Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence-Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation.

BACKGROUND: Despite the increasing availability of clinical decision support systems (CDSSs) and rising expectation for CDSSs based on artificial intelligence (AI), little is known about the acceptance of AI-based CDSS by physicians and its barriers and facilitators in emergency care settings. OBJECTIVE: We aimed to evaluate the acceptance, barriers, and facilitators to implementing AI-based CDSSs in the emergency care setting through the opinions of physicians on our newly developed, real-time AI-based CDSS, which alerts ED physicians by predicting aortic dissection based on numeric and text information from medical charts, by using the Unified Theory of Acceptance and Use of Technology (UTAUT; for quantitative evaluation) and the Consolidated Framework for Implementation Research (CFIR; for qualitative evaluation) frameworks. METHODS: This mixed methods study was performed from March to April 2021. Transitional year residents (n=6), emergency medicine residents (n=5), and emergency physicians (n=3) from two community, tertiary care hospitals in Japan were included. We first developed a real-time CDSS for predicting aortic dissection based on numeric and text information from medical charts (eg, chief complaints, medical history, vital signs) with natural language processing. This system was deployed on the internet, and the participants used the system with clinical vignettes of model cases. Participants were then involved in a mixed methods evaluation consisting of a UTAUT-based questionnaire with a 5-point Likert scale (quantitative) and a CFIR-based semistructured interview (qualitative). Cronbach α was calculated as a reliability estimate for UTAUT subconstructs. Interviews were sampled, transcribed, and analyzed using the MaxQDA software. The framework analysis approach was used during the study to determine the relevance of the CFIR constructs. RESULTS: All 14 participants completed the questionnaires and interviews. Quantitative analysis revealed generally positive responses for user acceptance with all scores above the neutral score of 3.0. In addition, the mixed methods analysis identified two significant barriers (System Performance, Compatibility) and two major facilitators (Evidence Strength, Design Quality) for implementation of AI-based CDSSs in emergency care settings. CONCLUSIONS: Our mixed methods evaluation based on theoretically grounded frameworks revealed the acceptance, barriers, and facilitators of implementation of AI-based CDSS. Although the concern of system failure and overtrusting of the system could be barriers to implementation, the locality of the system and designing an intuitive user interface could likely facilitate the use of optimal AI-based CDSS. Alleviating and resolving these factors should be key to achieving good user acceptance of AI-based CDSS.

Initial assessment in emergency departments by chief complaint and respiratory rate.

BACKGROUND: Understanding heterogeneity of the respiratory rate (RR) as a risk stratification marker across chief complaints is important to reduce misinterpretation of the risk posed by outcome events and to build accurate risk stratification tools. This study was conducted to investigate the associations between RR and clinical outcomes according to the five most frequent chief complaints in an emergency department (ED): fever, shortness of breath, altered mental status, chest pain, and abdominal pain. METHODS: This retrospective cohort study examined ED data of all adult patients who visited the ED of a tertiary medical center during April 2018-September 2019. The primary exposure was RR at the ED visit. Outcome measures were hospitalization and mechanical ventilation use. We used restrictive cubic spline and logistic regression models to assess the association of interest. RESULTS: Of 16 956 eligible ED patients, 4926 (29%) required hospitalization; 448 (3%) required mechanical ventilation. Overall, U-shaped associations were found between RR and the risk of hospitalization (eg, using RR = 16 as the reference, the odds ratio [OR] of RR = 32, 6.57 [95% CI 5.87-7.37]) and between RR and the risk of mechanical ventilation. This U-shaped association was driven by patients' association with altered mental status (eg, OR of RR = 12, 2.63 [95% CI 1.25-5.53]). For patients who have fever or shortness of breath, the risk of hospitalization increased monotonously with increased RR. CONCLUSIONS: U-shaped associations of RR with the risk of overall clinical outcomes were found. These associations varied across chief complaints.

Institution-Specific Machine Learning Models for Prehospital Assessment to Predict Hospital Admission: Prediction Model Development Study.

BACKGROUND: Although multiple prediction models have been developed to predict hospital admission to emergency departments (EDs) to address overcrowding and patient safety, only a few studies have examined prediction models for prehospital use. Development of institution-specific prediction models is feasible in this age of data science, provided that predictor-related information is readily collectable. OBJECTIVE: We aimed to develop a hospital admission prediction model based on patient information that is commonly available during ambulance transport before hospitalization. METHODS: Patients transported by ambulance to our ED from April 2018 through March 2019 were enrolled. Candidate predictors were age, sex, chief complaint, vital signs, and patient medical history, all of which were recorded by emergency medical teams during ambulance transport. Patients were divided into two cohorts for derivation (3601/5145, 70.0%) and validation (1544/5145, 30.0%). For statistical models, logistic regression, logistic lasso, random forest, and gradient boosting machine were used. Prediction models were developed in the derivation cohort. Model performance was assessed by area under the receiver operating characteristic curve (AUROC) and association measures in the validation cohort. RESULTS: Of 5145 patients transported by ambulance, including deaths in the ED and hospital transfers, 2699 (52.5%) required hospital admission. Prediction performance was higher with the addition of predictive factors, attaining the best performance with an AUROC of 0.818 (95% CI 0.792-0.839) with a machine learning model and predictive factors of age, sex, chief complaint, and vital signs. Sensitivity and specificity of this model were 0.744 (95% CI 0.716-0.773) and 0.745 (95% CI 0.709-0.776), respectively. CONCLUSIONS: For patients transferred to EDs, we developed a well-performing hospital admission prediction model based on routinely collected prehospital information including chief complaints.