Diagnostic Performance, Triage Safety, and Usability of a Clinical Decision Support System Within a University Hospital Emergency Department: Algorithm Performance and Usability Study.

Background: Computerized clinical decision support systems (CDSSs) are increasingly adopted in health care to optimize resources and streamline patient flow. However, they often lack scientific validation against standard medical care. Objective: The purpose of this study was to assess the performance, safety, and usability of a CDSS in a university hospital emergency department setting in Kuopio, Finland. Methods: Patients entering the emergency department were asked to voluntarily participate in this study. Patients aged 17 years or younger, patients with cognitive impairments, and patients who entered the unit in an ambulance or with the need for immediate care were excluded. Patients completed the CDSS web-based form and usability questionnaire when waiting for the triage nurse's evaluation. The CDSS data were anonymized and did not affect the patients' usual evaluation or treatment. Retrospectively, 2 medical doctors evaluated the urgency of each patient's condition by using the triage nurse's information, and urgent and nonurgent groups were created. The International Statistical Classification of Diseases, Tenth Revision diagnoses were collected from the electronic health records. Usability was assessed by using a positive version of the System Usability Scale questionnaire. Results: In total, our analyses included 248 patients. Regarding urgency, the mean sensitivities were 85% and 19%, respectively, for urgent and nonurgent cases when assessing the performance of CDSS evaluations in comparison to that of physicians. The mean sensitivities were 85% and 35%, respectively, when comparing the evaluations between the two physicians. Our CDSS did not miss any cases that were evaluated to be emergencies by physicians; thus, all emergency cases evaluated by physicians were evaluated as either urgent cases or emergency cases by the CDSS. In differential diagnosis, the CDSS had an exact match accuracy of 45.5% (97/213). The usability was good, with a mean System Usability Scale score of 78.2 (SD 16.8). Conclusions: In a university hospital emergency department setting with a large real-world population, our CDSS was found to be equally as sensitive in urgent patient cases as physicians and was found to have an acceptable differential diagnosis accuracy, with good usability. These results suggest that this CDSS can be safely assessed further in a real-world setting. A CDSS could accelerate triage by providing patient-provided data in advance of patients' initial consultations and categorize patient cases as urgent and nonurgent cases upon patients' arrival to the emergency department.

Value Mechanisms in the Implementation of Intelligent Patient Flow Management System - A Multiple Case Study.

The purpose of this study was to investigate the value mechanisms in implementing a digital health intervention (DHI) in different contexts and countries. We utilized realist evaluation and the CIMO logic (Context, Intervention, Mechanism, Outcome) to analyze the mechanisms explaining the value capture of Klinik Pro, an Intelligent Patient Flow Management system (IPFM), which is a DHI for seeking of treatment and triage purposes. The study was conducted as a multiple case study using semi-structured interviews to research four market expansions in three countries. In total, seven healthcare mechanisms were discovered: co-creation, proper competence level, coordination, evidence-based medicine, integration, proper timing, demand management. The first four mechanisms were the same in all cases. CIMO framework proved to be useful in the value formulation of the IPFM.

Intelligent Patient Flow Management System at a Primary Healthcare Center - The Effect on Service Use and Costs.

An intelligent patient flow management system (IPFM) was piloted at a large primary healthcare center in Finland in August 2017. The goals of the system are to help patients avoid unnecessary calls and visits to their health center and to enhance the use of professional resources through more streamlined patient pathways and the re-allocation of professionals from assessment tasks to actual patient care. These goals should be reflected in the decreased service costs through optimized contact forms. Using multiple regression analysis, we studied the associations between IPFM and patients' service utilization (17,943 patients; 73,038 service contacts) during the first five months of the pilot in 2017. The results indicated that the use of IPFM by the patient was associated with a decrease of EUR 31 in the total service costs of the patient in the study period. This decrease is 14% of patient's average total service cost.