Clinical support system for triage based on federated learning for the Korea triage and acuity scale.

Background and aims: This study developed a clinical support system based on federated learning to predict the need for a revised Korea Triage Acuity Scale (KTAS) to facilitate triage. Methods: This was a retrospective study that used data from 11,952,887 patients in the Korean National Emergency Department Information System (NEDIS) from 2016 to 2018 for model development. Separate cohorts were created based on the emergency medical center level in the NEDIS: regional emergency medical center (REMC), local emergency medical center (LEMC), and local emergency medical institution (LEMI). External and temporal validation used data from emergency department (ED) of the study site from 2019 to 2021. Patient features obtained during the triage process and the initial KTAS scores were used to develop the prediction model. Federated learning was used to rectify the disparity in data quality between EDs. The patient's demographic information, vital signs in triage, mental status, arrival information, and initial KTAS were included in the input feature. Results: 3,626,154 patients' visits were included in the regional emergency medical center cohort; 8,278,081 patients' visits were included in the local emergency medical center cohort; and 48,652 patients' visits were included in the local emergency medical institution cohort. The study site cohort, which is used for external and temporal validation, included 135,780 patients visits. Among the patients in the REMC and study site cohorts, KTAS level 3 patients accounted for the highest proportion at 42.4% and 45.1%, respectively, whereas in the LEMC and LEMI cohorts, KTAS level 4 patients accounted for the highest proportion. The area under the receiver operating characteristic curve for the prediction model was 0.786, 0.750, and 0.770 in the external and temporal validation. Patients with revised KTAS scores had a higher admission rate and ED mortality rate than those with unaltered KTAS scores. Conclusions: This novel system might accurately predict the likelihood of KTAS acuity revision and support clinician-based triage.

Feasibility Study of Federated Learning on the Distributed Research Network of OMOP Common Data Model.

OBJECTIVES: Since protecting patients' privacy is a major concern in clinical research, there has been a growing need for privacy-preserving data analysis platforms. For this purpose, a federated learning (FL) method based on the Observational Medical Outcomes Partnership (OMOP) common data model (CDM) was implemented, and its feasibility was demonstrated. METHODS: We implemented an FL platform on FeederNet, which is a distributed clinical data analysis platform based on the OMOP CDM in Korea. We trained it through an artificial neural network (ANN) using data from patients who received steroid prescriptions or injections, with the aim of predicting the occurrence of side effects depending on the prescribed dose. The ANN was trained using the FL platform with the OMOP CDMs of Kyung Hee University Medical Center (KHMC) and Ajou University Hospital (AUH). RESULTS: The area under the receiver operating characteristic curves (AUROCs) for predicting bone fracture, osteonecrosis, and osteoporosis using only data from each hospital were 0.8426, 0.6920, and 0.7727 for KHMC and 0.7891, 0.7049, and 0.7544 for AUH, respectively. In contrast, when using FL, the corresponding AUROCs were 0.8260, 0.7001, and 0.7928 for KHMC and 0.7912, 0.8076, and 0.7441 for AUH, respectively. In particular, FL led to a 14% improvement in performance for osteonecrosis at AUH. CONCLUSIONS: FL can be performed with the OMOP CDM, and FL often shows better performance than using only a single institution's data. Therefore, research using OMOP CDM has been expanded from statistical analysis to machine learning so that researchers can conduct more diverse research.

Federated Learning on Clinical Benchmark Data: Performance Assessment.

BACKGROUND: Federated learning (FL) is a newly proposed machine-learning method that uses a decentralized dataset. Since data transfer is not necessary for the learning process in FL, there is a significant advantage in protecting personal privacy. Therefore, many studies are being actively conducted in the applications of FL for diverse areas. OBJECTIVE: The aim of this study was to evaluate the reliability and performance of FL using three benchmark datasets, including a clinical benchmark dataset. METHODS: To evaluate FL in a realistic setting, we implemented FL using a client-server architecture with Python. The implemented client-server version of the FL software was deployed to Amazon Web Services. Modified National Institute of Standards and Technology (MNIST), Medical Information Mart for Intensive Care-III (MIMIC-III), and electrocardiogram (ECG) datasets were used to evaluate the performance of FL. To test FL in a realistic setting, the MNIST dataset was split into 10 different clients, with one digit for each client. In addition, we conducted four different experiments according to basic, imbalanced, skewed, and a combination of imbalanced and skewed data distributions. We also compared the performance of FL to that of the state-of-the-art method with respect to in-hospital mortality using the MIMIC-III dataset. Likewise, we conducted experiments comparing basic and imbalanced data distributions using MIMIC-III and ECG data. RESULTS: FL on the basic MNIST dataset with 10 clients achieved an area under the receiver operating characteristic curve (AUROC) of 0.997 and an F1-score of 0.946. The experiment with the imbalanced MNIST dataset achieved an AUROC of 0.995 and an F1-score of 0.921. The experiment with the skewed MNIST dataset achieved an AUROC of 0.992 and an F1-score of 0.905. Finally, the combined imbalanced and skewed experiment achieved an AUROC of 0.990 and an F1-score of 0.891. The basic FL on in-hospital mortality using MIMIC-III data achieved an AUROC of 0.850 and an F1-score of 0.944, while the experiment with the imbalanced MIMIC-III dataset achieved an AUROC of 0.850 and an F1-score of 0.943. For ECG classification, the basic FL achieved an AUROC of 0.938 and an F1-score of 0.807, and the imbalanced ECG dataset achieved an AUROC of 0.943 and an F1-score of 0.807. CONCLUSIONS: FL demonstrated comparative performance on different benchmark datasets. In addition, FL demonstrated reliable performance in cases where the distribution was imbalanced, skewed, and extreme, reflecting the real-life scenario in which data distributions from various hospitals are different. FL can achieve high performance while maintaining privacy protection because there is no requirement to centralize the data.