Automatic Classification of Thyroid Findings Using Static and Contextualized Ensemble Natural Language Processing Systems: Development Study.

BACKGROUND: In the case of Korean institutions and enterprises that collect nonstandardized and nonunified formats of electronic medical examination results from multiple medical institutions, a group of experienced nurses who can understand the results and related contexts initially classified the reports manually. The classification guidelines were established by years of workers' clinical experiences and there were attempts to automate the classification work. However, there have been problems in which rule-based algorithms or human labor-intensive efforts can be time-consuming or limited owing to high potential errors. We investigated natural language processing (NLP) architectures and proposed ensemble models to create automated classifiers. OBJECTIVE: This study aimed to develop practical deep learning models with electronic medical records from 284 health care institutions and open-source corpus data sets for automatically classifying 3 thyroid conditions: healthy, caution required, and critical. The primary goal is to increase the overall accuracy of the classification, yet there are practical and industrial needs to correctly predict healthy (negative) thyroid condition data, which are mostly medical examination results, and minimize false-negative rates under the prediction of healthy thyroid conditions. METHODS: The data sets included thyroid and comprehensive medical examination reports. The textual data are not only documented in fully complete sentences but also written in lists of words or phrases. Therefore, we propose static and contextualized ensemble NLP network (SCENT) systems to successfully reflect static and contextual information and handle incomplete sentences. We prepared each convolution neural network (CNN)-, long short-term memory (LSTM)-, and efficiently learning an encoder that classifies token replacements accurately (ELECTRA)-based ensemble model by training or fine-tuning them multiple times. Through comprehensive experiments, we propose 2 versions of ensemble models, SCENT-v1 and SCENT-v2, with the single-architecture-based CNN, LSTM, and ELECTRA ensemble models for the best classification performance and practical use, respectively. SCENT-v1 is an ensemble of CNN and ELECTRA ensemble models, and SCENT-v2 is a hierarchical ensemble of CNN, LSTM, and ELECTRA ensemble models. SCENT-v2 first classifies the 3 labels using an ELECTRA ensemble model and then reclassifies them using an ensemble model of CNN and LSTM if the ELECTRA ensemble model predicted them as "healthy" labels. RESULTS: SCENT-v1 outperformed all the suggested models, with the highest F1 score (92.56%). SCENT-v2 had the second-highest recall value (94.44%) and the fewest misclassifications for caution-required thyroid condition while maintaining 0 classification error for the critical thyroid condition under the prediction of the healthy thyroid condition. CONCLUSIONS: The proposed SCENT demonstrates good classification performance despite the unique characteristics of the Korean language and problems of data lack and imbalance, especially for the extremely low amount of critical condition data. The result of SCENT-v1 indicates that different perspectives of static and contextual input token representations can enhance classification performance. SCENT-v2 has a strong impact on the prediction of healthy thyroid conditions.

Deep-learning-based automated terminology mapping in OMOP-CDM.

OBJECTIVE: Accessing medical data from multiple institutions is difficult owing to the interinstitutional diversity of vocabularies. Standardization schemes, such as the common data model, have been proposed as solutions to this problem, but such schemes require expensive human supervision. This study aims to construct a trainable system that can automate the process of semantic interinstitutional code mapping. MATERIALS AND METHODS: To automate mapping between source and target codes, we compute the embedding-based semantic similarity between corresponding descriptive sentences. We also implement a systematic approach for preparing training data for similarity computation. Experimental results are compared to traditional word-based mappings. RESULTS: The proposed model is compared against the state-of-the-art automated matching system, which is called Usagi, of the Observational Medical Outcomes Partnership common data model. By incorporating multiple negative training samples per positive sample, our semantic matching method significantly outperforms Usagi. Its matching accuracy is at least 10% greater than that of Usagi, and this trend is consistent across various top-k measurements. DISCUSSION: The proposed deep learning-based mapping approach outperforms previous simple word-level matching algorithms because it can account for contextual and semantic information. Additionally, we demonstrate that the manner in which negative training samples are selected significantly affects the overall performance of the system. CONCLUSION: Incorporating the semantics of code descriptions more significantly increases matching accuracy compared to traditional text co-occurrence-based approaches. The negative training sample collection methodology is also an important component of the proposed trainable system that can be adopted in both present and future related systems.

Understanding Time Series Patterns of Weight and Meal History Reports in Mobile Weight Loss Intervention Programs: Data-Driven Analysis.

BACKGROUND: Mobile apps for weight loss provide users with convenient features for recording lifestyle and health indicators; they have been widely used for weight loss recently. Previous studies in this field generally focused on the relationship between the cumulative nature of self-reported data and the results in weight loss at the end of the diet period. Therefore, we conducted an in-depth study to explore the relationships between adherence to self-reporting and weight loss outcomes during the weight reduction process. OBJECTIVE: We explored the relationship between adherence to self-reporting and weight loss outcomes during the time series weight reduction process with the following 3 research questions: "How does adherence to self-reporting of body weight and meal history change over time?", "How do weight loss outcomes depend on weight changes over time?", and "How does adherence to the weight loss intervention change over time by gender?" METHODS: We analyzed self-reported data collected weekly for 16 weeks (January 2017 to March 2018) from 684 Korean men and women who participated in a mobile weight loss intervention program provided by a mobile diet app called Noom. Analysis of variance (ANOVA) and chi-squared tests were employed to determine whether the baseline characteristics among the groups of weight loss results were different. Based on the ANOVA results and slope analysis of the trend indicating participant behavior along the time axis, we explored the relationship between adherence to self-reporting and weight loss results. RESULTS: Adherence to self-reporting levels decreased over time, as previous studies have found. BMI change patterns (ie, absolute BMI values and change in BMI values within a week) changed over time and were characterized in 3 time series periods. The relationships between the weight loss outcome and both meal history and self-reporting patterns were gender-dependent. There was no statistical association between adherence to self-reporting and weight loss outcomes in the male participants. CONCLUSIONS: Although mobile technology has increased the convenience of self-reporting when dieting, it should be noted that technology itself is not the essence of weight loss. The in-depth understanding of the relationship between adherence to self-reporting and weight loss outcome found in this study may contribute to the development of better weight loss interventions in mobile environments.

Enchanted Life Space: Adding Value to Smart Health by Integrating Human Desires.

OBJECTIVES: Developments in advanced technology have unlocked an era of smart health, transforming healthcare practices inside and outside hospitals for both medical staff and patients. It is now possible for patients to collect detailed health data using smartphones and wearable devices, regardless of their physical location or time zone. The use of these patient-generated data holds great promise for future healthcare advancements in many ways; however, current strategies for smart-health technologies tend to focus on the smartness of the technology itself and on managing a particular disease or condition. Moreover, opportunities for people within the healthcare system to experience the benefits of these innovations are still limited. METHODS: An expert workshop was held to discuss the current limitations of smart health, where each expert gave a presentation on their particular expertise, followed by an exchange of ideas for the purpose of drawing conclusions. RESULTS: 'Smartness' should not be the ultimate value for patients using smart technologies; instead of focusing on individual smart devices, we should consider the space around people and their relation to each object so that the combination of space and objects brings an 'enchanted' experience to user. CONCLUSIONS: An 'enchanted' experience can only be possible when monitoring provides the user with a comfortable life and satisfies their needs and desires sufficiently. Only when the novelty of the device's smartness effectively connects people with the space around them and focuses on human desires can it be cost effective and value creating.

Learning representations for the early detection of sepsis with deep neural networks.

BACKGROUND: Sepsis is one of the leading causes of death in intensive care unit patients. Early detection of sepsis is vital because mortality increases as the sepsis stage worsens. OBJECTIVE: This study aimed to develop detection models for the early stage of sepsis using deep learning methodologies, and to compare the feasibility and performance of the new deep learning methodology with those of the regression method with conventional temporal feature extraction. METHOD: Study group selection adhered to the InSight model. The results of the deep learning-based models and the InSight model were compared. RESULTS: With deep feedforward networks, the area under the ROC curve (AUC) of the models were 0.887 and 0.915 for the InSight and the new feature sets, respectively. For the model with the combined feature set, the AUC was the same as that of the basic feature set (0.915). For the long short-term memory model, only the basic feature set was applied and the AUC improved to 0.929 compared with the existing 0.887 of the InSight model. CONCLUSIONS: The contributions of this paper can be summarized in three ways: (i) improved performance without feature extraction using domain knowledge, (ii) verification of feature extraction capability of deep neural networks through comparison with reference features, and (iii) improved performance with feedforward neural networks using long short-term memory, a neural network architecture that can learn sequential patterns.

Differences among admitting departments in alerts and alert overrides for drug-drug interaction.

OBJECTIVE: To determine differences in the incidence and risk factors of alerts for drug-drug interaction (DDI) and the rate of alert overrides by an admitting department. METHODS: A retrospective cohort study was performed using electronic health records of a Korean tertiary teaching hospital including all hospitalized adult patients for 18 months. The main outcome measures included incidence rates of alerts for DDI and their override, hazard ratios (HRs) for DDI alerts, and odds ratios (ORs) for alert overrides by admitting department (emergency department [ED], general ward [GW], and intensive care unit [ICU]) after adjusting for other known risk factors. RESULTS: Among 102 379 incident admissions, 6060 had alerts for DDI (5.4/person-year). After adjusting for covariates, patients admitted to the ED (HR, 4.02; confidence interval [CI], 3.69-4.38) or ICU (HR, 1.62; CI, 1.29-2.04) showed higher risks for DDI compared with those admitted to the GW. The alert-override rate was significantly higher in the ED (OR 1.68) than in the GW; however, there was no significant difference between GW and ICU. The prevalence of DDI alerts and their override rate were also demonstrated. DISCUSSION: The incidence of DDI and the alert-override rate differed by admitting department. The ED and ICU were associated with higher risks for alerts on DDI than did the GW after adjusting for other known risk factors. CONCLUSIONS: Admitting department was an independent risk factor for alerts and alert overrides. Strategies to reduce alerts and alert overrides should consider the admitting department.

Prediction of 4-year risk for coronary artery calcification using ensemble-based classification.

The progression of coronary artery calcification (CAC) has been regarded as an important risk factor of coronary artery disease (CAD), which is the biggest cause of death. Because CAC occurrence increases the risk of CAD by a factor of ten, the one whose coronary artery is calcified should pay more attention to the health management. However, performing the computerized tomography (CT) scan to check if coronary artery is calcified as a regular examination might be inefficient due to its high cost. Therefore, it is required to identify high risk persons who need regular follow-up checks of CAC or low risk ones who can avoid unnecessary CT scans. Due to this reason, we develop a 4-year prediction model for a new occurrence of CAC based on data collected by the regular health examination. We build the prediction model using ensemble-based methods to handle imbalanced dataset. Experimental results show that the developed prediction models provided a reasonable accuracy (AUC 75%), which is about 5% higher than the model built by the other imbalanced classification method.

Identifying relatively high-risk group of coronary artery calcification based on progression rate: statistical and machine learning methods.

Coronary artery calcification (CAC) score is an important predictor of coronary artery disease (CAD), which is the primary cause of death in advanced countries. Early prediction of high-risk of CAC based on progression rate enables people to prevent CAD from developing into severe symptoms and diseases. In this study, we developed various classifiers to identify patients in high risk of CAC using statistical and machine learning methods, and compared them with performance accuracy. For statistical approaches, linear regression based classifier and logistic regression model were developed. For machine learning approaches, we suggested three kinds of ensemble-based classifiers (best, top-k, and voting method) to deal with imbalanced distribution of our data set. Ensemble voting method outperformed all other methods including regression methods as AUC was 0.781.

Integration of heterogeneous clinical decision support systems and their knowledge sets: feasibility study with Drug-Drug Interaction alerts.

There exist limitations in both commercial and in-house clinical decision support systems (CDSSs) and issues related to the integration of different knowledge sources and CDSSs. We chose Standard-based Shareable Active Guideline Environment (SAGE) as a new architecture with knowledge integration and a centralized knowledge base which includes authoring/management functions and independent CDSS, and applied it to Drug-Drug Interaction (DDI) CDSS. The aim of this study was to evaluate the feasibility of the newly integrated DDI alerting CDSS into a real world hospital information system involving construction of an integrated CDSS derived from two heterogeneous systems and their knowledge sets. The proposed CDSS was successfully implemented and compensated for the weaknesses of the old CDSS from knowledge integration and management, and its applicability in actual situations was verified. Although the DDI CDSS was constructed as an example case, the new CDS architecture might prove applicable to areas of CDSSs.

High-Resolution Actigraphic Analysis of ADHD: A Wide Range of Movement Variability Observation in Three School Courses - A Pilot Study.

OBJECTIVES: This study was conducted to determine whether or not newly proposed high-resolution activity features could provide a superior analytic foundation compared to those commonly used to assess transitions in children's activities, under circumstances in which the types of courses attended exert different situational effects on activity levels. METHODS: From 153 children at a local elementary school, 10 subjects with attention deficit hyperactivity disorder (ADHD) and 7 controls were recruited. Their activity data was collected using an actigraph while they attended school. Ratios of partitioned activity ranges (0.5-2.8 G) during the entire activity were extracted during three classes: art, mathematics, and native language (Korean). Extracted activity features for each participant were compared between the two groups of children (ADHD and control) using graphs and statistical analysis. RESULTS: Activity distributions between ADHD and control groups for each class showed statistically significant differences spread through the entire range in art class compared to native language and mathematics classes. The ADHD group, but not the control group, experienced many significantly different intervals (> 50%) having low to very high activity acceleration regions during the art and languages courses. CONCLUSIONS: Class content appears to influence the activity patterns of ADHD children. Monitoring the actual magnitude and activity counts in a wide range of subjects could facilitate the examination of distributions or patterns of activities. Objective activity measurements made with an actigraph may be useful for monitoring changes in activities in children with ADHD in a timely manner.

A novel algorithm for detection of adverse drug reaction signals using a hospital electronic medical record database.

PURPOSE: Quantitative analytic methods are being increasingly used in postmarketing surveillance. However, currently existing methods are limited to spontaneous reporting data and are inapplicable to hospital electronic medical record (EMR) data. The principal objectives of this study were to propose a novel algorithm for detecting the signals of adverse drug reactions using EMR data focused on laboratory abnormalities after treatment with medication, and to evaluate the potential use of this method as a signal detection tool. METHODS: We developed an algorithm referred to as the Comparison on Extreme Laboratory Test results, which takes an extreme representative value pair according to the types of laboratory abnormalities on the basis of each patient's medication point. We used 10 years' EMR data from a tertiary teaching hospital, containing 32,033,710 prescriptions and 115,241,147 laboratory tests for 530,829 individual patients. Ten drugs were selected randomly for analysis, and 51 laboratory values were matched. The sensitivity, specificity, positive predictive value, and negative predictive value of the algorithm were calculated. RESULTS: The mean number of detected laboratory abnormality signals for each drug was 27 (±7.5). The sensitivity, specificity, positive predictive value, and negative predictive value of the algorithm were 64-100%, 22-76%, 22-75%, and 54-100%, respectively. CONCLUSION: The results of this study demonstrated that the Comparison on Extreme Laboratory Test results algorithm described herein was extremely effective in detecting the signals characteristic of adverse drug reactions. This algorithm can be regarded as a useful signal detection tool, which can be routinely applied to EMR data.

Prediction of Daily Patient Numbers for a Regional Emergency Medical Center using Time Series Analysis.

OBJECTIVES: To develop and evaluate time series models to predict the daily number of patients visiting the Emergency Department (ED) of a Korean hospital. METHODS: Data were collected from the hospital information system database. In order to develop a forecasting model, we used, 2 years of data from January 2007 to December 2008 data for the following 3 consecutive months were processed for validation. To establish a Forecasting Model, calendar and weather variables were utilized. Three forecasting models were established: 1) average; 2) univariate seasonal auto-regressive integrated moving average (SARIMA); and 3) multivariate SARIMA. To evaluate goodness-of-fit, residual analysis, Akaike information criterion and Bayesian information criterion were compared. The forecast accuracy for each model was evaluated via mean absolute percentage error (MAPE). RESULTS: The multivariate SARIMA model was the most appropriate for forecasting the daily number of patients visiting the ED. Because it's MAPE was 7.4%, this was the smallest among the models, and for this reason was selected as the final model. CONCLUSIONS: This study applied explanatory variables to a multivariate SARIMA model. The multivariate SARIMA model exhibits relativelyhigh reliability and forecasting accuracy. The weather variables play a part in predicting daily ED patient volume.

Toward an objective indexing system for ADHD-screening using children's activity monitoring.

Signs of ADHD are discernible in specific situations, and usually assessed according to subjective impressions. We performed a preliminary comparative study from children's activity at a natural classroom environment with 3-axis accelerator for a feasible objective index. From a total of 157 children (7-9 yrs) and clinically diagnosed 24 children out of them, variances in 1-min epoch mean activity had shown significant differences among the subgroups: (1) ADHD=.0194, Other Diseases=.0080, Normal=.0009; (2) ADHD=.0194, non-ADHD=.0057(p<.01, respectively). There were also significant differences in high-level activity (>1.6G) features among subgroups with the same order (p<.01, respectively). ADHD patients exhibited more dispersed activities and higher high-level activity ratio than normal. Activity features can be useful to build an objective indexing system for screening ADHD patients.