Comparison of deep learning models for natural language processing-based classification of non-English head CT reports.

PURPOSE: Natural language processing (NLP) can be used for automatic flagging of radiology reports. We assessed deep learning models for classifying non-English head CT reports. METHODS: We retrospectively collected head CT reports (2011-2018). Reports were signed in Hebrew. Emergency department (ED) reports of adult patients from January to February for each year (2013-2018) were manually labeled. All other reports were used to pre-train an embedding layer. We explored two use cases: (1) general labeling use case, in which reports were labeled as normal vs. pathological; (2) specific labeling use case, in which reports were labeled as with and without intra-cranial hemorrhage. We tested long short-term memory (LSTM) and LSTM-attention (LSTM-ATN) networks for classifying reports. We also evaluated the improvement of adding Word2Vec word embedding. Deep learning models were compared with a bag-of-words (BOW) model. RESULTS: We retrieved 176,988 head CT reports for pre-training. We manually labeled 7784 reports as normal (46.3%) or pathological (53.7%), and 7.1% with intra-cranial hemorrhage. For the general labeling, LSTM-ATN-Word2Vec showed the best results (AUC = 0.967 ± 0.006, accuracy 90.8% ± 0.01). For the specific labeling, all methods showed similar accuracies between 95.0 and 95.9%. Both LSTM-ATN-Word2Vec and BOW had the highest AUC (0.970). CONCLUSION: For a general use case, word embedding using a large cohort of non-English head CT reports and ATN improves NLP performance. For a more specific task, BOW and deep learning showed similar results. Models should be explored and tailored to the NLP task.

Promoting head CT exams in the emergency department triage using a machine learning model.

PURPOSE: In this study, we aimed to develop a novel prediction model to identify patients in need of a non-contrast head CT exam during emergency department (ED) triage. METHODS: We collected data of all adult ED visits in our institution for five consecutive years (1/2013-12/2017). Retrieved variables included the following: demographics, mode of arrival to the ED, comorbidities, home medications, structured and unstructured chief complaints, vital signs, pain scale score, emergency severity index, ED wing assignment, documentation of previous ED visits, hospitalizations and CTs, and current visit non-contrast head CT usage. A machine learning gradient boosting model was trained on data from the years 2013-2016 and tested on data from 2017. Area under the curve (AUC) was used as metrics. Single-variable AUCs were also determined. Youden's index evaluated optimal sensitivity and specificity of the models. RESULTS: The final cohort included 595,561 ED visits. Non-contrast head CT usage rate was 11.8%. Each visit was coded into an input vector of 171 variables. Single-variable analysis showed that chief complaint had the best single predictive analysis (AUC = 0.87). The best model showed an AUC of 0.93 (95% CI 0.931-0.936) for predicting non-contrast head CT usage at triage level. The model had a sensitivity of 88.1% and specificity of 85.7% for non-contrast head CT utilization. CONCLUSION: The developed model can identify patients that need to undergo head CT exam already in the ED triage level and by that allow faster diagnosis and treatment.