Deep Learning for Predictive Analysis of Pediatric Otolaryngology Personal Statements: A Pilot Study.

OBJECTIVE: The personal statement is often an underutilized aspect of pediatric otolaryngology fellowship applications. In this pilot study, we use deep learning language models to cluster personal statements and elucidate their relationship to applicant rank position and postfellowship research output. STUDY DESIGN: Retrospective cohort. SETTING: Single pediatric tertiary care center. METHODS: Data and personal statements from 115 applicants to our fellowship program were retrieved from San Francisco Match. BERT (Bidirectional Encoder Representations From Transformers) was used to generate document embeddings for clustering. Regression and machine learning models were used to assess the relationship of personal statements to number of postfellowship publications per year when controlling for publications, board scores, Alpha Omega Alpha status, gender, and residency. RESULTS: Document embeddings of personal statements were found to cluster into 4 distinct groups by K-means clustering: 2 focused on "training/research" and 2 on "personal/patient anecdotes." Training clusters 1 and 2 were associated with an applicant-organization fit by a single pediatric otolaryngology fellowship program on univariate but not multivariate analysis. Models utilizing document embeddings alone were able to equally predict applicant-organization fit (receiver operating characteristic areas under the curve, 0.763 and 0.750 vs 0.419; P values >.05) as compared with models utilizing applicant characteristics and personal statement clusters alone. All predictive models were poor predictors of postfellowship publications per year. CONCLUSION: We demonstrate ability for document embeddings to capture meaningful information in personal statements from pediatric otolaryngology fellowship applicants. A larger study can further differentiate personal statement clusters and assess the predictive potential of document embeddings.

Development and web deployment of an automated neuroradiology MRI protocoling tool with natural language processing.

BACKGROUND: A systematic approach to MRI protocol assignment is essential for the efficient delivery of safe patient care. Advances in natural language processing (NLP) allow for the development of accurate automated protocol assignment. We aim to develop, evaluate, and deploy an NLP model that automates protocol assignment, given the clinician indication text. METHODS: We collected 7139 spine MRI protocols (routine or contrast) and 990 head MRI protocols (routine brain, contrast brain, or other) from a single institution. Protocols were split into training (n = 4997 for spine MRI; n = 839 for head MRI), validation (n = 1071 for spine MRI, fivefold cross-validation used for head MRI), and test (n = 1071 for spine MRI; n = 151 for head MRI) sets. fastText and XGBoost were used to develop 2 NLP models to classify spine and head MRI protocols, respectively. A Flask-based web app was developed to be deployed via Heroku. RESULTS: The spine MRI model had an accuracy of 83.38% and a receiver operator characteristic area under the curve (ROC-AUC) of 0.8873. The head MRI model had an accuracy of 85.43% with a routine brain protocol ROC-AUC of 0.9463 and contrast brain protocol ROC-AUC of 0.9284. Cancer, infectious, and inflammatory related keywords were associated with contrast administration. Structural anatomic abnormalities and stroke/altered mental status were indicative of routine spine and brain MRI, respectively. Error analysis revealed increasing the sample size may improve performance for head MRI protocols. A web version of the model is provided for demonstration and deployment. CONCLUSION: We developed and web-deployed two NLP models that accurately predict spine and head MRI protocol assignment, which could improve radiology workflow efficiency.

High precision localization of pulmonary nodules on chest CT utilizing axial slice number labels.

BACKGROUND: Reidentification of prior nodules for temporal comparison is an important but time-consuming step in lung cancer screening. We develop and evaluate an automated nodule detector that utilizes the axial-slice number of nodules found in radiology reports to generate high precision nodule predictions. METHODS: 888 CTs from Lung Nodule Analysis were used to train a 2-dimensional (2D) object detection neural network. A pipeline of 2D object detection, 3D unsupervised clustering, false positive reduction, and axial-slice numbers were used to generate nodule candidates. 47 CTs from the National Lung Cancer Screening Trial (NLST) were used for model evaluation. RESULTS: Our nodule detector achieved a precision of 0.962 at a recall of 0.573 on the NLST test set for any nodule. When adjusting for unintended nodule predictions, we achieved a precision of 0.931 at a recall 0.561, which corresponds to 0.06 false positives per CT. Error analysis revealed better detection of nodules with soft tissue attenuation compared to ground glass and undeterminable attenuation. Nodule margins, size, location, and patient demographics did not differ between correct and incorrect predictions. CONCLUSIONS: Utilization of axial-slice numbers from radiology reports allowed for development of a lung nodule detector with a low false positive rate compared to prior feature-engineering and machine learning approaches. This high precision nodule detector can reduce time spent on reidentification of prior nodules during lung cancer screening and can rapidly develop new institutional datasets to explore novel applications of computer vision in lung cancer imaging.

An Open-Source, Vender Agnostic Hardware and Software Pipeline for Integration of Artificial Intelligence in Radiology Workflow.

Although machine learning (ML) has made significant improvements in radiology, few algorithms have been integrated into clinical radiology workflow. Complex radiology IT environments and Picture Archiving and Communication System (PACS) pose unique challenges in creating a practical ML schema. However, clinical integration and testing are critical to ensuring the safety and accuracy of ML algorithms. This study aims to propose, develop, and demonstrate a simple, efficient, and understandable hardware and software system for integrating ML models into the standard radiology workflow and PACS that can serve as a framework for testing ML algorithms. A Digital Imaging and Communications in Medicine/Graphics Processing Unit (DICOM/GPU) server and software pipeline was established at a metropolitan county hospital intranet to demonstrate clinical integration of ML algorithms in radiology. A clinical ML integration schema, agnostic to the hospital IT system and specific ML models/frameworks, was implemented and tested with a breast density classification algorithm and prospectively evaluated for time delays using 100 digital 2D mammograms. An open-source clinical ML integration schema was successfully implemented and demonstrated. This schema allows for simple uploading of custom ML models. With the proposed setup, the ML pipeline took an average of 26.52 s per second to process a batch of 100 studies. The most significant processing time delays were noted in model load and study stability times. The code is made available at " http://bit.ly/2Z121hX ". We demonstrated the feasibility to deploy and utilize ML models in radiology without disrupting existing radiology workflow.