Artificial Intelligence Outcome Prediction in Neonates with Encephalopathy (AI-OPiNE).

"Just Accepted" papers have undergone full peer review and have been accepted for publication in Radiology: Artificial Intelligence. This article will undergo copyediting, layout, and proof review before it is published in its final version. Please note that during production of the final copyedited article, errors may be discovered which could affect the content. Purpose To develop a deep learning algorithm to predict 2-year neurodevelopmental outcomes in neonates with hypoxic-ischemic encephalopathy (HIE) using MRI and basic clinical data. Materials and Methods In this study, MRI data of term neonates with encephalopathy in the High Dose Erythropoietin for Asphyxia (HEAL) trial (ClinicalTrials.gov: NCT02811263), who were enrolled from 17 institutions between January 25th, 2017 and October ninth, 2019, were retrospectively analyzed. The harmonized MRI protocol included T1-weighted, T2-weighted, and diffusion tensor imaging. Deep learning classifiers were trained to predict the primary outcome of the HEAL trial (death or any neurodevelopmental impairment [NDI] at 2 years) using multisequence MRI and basic clinical variables, including sex and gestational age at birth. Model performance was evaluated on a test sets comprising 10% of cases from 15 institutions (in-distribution test set, n = 41) and 100% of cases from 2 institutions (out-of-distribution test set, n = 41). Model performance in predicting additional secondary outcomes, including death alone, was also assessed. Results For the 414 neonates (mean gestational age, 39 weeks ± 1.4, 232 males, 182 females), in the study cohort, 198 (48%) died or had any NDI at 2 years. The deep learning model achieved an area under the receiver operating characteristic curve (AUC) of 0.74 (95% CI: 0.60-0.86) and 63% accuracy on the in-distribution test set and an AUC of 0.77 (95% CI: 0.63-0.90) and 78% accuracy on the out-of-distribution test set. Performance was similar or better for predicting secondary outcomes. Conclusion Deep learning analysis of neonatal brain MRI yielded high performance for predicting 2-year neurodevelopmental outcomes. ©RSNA, 2024.

Automated neonatal nnU-Net brain MRI extractor trained on a large multi-institutional dataset.

Brain extraction, or skull-stripping, is an essential data preprocessing step for machine learning approaches to brain MRI analysis. Currently, there are limited extraction algorithms for the neonatal brain. We aim to adapt an established deep learning algorithm for the automatic segmentation of neonatal brains from MRI, trained on a large multi-institutional dataset for improved generalizability across image acquisition parameters. Our model, ANUBEX (automated neonatal nnU-Net brain MRI extractor), was designed using nnU-Net and was trained on a subset of participants (N = 433) enrolled in the High-dose Erythropoietin for Asphyxia and Encephalopathy (HEAL) study. We compared the performance of our model to five publicly available models (BET, BSE, CABINET, iBEATv2, ROBEX) across conventional and machine learning methods, tested on two public datasets (NIH and dHCP). We found that our model had a significantly higher Dice score on the aggregate of both data sets and comparable or significantly higher Dice scores on the NIH (low-resolution) and dHCP (high-resolution) datasets independently. ANUBEX performs similarly when trained on sequence-agnostic or motion-degraded MRI, but slightly worse on preterm brains. In conclusion, we created an automatic deep learning-based neonatal brain extraction algorithm that demonstrates accurate performance with both high- and low-resolution MRIs with fast computation time.

Hypersensitivity to Distractors in Fragile X Syndrome from Loss of Modulation of Cortical VIP Interneurons.

Attention deficit is one of the most prominent and disabling symptoms in Fragile X syndrome (FXS). Hypersensitivity to sensory stimuli contributes to attention difficulties by overwhelming and/or distracting affected individuals, which disrupts activities of daily living at home and learning at school. We find that auditory or visual distractors selectively impair visual discrimination performance in humans and mice with FXS but not in typically developing controls. In both species, males and females were examined. Vasoactive intestinal polypeptide (VIP) neurons were significantly modulated by incorrect responses in the poststimulus period during early distractor trials in WT mice, consistent with their known role as error signals. Strikingly, however, VIP cells from Fmr1 -/- mice showed little modulation in error trials, and this correlated with their poor performance on the distractor task. Thus, VIP interneurons and their reduced modulatory influence on pyramidal cells could be a potential therapeutic target for attentional difficulties in FXS.SIGNIFICANCE STATEMENT Sensory hypersensitivity, impulsivity, and persistent inattention are among the most consistent clinical features of FXS, all of which impede daily functioning and create barriers to learning. However, the neural mechanisms underlying sensory over-reactivity remain elusive. To overcome a significant challenge in translational FXS research we demonstrate a compelling alignment of sensory over-reactivity in both humans with FXS and Fmr1 -/- mice (the principal animal model of FXS) using a novel analogous distractor task. Two-photon microscopy in mice revealed that lack of modulation by VIP cells contributes to susceptibility to distractors. Implementing research efforts we describe here can help identify dysfunctional neural mechanisms associated not only with sensory issues but broader impairments, including those in learning and cognition.

Deep learning augmented ECG analysis to identify biomarker-defined myocardial injury.

Chest pain is a common clinical complaint for which myocardial injury is the primary concern and is associated with significant morbidity and mortality. To aid providers' decision-making, we aimed to analyze the electrocardiogram (ECG) using a deep convolutional neural network (CNN) to predict serum troponin I (TnI) from ECGs. We developed a CNN using 64,728 ECGs from 32,479 patients who underwent ECG within 2 h prior to a serum TnI laboratory result at the University of California, San Francisco (UCSF). In our primary analysis, we classified patients into groups of TnI < 0.02 or ≥ 0.02 µg/L using 12-lead ECGs. This was repeated with an alternative threshold of 1.0 µg/L and with single-lead ECG inputs. We also performed multiclass prediction for a set of serum troponin ranges. Finally, we tested the CNN in a cohort of patients selected for coronary angiography, including 3038 ECGs from 672 patients. Cohort patients were 49.0% female, 42.8% white, and 59.3% (19,283) never had a positive TnI value (≥ 0.02 µg/L). CNNs accurately predicted elevated TnI, both at a threshold of 0.02 µg/L (AUC = 0.783, 95% CI 0.780-0.786) and at a threshold of 1.0 µg/L (AUC = 0.802, 0.795-0.809). Models using single-lead ECG data achieved significantly lower accuracy, with AUCs ranging from 0.740 to 0.773 with variation by lead. Accuracy of the multi-class model was lower for intermediate TnI value-ranges. Our models performed similarly on the cohort of patients who underwent coronary angiography. Biomarker-defined myocardial injury can be predicted by CNNs from 12-lead and single-lead ECGs.

Hypersensitivity to distractors in Fragile X syndrome from loss of modulation of cortical VIP interneurons.

Attention deficit is one of the most prominent and disabling symptoms in Fragile X Syndrome (FXS). Hypersensitivity to sensory stimuli contributes to attention difficulties by overwhelming and/or distracting affected individuals, which disrupts activities of daily living at home and learning at school. We find that auditory or visual distractors selectively impair visual discrimination performance in both humans and mice with FXS, but not their typically developing controls. Vasoactive intestinal polypeptide (VIP) neurons were significantly modulated by incorrect responses in the post-stimulus period during early distractor trials in WT mice, consistent with their known role as 'error' signals. Strikingly, however, VIP cells from Fmr1-/- mice showed little modulation in error trials, and this correlated with their poor performance on the distractor task. Thus, VIP interneurons and their reduced modulatory influence on pyramidal cells, could be a potential therapeutic target for attentional difficulties in FXS.

Application of a Domain-specific BERT for Detection of Speech Recognition Errors in Radiology Reports.

Purpose: To develop radiology domain-specific bidirectional encoder representations from transformers (BERT) models that can identify speech recognition (SR) errors and suggest corrections in radiology reports. Materials and Methods: A pretrained BERT model, Clinical BioBERT, was further pretrained on a corpus of 114 008 radiology reports between April 2016 and August 2019 that were retrospectively collected from two hospitals. Next, the model was fine-tuned on a training dataset of generated insertion, deletion, and substitution errors, creating Radiology BERT. This model was retrospectively evaluated on an independent dataset of radiology reports with generated errors (n = 18 885) and on unaltered report sentences (n = 2000) and prospectively evaluated on true clinical SR errors (n = 92). Correction Radiology BERT was separately trained to suggest corrections for detected deletion and substitution errors. Area under the receiver operating characteristic curve (AUC) and bootstrapped 95% CIs were calculated for each evaluation dataset. Results: Radiology-specific BERT had AUC values of >.99 (95% CI: >0.99, >0.99), 0.94 (95% CI: 0.93, 0.94), 0.98 (95% CI: 0.98, 0.98), and 0.97 (95% CI: 0.97, 0.97) for detecting insertion, deletion, substitution, and all errors, respectively, on the independently generated test set. Testing on unaltered report impressions revealed a sensitivity of 82% (28 of 34; 95% CI: 70%, 93%) and specificity of 88% (1521 of 1728; 95% CI: 87%, 90%). Testing on prospective SR errors showed an accuracy of 75% (69 of 92; 95% CI: 65%, 83%). Finally, the correct word was the top suggestion for 45.6% (475 of 1041; 95% CI: 42.5%, 49.3%) of errors. Conclusion: Radiology-specific BERT models fine-tuned on generated errors were able to identify SR errors in radiology reports and suggest corrections.Keywords: Computer Applications, Technology Assessment Supplemental material is available for this article. © RSNA, 2022See also the commentary by Abajian and Cheung in this issue.

Deep Learning to Predict Neonatal and Infant Brain Age from Myelination on Brain MRI Scans.

Background Assessment of appropriate brain myelination on T1- and T2-weighted MRI scans is based on gestationally corrected age (GCA) and requires subjective visual inspection of the brain with knowledge of normal myelination milestones. Purpose To develop a convolutional neural network (CNN) capable of estimating neonatal and infant GCA based on brain myelination on MRI scans. Materials and methods In this retrospective study from one academic medical center, brain MRI scans of patients aged 0-25 months with reported normal myelination were consecutively collected between January 1995 and June 2019. The GCA at MRI was manually calculated. After exclusion criteria were applied, T1- and T2-weighted MRI scans were preprocessed with skull stripping, linear registration, z scoring for normalization, and downsampling. A three-dimensional regression CNN was trained to predict GCA using mean absolute error (MAE) as its loss function. Attention maps were calculated using layer-wise relevance propagation. Models were validated on an external test set from the National Institutes of Health (NIH). Model MAEs were compared using Kruskal-Wallis and Mann-Whitney tests. Results A total of 518 neonates and infants (mean GCA, 67 weeks ± 33 [SD], 56% male) was included, comprising 469 T1-, 438 T2-, and 389 T1- and T2-weighted studies. Across 10 runs, MAEs of T1-, T2-, and T1- and T2-weighted networks were 9.8 ± 2.3, 9.1 ± 1.9, and 7.7 ± 1.7 weeks, respectively. Attention map analysis demonstrated increased network attention to the cerebellum, posterior white matter, and basal ganglia signal in neonates with GCA of less than 40 weeks and the anterior white matter signal in infants with GCA of more than 120 weeks, corresponding to the known progression of myelination. The T1- and T2-weighted network tested on the external NIH test set had an MAE of 9.1 weeks, which was reduced to 5.9 weeks with further training using half the external test set (P < .001). Conclusion A three-dimensional convolutional neural network can predict the gestationally corrected age of neonates and infants aged 0-25 months based on brain myelination patterns on T1- and T2-weighted MRI scans. © RSNA, 2022 Online supplemental material is available for this article.

Clinical language search algorithm from free-text: facilitating appropriate imaging.

BACKGROUND: The comprehensiveness and maintenance of the American College of Radiology (ACR) Appropriateness Criteria (AC) makes it a unique resource for evidence-based clinical imaging decision support, but it is underutilized by clinicians. To facilitate the use of imaging recommendations, we develop a natural language processing (NLP) search algorithm that automatically matches clinical indications that physicians write into imaging orders to appropriate AC imaging recommendations. METHODS: We apply a hybrid model of semantic similarity from a sent2vec model trained on 223 million scientific sentences, combined with term frequency inverse document frequency features. AC documents are ranked based on their embeddings' cosine distance to query. For model testing, we compiled a dataset of simulated simple and complex indications for each AC document (n = 410) and another with clinical indications from randomly sampled radiology reports (n = 100). We compare our algorithm to a custom google search engine. RESULTS: On the simulated indications, our algorithm ranked ground truth documents as top 3 for 98% of simple queries and 85% of complex queries. Similarly, on the randomly sampled radiology report dataset, the algorithm ranked 86% of indications with a single match as top 3. Vague and distracting phrases present in the free-text indications were main sources of errors. Our algorithm provides more relevant results than a custom Google search engine, especially for complex queries. CONCLUSIONS: We have developed and evaluated an NLP algorithm that matches clinical indications to appropriate AC guidelines. This approach can be integrated into imaging ordering systems for automated access to guidelines.

Development and web deployment of an automated neuroradiology MRI protocoling tool with natural language processing.

BACKGROUND: A systematic approach to MRI protocol assignment is essential for the efficient delivery of safe patient care. Advances in natural language processing (NLP) allow for the development of accurate automated protocol assignment. We aim to develop, evaluate, and deploy an NLP model that automates protocol assignment, given the clinician indication text. METHODS: We collected 7139 spine MRI protocols (routine or contrast) and 990 head MRI protocols (routine brain, contrast brain, or other) from a single institution. Protocols were split into training (n = 4997 for spine MRI; n = 839 for head MRI), validation (n = 1071 for spine MRI, fivefold cross-validation used for head MRI), and test (n = 1071 for spine MRI; n = 151 for head MRI) sets. fastText and XGBoost were used to develop 2 NLP models to classify spine and head MRI protocols, respectively. A Flask-based web app was developed to be deployed via Heroku. RESULTS: The spine MRI model had an accuracy of 83.38% and a receiver operator characteristic area under the curve (ROC-AUC) of 0.8873. The head MRI model had an accuracy of 85.43% with a routine brain protocol ROC-AUC of 0.9463 and contrast brain protocol ROC-AUC of 0.9284. Cancer, infectious, and inflammatory related keywords were associated with contrast administration. Structural anatomic abnormalities and stroke/altered mental status were indicative of routine spine and brain MRI, respectively. Error analysis revealed increasing the sample size may improve performance for head MRI protocols. A web version of the model is provided for demonstration and deployment. CONCLUSION: We developed and web-deployed two NLP models that accurately predict spine and head MRI protocol assignment, which could improve radiology workflow efficiency.

Domain specific word embeddings for natural language processing in radiology.

BACKGROUND: There has been increasing interest in machine learning based natural language processing (NLP) methods in radiology; however, models have often used word embeddings trained on general web corpora due to lack of a radiology-specific corpus. PURPOSE: We examined the potential of Radiopaedia to serve as a general radiology corpus to produce radiology specific word embeddings that could be used to enhance performance on a NLP task on radiological text. MATERIALS AND METHODS: Embeddings of dimension 50, 100, 200, and 300 were trained on articles collected from Radiopaedia using a GloVe algorithm and evaluated on analogy completion. A shallow neural network using input from either our trained embeddings or pre-trained Wikipedia 2014 + Gigaword 5 (WG) embeddings was used to label the Radiopaedia articles. Labeling performance was evaluated based on exact match accuracy and Hamming loss. The McNemar's test with continuity and the Benjamini-Hochberg correction and a 5×2 cross validation paired two-tailed t-test were used to assess statistical significance. RESULTS: For accuracy in the analogy task, 50-dimensional (50-D) Radiopaedia embeddings outperformed WG embeddings on tumor origin analogies (p < 0.05) and organ adjectives (p < 0.01) whereas WG embeddings tended to outperform on inflammation location and bone vs. muscle analogies (p < 0.01). The two embeddings had comparable performance on other subcategories. In the labeling task, the Radiopaedia-based model outperformed the WG based model at 50, 100, 200, and 300-D for exact match accuracy (p < 0.001, p < 0.001, p < 0.01, and p < 0.05, respectively) and Hamming loss (p < 0.001, p < 0.001, p < 0.01, and p < 0.05, respectively). CONCLUSION: We have developed a set of word embeddings from Radiopaedia and shown that they can preserve relevant medical semantics and augment performance on a radiology NLP task. Our results suggest that the cultivation of a radiology-specific corpus can benefit radiology NLP models in the future.

Bright-field holography: cross-modality deep learning enables snapshot 3D imaging with bright-field contrast using a single hologram.

Digital holographic microscopy enables the 3D reconstruction of volumetric samples from a single-snapshot hologram. However, unlike a conventional bright-field microscopy image, the quality of holographic reconstructions is compromised by interference fringes as a result of twin images and out-of-plane objects. Here, we demonstrate that cross-modality deep learning using a generative adversarial network (GAN) can endow holographic images of a sample volume with bright-field microscopy contrast, combining the volumetric imaging capability of holography with the speckle- and artifact-free image contrast of incoherent bright-field microscopy. We illustrate the performance of this "bright-field holography" method through the snapshot imaging of bioaerosols distributed in 3D, matching the artifact-free image contrast and axial sectioning performance of a high-NA bright-field microscope. This data-driven deep-learning-based imaging method bridges the contrast gap between coherent and incoherent imaging, and enables the snapshot 3D imaging of objects with bright-field contrast from a single hologram, benefiting from the wave-propagation framework of holography.

Author Correction: Impaired perceptual learning in a mouse model of Fragile X syndrome is mediated by parvalbumin neuron dysfunction and is reversible.

The original and corrected Acknowledgements are shown in the accompanying Author Correction.

Impaired perceptual learning in a mouse model of Fragile X syndrome is mediated by parvalbumin neuron dysfunction and is reversible.

To uncover the circuit-level alterations that underlie atypical sensory processing associated with autism, we adopted a symptom-to-circuit approach in the Fmr1-knockout (Fmr1-/-) mouse model of Fragile X syndrome. Using a go/no-go task and in vivo two-photon calcium imaging, we find that impaired visual discrimination in Fmr1-/- mice correlates with marked deficits in orientation tuning of principal neurons and with a decrease in the activity of parvalbumin interneurons in primary visual cortex. Restoring visually evoked activity in parvalbumin cells in Fmr1-/- mice with a chemogenetic strategy using designer receptors exclusively activated by designer drugs was sufficient to rescue their behavioral performance. Strikingly, human subjects with Fragile X syndrome exhibit impairments in visual discrimination similar to those in Fmr1-/- mice. These results suggest that manipulating inhibition may help sensory processing in Fragile X syndrome.