Qualifying Certainty in Radiology Reports through Deep Learning-Based Natural Language Processing.

BACKGROUND AND PURPOSE: Communication gaps exist between radiologists and referring physicians in conveying diagnostic certainty. We aimed to explore deep learning-based bidirectional contextual language models for automatically assessing diagnostic certainty expressed in the radiology reports to facilitate the precision of communication. MATERIALS AND METHODS: We randomly sampled 594 head MR imaging reports from an academic medical center. We asked 3 board-certified radiologists to read sentences from the Impression section and assign each sentence 1 of the 4 certainty categories: "Non-Definitive," "Definitive-Mild," "Definitive-Strong," "Other." Using the annotated 2352 sentences, we developed and validated a natural language-processing system based on the start-of-the-art bidirectional encoder representations from transformers (BERT), which can capture contextual uncertainty semantics beyond the lexicon level. Finally, we evaluated 3 BERT variant models and reported standard metrics including sensitivity, specificity, and area under the curve. RESULTS: A κ score of 0.74 was achieved for interannotator agreement on uncertainty interpretations among 3 radiologists. For the 3 BERT variant models, the biomedical variant (BioBERT) achieved the best macro-average area under the curve of 0.931 (compared with 0.928 for the BERT-base and 0.925 for the clinical variant [ClinicalBERT]) on the validation data. All 3 models yielded high macro-average specificity (93.13%-93.65%), while the BERT-base obtained the highest macro-average sensitivity of 79.46% (compared with 79.08% for BioBERT and 78.52% for ClinicalBERT). The BioBERT model showed great generalizability on the heldout test data with a macro-average sensitivity of 77.29%, specificity of 92.89%, and area under the curve of 0.93. CONCLUSIONS: A deep transfer learning model can be developed to reliably assess the level of uncertainty communicated in a radiology report.

Intracranial carotid artery calcification on head CT and its association with ischemic changes on brain MRI in patients presenting with stroke-like symptoms: retrospective analysis.

INTRODUCTION: Our purpose was to study the association between the intracranial arterial calcifications observed on head CT and brain infarcts demonstrated by MRI in patients presenting with acute stroke symptoms. METHODS: Institutional review board approval was obtained for this retrospective study which included 65 consecutive patients presenting acutely who had both head CT and MRI. Arterial calcifications of the vertebrobasilar system and the intracranial cavernous carotid arteries (intracranial carotid artery calcification, ICAC) were assigned a number (1 to 4) in the bone window images from CT scans. These four groups were then combined into high calcium (grades 3 and 4) and low calcium (grades 1 and 2) subgroups. Brain MRI was independently evaluated to identify acute and chronic large-vessel infarcts (LVI) and small-vessel infarcts (SVI). The relationship between ICAC and infarcts was evaluated before and after adjusting for demographics and cardiovascular risk factors. RESULTS: Statistical analysis could not be performed for the vertebrobasilar system due to an insufficient number of patients in the high calcium group. Of the 65 patients, 46 (71%) had a high ICAC grade on head CT. They were older and had a higher prevalence of cardiovascular risk factors. Acute SVI (P = 0.006), chronic SVI (P = 0.006) and acute LVI (P = 0.04) were associated with a high ICAC grade. After adjustment for age and other risk factors, only acute SVI was associated with a high ICAC grade (P = 0.002). CONCLUSION: Although age emerged as the most important determinant of ischemic cerebral changes, there were rather complex interactions among multiple risk factors with different infarct types. A high ICAC grade demonstrated a correlation with acute SVI in our patients independent of these risk factors.