Factors affecting the labelling accuracy of brain MRI studies relevant for deep learning abnormality detection.

Unlocking the vast potential of deep learning-based computer vision classification systems necessitates large data sets for model training. Natural Language Processing (NLP)-involving automation of dataset labelling-represents a potential avenue to achieve this. However, many aspects of NLP for dataset labelling remain unvalidated. Expert radiologists manually labelled over 5,000 MRI head reports in order to develop a deep learning-based neuroradiology NLP report classifier. Our results demonstrate that binary labels (normal vs. abnormal) showed high rates of accuracy, even when only two MRI sequences (T2-weighted and those based on diffusion weighted imaging) were employed as opposed to all sequences in an examination. Meanwhile, the accuracy of more specific labelling for multiple disease categories was variable and dependent on the category. Finally, resultant model performance was shown to be dependent on the expertise of the original labeller, with worse performance seen with non-expert vs. expert labellers.

Deep learning models for triaging hospital head MRI examinations.

The growing demand for head magnetic resonance imaging (MRI) examinations, along with a global shortage of radiologists, has led to an increase in the time taken to report head MRI scans in recent years. For many neurological conditions, this delay can result in poorer patient outcomes and inflated healthcare costs. Potentially, computer vision models could help reduce reporting times for abnormal examinations by flagging abnormalities at the time of imaging, allowing radiology departments to prioritise limited resources into reporting these scans first. To date, however, the difficulty of obtaining large, clinically-representative labelled datasets has been a bottleneck to model development. In this work, we present a deep learning framework, based on convolutional neural networks, for detecting clinically-relevant abnormalities in minimally processed, hospital-grade axial T2-weighted and axial diffusion-weighted head MRI scans. The models were trained at scale using a Transformer-based neuroradiology report classifier to generate a labelled dataset of 70,206 examinations from two large UK hospital networks, and demonstrate fast (< 5 s), accurate (area under the receiver operating characteristic curve (AUC) > 0.9), and interpretable classification, with good generalisability between hospitals (ΔAUC ≤ 0.02). Through a simulation study we show that our best model would reduce the mean reporting time for abnormal examinations from 28 days to 14 days and from 9 days to 5 days at the two hospital networks, demonstrating feasibility for use in a clinical triage environment.

Accurate brain-age models for routine clinical MRI examinations.

Convolutional neural networks (CNN) can accurately predict chronological age in healthy individuals from structural MRI brain scans. Potentially, these models could be applied during routine clinical examinations to detect deviations from healthy ageing, including early-stage neurodegeneration. This could have important implications for patient care, drug development, and optimising MRI data collection. However, existing brain-age models are typically optimised for scans which are not part of routine examinations (e.g., volumetric T1-weighted scans), generalise poorly (e.g., to data from different scanner vendors and hospitals etc.), or rely on computationally expensive pre-processing steps which limit real-time clinical utility. Here, we sought to develop a brain-age framework suitable for use during routine clinical head MRI examinations. Using a deep learning-based neuroradiology report classifier, we generated a dataset of 23,302 'radiologically normal for age' head MRI examinations from two large UK hospitals for model training and testing (age range = 18-95 years), and demonstrate fast (< 5 s), accurate (mean absolute error [MAE] < 4 years) age prediction from clinical-grade, minimally processed axial T2-weighted and axial diffusion-weighted scans, with generalisability between hospitals and scanner vendors (Δ MAE < 1 year). The clinical relevance of these brain-age predictions was tested using 228 patients whose MRIs were reported independently by neuroradiologists as showing atrophy 'excessive for age'. These patients had systematically higher brain-predicted age than chronological age (mean predicted age difference = +5.89 years, 'radiologically normal for age' mean predicted age difference = +0.05 years, p < 0.0001). Our brain-age framework demonstrates feasibility for use as a screening tool during routine hospital examinations to automatically detect older-appearing brains in real-time, with relevance for clinical decision-making and optimising patient pathways.

Deep learning to automate the labelling of head MRI datasets for computer vision applications.

OBJECTIVES: The purpose of this study was to build a deep learning model to derive labels from neuroradiology reports and assign these to the corresponding examinations, overcoming a bottleneck to computer vision model development. METHODS: Reference-standard labels were generated by a team of neuroradiologists for model training and evaluation. Three thousand examinations were labelled for the presence or absence of any abnormality by manually scrutinising the corresponding radiology reports ('reference-standard report labels'); a subset of these examinations (n = 250) were assigned 'reference-standard image labels' by interrogating the actual images. Separately, 2000 reports were labelled for the presence or absence of 7 specialised categories of abnormality (acute stroke, mass, atrophy, vascular abnormality, small vessel disease, white matter inflammation, encephalomalacia), with a subset of these examinations (n = 700) also assigned reference-standard image labels. A deep learning model was trained using labelled reports and validated in two ways: comparing predicted labels to (i) reference-standard report labels and (ii) reference-standard image labels. The area under the receiver operating characteristic curve (AUC-ROC) was used to quantify model performance. Accuracy, sensitivity, specificity, and F1 score were also calculated. RESULTS: Accurate classification (AUC-ROC > 0.95) was achieved for all categories when tested against reference-standard report labels. A drop in performance (ΔAUC-ROC > 0.02) was seen for three categories (atrophy, encephalomalacia, vascular) when tested against reference-standard image labels, highlighting discrepancies in the original reports. Once trained, the model assigned labels to 121,556 examinations in under 30 min. CONCLUSIONS: Our model accurately classifies head MRI examinations, enabling automated dataset labelling for downstream computer vision applications. KEY POINTS: • Deep learning is poised to revolutionise image recognition tasks in radiology; however, a barrier to clinical adoption is the difficulty of obtaining large labelled datasets for model training. • We demonstrate a deep learning model which can derive labels from neuroradiology reports and assign these to the corresponding examinations at scale, facilitating the development of downstream computer vision models. • We rigorously tested our model by comparing labels predicted on the basis of neuroradiology reports with two sets of reference-standard labels: (1) labels derived by manually scrutinising each radiology report and (2) labels derived by interrogating the actual images.

Computed tomographic features of the proximal petrous facial nerve canal in recurrent Bell's palsy.

OBJECTIVES: The primary objective was to determine whether the narrowest dimensions of the labyrinthine facial nerve (LFN) canal on the symptomatic side in patients with unilateral recurrent Bell's palsy (BP) differ from those on the contralateral side or in asymptomatic, age- and gender-matched controls on computed tomography (CT). The secondary objectives were to assess the extent of bony covering at the geniculate ganglion and to record inter-observer reliability of the CT measurements. METHODS: The dimensions of the LFN canal at its narrowest point perpendicular to the long axis and the extent of bony covering at the geniculate ganglion were assessed by two radiologists. Statistical analysis was performed using the Wilcoxon signed-rank and Mann-Whitney U tests (LFN canal dimensions) and the Chi-squared test (bony covering at the geniculate ganglion). Inter-observer reliability was evaluated using Intra-Class Correlation (ICC) and Cohen's kappa. RESULTS: The study included 21 patients with unilateral recurrent BP and 21 asymptomatic controls. There was no significant difference in the narrowest dimensions of the ipsilateral LFN canal when compared to the contralateral side or controls (P = .43-.94). Similarly, there was no significant difference in the extent of bony covering at the geniculate ganglion when compared to either group (P = .19-.8). Good inter-observer reliability was observed for LFN measurements (ICC = 0.75-0.88) but not for the bony covering at the geniculate ganglion (Cohen's kappa = 0.53). CONCLUSION: The narrowest dimensions of the LFN canal and the extent of bony covering at the geniculate ganglion do not differ in unilateral recurrent BP, casting doubt over their etiological significance. LEVEL OF EVIDENCE: Level IV.

What Role Can Process Mining Play in Recurrent Clinical Guidelines Issues? A Position Paper.

In the age of Evidence-Based Medicine, Clinical Guidelines (CGs) are recognized to be an indispensable tool to support physicians in their daily clinical practice. Medical Informatics is expected to play a relevant role in facilitating diffusion and adoption of CGs. However, the past pioneering approaches, often fragmented in many disciplines, did not lead to solutions that are actually exploited in hospitals. Process Mining for Healthcare (PM4HC) is an emerging discipline gaining the interest of healthcare experts, and seems able to deal with many important issues in representing CGs. In this position paper, we briefly describe the story and the state-of-the-art of CGs, and the efforts and results of the past approaches of medical informatics. Then, we describe PM4HC, and we answer questions like how can PM4HC cope with this challenge? Which role does PM4HC play and which rules should be employed for the PM4HC scientific community?