ABSTRACT
Purpose: To present a method that automatically detects, subtypes, and locates acute or subacute intracranial hemorrhage (ICH) on noncontrast CT (NCCT) head scans; generates detection confidence scores to identify high-confidence data subsets with higher accuracy; and improves radiology worklist prioritization. Such scores may enable clinicians to better use artificial intelligence (AI) tools. Materials and Methods: This retrospective study included 46 057 studies from seven "internal" centers for development (training, architecture selection, hyperparameter tuning, and operating-point calibration; n = 25 946) and evaluation (n = 2947) and three "external" centers for calibration (n = 400) and evaluation (n = 16 764). Internal centers contributed developmental data, whereas external centers did not. Deep neural networks predicted the presence of ICH and subtypes (intraparenchymal, intraventricular, subarachnoid, subdural, and/or epidural hemorrhage) and segmentations per case. Two ICH confidence scores are discussed: a calibrated classifier entropy score and a Dempster-Shafer score. Evaluation was completed by using receiver operating characteristic curve analysis and report turnaround time (RTAT) modeling on the evaluation set and on confidence score-defined subsets using bootstrapping. Results: The areas under the receiver operating characteristic curve for ICH were 0.97 (0.97, 0.98) and 0.95 (0.94, 0.95) on internal and external center data, respectively. On 80% of the data stratified by calibrated classifier and Dempster-Shafer scores, the system improved the Youden indexes, increasing them from 0.84 to 0.93 (calibrated classifier) and from 0.84 to 0.92 (Dempster-Shafer) for internal centers and increasing them from 0.78 to 0.88 (calibrated classifier) and from 0.78 to 0.89 (Dempster-Shafer) for external centers (P < .001). Models estimated shorter RTAT for AI-prioritized worklists with confidence measures than for AI-prioritized worklists without confidence measures, shortening RTAT by 27% (calibrated classifier) and 27% (Dempster-Shafer) for internal centers and shortening RTAT by 25% (calibrated classifier) and 27% (Dempster-Shafer) for external centers (P < .001). Conclusion: AI that provided statistical confidence measures for ICH detection on NCCT scans reliably detected and subtyped hemorrhages, identified high-confidence predictions, and improved worklist prioritization in simulation.Keywords: CT, Head/Neck, Hemorrhage, Convolutional Neural Network (CNN) Supplemental material is available for this article. © RSNA, 2022.
ABSTRACT
PURPOSE: The primary aim was to investigate the diagnostic performance of an Artificial Intelligence (AI) algorithm for pneumoperitoneum detection in patients with acute abdominal pain who underwent an abdominal CT scan. METHOD: This retrospective diagnostic test accuracy study used a consecutive patient cohort from the Acute High-risk Abdominal patient population at Herlev and Gentofte Hospital, Denmark between January 1, 2019 and September 25, 2019. As reference standard, all studies were rated for pneumoperitoneum (subgroups: none, small, medium, and large amounts) by a gastrointestinal radiology consultant. The index test was a novel AI algorithm based on a sliding window approach with a deep recurrent neural network at its core. The primary outcome was the area under the curve (AUC) of the receiver operating characteristic (ROC). RESULTS: Of 331 included patients (median age 68 years (Range 19-100; 180 women)) 31 patients (9%) had pneumoperitoneum (large: 16, moderate: 7, small: 8). The AUC was 0.77 (95% CI 0.66-0.87). At a specificity of 99% (297/300, 95% CI: 97-100%), sensitivity was 52% (16/31, 95% CI 29-65%), and positive likelihood ratio was 52 (95% CI 16-165). When excluding cases with smaller amounts of free air (<0.25 mL) the AUC increased to 0.96 (95% CI 0.89-1.0). At 99% specificity, sensitivity was 81% (13/16) and positive likelihood ratio was 82 (95% CI 27 - 254). CONCLUSIONS: An AI algorithm identified pneumoperitoneum on CT scans in a clinical setting with low sensitivity but very high specificity, supporting its role for ruling in pneumoperitoneum.
