Prospective Evaluation of Artificial Intelligence Triage of Incidental Pulmonary Emboli on Contrast-Enhanced CT Examinations of the Chest or Abdomen.

Background: Artificial intelligence (AI) algorithms improved detection of incidental pulmonary embolism (IPE) on contrast-enhanced CT (CECT) examinations in retrospective studies; however, prospective validation studies are lacking. Objective: To assess the effect on radiologists' real-world diagnostic performance and report turnaround times of a radiology department's clinical implementation of an AI triage system for detecting IPE on CECT examinations of the chest or abdomen. Methods: This prospective single-center study included consecutive adult patients who underwent CECT of the chest or abdomen for reasons other than PE detection from May 12, 2021 to June 30, 2021 (phase 1) or from July 1, 2021 to September 29, 2021 (phase 2). Before phase 1, the radiology department installed a commercially available AI triage algorithm for IPE detection that automatically processed CT examinations and notified radiologists of positive results through an interactive floating widget. In phase 1, the widget was inactive, and radiologists interpreted examinations without AI assistance. In phase 2, the widget was activated, and radiologists interpreted examinations with AI assistance. A review process involving a panel of radiologists was implemented to establish the reference standard for the presence of IPE. Diagnostic performance and report turnaround times were compared using Pearson Chi-square test and Wilcoxon rank-sum test, respectively. Results: Phase 1 included 1467 examinations in 1434 patients (mean age, 53.8±18.5 years; 753 male, 681 female); phase 2 included 3182 examinations in 2886 patients (mean age, 55.4±18.2 years; 1520 male, 1366 female). The frequency of IPE was 1.4% (20/1467) in phase 1 and 1.6% (52/3182) in phase 2. Radiologists without AI, in comparison with radiologists with AI, showed significantly lower sensitivity (80.0% vs 96.2%, P=.03), without a significant difference in specificity (99.1% vs 99.9%, P=.58), for detection of IPE. The mean report turnaround time for IPE-positive examinations was not significantly different between radiologists without AI and radiologists with AI (78.3 vs 64.6 min, P=.26). Conclusion: An AI triage system improved radiologists' sensitivity for IPE detection on CECT examinations of the chest or abdomen without significant change in report turnaround times. Clinical Impact: This prospective real-world study supports the use of AI assistance for maximizing IPE detection.

General-Purpose Large Language Models Versus a Domain-Specific Natural Language Processing Tool for Label Extraction From Chest Radiograph Reports.

GPT-4 outperformed a radiology domain-specific natural language processing model in classifying imaging findings from chest radiograph reports, both with and without predefined labels. Prompt engineering for context further improved performance. The findings indicate a role for large language models to accelerate artificial intelligence model development in radiology by automating data annotation.

Prospective Evaluation of AI Triage of Pulmonary Emboli on CT Pulmonary Angiograms.

Background Artificial intelligence (AI) algorithms have shown high accuracy for detection of pulmonary embolism (PE) on CT pulmonary angiography (CTPA) studies in academic studies. Purpose To determine whether use of an AI triage system to detect PE on CTPA studies improves radiologist performance or examination and report turnaround times in a clinical setting. Materials and Methods This prospective single-center study included adult participants who underwent CTPA for suspected PE in a clinical practice setting. Consecutive CTPA studies were evaluated in two phases, first by radiologists alone (n = 31) (May 2021 to June 2021) and then by radiologists aided by a commercially available AI triage system (n = 37) (September 2021 to December 2021). Sixty-two percent of radiologists (26 of 42 radiologists) interpreted studies in both phases. The reference standard was determined by an independent re-review of studies by thoracic radiologists and was used to calculate performance metrics. Diagnostic accuracy and turnaround times were compared using Pearson χ2 and Wilcoxon rank sum tests. Results Phases 1 and 2 included 503 studies (participant mean age, 54.0 years ± 17.8 [SD]; 275 female, 228 male) and 1023 studies (participant mean age, 55.1 years ± 17.5; 583 female, 440 male), respectively. In phases 1 and 2, 14.5% (73 of 503) and 15.9% (163 of 1023) of CTPA studies were positive for PE (P = .47). Mean wait time for positive PE studies decreased from 21.5 minutes without AI to 11.3 minutes with AI (P < .001). The accuracy and miss rate, respectively, for radiologist detection of any PE on CTPA studies was 97.6% and 12.3% without AI and 98.6% and 6.1% with AI, which was not significantly different (P = .15 and P = .11, respectively). Conclusion The use of an AI triage system to detect any PE on CTPA studies improved wait times but did not improve radiologist accuracy, miss rate, or examination and report turnaround times. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Murphy and Tee in this issue.