FDA-approved deep learning software application versus radiologists with different levels of expertise: detection of intracranial hemorrhage in a retrospective single-center study.

PURPOSE: To assess an FDA-approved and CE-certified deep learning (DL) software application compared to the performance of human radiologists in detecting intracranial hemorrhages (ICH). METHODS: Within a 20-week trial from January to May 2020, 2210 adult non-contrast head CT scans were performed in a single center and automatically analyzed by an artificial intelligence (AI) solution with workflow integration. After excluding 22 scans due to severe motion artifacts, images were retrospectively assessed for the presence of ICHs by a second-year resident and a certified radiologist under simulated time pressure. Disagreements were resolved by a subspecialized neuroradiologist serving as the reference standard. We calculated interrater agreement and diagnostic performance parameters, including the Breslow-Day and Cochran-Mantel-Haenszel tests. RESULTS: An ICH was present in 214 out of 2188 scans. The interrater agreement between the resident and the certified radiologist was very high (κ = 0.89) and even higher (κ = 0.93) between the resident and the reference standard. The software has delivered 64 false-positive and 68 false-negative results giving an overall sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of 68.2%, 96.8%, 69.5%, 96.6%, and 94.0%, respectively. Corresponding values for the resident were 94.9%, 99.2%, 93.1%, 99.4%, and 98.8%. The accuracy of the DL application was inferior (p < 0.001) to that of both the resident and the certified neuroradiologist. CONCLUSION: A resident under time pressure outperformed an FDA-approved DL program in detecting ICH in CT scans. Our results underline the importance of thoughtful workflow integration and post-approval validation of AI applications in various clinical environments.

The "central vein sign": is there a place for susceptibility weighted imaging in possible multiple sclerosis?

OBJECTIVES: Susceptibility weighted imaging (SWI) may have the potential to depict the perivenous extent of white matter lesions (WMLs) in multiple sclerosis (MS). We aimed to assess the discriminatory value of the "central vein sign" (CVS). METHODS: In a 3-T magnetic resonance imaging (MRI) study, 28 WMLs in 14 patients with at least one circumscribed lesion >5 mm and not more than eight non-confluent lesions >3 mm were prospectively included. Only WMLs in FLAIR images with a maximum diameter of >5 mm were correlated to their SWI equivalent for CVS evaluation. RESULTS: Five patients fulfilled the revised McDonald criteria for MS and nine patients were given alternative diagnoses. Nineteen MS-WMLs and nine non-MS-WMLs >5 mm were detected. Consensus reading found a central vein in 16 out of 19 MS-WMLs (84 %) and in one out of nine non-MS-WMLs (11 %), respectively. The CVS proved to be a highly significant discriminator (P < 0.001) between MS-WMLs and non-MS-WMLs with a sensitivity, specificity, positive and negative predictive value and accuracy of 84 %, 89 %, 94 %, 73 % and 86 %, respectively. Inter-rater agreement was good (κ = 0.77). CONCLUSIONS: Even though the CVS is not exclusively found in MS-WMLs, SWI may be a useful adjunct in patients with possible MS. KEY POINTS: • MRI continues to yield further information concerning MS lesions. • SWI adds diagnostic information in patients with possible MS. • The "central vein sign" was predominantly seen in MS lesions. • The "central vein sign" helps discriminate between MS and non-MS lesions.