ABSTRACT
OBJECTIVES: To evaluate an artificial intelligence (AI)-assisted double reading system for detecting clinically relevant missed findings on routinely reported chest radiographs. METHODS: A retrospective study was performed in two institutions, a secondary care hospital and tertiary referral oncology centre. Commercially available AI software performed a comparative analysis of chest radiographs and radiologists' authorised reports using a deep learning and natural language processing algorithm, respectively. The AI-detected discrepant findings between images and reports were assessed for clinical relevance by an external radiologist, as part of the commercial service provided by the AI vendor. The selected missed findings were subsequently returned to the institution's radiologist for final review. RESULTS: In total, 25,104 chest radiographs of 21,039 patients (mean age 61.1 years ± 16.2 [SD]; 10,436 men) were included. The AI software detected discrepancies between imaging and reports in 21.1% (5289 of 25,104). After review by the external radiologist, 0.9% (47 of 5289) of cases were deemed to contain clinically relevant missed findings. The institution's radiologists confirmed 35 of 47 missed findings (74.5%) as clinically relevant (0.1% of all cases). Missed findings consisted of lung nodules (71.4%, 25 of 35), pneumothoraces (17.1%, 6 of 35) and consolidations (11.4%, 4 of 35). CONCLUSION: The AI-assisted double reading system was able to identify missed findings on chest radiographs after report authorisation. The approach required an external radiologist to review the AI-detected discrepancies. The number of clinically relevant missed findings by radiologists was very low. CLINICAL RELEVANCE STATEMENT: The AI-assisted double reader workflow was shown to detect diagnostic errors and could be applied as a quality assurance tool. Although clinically relevant missed findings were rare, there is potential impact given the common use of chest radiography. KEY POINTS: ⢠A commercially available double reading system supported by artificial intelligence was evaluated to detect reporting errors in chest radiographs (n=25,104) from two institutions. ⢠Clinically relevant missed findings were found in 0.1% of chest radiographs and consisted of unreported lung nodules, pneumothoraces and consolidations. ⢠Applying AI software as a secondary reader after report authorisation can assist in reducing diagnostic errors without interrupting the radiologist's reading workflow. However, the number of AI-detected discrepancies was considerable and required review by a radiologist to assess their relevance.
ABSTRACT
BACKGROUND With the advent of numerous new-generation disease-modifying drugs for multiple sclerosis (MS), the discrimination between relapsing-remitting MS (RRMS) and secondary progressive MS (SPMS) has become a problem of high importance. The aim of our study was to find a simple way to accurately discriminate between RRMS and SPMS that is applicable in clinical practice as a composite marker, using the linear measures of magnetic resonance imaging (MRI) and the results of cognitive tests. MATERIAL AND METHODS We included 88 MS patients in the study: 43 participants had RRMS and 45 had SPMS. A battery consisting of 11 tests was used to evaluate cognitive function. We used 11 linear MRI measures and 7 indexes to assess brain atrophy. RESULTS Four cognitive tests and 3 linear MRI measures were able to distinguish RRMS from SPMS with the AUC >0.8 based on ROC analysis. Multiple logistic regression models were constructed to identify the best set of cognitive and MRI markers. The model, using the Rey Auditory Verbal Learning Test (RAVLT), Digit Symbol Substitution Test (DSST), and Huckman Index, showed the highest predictive ability: AUC=0.921 (p<0.001). We constructed a simple remission-progression index from the same 3 variables, which discriminated well between RRMS and SPMS: AUC=0.920 (p<0.001), maximal Youden Index=0.702, cut-off=1.68, sensitivity=79.1%, and specificity=91.1%. CONCLUSIONS The composite remission-progression index, using the RAVLT test, DSST test, and MRI Huckman Index, is highly accurate in discriminating between RRMS and SPMS.
