Minimum perceivable size difference: how well can radiologists visually detect a change in lung nodule size from CT images?

OBJECTIVE: The purpose of this study was to determine how well radiologists could visually detect a change in lung nodule size on the basis of visual image perception alone. SUBJECTS AND METHODS: Under IRB approval, 109 standard chest CT image series were anonymized and exported from PACS. Nine hundred forty virtual lung nodule pairs (six baseline diameters, six relative volume differences, two nodule types-solid and ground glass-and 14 repeats) were digitally inserted into the chest CT image series (same location, different sizes between the pair). These digitally altered CT image pairs were shown to nine radiologists who were tasked to visually determine which image contained the larger nodule using a two-alternative forced-choice perception experimental design. These data were statistically analyzed using a generalized linear mixed effects model to determine how accurately the radiologists were able to correctly identify the larger nodule. RESULTS: Nominal baseline nodule diameter, relative volume difference, and nodule type were found to be statistically significant factors (p < 0.001) in influencing the radiologists' accuracy. For solid (ground-glass) nodules, the baseline diameter needed to be at least 6.3 mm (13.2 mm) to be able to visually detect a 25% change in volume with 95 ± 1.4% accuracy. Accuracy was lowest for the nodules with the smallest baseline diameters and smallest relative volume differences. Additionally, accuracy was lower for ground-glass nodules compared to solid nodules. CONCLUSIONS: Factors that impacted visual size assessment were baseline nodule diameter, relative volume difference, and solid versus non-solid nodule type, with larger and more solid lesions offering a more precise assessment of change. KEY POINTS: • For solid nodules, radiologists could visually detect a 25% change in volume with 95% accuracy for nodules having greater than 6.3-mm baseline diameter. • For ground-glass nodules, radiologists could visually detect a 25% change in volume with 95% accuracy for nodules having greater than 13.2-mm baseline diameter. • Accuracy in detecting a change in nodule size began to stabilize around 90-100% for nodules with larger baseline diameters (> 8 mm for solid nodules, > 12 mm for ground-glass nodules) and larger relative volume differences (>15% for solid nodules, > 25% for ground-glass nodules).

A Deep-Learning Diagnostic Support System for the Detection of COVID-19 Using Chest Radiographs: A Multireader Validation Study.

MATERIALS AND METHODS: Five publicly available databases comprising normal CXR, confirmed COVID-19 pneumonia cases, and other pneumonias were used. After the harmonization of the data, the training set included 7966 normal cases, 5451 with other pneumonia, and 258 CXRs with COVID-19 pneumonia, whereas in the testing data set, each category was represented by 100 cases. Eleven blinded radiologists with various levels of expertise independently read the testing data set. The data were analyzed separately with the newly proposed artificial intelligence-based system and by consultant radiologists and residents, with respect to positive predictive value (PPV), sensitivity, and F-score (harmonic mean for PPV and sensitivity). The χ2 test was used to compare the sensitivity, specificity, accuracy, PPV, and F-scores of the readers and the system. RESULTS: The proposed system achieved higher overall diagnostic accuracy (94.3%) than the radiologists (61.4% ± 5.3%). The radiologists reached average sensitivities for normal CXR, other type of pneumonia, and COVID-19 pneumonia of 85.0% ± 12.8%, 60.1% ± 12.2%, and 53.2% ± 11.2%, respectively, which were significantly lower than the results achieved by the algorithm (98.0%, 88.0%, and 97.0%; P < 0.00032). The mean PPVs for all 11 radiologists for the 3 categories were 82.4%, 59.0%, and 59.0% for the healthy, other pneumonia, and COVID-19 pneumonia, respectively, resulting in an F-score of 65.5% ± 12.4%, which was significantly lower than the F-score of the algorithm (94.3% ± 2.0%, P < 0.00001). When other pneumonia and COVID-19 pneumonia cases were pooled, the proposed system reached an accuracy of 95.7% for any pathology and the radiologists, 88.8%. The overall accuracy of consultants did not vary significantly compared with residents (65.0% ± 5.8% vs 67.4% ± 4.2%); however, consultants detected significantly more COVID-19 pneumonia cases (P = 0.008) and less healthy cases (P < 0.00001). CONCLUSIONS: The system showed robust accuracy for COVID-19 pneumonia detection on CXR and surpassed radiologists at various training levels.