Multiple diagnostic task performance in CT examination of the chest.

OBJECTIVES: In three experiments, we studied the detection of multiple abnormality types using the satisfaction of search (SOS) paradigm, the provision of a computer-aided detection (CAD) of pulmonary nodules and a focused nodule detection task. METHODS: 51 chest CT examinations (24 that demonstrated subtle pulmonary nodules and 27 that demonstrated no pulmonary nodules) were read by 15 radiology residents and fellows under two experimental conditions: (1) when there were no other abnormalities present except test abnormalities in the exams (non-SOS condition), and (2) when other abnormalities were present in the exams (SOS condition). Trials from the two conditions were intermixed. Readers were invited to return for two sessions: one in which the SOS condition was repeated with a simulated CAD; another in which only the non-SOS condition was presented. Detection accuracy was measured using receiver operating characteristic (ROC) analysis. RESULTS: An SOS effect (reduced detection accuracy for the test nodules in the presence of the diverse added abnormalities) was not found. Average accuracy was much higher when the CAD prompt was provided, without cost in the detection of the added abnormalities. Accuracy for detecting nodules appearing without intermixed SOS trials was also substantially improved. CONCLUSIONS: CT interpretation was highly task dependent. Nodule detection was poor in the general search task. Therefore, CAD may offer a greater performance improvement than demonstrated in experiments assessing CAD using focused search. The absence of SOS may be due to limited nodule detection even without other abnormalities. Advances in knowledge CAD prompts of nodules increase the detection accuracy of nodules and decrease the time to detection-without impairing the detection accuracy-of non-nodule abnormalities.

Multiple diagnostic task performance in CT examination of the chest.

Objectives In three experiments, we studied the detection of multiple abnormality types using the satisfaction of search (SOS) paradigm, the provision of a computer-aided detection (CAD) of pulmonary nodules and a focused nodule detection task. Methods 51 chest CT examinations (24 that demonstrated subtle pulmonary nodules and 27 that demonstrated no pulmonary nodules) were read by 15 radiology residents and fellows under two experimental conditions: (1) when there were no other abnormalities present except test abnormalities in the exams (non-SOS condition), and (2) when other abnormalities were present in the exams (SOS condition). Trials from the two conditions were intermixed. Readers were invited to return for two sessions: one in which the SOS condition was repeated with a simulated CAD; another in which only the non-SOS condition was presented. Detection accuracy was measured using receiver operating characteristic (ROC) analysis. Results An SOS effect (reduced detection accuracy for the test nodules in the presence of the diverse added abnormalities) was not found. Average accuracy was much higher when the CAD prompt was provided, without cost in the detection of the added abnormalities. Accuracy for detecting nodules appearing without intermixed SOS trials was also substantially improved. Conclusions CT interpretation was highly task dependent. Nodule detection was poor in the general search task. Therefore, CAD may offer a greater performance improvement than demonstrated in experiments assessing CAD using focused search. The absence of SOS may be due to limited nodule detection even without other abnormalities. Advances in knowledge CAD prompts of nodules increase the detection accuracy of nodules and decrease the time to detection-without impairing the detection accuracy-of non-nodule abnormalities.

Does computer-aided diagnosis for lung tumors change satisfaction of search in chest radiography?

RATIONALE AND OBJECTIVES: Computer-aided diagnosis (CAD) has been developed to ensure that the radiologist considers suspect focal opacities that may represent cancer in chest radiography. Although CAD was not developed to counteract the satisfaction of search (SOS) effect, it may be an effective intervention to do so. The objective of this study is to determine whether an idealized CAD can reduce SOS effects in chest radiography. MATERIALS AND METHODS: Fifty-seven chest radiographs, half of which demonstrated diverse, native abnormalities were read twice by 16 observers, once with and once without the addition of a simulated pulmonary nodule. Simulated CAD prompts were provided during the interpretation, which unerringly pointed to the added simulated nodule. Area under the ROC curve for detecting the native abnormalities was estimated for each observer in each treatment condition. In addition to testing for the SOS effect in the presence of CAD prompts, results were compared to those of a previous SOS study. RESULTS: Significantly more nodules were reported in the SOS with CAD experiment than in the original SOS experiment (49 versus 43, P < .01). An SOS effect was found even when CAD prompts were provided; ROC areas for detecting native abnormalities were reduced with added nodules [0.68 versus 0.65, P (one-tailed) < .05]. Comparison of the current experiment with CAD and the previous SOS experiments failed to show a significant difference of the magnitude of the SOS effect (P = .52). The threshold for reporting was more conservative with CAD prompts than in SOS studies (P = .052). CONCLUSION: Our results indicate that the CAD prompts, even those that always point to their target lesion without false-positive error, fail to counteract SOS in chest radiography. The stricter decision thresholds with CAD prompts may indicate less visual search for native abnormalities.

Can order of report prevent satisfaction of search in abdominal contrast studies?

RATIONALE AND OBJECTIVE: A previous receiver operating characteristic (ROC) study showed a systematic shift in decision thresholds for detecting plain film abnormalities on contrast examinations rather than plain radiographs. A previous eye-position study showed that this shift was based on a relative visual neglect of plain film regions on the contrast studies. We now determine whether an intervention that changes visual search can reduce this search-based satisfaction of search effect in contrast studies of the abdomen. MATERIALS AND METHODS: The authors measured detection of 23 plain film abnormalities in 44 patients who had plain film and contrast examinations. In 2 experiments, each plain-film and contrast study was examined independently in different sessions with observers providing a confidence rating of abnormality for each interpretation. There were 13 observers in the first experiment and 10 in the second experiment. The intervention required that for the contrast studies, observers first report abnormalities in the noncontrast region of the radiograph before reporting contrast findings. ROC curve areas for each observer in each treatment condition were estimated by using a proper ROC model. The analysis focused on changes in decision thresholds among the treatment conditions. RESULTS: The SOS effect on decision thresholds in abdominal contrast studies was replicated. Although reduced, the shift in decision thresholds in detecting plain film abnormalities on contrast examinations remained when observers were required to report those abnormalities before contrast findings. CONCLUSION: Reporting plain film abnormalities before reporting abnormalities demonstrated by contrast reduced somewhat the satisfaction of search effect on decision thresholds produced by a visual neglect of noncontrast regions on contrast examinations. This suggests that interventions that direct visual search do not offer protection against satisfaction of search effects that are based on faulty visual search.

An empirical comparison of discrete ratings and subjective probability ratings.

RATIONALE AND OBJECTIVES: The authors compared receiver operating characteristic (ROC) data from a five-category discrete scale with that from a 101-category subjective probability scale to determine how well the latter categories define the ROC curve. MATERIALS AND METHODS: The authors analyzed data from a pilot study performed for another purpose in which 10 radiologists provided both a five-point confidence rating and a subjective probability rating of abnormality for each interpretation. ROC operating points were plotted for a five-category scale and a 101-category scale to determine how well the observed points covered the range of false-positive probabilities. ROC curves were fitted to the subjective probability data according to the standard ROC model. RESULTS: For these data, subjective probability ratings were somewhat more effective in populating the range of false-positive probability with ROC points. For three observers, the ROC curves inappropriately crossed the chance line. For another four, prevention of such crossing seemed to depend on one or two ROC points near the upper right corner of the ROC space, points based on discriminations within the discrete category "no abnormality to report." CONCLUSION: Subjective probability rating should provide substantially better coverage of the ROC space with operating points, preventing inappropriate crossing of the chance line. Unfortunately, the protection offered by subjective probability ratings was unreliable and depended on ROC points derived from discriminations not directly related to apparent abnormality. The use of proper ROC models to fit data may offer a better solution.

Gaze dwell times on acute trauma injuries missed because of satisfaction of search.

RATIONALE AND OBJECTIVES: The authors performed this study to determine whether satisfaction of search (SOS) errors in patients with multiple traumas are caused by faulty visual scanning, faulty recognition, or faulty decision making. MATERIALS AND METHODS: A series of radiographs were obtained in patients with multiple traumas. Radiologists interpreted each series under two experimental conditions: when the first radiograph in the series included a fracture, and when it did not. In the first experiment, the initial radiographs showed nondisplaced fractures of the extremities (minor fractures); in the second experiment, the initial radiographs showed abnormalities of greater clinical importance (major fractures). Each series also included a radiograph with a subtle (test) fracture and a normal radiograph on which detection accuracy was measured. In each experiment, gaze dwell time was recorded as 10 radiologists reviewed images from 10 simulated cases of multiple trauma. RESULTS: An SOS effect could be demonstrated only in the second experiment. Analysis of dwell times showed that search on subsequent radiographs was shortened when the initial radiograph contained a fracture; however, the errors were not based on faulty scanning. CONCLUSION: The SOS effect in musculoskeletal trauma is not caused by faulty scanning. Demonstration of an SOS effect on test fractures with major but not minor additional fractures suggests that detection of other fractures is inversely related to the severity of the detected fracture.

Role of faulty decision making in the satisfaction of search effect in chest radiography.

RATIONALE AND OBJECTIVES: The authors performed this study to determine whether defective pattern recognition or defective decision making is more to blame for satisfaction of search (SOS) errors in chest radiography. MATERIALS AND METHODS: Fifty-eight chest radiographs-half of which demonstrated diverse, native abnormalities-were read by 20 observers. The radiographs were read twice, once with and once without the addition of a simulated pulmonary nodule. Observers provided a verbal account of their focus of attention, indicating suspicious features and regions considered during their inspection of the radiograph. Observers also provided a separate account of the abnormalities they would include in a radiologic report. RESULTS: When the authors considered only those reports that did not refer to the simulated nodules, they found no reduction in the area under the proper receiver operating characteristic (ROC) curves in cases that contained nodules. A smaller SOS effect, however, was demonstrated with analysis of events in which the native abnormality was missed in one condition but not the other. Verbal protocols suggested that the SOS errors were mainly caused by recognition failure rather than faulty decision making. CONCLUSION: Describing their focus of attention may have prompted observers to inspect the radiographs in a more deliberate, systematic way, thus reducing the SOS effect. More residual SOS errors were caused by defective pattern recognition than by faulty decision making.

Proper ROC analysis and joint ROC analysis of the satisfaction of search effect in chest radiology.

RATIONALE AND OBJECTIVES: The purpose of this study was to test whether the satisfaction of search (SOS) effect in chest radiology could be demonstrated with proper receiver operating characteristic (ROC) curves and with joint detection and localization ROC curves. MATERIAL AND METHODS: Data from an earlier ROC study of SOS in chest radiology were analyzed with three proper ROC models and one ROC model for joint detection and localization. Fits of the models were compared on the basis of likelihood-ratio chi-squared statistics (G2). To examine further the validity of the SOS effect in chest radiology, the authors also replicated the earlier study with a new sample of readers, analyzing the new data with the same methods. RESULTS: The proper contaminated binormal model fit the data better than the other two proper ROC models. Contaminated binormal analysis of the earlier and the replication experiment demonstrated an SOS effect: a reduction in area under the ROC curve for detection of the native abnormalities with the addition of nodules. Similarly, joint ROC analysis producing curves that appropriately cross the chance line gave similar results. CONCLUSION: Preventing inappropriate chance line crossing reduces measurement error and provides more powerful statistical tests. Results of both experiments showed that the SOS effect in chest radiology can be demonstrated with ROC methods that avoid inappropriate crossing of the chance line.

Role of faulty visual search in the satisfaction of search effect in chest radiography.

RATIONALE AND OBJECTIVES: The authors tested the hypothesis that satisfaction of search effect, which is associated with the failure to detect native chest abnormalities in the presence of simulated nodules, is caused by reduced gaze on the native abnormalities. MATERIALS AND METHODS: Gaze dwell time of 20 radiologists was recorded for the region around abnormalities on images. Ten radiographs were reviewed, nine of which contained native abnormalities. Each image was seen with and without a simulated nodule. RESULTS: The decrease in the rate of true-positive findings in the detection of native abnormalities on images that contained simulated nodules confirmed the occurrence of a satisfaction of search effect. Gaze times on native abnormalities (up to the time of report of the abnormalities) were the same for images with nodules in which native abnormalities were missed (gaze time, 9.4 seconds) as they were for images without nodules in which native abnormalities were detected (gaze time, 9.5 seconds). Gaze time on missed native abnormalities was not affected by the presence (7.80 seconds) or absence (7.45 seconds) of nodules. CONCLUSION: Reduction in gaze dwell time on the missed abnormalities is not the cause of satisfaction of search errors in chest radiographs.

Teleradiology consultation for a rural hospital: patterns of use.

RATIONALE AND OBJECTIVES: The authors documented quantitatively how teleradiology was used for contemporaneous consultation with a radiologist by physicians at a small rural hospital. MATERIALS AND METHODS: The authors collected 31/2 years worth of data at a small hospital on all patients (n = 327) for whom a radiologist's consultation by teleradiology was requested and compared the data with those from a control group of 309 patients whose studies were interpreted at the same hospital. RESULTS: Teleradiology consultation was requested by the attending physician in 3.6% (519 of 14,586) of all examinations performed during the study period. Physicians requested teleradiology consultation most often for patients with multiple examinations (average 1.59 vs 1.35 for controls). Examinations of infants were 10 times more prevalent in the teleradiology group than in the control group. The indication for most (52%) teleradiology consultations was trauma. Requests for interpretation of spine and abdominal radiographs were relatively more frequent than were those of other studies. CONCLUSION: Physicians in this rural practice used contemporaneous radiologic consultation for selected specific examinations, with emphasis on examinations for trauma, spine, abdomen, and the infant age group.

Pediatric radiology at a rural hospital: value of teleradiology and subspecialty consultation.

OBJECTIVE: We performed a field study and subsequent laboratory investigation of pediatric radiology at a small rural hospital. Our investigation had three components: to describe the characteristics of pediatric radiology in a rural primary care facility, to test the diagnostic accuracy of interpretation of pediatric images transmitted by teleradiology, and to compare relative diagnostic accuracy of general and pediatric radiologists who interpreted pediatric images at a rural institution. MATERIALS AND METHODS: All 196 pediatric radiographs obtained during a 4-month period comprised the database from which we determined practice characteristics. Reports of 153 unselected cases interpreted by general radiologists using teleradiology were compared with interpretation of the same cases by a pediatric radiologist who interpreted the original radiographs. Discrepant cases were further investigated by a receiver-operating-characteristic curve experiment in which general and pediatric radiologists interpreted each case twice: once viewing teleradiologic images and once viewing the original radiographs at another setting. We then compared interpretive accuracy of observers and techniques. RESULTS: The pediatric radiographs were predominantly simple examinations for common acute disease, particularly pneumonia and fractures. Discrepancies of interpretation between teleradiology and original radiographs, which occurred in 13% of images, showed no significant difference in accuracy of interpretation for either teleradiologic images or original radiographs. Likewise, we found no significant advantage for accuracy of interpretation by general or pediatric radiologists. Receiver-operating-characteristic analysis of 18 discrepant cases showed slightly increased accuracy for interpretation of original radiographs by pediatric subspecialists. CONCLUSION: Simple pediatric radiographs obtained at a rural primary care institution and transmitted by teleradiology can be adequately interpreted by general radiologists.

Cause of satisfaction of search effects in contrast studies of the abdomen.

RATIONALE AND OBJECTIVES: Extraintestinal abnormalities visible without contrast material on abdominal radiographs are reported less frequently when contrast examinations are performed. Gaze dwell time was used to determine whether this difference is due to failure by observers to scan plain-film regions of contrast studies or discounting of plain-film abnormalities that were actually scanned. METHODS: Patients were included whose contrast studies had elicited the largest reductions in positive responses compared with their plain-film studies in a previous detection experiment. Gaze of 10 radiologists was studied. RESULTS: Significantly less time was spent gazing at non-contrast regions of contrast examinations than at the corresponding regions of radiographs. Errors with radiographs were based primarily on failures of recognition and decision making, whereas errors with contrast studies were based primarily on faulty scanning. CONCLUSION: Satisfaction of search errors on contrast examinations are caused by reduction in scanning of noncontrast regions.

Imaging techniques in the diagnosis of gastrointestinal and hepatic disease.

Abdominal imaging continues to grow in importance for diagnosis, anatomic staging, and functional analysis of gastrointestinal ailments. In this review, we attempt to delineate those publications, both original investigations and summary articles, that have advanced our understanding of abdominal imaging in infants and children. Special attention is given to the newer modalities that provide previously unavailable methods to analyze abnormalities. These include Doppler ultrasonography and magnetic resonance imaging as well as CT for abdominal trauma. Interventional radiology techniques useful for treatment of gastrointestinal diseases are also discussed.