Multiple diagnostic task performance in CT examination of the chest.

OBJECTIVES: In three experiments, we studied the detection of multiple abnormality types using the satisfaction of search (SOS) paradigm, the provision of a computer-aided detection (CAD) of pulmonary nodules and a focused nodule detection task. METHODS: 51 chest CT examinations (24 that demonstrated subtle pulmonary nodules and 27 that demonstrated no pulmonary nodules) were read by 15 radiology residents and fellows under two experimental conditions: (1) when there were no other abnormalities present except test abnormalities in the exams (non-SOS condition), and (2) when other abnormalities were present in the exams (SOS condition). Trials from the two conditions were intermixed. Readers were invited to return for two sessions: one in which the SOS condition was repeated with a simulated CAD; another in which only the non-SOS condition was presented. Detection accuracy was measured using receiver operating characteristic (ROC) analysis. RESULTS: An SOS effect (reduced detection accuracy for the test nodules in the presence of the diverse added abnormalities) was not found. Average accuracy was much higher when the CAD prompt was provided, without cost in the detection of the added abnormalities. Accuracy for detecting nodules appearing without intermixed SOS trials was also substantially improved. CONCLUSIONS: CT interpretation was highly task dependent. Nodule detection was poor in the general search task. Therefore, CAD may offer a greater performance improvement than demonstrated in experiments assessing CAD using focused search. The absence of SOS may be due to limited nodule detection even without other abnormalities. Advances in knowledge CAD prompts of nodules increase the detection accuracy of nodules and decrease the time to detection-without impairing the detection accuracy-of non-nodule abnormalities.

Multiple diagnostic task performance in CT examination of the chest.

Objectives In three experiments, we studied the detection of multiple abnormality types using the satisfaction of search (SOS) paradigm, the provision of a computer-aided detection (CAD) of pulmonary nodules and a focused nodule detection task. Methods 51 chest CT examinations (24 that demonstrated subtle pulmonary nodules and 27 that demonstrated no pulmonary nodules) were read by 15 radiology residents and fellows under two experimental conditions: (1) when there were no other abnormalities present except test abnormalities in the exams (non-SOS condition), and (2) when other abnormalities were present in the exams (SOS condition). Trials from the two conditions were intermixed. Readers were invited to return for two sessions: one in which the SOS condition was repeated with a simulated CAD; another in which only the non-SOS condition was presented. Detection accuracy was measured using receiver operating characteristic (ROC) analysis. Results An SOS effect (reduced detection accuracy for the test nodules in the presence of the diverse added abnormalities) was not found. Average accuracy was much higher when the CAD prompt was provided, without cost in the detection of the added abnormalities. Accuracy for detecting nodules appearing without intermixed SOS trials was also substantially improved. Conclusions CT interpretation was highly task dependent. Nodule detection was poor in the general search task. Therefore, CAD may offer a greater performance improvement than demonstrated in experiments assessing CAD using focused search. The absence of SOS may be due to limited nodule detection even without other abnormalities. Advances in knowledge CAD prompts of nodules increase the detection accuracy of nodules and decrease the time to detection-without impairing the detection accuracy-of non-nodule abnormalities.

Factors influencing visualization of vertebral metastases on MR imaging versus bone scintigraphy.

OBJECTIVE: The purpose of this study was to investigate whether the location and size of vertebral body metastases influence the difference in detection rates between MR imaging and bone scintigraphy. MATERIALS AND METHODS: We retrospectively evaluated the vertebral body lesions detected on MR imaging in 74 patients with known widely disseminated metastatic disease. Three radiologists independently reviewed the MR images and bone scintigraphs. MR imaging findings included lesion size and its spatial relationship to the bony cortex (intramedullary, subcortical, and transcortical) and results were correlated with those of planar technetium 99m bone scintigraphy. RESULTS: Findings on bone scans were negative for all intramedullary lesions without cortical involvement shown on MR imaging, regardless of their size. Findings on bone scans (71.3% for transcortical and 33.8% for subcortical) were frequently positive for lesions with cortical involvement (trans- or subcortical), and the probability of positive findings on bone scans was also influenced by the lesion size. Statistical analysis showed a positive correlation among cortical involvement, lesion size, and positive findings on bone scintigraphy (p < 0.0001). CONCLUSION: Location (the presence of cortical bone involvement on MR imaging) and size of the vertebral body metastases appear to be important contributing factors to the difference in detection rates between MR imaging and bone scintigraphy. Cortical involvement is likely the cause of positive findings on bone scans. Early vertebral metastases tend to be small and located in the medullary cavity without cortical involvement, and therefore, findings may be positive on MR images but negative on bone scans.

Gaze dwell times on acute trauma injuries missed because of satisfaction of search.

RATIONALE AND OBJECTIVES: The authors performed this study to determine whether satisfaction of search (SOS) errors in patients with multiple traumas are caused by faulty visual scanning, faulty recognition, or faulty decision making. MATERIALS AND METHODS: A series of radiographs were obtained in patients with multiple traumas. Radiologists interpreted each series under two experimental conditions: when the first radiograph in the series included a fracture, and when it did not. In the first experiment, the initial radiographs showed nondisplaced fractures of the extremities (minor fractures); in the second experiment, the initial radiographs showed abnormalities of greater clinical importance (major fractures). Each series also included a radiograph with a subtle (test) fracture and a normal radiograph on which detection accuracy was measured. In each experiment, gaze dwell time was recorded as 10 radiologists reviewed images from 10 simulated cases of multiple trauma. RESULTS: An SOS effect could be demonstrated only in the second experiment. Analysis of dwell times showed that search on subsequent radiographs was shortened when the initial radiograph contained a fracture; however, the errors were not based on faulty scanning. CONCLUSION: The SOS effect in musculoskeletal trauma is not caused by faulty scanning. Demonstration of an SOS effect on test fractures with major but not minor additional fractures suggests that detection of other fractures is inversely related to the severity of the detected fracture.

Pixel analysis of MR perfusion imaging in predicting radiation therapy outcome in cervical cancer.

The purpose of this study was to assess heterogeneity of tumor microcirculation determined by dynamic contrast-enhanced magnetic resonance (MR) imaging and its prognostic value for tumor radiosensitivity and long-term tumor control using pixel-by-pixel analysis of the dynamic contrast enhancement. Sixteen patients with advanced cervical cancer were examined with dynamic contrast-enhanced MR imaging at the time of radiation therapy. Pixel-by-pixel statistical analysis of the ratio of post- to precontrast relative signal intensity (RSI) values in the tumor region was performed to generate pixel RSI distributions of dynamic enhancement patterns. Histogram parameters were correlated with subsequent tumor control based on long-term cancer follow-up (median follow-up 4.6 years; range 3.8-5.2 years). The RSI distribution histograms showed a wide spectrum of heterogeneity in the dynamic enhancement pattern within the tumor. The quantity of low-enhancement regions (10th percentile RSI < 2.5) significantly predicted subsequent tumor recurrence (88% vs. 0%, P = 0.0004). Discriminant analysis based on both 10th percentile RSI and pixel number (reflective of tumor size) further improved the prediction rate (100% correct prediction of subsequent tumor control vs. recurrence). These preliminary results suggest that quantification of the extent of poor vascularity regions within the tumor may be useful in predicting long-term tumor control and treatment outcome in cervical cancer. J. Magn. Reson. Imaging 2000;12:1027-1033.

Role of faulty decision making in the satisfaction of search effect in chest radiography.

RATIONALE AND OBJECTIVES: The authors performed this study to determine whether defective pattern recognition or defective decision making is more to blame for satisfaction of search (SOS) errors in chest radiography. MATERIALS AND METHODS: Fifty-eight chest radiographs-half of which demonstrated diverse, native abnormalities-were read by 20 observers. The radiographs were read twice, once with and once without the addition of a simulated pulmonary nodule. Observers provided a verbal account of their focus of attention, indicating suspicious features and regions considered during their inspection of the radiograph. Observers also provided a separate account of the abnormalities they would include in a radiologic report. RESULTS: When the authors considered only those reports that did not refer to the simulated nodules, they found no reduction in the area under the proper receiver operating characteristic (ROC) curves in cases that contained nodules. A smaller SOS effect, however, was demonstrated with analysis of events in which the native abnormality was missed in one condition but not the other. Verbal protocols suggested that the SOS errors were mainly caused by recognition failure rather than faulty decision making. CONCLUSION: Describing their focus of attention may have prompted observers to inspect the radiographs in a more deliberate, systematic way, thus reducing the SOS effect. More residual SOS errors were caused by defective pattern recognition than by faulty decision making.

Proper ROC analysis and joint ROC analysis of the satisfaction of search effect in chest radiology.

RATIONALE AND OBJECTIVES: The purpose of this study was to test whether the satisfaction of search (SOS) effect in chest radiology could be demonstrated with proper receiver operating characteristic (ROC) curves and with joint detection and localization ROC curves. MATERIAL AND METHODS: Data from an earlier ROC study of SOS in chest radiology were analyzed with three proper ROC models and one ROC model for joint detection and localization. Fits of the models were compared on the basis of likelihood-ratio chi-squared statistics (G2). To examine further the validity of the SOS effect in chest radiology, the authors also replicated the earlier study with a new sample of readers, analyzing the new data with the same methods. RESULTS: The proper contaminated binormal model fit the data better than the other two proper ROC models. Contaminated binormal analysis of the earlier and the replication experiment demonstrated an SOS effect: a reduction in area under the ROC curve for detection of the native abnormalities with the addition of nodules. Similarly, joint ROC analysis producing curves that appropriately cross the chance line gave similar results. CONCLUSION: Preventing inappropriate chance line crossing reduces measurement error and provides more powerful statistical tests. Results of both experiments showed that the SOS effect in chest radiology can be demonstrated with ROC methods that avoid inappropriate crossing of the chance line.

A contaminated binormal model for ROC data: Part I. Some interesting examples of binormal degeneracy.

RATIONALE AND OBJECTIVES: Receiver operating characteristic (ROC) data with false-positive fractions of 0 are often difficult to fit with standard ROC methods and are sometimes discarded. Some extreme examples of such data were analyzed to evaluate the nature of these difficulties. MATERIALS AND METHODS: Rating reports of fracture for single-view ankle radiographs were analyzed with the binormal ROC model and with two ROC models that keep the ROC curve from crossing the chance line. Because fractures were almost never reported that were not present, some views and locations yielded only ROC points with false-positive fractions of 0, while others yielded at least one ROC point with a non-0 false-positive fraction. RESULTS: The models tended to yield ROC areas close to or equal to 1. ROC areas of 1 imply a true-positive fraction close to 1; yet the data contained no such fractions. When all false-positive fractions were 0, the true-positive fraction could be much higher for one view than another for all observers. ROC areas gave little or no hint of these unmistakable differences in performance. CONCLUSION: These data challenge the validity and robustness of current ROC models. A key aspect of ankle fractures is that some may be visible on one view but not at all visible on another.

A contaminated binormal model for ROC data: Part II. A formal model.

RATIONALE AND OBJECTIVES: A contaminated binormal receiver operating characteristic (ROC) model is proposed to account for ROC data that have very few false-positive reports even though many healthy subjects are sampled. The model assumes that no signal information is captured for a proportion of abnormalities, and that these abnormalities have the same distribution as noise along the latent decision axis. MATERIALS AND METHODS: The authors developed a formal psychophysical model, presented here in detail. They have specified the psychophysical assumptions of the theory, and have provided proofs that include all essential details, from assumptions to implications. With the technical details that are provided, this theory can be implemented with computer programs to fit data. RESULTS: The new model can fit ROC data in which some or all of the ROC points have false-positive fractions of 0 and true-positive fractions of less than 1, without implying that performance is perfect. The resulting ROC curves are always proper, never exhibiting inappropriate chance line crossings. The model predicts that, under certain conditions, a bimodal categorical rating histogram will be observed for the signal distribution. The model predicts a relationship between the mean and standard deviation of the signal distribution and holds that, for expert decision makers, there are situations in which the prevalence and utility matrix preclude operating points in some ROC regions. The model has a straightforward extension to the joint detection and localization ROC curve. CONCLUSION: The contaminated binormal model accounts for ROC data with few or no false-positive reports.

A contaminated binormal model for ROC data: Part III. Initial evaluation with detection ROC data.

RATIONALE AND OBJECTIVES: The authors' purpose was to evaluate how well the contaminated binormal receiver operating characteristic (ROC) model fits (a) degenerate data for which standard ROC models commonly fail and (b) nondegenerate data from exemplary experiments, for which the standard binormal model should be appropriate. MATERIALS AND METHODS: The authors studied two examples of binormally degenerate data, with and without interior points, and ROC rating data from four experiments in visual psychophysics and radiology. The plots of contaminated binormal ROC curves of the binormal degenerate data were examined. For ROC data with at least one interior point, the new model was compared with conventional models on the basis of likelihood-ratio chi2 statistics (G2). RESULTS: With no interior points, the contaminated binormal model gave results consistent with the fundamental principle underlying ROC analysis, that is, for a fixed false-positive probability, the higher the true-positive probability, the better the diagnostic performance. Contaminated binormal ROC curves go through the empirical ROC points of the degenerate data without crossing the chance line or climbing far above the true-positive fractions of the points. For several model ROC studies, the contaminated binormal model gave smaller G2 results than conventional ROC models, although the differences tended to be small, usually with little difference in ROC area. CONCLUSION: The contaminated binormal model fits binormal degenerate data better than conventional ROC models, and it offers an explanation for the degeneracy. The lower G2 values on some classic, nondegenerate ROC data suggest that contamination may not be limited to degenerate ROC data.

Contribution of individual projections alone and in combination for radiographic detection of ankle fractures.

OBJECTIVE: We wanted to determine whether the standard three-view ankle radiographic series could be replaced by a two-view combination, and if so, which two-view combination (anteroposterior with lateral or mortise with lateral) would be superior. MATERIALS AND METHODS: During a 12-month period, we retrospectively reviewed 556 consecutive ankle radiographic studies consisting of anteroposterior, mortise, and lateral views. One hundred twenty patients with at least one ankle fracture were paired with 140 healthy control subjects. Each image in the three-view examination was separated and sorted by view and studied independently; all images were reviewed by two skeletal radiologists and two orthopedic surgeons. Each radiograph was evaluated for fracture of the medial, lateral, and posterior malleoli and the foot using a five-point confidence rating. Performance of each view and modeled two- and three-view combinations of views was evaluated with modified receiver operating characteristic analysis. RESULTS: The data provide little support for preferring either two-view combination (anteroposterior-lateral or mortise-lateral) for any type of fracture. The three-view combination does detect significantly more fractures than some two-view combinations in some locations, and there is a statistically significant cost in diagnostic accuracy for eliminating the anteroposterior or mortise view. CONCLUSION: Reducing the ankle radiographic series from three to two views would result in a small but significant decrease in the detection of fractures of the ankle and foot. Both two-view combinations are equivalent for fracture detection.

Adjunctive non-pharmacological analgesia for invasive medical procedures: a randomised trial.

BACKGROUND: Non-pharmacological behavioural adjuncts have been suggested as efficient safe means in reducing discomfort and adverse effects during medical procedures. We tested this assumption for patients undergoing percutaneous vascular and renal procedures in a prospective, randomised, single-centre study. METHODS: 241 patients were randomised to receive intraoperatively standard care (n=79), structured attention (n=80), or self-hypnotic relaxation (n=82). All had access to patient-controlled intravenous analgesia with fentanyl and midazolam. Patients rated their pain and anxiety on 0-10 scales before, every 15 min during and after the procedures. FINDINGS: Pain increased linearly with procedure time in the standard group (slope 0.09 in pain score/15 min, p<0.0001), and the attention group (slope 0.04/15 min; p=0.0425), but remained flat in the hypnosis group. Anxiety decreased over time in all three groups with slopes of -0.04 (standard), -0.07 (attention), and -0.11 (hypnosis). Drug use in the standard group (1.9 units) was significantly higher than in the attention and hypnosis groups (0.8 and 0.9 units, respectively). One hypnosis patient became haemodynamically unstable compared with ten attention patients (p=0.0041), and 12 standard patients (p=0.0009). Procedure times were significantly shorter in the hypnosis group (61 min) than in the standard group (78 min, p=0.0016) with procedure duration of the attention group in between (67 min). INTERPRETATION: Structured attention and self-hypnotic relaxation proved beneficial during invasive medical procedures. Hypnosis had more pronounced effects on pain and anxiety reduction, and is superior, in that it also improves haemodynamic stability.

Correcting errors in eye-position data arising from the distortion of magnetic fields by display devices.

Eye-position recordings have been used to produce detailed information about sampling and search behavior. The melding of magnetic head-tracking technology with eye-tracking systems has allowed for freer head movement, but not without problems. Although the requisite calibrations made before and after a reading might indicate good eye-tracking accuracy, point-of-gaze errors can occur if the reader leans toward a display, such as a film alternator. The error results from the characteristic that large metal masses can distort magnetic fields, leading to false magnetic sensor data. Corrections for this type of error have involved mapping the exact location and orientation of the magnetic sensor in the magnetic field by using a precision positioning device. We have devised a much cheaper method that relies on a fixture that holds the magnetic sensor in identical positions, with and without the distorting effect.

A survey of post-discharge side effects of conscious sedation using chloral hydrate in pediatric CT and MR imaging.

BACKGROUND: Limited information is available on post-discharge side effects of chloral hydrate sedation in pediatric imaging. OBJECTIVE: To prospectively study the post-discharge side effects of chloral hydrate sedation in pediatric CT and MR imaging. MATERIALS AND METHODS: A total of 119 children undergoing CT and MRI were sedated using chloral hydrate with 89 % success (mean initial dose, 72 mg/kg body weight) and 98 % success after augmentation (mean total, 78 mg/kg body weight). The frequency of each post-discharge side effect was correlated with other side effects and 12 patient/technical parameters. RESULTS: The survey was completed in 80 children. Sleepiness lasted for > 4 h in 28 %. Unsteadiness occurred in 68 % and hyperactivity in 29 %. Appetite became poor in 14 % and vomiting occurred in 15 %. Normal activity was resumed after > 4 h in 54 %. Sleep deprivation did not result in increased success or earlier onset of sedation and might be associated with hyperactivity. A higher dose did not result in an increased success rate or earlier onset of sedation within the dose range used in this study. CONCLUSION: Data on the post-discharge side effects of chloral hydrate sedation will be useful to radiologists, technologists, and nurses explaining to parents about sedation using this agent.

Monte Carlo validation of a multireader method for receiver operating characteristic discrete rating data: factorial experimental design.

RATIONALE AND OBJECTIVES: The authors conducted a series of null-case Monte Carlo simulations to evaluate the Dorfman-Berbaum-Metz (DBM) method for comparing modalities with multireader receiver operating characteristic (ROC) discrete rating data. MATERIALS AND METHODS: Monte Carlo simulations were performed by using discrete ratings on fully crossed factorial designs with two modalities and three, five, and 10 hypothetical readers. The null hypothesis was true for all simulations. The population ROC areas, latent variable structures, case sample sizes, and normal/abnormal case sample ratios used in another study were used in these simulations. RESULTS: For equal allocation ratios and small (Az = 0.702) and moderate (Az = 0.855) ROC areas, the empirical type I error rate closely matched the nominal alpha level. For very large ROC areas (Az = 0.961), however, the empirical type I error rate was somewhat smaller than the nominal alpha level. This conservatism increased with decreasing case sample size and asymmetric normal/abnormal case allocation ratio. The empirical type I error rate was sometimes slightly larger than the nominal alpha level with many cases and few readers, where there was large residual, relatively small treatment-by-case interaction and relatively large treatment-by-reader interaction. CONCLUSION: The results suggest that the DBM method provides trustworthy alpha levels with discrete ratings when the ROC area is not too large and case and reader sample sizes are not too small. In other situations, the test tends to be somewhat conservative or slightly liberal.

Determinants of intravenous conscious sedation for arteriography.

PURPOSE: To assess factors that determine the amount of drugs given for intravenous conscious sedation during arteriography. MATERIALS AND METHODS: Data from 254 patients undergoing infradiaphragmatic arteriography at three institutions were evaluated. The effect of age, sex, procedure time, attending physician, and institution on drug use was assessed by analysis of variance and covariance with repeated measures. In a subset of 34 patients, pain and anxiety scores before and after medication were correlated with drug scores. RESULTS: Institution identity and procedure time significantly affected the amount of medication used (both, P = .000). Patient's age and sex, and identity of the physician had no significant effects. While drug use was relatively constant in each institution among different staff physicians, the institutional differences prevailed when the same physicians performed procedures at different institutions. Drug deliveries did not correlate with anxiety and pain scores before or after medication. CONCLUSION: Habits and philosophies of particular institutions, rather than physician guidance or patients' needs, tend to govern the use of intravenous sedatives and analgesics. There is a need for a more patient-oriented standardization of intravenous conscious sedation and analgesia.

Identifying left lower lobe pneumonia at chest radiography: performance of family practice residents before and after a didactic session.

RATIONALE AND OBJECTIVES: The authors sought to determine whether the lateral chest radiograph is helpful in identifying left lower lobe pneumonia among inexperienced readers. MATERIALS AND METHODS: The authors selected all patients who presented to a family practice training program with radiologic and clinical evidence of left lower lobe pneumonia (n = 65). They then selected an equal number of patients in whom chest radiographs were taken to "rule out pneumonia" and were found to be normal. Eight 1st-year family practice residents were asked to read the radiographs before and after a didactic session that emphasized lateral chest radiograph interpretation. The radiographs were presented under two viewing conditions: posteroanterior (PA) only versus PA and lateral. Receiver operating characteristic (ROC) curve methods were used to compare the effect of both the didactic session and the viewing condition on diagnostic accuracy. RESULTS: There were no significant differences in performance before and after the didactic session and no differences between the two viewing conditions. After including only abnormal radiographs that demonstrated the "spine sign" (an apparent increased opacification of the lower vertebral bodies on the lateral view), the residents performed better when presented with both PA and lateral radiographs than when presented with the PA radiograph only (area under ROC curve, .8158 vs .7418, respectively; P = 0.24). CONCLUSION: In patients with left lower lobe pneumonia whose radiographs demonstrated the spine sign, diagnostic accuracy improved when the lateral chest radiograph was viewed.

Use of reports of previous radiologic studies.

RATIONALE AND OBJECTIVES: The purpose of this study was to determine the length of time over which previous radiologic reports were useful during current interpretations and the types of information gathered from the old reports. MATERIALS AND METHODS: The authors directly observed the use of 209 old reports by radiologists during current radiologic interpretation. They recorded the date of the old report, modality used, and type of information sought. The interpreting radiologists then rated the value of the information in the old report on a Likert scale of 1 (high value) to 4 (minimal value). RESULTS: In 85% of cases, the most valuable information obtained was found in reports that were less than 2 years old. The most frequently sought types of information were clinical history or the previous radiologist's interpretation. CONCLUSION: Old reports add value to current interpretation by providing a form of "second reading" and providing clinical history. Two years is sufficient old report access in 85% of situations.

Role of faulty visual search in the satisfaction of search effect in chest radiography.

RATIONALE AND OBJECTIVES: The authors tested the hypothesis that satisfaction of search effect, which is associated with the failure to detect native chest abnormalities in the presence of simulated nodules, is caused by reduced gaze on the native abnormalities. MATERIALS AND METHODS: Gaze dwell time of 20 radiologists was recorded for the region around abnormalities on images. Ten radiographs were reviewed, nine of which contained native abnormalities. Each image was seen with and without a simulated nodule. RESULTS: The decrease in the rate of true-positive findings in the detection of native abnormalities on images that contained simulated nodules confirmed the occurrence of a satisfaction of search effect. Gaze times on native abnormalities (up to the time of report of the abnormalities) were the same for images with nodules in which native abnormalities were missed (gaze time, 9.4 seconds) as they were for images without nodules in which native abnormalities were detected (gaze time, 9.5 seconds). Gaze time on missed native abnormalities was not affected by the presence (7.80 seconds) or absence (7.45 seconds) of nodules. CONCLUSION: Reduction in gaze dwell time on the missed abnormalities is not the cause of satisfaction of search errors in chest radiographs.