Effect of fatigue on reading computed tomography examination of the multiply injured patient.

Our goal was to ascertain how fatigue affects performance in reading computed tomography (CT) examinations of patients with multiple injuries. CT images with multiple fractures from a previous study of satisfaction of search (SOS) were read by radiologists after a day of clinical work. Performance in this study with fatigued readers was compared to a previous study in which readers were not fatigued. Detection accuracy for obvious injuries was not affected by fatigue, but accuracy for subtle fractures was reduced ([Formula: see text]). An SOS effect on decision thresholds was evident mirroring recent studies. Without fatigue, readers spent more time interpreting and reporting findings as the number of the injuries increased. When fatigued, readers did not increase reading time as fracture number increased. Without fractures, reading time for not-fatigued and fatigued readers was the same ([Formula: see text]) but was significant ([Formula: see text]) with an added subtle fracture. The difference increased with a major injury ([Formula: see text]) and increased further with both a major injury and subtle fracture ([Formula: see text]). Fatigue and multiple abnormalities have independent effects on detection performance but do interact in determining search time.

The Impact of Fatigue on Satisfaction of Search in Chest Radiography.

RATIONALE AND OBJECTIVES: To assess the nature of the satisfaction of search (SOS) effect in chest radiography when observers are fatigued; determine if we could replicate recent findings that have documented the nature of the SOS effect to be due to a threshold shift rather than a change in diagnostic accuracy as in earlier film-based studies. MATERIALS AND METHODS: Nearing or at the end of a clinical workday, 20 radiologists read 64 chest images twice, once with and once without the addition of a simulated pulmonary nodule. Half of the images had different types of "test" abnormalities. Decision thresholds were analyzed using the center of the range of false-positive (FP) and true-positive (TP) fractions associated with each receiver operating characteristic (ROC) point for reporting test abnormalities. Detection accuracy was assessed with ROC technique and inspection time was recorded. RESULTS: The SOS effect was confirmed to be a reduction in willingness to respond (threshold shift). The center of the FP range was significantly reduced (FP = 0.10 without added nodules, FP = 0.05 with added nodules, F(1,18) = 19.85, P = 0.0003). The center of the TP range was significantly reduced (TP = 0.39 without added nodules, TP = 0.33 with added nodules, F(1,18) = 10.81, P = 0.004). CONCLUSIONS: This study suggests that fatigue does not change the nature of the SOS effect, but rather may be additive with the SOS effect. SOS reduces both TP and FP responses, whereas fatigue reduces TPs more than FPs.

Trauma in CT: The Role of Severe Injury on Satisfaction of Search Revised.

PURPOSE: The satisfaction-of-search (SOS) effect occurs when an abnormality on an image is missed because another is found. The aim of this experiment was to test whether severe distracting fractures control the magnitude of SOS on other fractures when both appear in a single CT image. METHODS: The institutional review board approved this study. The experimental (SOS) condition included 35 cervical spine CT cases, all of which contained severe cervical spine injuries. For each of these cases, a similar case was found that had no injuries. Image modification software was developed to add simulated fractures to each pair of cases, with and without a major injury. Sixteen different minor fractures were added to 16 of the 35 pairs of images. The 35 cases without native injuries constituted a control (non-SOS) condition mixed in a random order. Twenty radiologists read 35 mixed cases in each of two sessions. False-positive evaluations were collected only for cases without simulated fractures. RESULTS: An SOS effect on the detection of simulated fractures was not observed. There was a nonsignificant (P = .07) finding of poorer detection in the presence of cases with severe injuries. However, the magnitude of the effect was no greater than has been observed for less severe distracting injuries. CONCLUSIONS: The outcome agrees with the results of two previous experiments that failed to yield an SOS effect associated with detecting severe injuries, suggesting that the severity of a distracting injury does not determine whether a second injury is discovered.

The Influence of a Vocalized Checklist on Detection of Multiple Abnormalities in Chest Radiography.

RATIONALE AND OBJECTIVES: Although a checklist has been recommended for preventing satisfaction of search (SOS) errors, a previous research study did not demonstrate that benefit. However, observers in that study had to turn away from the image display to use the checklist. The current study tested a vocalized checklist to avoid this constraint. MATERIALS AND METHODS: A total of 64 chest computed radiographs, half containing various "test" abnormalities, were read twice by 20 radiologists, once with and once without the addition of a simulated pulmonary nodule. Readers used a vocalized checklist-directing search. Receiver operating characteristic (ROC) detection accuracy and decision thresholds were analyzed to study the effects of adding the nodule on detecting the test abnormalities. RESULTS: Adding nodules induced a substantial reluctance to report the other abnormalities (P < 0.001), as had been the case in the most recent study of the SOS effect in radiography. CONCLUSIONS: The vocalized checklist did not reduce nor eliminate the SOS effect on readiness to report further abnormalities. Although useful for organizing search and reporting, particularly among students, a vocalized checklist does not prevent SOS effects.

Satisfaction of Search in Chest Radiography 2015.

RATIONALE AND OBJECTIVES: Two decades have passed since the publication of laboratory studies of satisfaction of search (SOS) in chest radiography. Those studies were performed using film. The current investigation tests for SOS effects in computed radiography of the chest. METHODS: Sixty-four chest computed radiographs half demonstrating various "test" abnormalities were read twice by 20 radiologists, once with and once without the addition of a simulated pulmonary nodule. Receiver-operating characteristic detection accuracy and decision thresholds were analyzed to study the effects of adding the nodule on detecting the test abnormalities. Results of previous studies were reanalyzed using similar modern techniques. RESULTS: In the present study, adding nodules did not influence detection accuracy for the other abnormalities (P = .93), but did induce a reluctance to report them (P < .001). Adding nodules did not affect inspection time (P = .58) so the reluctance to report was not associated with reduced search. Reanalysis revealed a similar decision threshold shift that had not been recognized in the early studies of SOS in chest radiography (P < .01) in addition to reduced detection accuracy (P < .01). CONCLUSIONS: The nature of SOS in chest radiography has changed, but it is not clear why. ADVANCES IN KNOWLEDGE: SOS may be changing as a function of changes in radiology education and practice.

Satisfaction of search from detection of pulmonary nodules in computed tomography of the chest.

RATIONALE AND OBJECTIVES: We tested whether satisfaction of search (SOS) effects that occur in computed tomography (CT) examination of the chest on detection of native abnormalities are produced by the addition of simulated pulmonary nodules. MATERIALS AND METHODS: Two experiments were conducted. In the first experiment, 70 CT examinations, half that demonstrated diverse, subtle abnormalities and half that demonstrated no native lesions, were read by 18 radiology residents and fellows under two experimental conditions: presented with and without pulmonary nodules. In a second experiment, many of the examinations were replaced to include more salient native abnormalities. This set was read by 14 additional radiology residents and fellows. In both experiments, detection of the natural abnormalities was studied. Receiver operating characteristic (ROC) curve areas for each reader-treatment combination were estimated using empirical and proper ROC models. Additional analyses focused on decision thresholds and visual search time on abnormality-free CT slice ranges. Institutional review board approval and informed consent from 32 participants were obtained. RESULTS: Observers more often missed diverse native abnormalities when pulmonary nodules were added, but also made fewer false-positive responses. There was no change in ROC area, but decision criteria grew more conservative. The SOS effect on decision thresholds was accompanied by a reduction in search time on abnormality-free CT slice ranges. CONCLUSION: The SOS effect in CT examination of the chest is similar to that found in contrast examination of the abdomen, involving induced visual neglect.

Clinical role of radiography for thoracic spine fractures in daily practice in the MDCT era: a retrospective review of 255 trauma patients.

PURPOSE: To retrospectively assess the diagnostic efficacy of radiography in detecting vertebral body fractures of the thoracic spine compared with MDCT, to assess the confounding factors reducing the diagnostic efficacy, and to investigate the outcomes of radiographically overlooked patients. MATERIALS AND METHODS: Two hundred fifty-five patients suspected of thoracic spine fractures were enrolled. We assessed the diagnostic efficacy of radiography for the patients sub-grouped based on five confounding factors: chest abnormalities, head injuries, cervical spine fractures, upper extremity injuries, and age of 65 years or older. We investigated the outcomes of radiographically overlooked patients. RESULTS: Three hundred fifty-one vertebral body fractures were detected. The per-fracture sensitivities and specificities were 55 % and 94 % for vertebral body fractures and 41 and 99 % for unstable fractures. In patients with upper extremity injuries or aged 65 years or older, radiography was less sensitive in detecting the unstable fractures (P < 0.05). Nineteen patients were overlooked by radiography; two had neurological deficits and needed surgical fixation; 15 with no neurological deficit were conservatively treated with uneventful outcomes. CONCLUSION: Radiography had low sensitivity but high specificity. In daily practice, primary use of MDCT is beneficial for patients with neurological deficit or upper extremity injuries or elderly patients.

Satisfaction of search for subtle skeletal fractures may not be induced by more serious skeletal injury.

PURPOSE: The aim of this experiment was to test whether radiographs of major injuries, those having serious consequences for life and limb, produce a satisfaction-of-search (SOS) effect on the detection of subtle, nondisplaced test fractures. METHODS: Institutional review board approval and informed consent from 24 participants were obtained. Seventy simulated patients with multiple trauma injuries were constructed from radiographs of 3 different anatomic areas demonstrated only skeletal injuries. Readers evaluated each patient under 2 conditions: first, in the non-SOS condition, no injuries were present in the first anatomic images, and second, in the SOS condition, the first anatomic images included major injuries requiring immediate medical intervention. The SOS effect was measured on detection accuracy using receiver operating characteristic analysis for subtle test fractures presented on examinations of the second or third anatomic areas. RESULTS: Satisfaction-of-search reduction in receiver operating characteristic experiments for detecting subtle test fractures with the addition of a major injury was not observed. CONCLUSIONS: Satisfaction of search was absent when major injuries were presented on radiographs. This finding rejects the hypothesis that SOS arises primarily from injuries requiring major intervention. Similar results have been found previously when major injuries were presented on CT but test fractures were presented on radiographs. This new finding rejects the possibility that SOS is absent because added and test fractures appear on different imaging modalities.

Do long radiology workdays affect nodule detection in dynamic CT interpretation?

PURPOSE: A previous study demonstrated decreased diagnostic accuracy for finding fractures and decreased ability to focus on skeletal radiographs after a long working day. Skeletal radiographic examinations commonly have images that are displayed statically. The aim of this study was to investigate whether diagnostic accuracy for detecting pulmonary nodules on CT of the chest displayed dynamically would be similarly affected by fatigue. METHODS: Twenty-two radiologists and 22 residents were given 2 tests searching CT chest sequences for a solitary pulmonary nodule before and after a day of clinical reading. To measure search time, 10 lung CT sequences, each containing 20 consecutive sections and a single nodule, were inspected using free search and navigation. To measure diagnostic accuracy, 100 CT sequences, each with 20 sections and half with nodules, were displayed at preset scrolling speed and duration. Accuracy was measured using receiver operating characteristic curve analysis. Visual strain was measured via dark vergence, an indicator of the ability to keep the eyes focused on the display. RESULTS: Diagnostic accuracy was reduced after a day of clinical reading (P = .0246), but search time was not affected (P > .05). After a day of reading, dark vergence was significantly larger and more variable (P = .0098), reflecting higher levels of visual strain, and subjective ratings of fatigue were also higher. CONCLUSIONS: After their usual workday, radiologists experience increased fatigue and decreased diagnostic accuracy for detecting pulmonary nodules on CT. Effects of fatigue may be mitigated by active interaction with the display.

Synthetic positron emission tomography-computed tomography images for use in perceptual studies.

To better understand fundamental issues, perception studies of the fusion display would best be performed with a panel of lesions of variable location, size, intensity, and background. There are compelling reasons to use synthetic images that contain artificial lesions for perception research. A consideration of how to obtain this panel of lesions is the nucleus of the present review. This article is a conjoint effort of 3 groups that have joined together to review results from work that they and others have performed. The techniques we review include (1) substitution of lesions into a preexisting image matrix (either using actual prior patient-derived lesions or mathematically modeled artificial lesions), (2) addition of images (either in the attenuation-corrected image space or at an earlier stage before image reconstruction), and (3) simulation of the entire patient image. A judicious combination of the techniques discussed in this review may represent the most efficient pathway of simulating statistically varied but realistic appearing lesions.

Power estimation for multireader ROC methods an updated and unified approach.

RATIONALE AND OBJECTIVES: We describe a step-by-step procedure for estimating power and sample size for planned multireader receiver operating characteristic (ROC) studies that will be analyzed using either the Dorfman-Berbaum-Metz (DBM) or Obuchowski-Rockette (OR) method. This procedure updates previous approaches by incorporating recent methodological developments and unifies the approaches by allowing inputs to be conjectured parameter values or outputs from either a DBM or OR pilot-study analysis. MATERIALS AND METHODS: Power computations are described in a step-by-step procedure and the theoretical basis for the procedure is described. Updates include using the currently recommended denominator degrees of freedom, accounting for different pilot and planned study normal-to-abnormal case ratios, and a new method for computing the OR test-by-reader variance component. RESULTS: Using a real dataset we illustrate how to compute the power for two planned studies, one having the same normal-to-abnormal case ratio as the pilot study and the other having a different ratio. In a simulation study, we show that the proposed procedure gives mean power estimates close to the true power. CONCLUSIONS: Application of the updated procedure is straightforward. It is important that pilot data be comparable to the planned study with respect to the modalities, reader expertise, and case selection. Variability of the power estimates warrants further investigation.

Using the mean-to-sigma ratio as a measure of the improperness of binormal ROC curves.

RATIONALE AND OBJECTIVES: A basic assumption for a meaningful diagnostic decision variable is that there is a monotone relationship between the decision variable and the likelihood of disease. This relationship, however, generally does not hold for the binormal model. As a result, receiver operating characteristic (ROC)-curve estimation based on the binormal model produces improper ROC curves that are not concave over the entire domain and cross the chance line. Although in practice the "improperness" is typically not noticeable, there are situations where the improperness is evident. Presently, standard statistical software does not provide diagnostics for assessing the magnitude of the improperness. MATERIALS AND METHODS: We show how the mean-to-sigma ratio can be a useful, easy-to-understand and easy-to-use measure for assessing the magnitude of the improperness of a binormal ROC curve by showing how it is related to the chance-line crossing. We suggest an improperness criterion based on the mean-to-sigma ratio. RESULTS: Using a real-data example, we illustrate how the mean-to-sigma ratio can be used to assess the improperness of binormal ROC curves, compare the binormal method with an alternative proper method, and describe uncertainty in a fitted ROC curve with respect to improperness. CONCLUSIONS: By providing a quantitative and easily computable improperness measure, the mean-to-sigma ratio provides an easy way to identify improper binormal ROC curves and facilitates comparison of analysis strategies according to improperness categories in simulation and real-data studies.

Long radiology workdays reduce detection and accommodation accuracy.

PURPOSE: The aim of this study was to measure the diagnostic accuracy of fracture detection, visual accommodation, reading time, and subjective ratings of fatigue and visual strain before and after a day of clinical reading. METHODS: Forty attending radiologists and radiology residents viewed 60 deidentified, HIPAA-compliant bone examinations, half with fractures, once before any clinical reading (early) and once after a day of clinical reading (late). Reading time was recorded. Visual accommodation (the ability to maintain focus) was measured before and after each reading session. Subjective ratings of symptoms of fatigue and oculomotor strain were collected. The study was approved by local institutional review boards. RESULTS: Diagnostic accuracy was reduced significantly after a day of clinical reading, with average areas under the receiver operating characteristic curves of 0.885 for early reading and 0.852 for late reading (P < .05). After a day of image interpretation, visual accommodation was no more variable, though error in visual accommodation was greater (P < .01), and subjective ratings of fatigue were higher. CONCLUSIONS: After a day of clinical reading, radiologists have reduced ability to focus, increased symptoms of fatigue and oculomotor strain, and reduced ability to detect fractures. Radiologists need to be aware of the effects of fatigue on diagnostic accuracy and take steps to mitigate these effects.

On the convexity of ROC curves estimated from radiological test results.

RATIONALE AND OBJECTIVES: Although an ideal observer's receiver operating characteristic (ROC) curve must be convex-ie, its slope must decrease monotonically-published fits to empirical data often display "hooks." Such fits sometimes are accepted on the basis of an argument that experiments are done with real, rather than ideal, observers. However, the fact that ideal observers must produce convex curves does not imply that convex curves describe only ideal observers. This article aims to identify the practical implications of nonconvex ROC curves and the conditions that can lead to empirical or fitted ROC curves that are not convex. MATERIALS AND METHODS: This article views nonconvex ROC curves from historical, theoretical, and statistical perspectives, which we describe briefly. We then consider population ROC curves with various shapes and analyze the types of medical decisions that they imply. Finally, we describe how sampling variability and curve-fitting algorithms can produce ROC curve estimates that include hooks. RESULTS: We show that hooks in population ROC curves imply the use of an irrational decision strategy, even when the curve does not cross the chance line, and therefore usually are untenable in medical settings. Moreover, we sketch a simple approach to improve any nonconvex ROC curve by adding statistical variation to the decision process. Finally, we sketch how to test whether hooks present in ROC data are likely to have been caused by chance alone and how some hooked ROCs found in the literature can be easily explained as fitting artifacts or modeling issues. CONCLUSION: In general, ROC curve fits that show hooks should be looked on with suspicion unless other arguments justify their presence.

Measurement of visual strain in radiologists.

RATIONALE AND OBJECTIVES: The authors hypothesized that the current practice of radiology produces oculomotor fatigue that reduces diagnostic accuracy. MATERIALS AND METHODS: Testing this hypothesis required an ability to measure eyestrain. This capability was developed by measuring the visual accommodation of radiologists before and after diagnostic viewing work using an autorefractor that was capable of making multiple measurements of accommodation per second. Three radiologists and three residents focused on a simple target placed at near to far distances while accommodation was measured. The target distances varied from 20 to 183 cm from the eye. The data were collected prior to and after a day of digital diagnostic viewing. RESULTS: The results indicated that accommodation at near distances was significantly worse overall compared to far distances and was significantly worse after a day of digital reading at all distances. CONCLUSIONS: Because diagnostic image interpretation is performed at near viewing distances, this inability to maintain focus on an image could affect diagnostic accuracy. As expected, younger residents had better accommodative accuracy than older radiologists.

Large-core breast biopsy: abnormal salivary cortisol profiles associated with uncertainty of diagnosis.

PURPOSE: To determine whether uncertainty of the diagnosis after large-core breast biopsy (LCBB) adversely affects biochemical stress levels. MATERIALS AND METHODS: This study was institutional review board approved and HIPAA compliant, and all patients gave written informed consent. One hundred fifty women aged 18-86 years collected four salivary cortisol samples per day for 5 days after LCBB. t Tests were used to compare diurnal cortisol slopes among three groups: patients who did not have a final diagnosis (uncertain group), patients who knew they had cancer (known malignant group), and patients who knew they had benign disease (known benign group). RESULTS: Women learned their diagnosis on days 1-6 (mean, day 2.4) after LCBB. Analysis was truncated at day 5, when the data from a sufficient number of patients from each group were available for meaningful analysis: 16 patients from the known malignant group, 37 from the known benign group, and 73 from the uncertain group, which totaled 126 patients. The mean cortisol slope for the women with an uncertain diagnosis (-0.092 ln [microg/dL]/hr; 95% confidence interval [CI]: -0.113 ln [microg/dL]/hr, -0.072 ln [microg/dL]/hr) was significantly flatter (less desirable) than that for the women who learned that they had benign disease (-0.154 ln [microg/dL]/hr; 95% CI: -0.197 ln [microg/dL]/hr, -0.111 ln [microg/dL]/hr; P = .014) but not significantly different from that for the women who learned that they had malignant disease (-0.110 ln [microg/dL]/hr; 95% CI: -0.147 ln [microg/dL]/hr, -0.073 ln [microg/dL]/hr; P = .421). CONCLUSION: Uncertainty about the final diagnosis after LCBB is associated with substantial biochemical distress, which may have adverse effects on immune defense and wound healing. Results indicate the need for more rapid communication of biopsy results.

Beneficial effects of hypnosis and adverse effects of empathic attention during percutaneous tumor treatment: when being nice does not suffice.

PURPOSE: To determine how hypnosis and empathic attention during percutaneous tumor treatments affect pain, anxiety, drug use, and adverse events. MATERIALS AND METHODS: For their tumor embolization or radiofrequency ablation, 201 patients were randomized to receive standard care, empathic attention with defined behaviors displayed by an additional provider, or self-hypnotic relaxation including the defined empathic attention behaviors. All had local anesthesia and access to intravenous medication. Main outcome measures were pain and anxiety assessed every 15 minutes by patient self-report, medication use (with 50 mug fentanyl or 1 mg midazolam counted as one unit), and adverse events, defined as occurrences requiring extra medical attention, including systolic blood pressure fluctuations (> or =50 mm Hg change to >180 mm Hg or <105 mm Hg), vasovagal episodes, cardiac events, and respiratory impairment. RESULTS: Patients treated with hypnosis experienced significantly less pain and anxiety than those in the standard care and empathy groups at several time intervals and received significantly fewer median drug units (mean, 2.0; interquartile range [IQR], 1-4) than patients in the standard (mean, 3.0; IQR, 1.5-5.0; P = .0147) and empathy groups (mean, 3.50; IQR, 2.0-5.9; P = .0026). Thirty-one of 65 patients (48%) in the empathy group had adverse events, which was significantly more than in the hypnosis group (eight of 66; 12%; P = .0001) and standard care group (18 of 70; 26%; P = .0118). CONCLUSIONS: Procedural hypnosis including empathic attention reduces pain, anxiety, and medication use. Conversely, empathic approaches without hypnosis that provide an external focus of attention and do not enhance patients' self-coping can result in more adverse events. These findings should have major implications in the education of procedural personnel.

Recent developments in the Dorfman-Berbaum-Metz procedure for multireader ROC study analysis.

RATIONALE AND OBJECTIVES: The Dorfman-Berbaum-Metz (DBM) method has been one of the most popular methods for analyzing multireader receiver-operating characteristic (ROC) studies since it was proposed in 1992. Despite its popularity, the original procedure has several drawbacks: it is limited to jackknife accuracy estimates, it is substantially conservative, and it is not based on a satisfactory conceptual or theoretical model. Recently, solutions to these problems have been presented in three papers. Our purpose is to summarize and provide an overview of these recent developments. MATERIALS AND METHODS: We present and discuss the recently proposed solutions for the various drawbacks of the original DBM method. RESULTS: We compare the solutions in a simulation study and find that they result in improved performance for the DBM procedure. We also compare the solutions using two real data studies and find that the modified DBM procedure that incorporates these solutions yields more significant results and clearer interpretations of the variance component parameters than the original DBM procedure. CONCLUSIONS: We recommend using the modified DBM procedure that incorporates the recent developments.