Prostate cancer: relative effects of demographic, clinical, histologic, and MR imaging variables on the accuracy of staging.

PURPOSE: To determine the effects on the accuracy of staging prostate gland cancer of diagnostic prediction rules based on demographic, clinical, histologic, and magnetic resonance (MR) image variables. MATERIALS AND METHODS: A total of 200 cases from four medical centers were evaluated by nine radiologists experienced in MR imaging. The accuracies of the four diagnostic variables (age, prostate specific antigen level, Gleason tumor grade, and MR imaging findings) were measured, both singly and combined in a particular sequence, by calculating the area index of the receiver operating characteristic curve. RESULTS: The accuracy of staging with single variables (age, 0.58; prostate specific antigen level, 0.74; Gleason grade 0.73, MR image findings, 0.74) increased as the variables were optimally merged. The first two variables combined to yield an accuracy of 0.74; the first three combined to yield an accuracy of 0.81; and all four variables resulted in an accuracy of 0.86. In a clinically important subset of 69 cases for which antigen level and Gleason grade together were inconclusive for the purposes of staging, the addition of MR imaging findings resulted in an increase in accuracy from 0.55 to 0.73. CONCLUSION: Optimal merging of diagnostic test results yields an improvement in the accuracy of prostate cancer staging.

Staging prostate cancer with MR imaging: a combined radiologist-computer system.

PURPOSE: To test the accuracy of a combined radiologist-computer system in the diagnosis with magnetic resonance (MR) imaging of cancer of the prostate gland. MATERIALS AND METHODS: The combined system was developed and tested by four specialists in prostate MR imaging and five radiologists expert in body MR imaging. Each group read MR images obtained in 100 proved cases of prostate cancer. The images were obtained from two sources, and all were obtained with an endorectal surface coil. Prostate MR specialists ranked imaging features of cases to develop a checklist for image interpretation. Features with greatest diagnostic value were incorporated in the combined system. Accuracy measures were derived from the area index of the receiver operating characteristic curve for the combined system and compared with those of radiologists working alone. RESULTS: Body MR radiologists had a mean baseline accuracy of 0.67; mean accuracy of their combined system was 0.80. The prostate MR specialists, when they rated the features in each case, had a mean accuracy of 0.81; the accuracy of their combined system was 0.87. CONCLUSIONS: A combined radiologist-computer system substantially improved accuracy of body MR radiologists in the diagnosis of prostate cancer. High levels of accuracy were also achieved by the system with prostate MR specialists.

Combining evidence from multiple imaging modalities: a feature-analysis method.

This study was designed to develop methods to improve radiologists' ability to detect and diagnose breast cancer. We evaluated the ability of a feature-analysis method to help radiologists merge judgements constructively from two rather disparate breast imaging tests. To accomplish these goals, we developed a list of perceptual features and quantitated the importance of each in the diagnosis of patients having both diaphanography (Test 1) and mammography (Test 2). Then, two decision aids were developed: One was a checklist of the critical diagnostic visual features from both tests that also assisted readers in rating these features numerically. The second was a computer-based classifier that assisted readers in merging the assessments of the two tests into one overall diagnostic probability. The value of these aids was assessed by comparing radiologists' accuracy in reading a set of proven cases in their standard fashion with their accuracy when reading in an enhanced mode, utilizing the checklist and computer classifier. When Test 1 was read adjunctively with Test 2, use of the decision aids led to a significant improvement in accuracy (p = .013) over the unenhanced, combined readings. For Test 1 alone, the aids led to a significant improvement over its low level of unenhanced reading (p = .046). For Test 2 alone, the enhancements provided little gain in accuracy over an already high level of performance on the full case set (p = .081), although significant gains were realized on the most difficult ones. We conclude that methods to aid standardization and merging of feature-based judgements can improve radiologists performance on complex diagnostic tasks.

Reading and decision aids for improved accuracy and standardization of mammographic diagnosis.

Image-reading and decision aids were designed to improve the accuracy of mammogram interpretation. The reading aid was a list of diagnostic radiographic features and scales for quantification of each feature. The decision aid, a computer program, converted the reader's scaled values, weighted for predictive power, into an advisory estimate of the probability of malignancy. The features were identified and their importance was assigned in four steps: (a) interviews of five expert readers to establish an initial set of features, (b) perceptual tests to refine the feature set, (c) a consensus meeting to refine this set and establish nomenclature and scales, and (d) the expert's scaling of each feature in a set of 150 mammograms. Those scaled judgments were analyzed to provide the final list of features and their relative importance and to program the computer decision aid. To test the enhancement effect, six other radiologists interpreted a different set of mammograms without, and later with, the two aids. Receiver operating characteristic analysis showed a gain of approximately 0.05 in sensitivity or specificity when the other value remained at 0.85. In a subset of the more difficult cases, the enhancement effect was approximately 0.15 in either sensitivity or specificity.

Enhancing and evaluating diagnostic accuracy.

Techniques that may enhance diagnostic accuracy in clinical settings were tested in the context of mammography. Statistical information about the relevant features among those visible in a mammogram and about their relative importances in the diagnosis of breast cancer was the basis of two decision aids for radiologists: a checklist that guides the radiologist in assigning a scale value to each significant feature of the images of a particular case, and a computer program that merges those scale values optimally to estimate a probability of malignancy. A test set of approximately 150 proven cases (including normals and benign and malignant lesions) was interpreted by six radiologists, first in their usual manner and later with the decision aids. The enhancing effect of these feature-analytic techniques was analyzed across subsets of cases that were restricted progressively to more and more difficult cases, where difficulty was defined in terms of the radiologists' judgements in the standard reading condition. Accuracy in both standard and enhanced conditions decreased regularly and substantially as case difficulty increased, but differentially, such that the enhancement effect grew regularly and substantially. For the most difficult case sets, the observed increases in accuracy translated into an increase of about 0.15 in sensitivity (true-positive proportion) for a selected specificity (true-negative proportion) of 0.85 or a similar increase in specificity for a selected sensitivity of 0.85. That measured accuracy can depend on case-set difficulty to different degrees for two diagnostic approaches has general implications for evaluation in clinical medicine. Comparative, as well as absolute, assessments of diagnostic performances--for example, of alternative imaging techniques--may be distorted by inadequate treatments of this experimental variable. Subset analysis, as defined and illustrated here, can be useful in alleviating the problem.

An enriched set of features of nuclear cataract identified by multidimensional scaling.

Development of an improved system for visual classification of cataracts requires a three-step procedure: first, to identify the full range of visible features of cataracts; second, to develop and test scales for the visual assessment of each feature; and third, to establish the epidemiological or clinical validity of each scale for cataract classification. This paper focuses on the first step, applying a powerful psychometric technique for identifying the visible features of nuclear cataracts. New visual features of nuclear cataract were identified using the psychometric procedure of multidimensional scaling (MDS). Each of 5 observers independently examined pairings of slitlamp photographs of 24 cases of pure nuclear cataract, making two different ratings of dissimilarity of each of the 276 possible pairs. The two dissimilarity ratings were, first, of nuclear color and, second, of nuclear structure. MDS analysis of the dissimilarity ratings of nuclear color revealed two major visual features underlying the judgments: one a combination of hue and saturation, and the other brightness. Analysis of the ratings of nuclear structure identified a total of nine features: one distinguishing between immature and mature cataracts, four describing features of the immature cataracts (aspect ratio, background haze, clarity of the embryonal nucleus, and clarity of the outer nuclear shell), and four describing features of the mature cataracts (opalescence, aspect ratio, color of the nucleus, and symmetry). We conclude that there are many more systematic distinctions to be made in the appearance of nuclear cataracts than are now recognized in clinical practice.

Enhanced interpretation of diagnostic images.

In radiology, as in various other fields, observers study images to detect and diagnose underlying conditions. They make assessments of several image features and merge them into an overall decision. Demonstration is given here, in the context of mammography, that objective aids to this interpretative process can substantially improve accuracy, even for sophisticated and motivated radiologists. The aids are a checklist that solicits explicit, quantitative, systematic assessments of the important features of an image and a computer program that merges those assessments with optimal weights. The computer issues estimates of the likelihoods that specified conditions are present (in this study, the likelihood that a localized abnormality is malignant), and the radiologist benefits from taking those estimates as guidance.

Assessment of diagnostic technologies.

A general protocol for rigorous evaluation of diagnostic systems in medicine was applied successfully in a comparative study of two radiologic techniques. Accuracies of computed tomography and radionuclide scanning in detecting, localizing, and diagnosing brain lesions were assessed with a sample of patients in whom tumor had been suspected. The principal means of analysis was the "relative operating characteristic," which is unique in providing a measure of accuracy that is largely independent of decision biases. Computed tomography was found to be substantially more accurate than radionuclide scanning.