SPIE Medical Imaging 50th anniversary: history of the Image Perception, Observer Performance, and Technology Assessment Conference.

As SPIE Medical Imaging celebrates its 50th anniversary, we reflect on the history of the Image Perception, Observer Performance, and Technology Assessment Conference and its importance within the SPIE Medical Imaging Symposium and the greater medical imaging community.

Using gaze-tracking data and mixture distribution analysis to support a holistic model for the detection of cancers on mammograms.

RATIONALE AND OBJECTIVES: Use data collected independently at three institutions to compare time to first fixate the true lesion in searching for cancers on mammograms. Examine the fit of the results to a holistic model of visual perception. MATERIALS AND METHODS: The time required to first fixate a cancer on a mammogram was extracted from 400 eye-tracking records collected independently from three institutions. The time was used as an indicator of the initial perception of cancer. The distribution of first fixation times was partitioned into two normally distributed components using mixture distribution analysis. The true-positive fraction of each component was calculated. RESULTS: About 57% of the cancers had a 95% chance of being fixated in the first second of viewing. The remainder took longer (range, 1.0 to 15.2 seconds). The true-positive fraction was larger for the lesions hit immediately for most of the readers (TPF = 0.63 vs. 0.52, F = 5.88, P = .02) in 68% (13/19) of the readers. CONCLUSIONS: The initial detection occurs before visual scanning and, therefore, must be the result of a parallel "global" analysis of the image resulting in an initial holistic, gestalt-like perception. The development of expertise in medical image analysis may consist of a shift in the recognition mechanism from scan-look-detect to look-detect-scan.

Holistic component of image perception in mammogram interpretation: gaze-tracking study.

PURPOSE: To test the hypothesis that rapid and accurate performance of the proficient observer in mammogram interpretation involves a shift in the mechanism of image perception from a relatively slow search-to-find mode to a relatively fast holistic mode. MATERIALS AND METHODS: This HIPAA-compliant study had institutional review board approval, and participant informed consent was obtained; patient informed consent was not required. The eye positions of three full-time mammographers, one attending radiologist, two mammography fellows, and three radiology residents were recorded during the interpretation of 20 normal and 20 subtly abnormal mammograms. The search time required to first locate a cancer, as well as the initial eye scan path, was determined and compared with diagnostic performance as measured with receiver operating characteristic (ROC) analysis. RESULTS: The median time for all observers to fixate a cancer, regardless of the decision outcome, was 1.13 seconds, with a range of 0.68 second to 3.06 seconds. Even though most of the lesions were fixated, recognition of them as cancerous ranged from 85% (17 of 20) to 10% (two of 20), with corresponding areas under the ROC curve of 0.87-0.40. The ROC index of detectability, d(a), was linearly related to the time to first fixate a cancer with a correlation (r(2)) of 0.81. CONCLUSION: The rapid initial fixation of a true abnormality is evidence for a global perceptual process capable of analyzing the visual input of the entire retinal image and pinpointing the spatial location of an abnormality. It appears to be more highly developed in the most proficient observers, replacing the less efficient initial search-to-find strategies.

Pulmonary nodule detection in CT images with quantized convergence index filter.

A novel filter termed quantized convergence index filter (QCI filter) that is capable of enhancing the conspicuity of rounded lesions is proposed as part of a CAD (computer-aided diagnosis) scheme for detecting pulmonary nodules in computed tomography (CT) images. In this filter and its predecessor, the convergence index filter (CI filter), the output at a pixel represents the degree of convergence toward the pixel shown by the directions of gray-level gradients at surrounding pixels. The QCI filter and the CAD scheme were evaluated using five clinical datasets containing 50 nodules. With the support region of 9 x 9 pixels, the QCI filter showed more selective response to the nodules than the CI filter. In the CAD scheme, intermediate nodule candidates are generated based on the QCI filter output and then classified using linear discriminant analysis of eight features that are attributed to each intermediate nodule candidate. The QCI filter output level itself was used as one of the features. The scheme achieved a sensitivity of 90% with 1.67 false positives per slice. The QCI filter output level was most effective among the features in correctly classifying intermediate nodule candidates. The QCI filter is promising as a tool of preprocessing for automated pulmonary nodule detection in CT images.

History of research in medical image perception.

Human observers engage in 2 interrelated processes when interpreting medical images: perception and analysis. Perception is the unified awareness of the content of a displayed image that is present while the stimulus is on. Analysis is determining the meaning of the perception in the context of the medical problem that initiated the acquisition of the image. Radiologists have, correctly, regarded image analysis as their primary field of research. They have naively assumed that what they perceive in images is a faithful representation of the images' information content and have not been concerned with perception unless it fails. Failures have stimulated research on quantifying observer performance, defining image quality, and understanding perceptual error. This article traces the historical development of the use of receiver operating characteristic analysis for describing performance, the development of signal-to-noise ratio psychophysical models for defining task-dependent image quality, studies of error in small lesion detection, and the beginnings of studies of the nature of expertise in image interpretation. The history is traced through published articles.

The place of medical image perception in 21st-century health care.

An image that is not perceived and interpreted can have no positive impact on health care. In this article, the authors review publicly available data and the published literature concerning the unitary event of the perception and interpretation of medical images. Their review shows that this event occurs as frequently as do major medical, public health, and public safety events in the United States; constitutes a significant economic activity; and makes up a significant portion of hospital-based health care in the United States. Yet despite its central importance to the economy and to health care, the authors' analysis found that research in the perception and interpretation of medical images has been awarded minimal support by National Institutes of Health extramural funding: fewer than 5% of all National Institutes of Health-funded grants related to radiology during the 10-year period from 1994 to 2003 focused on human perception and interpretation. The increased funding of medical image perception and interpretation research could lead to important improvements in overall health care thanks to the pervasive and vital role imaging plays in modern medicine.

Perceptually based FROC analysis.

RATIONALE AND OBJECTIVES: Analysis of reading data when cases have multiple targets and/or the reader is required to localize targets is difficult. One approach to this free-response operating characteristic (FROC) problem is for images to be segmented (eg, with quadrants) by the investigator and a segment-level analysis be conducted with the case as a nesting factor. In this report, we introduce an alternative method that uses the visual scan path of the reader to segment the image. We evaluate the new method by applying it to data from a mammography reading experiment. MATERIALS AND METHODS: The gaze scan path of one radiologist was recorded as she scanned 40 mammograms for masses and microcalcifications. The observer is an experienced mammographer and was not one of the authors. In addition, the reader provided a rating indicating the degree of suspicion for any suspected targets she identified and localized. We then established "perceptual regions" by using a clustering algorithm on the visual fixations. We combined ratings given to specific locations indicated by the reader with the segmentation from the visual scan to generate a series of ratings classified for whether the perceptually based region associated with the rating contained or did not contain a known target. We analyzed data generated by our method from all 40 cases by using the conventional maximum-likelihood method based on the binormal model. Finally, we tested goodness-of-fit of the binormal model to the data by using chi-square. RESULTS: Maximum-likelihood estimation led to a model that did not fit the data (P < .001). However, examination of the observed and expected counts suggests that the binormal assumption does not hold for segments that contain targets and a bimodal distribution model might be preferred. CONCLUSION: Our new method provides an alternative approach to analysis of the FROC experiment. It needs to be developed further. Specifically, we propose that a mixture model extension of the binormal model be developed for ratings data arising from perceptually based FROC experiments. A disadvantage to our method is the requirement to record the scan path of the reader. However, we believe that adding such information to receiver operating characteristic (ROC) curve analysis will pay off when appropriate statistical models have been identified because we believe our data support our hypothesis that the perceptual scanning of images by humans deconvolves interpretation correlation. If true, this hypothesis implies that conventional statistical methods for ROC analysis based on independent data can be applied to the analysis of FROC data after conditioning on the scan path of the observer.

Gadodiamide T1 relaxivity in brain tissue in vivo is lower than in saline.

In vivo measurements of gadodiamide (Gd-DTPA-BMA) T(1) relaxivity were performed at 4.7 T in injured and normal rat brains. Cerebral lesions were induced in nine rats by a localized freezing method. T(1) maps of the lesions were generated before and after injection of Gd-DTPA-BMA (0.1-0.6 mmol/kg). Samples of normal and necrotic brain were collected postmortem; the wet and dry weights were determined, and Gd content was measured by inductively coupled plasma mass spectroscopy. The in vivo relaxivity was determined by a linear fit of a plot of the change in relaxation rate following injection of the contrast agent as a function of Gd content. This analysis yielded a relaxivity in the injured brain of 2.8 sec(-1) mmol(-1) kg tissue water at 36 degrees C. The water weight fraction was 0.90 +/- SD 0.02 wt/wt in injured brain and 0.79 +/- 0.02 in normal brain. Relaxivity measurements were also performed on solutions of Gd-DTPA-BMA (0.0-0.6 mmol) and albumin (0-30% wt/wt) in normal saline at room and physiologic temperatures. The relaxivity in the albumin/saline increased with increasing solids content with values of 4.0-4.9 sec(-1) mmol(-1)kg at 21 degrees C and 3.4-4.5 sec(-1) mmol(-1) kg at 37 degrees C. The relaxivity of the tissues differed significantly from that of the saline solutions of comparable solids content, suggesting that the solids content of a tissue is not the only factor that determines in vivo relaxivity.

A novel method for analysis of TOMROP data.

PURPOSE: To develop an efficient method for extracting maps of the corrected T1 from images generated using the T One by Multiple Read Out Pulses (TOMROP) sequence. MATERIALS AND METHODS: An expression is developed for the true T1 in terms of the parameters from a three-parameter fit of the TOMROP data. Solutions of gadodiamide in normal saline with concentrations of 0.0, 0.06, 0.11, 0.23, 0.46, and 0.91 mM were prepared and T1 measurements were performed using both the inversion recovery (IR) and the TOMROP methods. The TOMROP data were analyzed using the proposed technique and the results compared to those from the IR measurements. RESULTS: The T1 estimates generated from the TOMROP data using the proposed method were consistent with the IR results. However, systematic errors were observed in the T1 estimates when the repetition time was not sufficient for full recovery of axial magnetization. Relaxation times determined using the proposed method were within 1% of the spectroscopiclly determined values for T1 values in the range of 0.28-2.8 seconds when a suitable delay was employed. CONCLUSION: The proposed method of analysis was found to yield accurate T1 estimates when the assumptions used in the analysis were not violated.

Mixture distribution analysis of a computer assisted diagnostic method for the evaluation of pulmonary nodules on computed tomography scan.

RATIONALE AND OBJECTIVES: To compare the effectiveness of a new computational scheme for pulmonary nodule detection in computed tomography images against human observers. MATERIALS AND METHODS: The study involved evaluation of 81 potential nodules by four radiologists. Each radiologist separately evaluated the potential nodules and provided a confidence level for the presence of pulmonary nodules. Their performance was compared with that of the new computational scheme by mixture distribution analysis. RESULTS: Mixture distribution analysis of the results of the four radiologists demonstrated a relative proportion agreement of 0.84. The kappa statistic was used to compare the agreement of the computational scheme with the results of the four radiologists. A kappa value of .65 (se = .11) was shown to be significantly different from chance (P = .99). CONCLUSION: The new computational scheme correlates well with the radiologists' subjective rankings of pulmonary nodules on computed tomography scans and may prove a useful tool in the evaluation of algorithms for the screening and diagnosis of lung cancer.

The perception of breast cancers--a spatial frequency analysis of what differentiates missed from reported cancers.

The primary detector of breast cancer is the human eye. Radiologists read mammograms by mapping exogenous and endogenous factors, which are based on the image and observer, respectively, into observer-based decisions. These decisions rely on an internal schema that contains a representation of possible malignant and benign findings. Thus, to understand the hits and misses made by the radiologists, it is important to model the interactions between the measurable image-based elements contained in the mammogram and the decisions made. The image-based elements can be of two types, i.e., areas that attracted the visual attention of the radiologist, but did not yield a report, and areas where the radiologist indicated the presence of an abnormal finding. In this way, overt and covert decisions are made when reading a mammogram. In order to model this decision-making process, we use a system that is based upon the processing done by the human visual system, which decomposes the areas under scrutiny in elements of different sizes and orientations. In our system, this decomposition is done using wavelet packets (WPs). Nonlinear features are then extracted from the WP coefficients, and an artificial neural network is trained to recognize the patterns of decisions made by each radiologist. Afterwards, the system is used to predict how the radiologist will respond to visually selected areas in new mammogram cases.

Measurement of observer agreement.

Statistical measures are described that are used in diagnostic imaging for expressing observer agreement in regard to categorical data. The measures are used to characterize the reliability of imaging methods and the reproducibility of disease classifications and, occasionally with great care, as the surrogate for accuracy. The review concentrates on the chance-corrected indices, kappa and weighted kappa. Examples from the imaging literature illustrate the method of calculation and the effects of both disease prevalence and the number of rating categories. Other measures of agreement that are used less frequently, including multiple-rater kappa, are referenced and described briefly.

The perception of breast cancer: what differentiates missed from reported cancers in mammography?

RATIONALE AND OBJECTIVES: Mammographers map endogenous and exogenous factors into decisions whether to report the presence of a malignant finding in a mammogram case. Thus, to understand how image-based elements are translated into observer-based decisions, the authors used spatial frequency analysis to model the areas on mammograms that attracted visual attention, in addition to the areas localized as abnormal. MATERIALS AND METHODS: Four mammographers read 40 two-view mammogram cases, of which 30 contained at least one malignant lesion visible on one or two views. Their eye positions were recorded during visual search. Once the mammographer felt confident enough to provide an initial impression of the case ("normal" or "abnormal"), the eye position monitoring was turned off and the mammographer indicated, with a mouse-controlled cursor, the location and nature of any malignant findings. Regions that elicited an overt or a covert response by the mammographers were extracted for processing by means of wavelet packets and artificial neural networks. RESULTS: Different decision outcomes yielded different energy representations, in the spatial frequency domain. These energy representations were used by an artificial neural network to predict decision outcome in areas of interest, derived from eye position analysis, on mammograms from new cases. Individual trends were observed for each mammographer. CONCLUSION: Spatial frequency representation of regions that attracted a given mammographer's visual attention may be useful for characterizing how that mammographer will respond to the visually selected areas.

Time course of perception and decision making during mammographic interpretation.

OBJECTIVE: This article describes the time course of lesion detection on digital mammograms using data about both eye position and decision time to compare performance between experienced mammographers and trainees. Research indicates that a longer decision time works against performance in the interpretation of chest radiographs because the likelihood of error is increased, particularly for trainees. Is this relation between decision time and performance also true for interpreting mammograms? Is there an optimal decision time-performance trade-off for detecting breast lesions? MATERIALS AND METHODS: Six radiology trainees (experience, 302-976 cases) and three mammographers (experience, 3000-5000 cases per year) reviewed 40 test cases. Each test case was represented by two mammograms that showed different views of the same breast. Twenty breasts contained suspicious lesions, and 20 were lesion-free. An interactive computer display system with an eye-head tracker measured the timing of decisions, where visual attention was directed, and how much time was spent fixating on a region of interest for each decision. Eye position was monitored during an initial-decision phase, and decision times were measured throughout a final-decision phase during which suspicious lesions recognized initially were interpreted and localized. Performance was analyzed using localization receiver operating characteristic curves. RESULTS: The time course of interpreting mammograms is similar to that for interpreting chest radiographs. Mammographers detected 71% of the true lesions within 25 sec, and trainees detected 46% within 40 sec. Both a fixation dwell time of 1000 msec and a high level of confidence in the decision were associated with the detection of true lesions for the mammographers but not for the trainees. CONCLUSION: Mammographers detected most breast lesions by global recognition within 25 sec, but trainees took more time. Prolonging one's search beyond the global recognition phase yielded few new lesions and increased the risk of error.

Radiology services in emergency medicine residency programs: a national survey.

OBJECTIVES: To determine who reads plain film radiographs, how quickly radiologists' interpretations are available, how many initial readings require correction, and how satisfied emergency physicians (EPs) are with radiology in emergency departments (EDs) with emergency medicine (EM) residency programs. METHODS: A questionnaire was sent to the chairs of all U.S. EM residencies, asking about EM radiology services. RESULTS: Of 120 sites surveyed, 97 (81%) responded. Respondents reported that, on weekday days, EM attendings or residents performed the radiograph interpretation used for clinical decision making at 66% of sites; on nights and weekends, EPs performed the clinically relevant readings at 79% of sites. Twenty-one percent of sites reported that no radiologist reviewed images before patients left the ED on nights and weekends. Only 39% of sites reported that all images were read within four hours on weekday days, and only 19% of sites reported readings within this time frame on nights and weekends. Median misinterpretation rates were reported as 1% on weekday days and 1.5% at other times. Overall, EPs were satisfied with their interactions with radiology at 63% of EDs. CONCLUSIONS: This study summarizes the perceptions of EPs regarding radiology services; the findings must be interpreted with caution, given the lack of external validation. Nevertheless, EPs report that many EM residency programs depend on EPs' interpretations of radiographs. Emergency physicians report that attending radiologists rarely read images on nights and weekends and that images are misread more frequently at these times. Although EPs were satisfied with many aspects of radiology, EPs expressed the most dissatisfaction with turnaround times and misreads.