A review of the PERFORMS scheme in breast screening.

This review details the aetiology of the PERFORMS self-assessment scheme in breast screening, together with its subsequent development, current implementation and future function. The purpose of the scheme is examined and the importance of its continuing role in a changing screening service described, together with current evolution.

Do prevalence expectations affect patterns of visual search and decision-making in interpreting CT colonography endoluminal videos?

OBJECTIVE: To assess the effect of expected abnormality prevalence on visual search and decision-making in CT colonography (CTC). METHODS: 13 radiologists interpreted endoluminal CTC fly-throughs of the same group of 10 patient cases, 3 times each. Abnormality prevalence was fixed (50%), but readers were told, before viewing each group, that prevalence was either 20%, 50% or 80% in the population from which cases were drawn. Infrared visual search recording was used. Readers indicated seeing a polyp by clicking a mouse. Multilevel modelling quantified the effect of expected prevalence on outcomes. RESULTS: Differences between expected prevalence were not statistically significant for time to first pursuit of the polyp (median 0.5 s, each prevalence), pursuit rate when no polyp was on screen (median 2.7 s(-1), each prevalence) or number of mouse clicks [mean 0.75/video (20% prevalence), 0.93 (50%), 0.97 (80%)]. There was weak evidence of increased tendency to look outside the central screen area at 80% prevalence and reduction in positive polyp identifications at 20% prevalence. CONCLUSION: This study did not find a large effect of prevalence information on most visual search metrics or polyp identification in CTC. Further research is required to quantify effects at lower prevalence and in relation to secondary outcome measures. ADVANCES IN KNOWLEDGE: Prevalence effects in evaluating CTC have not previously been assessed. In this study, providing expected prevalence information did not have a large effect on diagnostic decisions or patterns of visual search.

The use of lower resolution viewing devices for mammographic interpretation: implications for education and training.

AIMS: To establish whether lower resolution, lower cost viewing devices have the potential to deliver mammographic interpretation training. METHODS: On three occasions over eight months, fourteen consultant radiologists and reporting radiographers read forty challenging digital mammography screening cases on three different displays: a digital mammography workstation, a standard LCD monitor, and a smartphone. Standard image manipulation software was available for use on all three devices. Receiver operating characteristic (ROC) analysis and ANOVA (Analysis of Variance) were used to determine the significance of differences in performance between the viewing devices with/without the application of image manipulation software. The effect of reader's experience was also assessed. RESULTS: Performance was significantly higher (p < .05) on the mammography workstation compared to the other two viewing devices. When image manipulation software was applied to images viewed on the standard LCD monitor, performance improved to mirror levels seen on the mammography workstation with no significant difference between the two. Image interpretation on the smartphone was uniformly poor. Film reader experience had no significant effect on performance across all three viewing devices. CONCLUSION: Lower resolution standard LCD monitors combined with appropriate image manipulation software are capable of displaying mammographic pathology, and are potentially suitable for delivering mammographic interpretation training. KEY POINTS: • This study investigates potential devices for training in mammography interpretation. • Lower resolution standard LCD monitors are potentially suitable for mammographic interpretation training. • The effect of image manipulation tools on mammography workstation viewing is insignificant. • Reader experience had no significant effect on performance in all viewing devices. • Smart phones are not suitable for displaying mammograms.

The effect of computer-aided detection markers on visual search and reader performance during concurrent reading of CT colonography.

OBJECTIVE: We aimed to identify the effect of computer-aided detection (CAD) on visual search and performance in CT Colonography (CTC) of inexperienced and experienced readers. METHODS: Fifteen endoluminal CTC examinations were recorded, each with one polyp, and two videos were generated, one with and one without a CAD mark. Forty-two readers (17 experienced, 25 inexperienced) interpreted the videos during infrared visual search recording. CAD markers and polyps were treated as regions of interest in data processing. This multi-reader, multi-case study was analysed using multilevel modelling. RESULTS: CAD drew readers' attention to polyps faster, accelerating identification times: median 'time to first pursuit' was 0.48 s (IQR 0.27 to 0.87 s) with CAD, versus 0.58 s (IQR 0.35 to 1.06 s) without. For inexperienced readers, CAD also held visual attention for longer. All visual search metrics used to assess visual gaze behaviour demonstrated statistically significant differences when "with" and "without" CAD were compared. A significant increase in the number of correct polyp identifications across all readers was seen with CAD (74 % without CAD, 87 % with CAD; p < 0.001). CONCLUSIONS: CAD significantly alters visual search and polyp identification in readers viewing three-dimensional endoluminal CTC. For polyp and CAD marker pursuit times, CAD generally exerted a larger effect on inexperienced readers. KEY POINTS: • Visual gaze is attracted by computer-assisted detection (CAD) marks on polyps • Inexperienced readers' gaze is affected more by CAD than experienced readers. • CAD marks could mean that the unannotated endoluminal surface is relatively neglected. • Correct polyp identification is increased significantly by CAD.

Retrospective review of the drop in observer detection performance over time in lesion-enriched experimental studies.

The vigilance decrement describes a decrease in sensitivity or increase in specificity with time on task. It has been observed in a variety of repetitive visual tasks, but little is known about these patterns in radiologists. We investigated whether there is systematic variation in performance over the course of a radiology reading session. We re-analyzed data from six previous lesion-enriched radiology studies. Studies featured 8-22 participants assessing 27-100 cases (including mammograms, chest CT, chest x-ray, and bone x-ray) in a reading session. Changes in performance and speed as the reading session progressed were analyzed using mixed effects models. Time taken per case decreased 9-23% as the reading session progressed (p < 0.005 for every study). There was a sensitivity decrease or specificity increase over the course of reading 100 chest x-rays (p = 0.005), 60 bone fracture x-rays (p = 0.03), and 100 chest CT scans (p < 0.0001). This effect was not found in the shorter mammography sessions with 27 or 50 cases. We found evidence supporting the hypothesis that behavior and performance may change over the course of reading an enriched test set. Further research is required to ascertain whether this effect is present in radiological practice.

Tracking eye gaze during interpretation of endoluminal three-dimensional CT colonography: visual perception of experienced and inexperienced readers.

PURPOSE: To identify and compare key stages of the visual process in experienced and inexperienced readers and to examine how these processes are used to search a moving three-dimensional ( 3D three-dimensional ) image and their relationship to false-negative errors. MATERIALS AND METHODS: Institutional review board research ethics approval was granted to use anonymized computed tomographic (CT) colonographic data from previous studies and to obtain eye-tracking data from volunteers. Sixty-five radiologists (27 experienced, 38 inexperienced) interpreted 23 endoluminal 3D three-dimensional CT colonographic videos. Eye movements were recorded by using eye tracking with a desk-mounted tracker. Readers indicated when they saw a polyp by clicking a computer mouse. Polyp location and boundary on each video frame were quantified and gaze data were related to the polyp boundary for each individual reader and case. Predefined metrics were quantified and used to describe and compare visual search patterns between experienced and inexperienced readers by using multilevel modeling. RESULTS: Time to first pursuit was significantly shorter in experienced readers (hazard ratio, 1.22 [95% confidence interval: 1.04, 1.44]; P = .017) but other metrics were not significantly different. Regardless of expertise, metrics such as assessment, identification period, and pursuit times were extended in videos where polyps were visible on screen for longer periods of time. In 97% (760 of 787) of observations, readers correctly pursued polyps. CONCLUSION: Experienced readers had shorter time to first eye pursuit, but many other characteristics of eye tracking were similar between experienced and inexperienced readers. Readers pursued polyps in 97% of observations, which indicated that errors during interpretation of 3D three-dimensional CT colonography in this study occurred in either the discovery or the recognition phase, but rarely in the scanning phase of radiologic image inspection.

Method for tracking eye gaze during interpretation of endoluminal 3D CT colonography: technical description and proposed metrics for analysis.

PURPOSE: To develop an eye-tracking method applicable to three-dimensional (3D) images, where the abnormality is both moving and changing in size. MATERIALS AND METHODS: Research ethics committee approval was granted to record eye-tracking data from six inexperienced readers who inspected eight short (<30 seconds) endoluminal fly-through videos extracted from computed tomographic (CT) colonography examinations. Cases included true-positive and false-positive polyp detections from a previous study (polyp diameters, 5-25 mm). Eye tracking was performed with a desk-mounted tracker, and readers indicated when they saw a polyp with a mouse click. The polyp location on each video frame was quantified subsequently by using a circular mask. Gaze data related to each video frame were calculated relative to the visible polyp boundary and used to identify eye movements that pursue a polyp target as it changes size and position during fly-through. Gaze data were then related to positive polyp detections by readers. RESULTS: Tracking eye gaze on moving 3D images was technically feasible. Gaze was successfully classified by using pursuit analysis, and pursuit-based gaze metrics were able to help discriminate different reader search behaviors and methods of allocating visual attention during polyp identification. Of a total of 16 perceptual errors, 15 were recognition errors. There was only one visual search error. The largest polyp (25 mm) was seen but not recognized by five of six readers. CONCLUSION: Tracking a reader's gaze during endoluminal interpretation of 3D data sets is technically feasible and can be described with pursuit-based metrics. Perceptual errors can be classified into visual search errors and recognition errors. Recognition errors are more frequent in inexperienced readers.

Eye movement strategies involved in face perception.

Recordings were made of the eye fixations of three subjects in two tasks involving black-and-white photographs of faces. In the first task, subjects matched a test face with a previously viewed target face; in the second task, subjects compared two simultaneously presented faces. The eye movements were recorded with a corneal reflection technique. Each subject showed an individual fixation strategy for the tasks; in particular each subject had one or more preferred facial features which were viewed foveally in both tasks. The subjects also showed some tendency to use a regular sequential pattern of eye movements. However, the sequences used differed from one task to the other. Although some aspects of the results support the scanpath hypothesis of Noton, it is suggested that an alternative interpretation is possible.

The influence of 'Tall Man' lettering on errors of visual perception in the recognition of written drug names.

Visual errors in the perception of written drug names can reflect orthographic similarity amongst certain names. Drug names are typically printed in lowercase text. 'Tall Man' lettering, the capitalisation of the portions that differ amongst orthographically similar drug names, is employed in the field of medication labelling and prescribing to reduce medication errors by highlighting the area most likely to prevent confusion. The influence of textual format on visual drug name perception was tested amongst healthcare professionals (n = 133) using the Reicher-Wheeler task. Relative to lowercase text, Tall Man lettering improved accuracy in drug name perception. However, an equivalent improvement in accuracy was obtained using entirely uppercase text. Thus, character size may be a key determinant of perceptual accuracy for Tall Man lettering. Specific considerations for the manner in which Tall Man lettering might be best formatted and implemented in practice to reduce medication errors are discussed. STATEMENT OF RELEVANCE: Tall Man lettering aims to prevent medication errors by reducing visual confusions amongst orthographically similar drug names. It was found that, compared to lowercase text, Tall Man lettering improved accuracy in drug name perception. Character size appeared to be a key determinant of perceptual accuracy for Tall Man lettering.

The influence of tall man lettering on drug name confusion: a laboratory-based investigation in the UK using younger and older adults and healthcare practitioners.

BACKGROUND: Medication errors commonly involve confusion between drugs with similar names. One possible method of reducing error is to emphasize differences between the names using 'Tall Man' (uppercase) letters (e.g. cefTAZidime vs cefUROxime). Previous studies investigating this issue have been conducted mainly on university students, and results have been mixed. OBJECTIVE: To investigate the influence of Tall Man lettering on drug name confusion in other key participant groups. STUDY DESIGN: Two separate experiments were conducted. In Experiment 1 (conducted at the University of Glasgow, Scotland, between January 2008 and May 2008), younger and older adults performed a same/different judgement task. In Experiment 2 (conducted at various sites in England between December 2008 and February 2009), healthcare practitioners performed a task based on electronic prescribing. RESULTS: In Experiment 1, both younger and older adults made fewer name confusion errors when names contained Tall Man letters. Response times suggested that Tall Man lettering drew participants' attention to those letters, but that readers did not solely rely on these letters in making their response. In Experiment 2, healthcare practitioners made fewer name confusion errors when the names contained Tall Man letters. CONCLUSIONS: Overall, results showed that Tall Man lettering reduced drug name confusion errors in a series of laboratory-based tasks, in both younger and older adults, and healthcare practitioners. Thus, the current findings offer some support for the use of Tall Man letters as a possible systems change that could be made by both pharmacies and manufacturers in an effort to reduce error caused by drug name confusion.

Labeling of medicines and patient safety: evaluating methods of reducing drug name confusion.

OBJECTIVE: We report three experiments evaluating the proposal that highlighting sections of drug names using uppercase ("tall man") lettering and/or color may reduce the confusability of similar drug names. BACKGROUND: Medication errors commonly involve drug names that look or sound alike. One potential method of reducing these errors is to highlight sections of names on labels in order to emphasize the differences between similar products. METHOD: In Experiments 1 and 2, participants were timed as they decided whether similar name pairs were the same name or two different names. Experiment 3 was a recognition memory task. RESULTS: Results from Experiments 1 and 2 showed that highlighting sections of words using tall man lettering can make similar names easier to distinguish if participants are aware that this is the purpose of the intervention. Results from Experiment 3 suggested that tall man lettering and/or color does not make names less confusable in memory but that tall man letters may increase attention. CONCLUSION: These findings offer some support for the use of tall man letters in order to reduce errors caused by confusion between drug products with look-alike names. APPLICATION: The use of tall man letters could be applied in a variety of visual presentations of drug names--for example, by manufacturers on packaging, labeling, and computer software, and in pharmacies on shelf labels. Additionally, this paper demonstrates two meaningful behavioral measures that can be used during product design to objectively assess confusability of packaging and labeling.

Drug name confusion: evaluating the effectiveness of capital ("Tall Man") letters using eye movement data.

Medication errors commonly involve confusion between drug names that look or sound alike. One possible method of reducing these errors is to print sections of the names in "Tall Man" (capital) letters, in order to emphasise differences between similar products. This paper reports an eye-tracking experiment that evaluates this strategy. Participants had their eye movements monitored while they searched for a target product amongst an array of product packs. The target pack was replaced by a similar distractor in the array. Participants made fewer errors when the appearance of the names had been altered, that is, they were less likely to incorrectly identify a distractor as the target drug. This result was reflected in the eye movement data.

Inspection times and the selection task: what do eye-movements reveal about relevance.

Three experiments are reported that used eye-movement tracking to investigate the inspection-time effect predicted by Evans' (1996) heuristic-analytic account of the Watson selection task. Evans' account proposes that card selections are based on the operation relevance -determining heuristics, whilst analytic processing only rationalizes selections. As such, longer inspection times should be associated with selected cards (which are subjected to rationalization) than with rejected cards. Evidence for this effect has been provided by Evans (1996) using computer-presented selection tasks and instructions for participants to indicate (with a mouse pointer) cards under consideration. Roberts (1996) has argued that mouse pointing gives rise to artefactual support for Evans' predictions because of biases associated with the task format and the use of mouse pointing. We eradicated all sources of artefact by combining careful task constructions with eye-movement tracking to measure directly on-line attentional processing. All three experiments produced good evidence for the robustness of the inspection-time effect, supporting predictions of the heuristic-analytic account.

Bridge strike reduction: optimising the design of markings.

Cases of high-sided vehicles striking low bridges is a large problem in many countries, especially the UK. This paper describes an experiment to evaluate a new design of markings for low bridges. A full size bridge was constructed which was capable of having its overhead clearance adjusted. Subjects sat in a truck cab as it drove towards the bridge and were asked to judge whether the vehicle could pass safely under the bridge. The main objective of the research was to determine whether marking the bridge with a newly devised experimental marking would result in more cautious decisions from subjects regarding whether or not the experimental bridge structure could be passed under safely compared with the currently used UK bridge marking standard. The results show that the type of bridge marking influenced the level of caution associated with decisions regarding bridge navigation, with the new marking design producing the most cautious decisions for the two different bridge heights used, at all distances away from the bridge structure. Additionally, the distance before the bridge at which decisions were given had an effect on the level of caution associated with decisions regarding bridge navigation (the closer to the bridge, the more cautious the decisions became, irrespective of the marking design). The implications of these results for reducing the number of bridge strikes are discussed.

Improving the accuracy of mammography: volume and outcome relationships.

BACKGROUND: Countries with centralized, high-volume mammography screening programs, such as the U.K. and Sweden, emphasize high specificity (low percentage of false positives) and high sensitivity (high percentage of true positives). By contrast, the United States does not have centralized, high-volume screening programs, emphasizes high sensitivity, and has lower average specificity. We investigated whether high sensitivity can be achieved in the context of high specificity and whether the number of mammograms read per radiologist (reader volume) drives both sensitivity and specificity. METHODS: The U.K.'s National Health Service Breast Screening Programme uses the PERFORMS 2 test as a teaching and assessment tool for radiologists. The same 60-film PERFORMS 2 test was given to 194 high-volume U.K. radiologists and to 60 U.S. radiologists, who were assigned to low-, medium-, or high-volume groups on the basis of the number of mammograms read per month. The standard binormal receiver-operating characteristic (ROC) model was fitted to the data of individual readers. Detection accuracy was measured by the sensitivity at specificity = 0.90, and differences among sensitivities were determined by analysis of variance. RESULTS: The average sensitivity at specificity = 0.90 was 0.785 for U.K. radiologists, 0.756 for high-volume U.S. radiologists, 0.702 for medium-volume U.S. radiologists, and 0.648 for low-volume U.S. radiologists. At this specificity, low-volume U.S. radiologists had statistically significantly lower sensitivity than either high-volume U.S. radiologists or U.K. radiologists, and medium-volume U.S. radiologists had statistically significantly lower sensitivity than U.K. radiologists (P<.001, for all comparisons). CONCLUSIONS: Reader volume is an important determinant of mammogram sensitivity and specificity. High sensitivity (high cancer detection rate) can be achieved with high specificity (low false-positive rate) in high-volume centers. This study suggests that there is great potential for optimizing mammography screening.

Eye movements of large populations: I. Implementation and performance of an autonomous public eye tracker.

This paper details the design and construction of an autonomous public eye tracker exhibit, which was installed at the National Gallery, London, in 2000/2001. For over 3 months, it functioned both as an informative exhibit and as a controlled eye movement experiment, gathering data from over 5,000 participants. The issues associated with automatic unattended recording of the eye movements of members of the public are discussed. The performance of the exhibit is examined, and its successes and problem areas are highlighted with regard to potential applications and future exhibits. The success of the project proves the viability of autonomous public eye trackers as both data-gatherers and public exhibits.