Telereporting in Breast Imaging Involving More Than 1 Country Requires Careful Consideration.

PURPOSE: To examine the effect that a test set from 1 region of the world has on the performance of expert breast radiologists from that region and from a distant region. METHODS: Forty-seven Australian and 15 Singaporean breast radiologists interpreted the same test set (ie, 60 mammograms of Australian women consisting of 40 typical and 20 atypical cases) in their respective locations. The radiologists assigned each case a confidence score of 1 to 5, where 1 was typical, 2 was benign, and 3 to 5 indicated identifiable cancer. Localization-response operating characteristic area under the curve (AUC), receiver operating characteristic AUC, location sensitivity, case sensitivity, and specificity were calculated for each radiologist and compared between Australian and Singaporean readers using nonparametric 2-tailed Mann-Whitney U tests. RESULTS: Australian readers performed better than did their Singaporean counterparts with respect to localization-response operating characteristic AUC (.738 vs .642, P = .0395) and location sensitivity (.775 vs .660, P = .0185). DISCUSSION: Technological advancement has made possible telereporting, in which radiologists report on mammograms originating from other countries. However, those readers' levels of performance might be affected by interpreting images from populations that are different from those with which they are familiar. CONCLUSION: Radiologist performance differed between the groups when localization of lesions was taken into account. This difference might be because of the Australians' reading mammograms from populations with which they were familiar, in contrast to the Singaporeans.

Improvement of Cancer Detection on Mammograms via BREAST Test Sets.

BACKGROUND: Breast Screen Reader Assessment Strategy (BREAST) is an innovative training and research program for radiologists in Australia and New Zealand. The aim of this study is to evaluate the efficacy of BREAST test sets in improving readers' performance in detecting cancers on mammograms. MATERIALS AND METHODS: Between 2011 and 2018, 50 radiologists (40 fellows, 10 registrars) completed three BREAST test sets and 17 radiologists completed four test sets. Each test set contained 20 biopsy-proven cancer and 40 normal cases. Immediate image-based feedback was available to readers after they completed each test set which allowed the comparison of their selections with the truth. Case specificity, case sensitivity, lesion sensitivity, the Receiver Operating Characteristic (ROC) Area Under the Curve (AUC) and Jackknife Free-Response Receiver Operating Characteristic (JAFROC) Figure of Merit (FOM) were calculated for each reader. Kruskal-Wallis test was utilized to compare scores of the radiologist and registrars across all test-sets whilst Wilcoxon signed rank test was to compare the scores between pairs of test sets. RESULTS: Significant improvements in lesion sensitivity ranging from 21% to 31% were found in radiologists completing later test sets compared to first test set (p ≤ 0.01). Eighty three percent of radiologists achieved higher performance in lesion sensitivity after they completed the first read. Registrars had significantly better scores in the third test set compared to the first set with mean increases of 79% in lesion sensitivity (p = 0.005) and 37% in JAFROC (p = 0.02). Sixty percent and 100% of registrars increased their scores in lesion sensitivity in the second and third reads compared to the first read while the percentage of registrars with higher scores in JAFROC was 80%. CONCLUSION: Introduction of BREAST into national training programs appears to have an important impact in promoting diagnostic efficacy amongst radiologists and radiology registrars undergoing mammographic readings.

Observer Variability in Breast Cancer Diagnosis between Countries with and without Breast Screening.

RATIONAL AND OBJECTIVES: Image reporting is a vital component of patient management depending on individual radiologists' performance. Our objective was to explore mammographic diagnostic efficacy in a country where breast cancer screening does not exist. MATERIALS AND METHODS: Two mammographic test sets were used: a typical screening (TS) and high-difficulty (HD) test set. Nonscreening (NS) radiologists (n = 11) read both test sets, while 52 and 49 screening radiologists read the TS and HD test sets, respectively. The screening radiologists were classified into two groups: a less experienced (LE) group with ≤5 years' experience and a more experienced (ME) group with ≥5 years' experience. A Kruskal-Wallis and Tukey-Kramer post hoc test were used to compare reading performance among reader groups, and the Wilcoxon matched pairs tests was used to compare TS and ND test sets for the NS radiologists. RESULTS: Across the three reader groups, there were significant differences in case sensitivity (χ2 [2] = 9.4, P = .008), specificity (χ2 [2] = 10.3, P = .006), location sensitivity (χ2 [2] = 19.8, P < .001), receiver operating characteristics, area under the curve (χ2 [2] = 19.7, P < .001) and jack-knife free-response receiver operating characteristics (JAFROCs) (χ2 [2] = 18.1, P < .001). NS performance for all measured scores was significantly lower than those for the ME readers (P < .006), while only location sensitivity was lower (χ2 [2] = 17.5, P = .026) for the NS compared to the LE group. No other significant differences were observed. CONCLUSION: Large variations in mammographic performance exist between radiologists from screening and nonscreening countries.

Mammographic detection of breast cancer in a non-screening country.

OBJECTIVE:: To compare the diagnostic accuracy between radiologists' from a country with and without breast cancer screening. METHODS:: All participating radiologists gave informed consent. A test-set involving 60 mammographic cases (20 cancer and 40 non-cancer) were read by 11 radiologists from a non-screening (NS) country during a workshop in July 2016. 52 radiologists from a screening country read the same test-set at the Royal Australian and New Zealand College of Radiologists' meetings in July 2015. The screening radiologists were classified into two groups: those with less than or equal to 5 years of experience; those with more than 5 years of experience, and each group was compared to the group of NS radiologists. A Kruskal-Wallis test followed by post-hoc multiple comparisons test were used to compare measures of diagnostic accuracy among the reader groups. RESULTS:: The diagnostic accuracy of the NS radiologists was significantly lower in terms of sensitivity [mean = 54.0; 95% confidence interval (CI) (40.0-67.0)], location sensitivity [mean = 26.0; 95% CI (16.0-37.0)], receive roperating characteristic area under curve [mean = 73.0; 95% CI (66.5-81.0)] and Jackknifefree-response receiver operating characteristics figure-of-merit [mean = 45.0; 95% CI (40.0-50.0)] when compared with the less and more experienced screening radiologists, whilst no difference in specificity [mean = 75.0; 95% CI (70.0- 81.0)] was found. No significant differences in all measured diagnostic accuracy were found between the two groups of screening radiologists. CONCLUSION:: The mammographic performance of a group of radiologists from a country without screening program was suboptimal compared with radiologists from Australia. ADVANCES IN KNOWLEDGE:: Identifying mammographic performance in developing countries is required to optimize breast cancer diagnosis.

Classification of normal screening mammograms is strongly influenced by perceived mammographic breast density.

INTRODUCTION: To investigate how breast screen readers classify normal screening cases using descriptors of normal mammographic features and to assess test cases for suitability for a single reading strategy. METHODS: Fifteen breast screen readers interpreted a test set of 29 normal screening cases and classified them by firstly rating their perceived difficulty to reach a 'normal' decision, secondly identifying the cases' salient normal mammographic features and thirdly assessing the cases' suitability for a single reading strategy. RESULTS: The relationship between the perceived difficulty in making 'normal' decisions and the normal mammographic features was investigated. Regular ductal pattern (Tb  = -0.439, P = 0.001), uniform density (Tb  = -0.527, P < 0.001), non-dense breasts (Tb  = -0.736, P < 0.001), symmetrical mammographic features (Tb  = -0.474, P = 0.001) and overlapped density (Tb  = 0.630, P < 0.001) had a moderate to strong correlation with the difficulty to make 'normal' decisions. Cases with regular ductal pattern (Tb  = 0.447, P = 0.002), uniform density (Tb  = 0.550, P < 0.001), non-dense breasts (Tb  = 0.748, P < 0.001) and symmetrical mammographic features (Tb  = 0.460, P = 0.001) were considered to be more suitable for single reading, whereas cases with overlapped density were not (Tb  = -0.679, P < 0.001). CONCLUSION: The findings suggest that perceived mammographic breast density has a major influence on the difficulty for readers to classify cases as normal and hence their suitability for single reading.

Impact of Breast Reader Assessment Strategy on mammographic radiologists' test reading performance.

INTRODUCTION: The detection of breast cancer is somewhat limited by human factors, and thus there is a need to improve reader performance. This study assesses whether radiologists who regularly undertake the education in the form of the Breast Reader Assessment Strategy (BREAST) demonstrate any changes in mammography interpretation performance over time. METHODS: In 2011, 2012 and 2013, 14 radiologists independently assessed a year-specific BREAST mammographic test-set. Radiologists read a different single test-set once each year, with each comprising 60 digital mammogram cases. Radiologists marked the location of suspected lesions without computer-aided diagnosis (CAD) and assigned a confidence rating of 2 for benign and 3-5 for malignant lesions. The mean sensitivity, specificity, location sensitivity, JAFROC FOM and ROC AUC were calculated. A Kruskal-Wallis test was used to compare the readings for the 14 radiologists across the 3 years. Wilcoxon signed rank test was used to assess comparison between pairs of years. Relationships between changes in performance and radiologist characteristics were examined using a Spearman's test. RESULTS: Significant increases were noted in mean sensitivity (P = 0.01), specificity (P = 0.01), location sensitivity (P = 0.001) and JAFROC FOM (P = 0.001) between 2011 and 2012. Between 2012 and 2013, significant improvements were noted in mean sensitivity (P = 0.003), specificity (P = 0.002), location sensitivity (P = 0.02), JAFROC FOM (P = 0.005) and ROC AUC (P = 0.008). No statistically significant correlations were shown between the levels of improvement and radiologists' characteristics. CONCLUSION: Radiologists' who undertake the BREAST programme demonstrate significant improvements in test-set performance during a 3-year period, highlighting the value of ongoing education through the use of test-set.

Personal and Network Dynamics in Performance of Knowledge Workers: A Study of Australian Breast Radiologists.

MATERIALS AND METHODS: In this paper, we propose a theoretical model based upon previous studies about personal and social network dynamics of job performance. We provide empirical support for this model using real-world data within the context of the Australian radiology profession. An examination of radiologists' professional network topology through structural-positional and relational dimensions and radiologists' personal characteristics in terms of knowledge, experience and self-esteem is provided. Thirty one breast imaging radiologists completed a purpose designed questionnaire regarding their network characteristics and personal attributes. These radiologists also independently read a test set of 60 mammographic cases: 20 cases with cancer and 40 normal cases. A Jackknife free response operating characteristic (JAFROC) method was used to measure the performance of the radiologists' in detecting breast cancers. RESULTS: Correlational analyses showed that reader performance was positively correlated with the social network variables of degree centrality and effective size, but negatively correlated with constraint and hierarchy. For personal characteristics, the number of mammograms read per year and self-esteem (self-evaluation) positively correlated with reader performance. Hierarchical multiple regression analysis indicated that the combination of number of mammograms read per year and network's effective size, hierarchy and tie strength was the best fitting model, explaining 63.4% of the variance in reader performance. The results from this study indicate the positive relationship between reading high volumes of cases by radiologists and expertise development, but also strongly emphasise the association between effective social/professional interactions and informal knowledge sharing with high performance.

An Investigation into the Consistency in Mammographic Density Identification by Radiologists: Effect of Radiologist Expertise and Mammographic Appearance.

The aim of this work is to investigate how radiologist expertise and image appearance may have an impact on inter-reader variability of mammographic density (MD) identification. Seventeen radiologists, divided into three expertise groups, were asked to manually segment the areas they consider to be MD in 40 clinical images. The variation in identification of MD for each image was quantified by finding the range of segmentation areas. The impact of radiologist expertise and image appearance on this variation was explored. The range of areas chosen by participating radiologists varied from 7 to 73% across the 40 images, with a mean range of 35 ± 13%. Participants with high expertise were more likely to choose similar areas to one another, compared to participants with medium and low expertise levels (mean range were 19 ± 10%, 29 ± 13% and 25 ± 14 %, respectively, p < 0.0001). There was a significantly higher average grey level for the area segmented by all radiologists as MD compared to the area of variation, with mean grey level value for 8-bit images being 146 ± 19 vs. 99 ± 14, respectively. MD segmentation borders were consistent in areas where there was a sharp intensity change within a short distance. In conclusion, radiologists with high expertise tend to have a higher agreement when identifying MD. Tissues which have a lower contrast and a less visually sharp gradient change at the interface between high density tissue and adipose background lead to inter-reader variation in choosing mammographic density.

Evaluation of the effect of zoom function on lesion detection by soft-copy reading of screening mammograms.

INTRODUCTION: This study aims to evaluate the effectiveness of zooming in improving screen-reader performance in reporting digital mammograms. METHOD: Two experiments were conducted. In the first experiment, 5 readers were asked to report 59 two-view bilateral mammograms retrospectively with zooming function turned off. The second session was similar to the first one except that zooming was enabled. The task of readers was to assess if the mammograms were normal or abnormal and rate the confidence levels for each of the lesion they detected. The reader performances were evaluated via case sensitivity, lesion sensitivity, specificity, receiver operating characteristics (ROC) area under the curve (AUC) and jackknife free-response receiver operating characteristics (JAFROC) figure of merit (FOM). RESULTS: There was no significant improvement in overall reader performance in detecting abnormalities in zooming condition compared with no zooming in terms of case sensitivity (96% and 87%, P = 0.285) or lesion sensitivity (88% and 81%, P = 0.224). However, differences in ROC AUC and JAFROC FOM (P ≤ 0.05) were found in two readers when they performed the test set with zooming function. CONCLUSION: The results suggested that the use of the zooming function did improve the performance of some readers in detecting abnormal cases.

Certain performance values arising from mammographic test set readings correlate well with clinical audit.

INTRODUCTION: Test sets have been increasingly utilised to augment clinical audit in breast screening programmes; however, their relationship has never been satisfactorily understood. This study examined the relationship between mammographic test set performance and clinical audit data. METHODS: Clinical audit data over a 2-year period was generated for each of 20 radiologists. Sixty mammographic examinations, consisting of 40 normal and 20 cancer cases, formed the test set. Readers located any identifiable cancer, and levels of confidence were scored from 2 to 5, where a score of 3 and above is considered a recall rating. Jackknifing free response operating characteristic (JAFROC) figure-of-merit (FOM), location sensitivity and specificity were calculated for individual readers and then compared with clinical audit values using Spearman's rho. RESULTS: JAFROC FOM showed significant correlations to: recall rate at a first round of screening (r = 0.51; P = 0.02); rate of small invasive cancers per 10 000 reads (r = 0.5; P = 0.02); percentage of all cancers read that were not recalled (r = -0.51; P = 0.02); and sensitivity (r = 0.51; P = 0.02). Location sensitivity demonstrated significant correlations with: rate of small invasive cancers per 10 000 reads (r = 0.46; P = 0.04); rate of DCIS (ductal carcinoma in situ) per 10 000 reads (r = 0.44; P = 0.05); detection rate of all invasive cancers and DCIS per 10 000 reads (r = 0.54; P = 0.01); percentage of all cancers read that were not recalled (r = -0.57; P = 0.009); and sensitivity (r = 0.57; P = 0.009). No other significant relationships were noted. CONCLUSION: Performance indicators from test set demonstrate significant correlations with specific aspects of clinical performance, although caution needs to be exercised when generalising test set specificity to the clinical situation.

Mammographic density and cancer detection: does digital imaging challenge our current understanding?

RATIONALE AND OBJECTIVES: To investigate the impact of breast density on the performance of radiologists when mammograms are digitally acquired and displayed. MATERIALS AND METHODS: A total of 150 craniocaudal digital mammograms including 75 cases with cancer were examined by 14 radiologists divided into two groups: those who read more (six) and less (eight) than 2000 mammograms per year. Cases were classified as low or high mammographic density. For both types of cases, detection of cancers within and outside the dense fibroglandular tissue was investigated. The performance of radiologist was measured using jack-knife free-response receiver operating characteristic (JAFROC) figure of merit (FOM). RESULTS: Radiologists with over 2000 annual reads had significantly higher JAFROC FOM (P = .03) for high (0.76) mammographic density compared to low (0.70) mammographic density cases. When lesions overlaid the fibroglandular tissue, cases with high mammographic density compared to low mammographic density displayed increased location sensitivity for all radiologists (P = .03) and for those radiologists reading more than 2000 mammograms annually (P = .04), whereas JAFROC FOMs increased for all radiologists (P = .05). No significant changes were observed when the lesion was outside the fibroglandular region. CONCLUSIONS: Increased mammographic density improves the performance of experienced radiologists when using digital mammograms. This finding, which does not align with those previously reported for film screen systems, may be because of windowing/leveling opportunities available with digital images.

How mammographic breast density affects radiologists' visual search patterns.

RATIONALE AND OBJECTIVES: To determine the impact of mammographic breast density on the visual search process of radiologists when reading digital mammograms. MATERIALS AND METHODS: Institutional review board approval was obtained. A set of 149 craniocaudal digital mammograms were read by seven radiologists, and observer search patterns were recorded. Total time examining each case, time to first hit the lesion, dwell time, and number of hits per area were calculated. The nonparametric Mann-Whitney U test was used for statistical evaluation. RESULTS: In both low- and high-mammographic density cases, significant increases were observed in the time to first hit lesions when they were located outside, compared to overlying fibroglandular dense tissue (P = .001). Significantly longer dwell time (P = .003) and greater number of fixations (P = .0003) were observed when the lesions were situated within--rather than outside--the dense fibroglandular tissue. CONCLUSIONS: Increased mammographic breast density changes radiologists' visual search patterns. Dense areas of the parenchyma attracted greater visual attention in both high- and low-mammographic density cases, resulting in faster detection of lesions overlying the fibroglandular dense tissue, along with longer dwell times and greater number of fixations, as compared to lesions located outside the dense fibroglandular regions.

Markers of good performance in mammography depend on number of annual readings.

PURPOSE: To explore relationships between reader performance and reader characteristics in mammography for specific radiologist groupings on the basis of annual number of readings. MATERIALS AND METHODS: The institutional review board approved the study and waived the need for patient consent to use all images. Readers gave informed consent. One hundred sixteen radiologists independently reviewed 60 mammographic cases: 20 cases with cancer and 40 cases with normal findings. Readers located any visualized cancer, and levels of confidence were scored from 1 to 5. A jackknifing free response operating characteristic (JAFROC) method was used, and figures of merit along with sensitivity and specificity were correlated with reader characteristics by using Spearman techniques and standard multiple regressions. RESULTS: Reader performance was positively correlated with number of years since qualification as a radiologist (P ≤ .01), number of years reading mammograms (P ≤ .03), and number of readings per year (P ≤ .0001). The number of years since qualification as a radiologist (P ≤ .004) and number of years of reading mammograms (P ≤ .002) were negatively related to JAFROC values for radiologists with annual volumes of less than 1000 mammographic readings. For individuals with more than 5000 mammographic readings per year, JAFROC values were positively related to the number of years that the reader was qualified as a radiologist (P ≤ .01), number of years of reading mammograms (P ≤ .002), and number of hours per week of reading mammograms (P ≤ .003). Number of mammographic readings per year was positively related with JAFROC scores for readers with an annual volume between 1000 and 5000 readings (P ≤ .03). Differences in JAFROC scores appear to be more related to specificity than location sensitivity, with the former demonstrating significant relationships with four of the five characteristics analyzed, whereas no relationships were shown for the latter. CONCLUSION: Radiologists' determinants of performance are associated with annual reading volumes. Ability to recognize normal images is a discriminating factor in individuals with a high volume of mammographic readings.

Screening mammography: test set data can reasonably describe actual clinical reporting.

PURPOSE: To establish the extent to which test set reading can represent actual clinical reporting in screening mammography. MATERIALS AND METHODS: Institutional ethics approval was granted, and informed consent was obtained from each participating screen reader. The need for informed consent with respect to the use of patient materials was waived. Two hundred mammographic examinations were selected from examinations reported by 10 individual expert screen readers, resulting in 10 reader-specific test sets. Data generated from actual clinical reports were compared with three test set conditions: clinical test set reading with prior images, laboratory test set reading with prior images, and laboratory test set reading without prior images. A further set of five expert screen readers was asked to interpret a common set of images in two identical test set conditions to establish a baseline for intraobserver variability. Confidence scores (from 1 to 4) were assigned to the respective decisions made by readers. Region-of-interest (ROI) figures of merit (FOMs) and side-specific sensitivity and specificity were described for the actual clinical reporting of each reader-specific test set and were compared with those for the three test set conditions. Agreement between pairs of readings was performed by using the Kendall coefficient of concordance. RESULTS: Moderate or acceptable levels of agreement were evident (W = 0.69-0.73, P < .01) when describing group performance between actual clinical reporting and test set conditions that were reasonably close to the established baseline (W = 0.77, P < .01) and were lowest when prior images were excluded. Higher median values for ROI FOMs were demonstrated for the test set conditions than for the actual clinical reporting values; this was possibly linked to changes in sensitivity. CONCLUSION: Reasonable levels of agreement between actual clinical reporting and test set conditions can be achieved, although inflated sensitivity may be evident with test set conditions.

Quantitative measures confirm the inverse relationship between lesion spiculation and detection of breast masses.

OBJECTIVE: To identify specific mammographic appearances that reduce the mammographic detection of breast cancer. MATERIALS AND METHODS: This study received institutional board review approval and all readers gave informed consent. A set of 60 mammograms each consisting of craniocaudal and mediolateral oblique projections were presented to 129 mammogram Breastscreen readers. The images consisted of 20 positive cases with single and multicentric masses in 16 and 4 cases, respectively (resulting in a total of 24 cancers), and readers were asked to identify and locate the lesions. Each lesion was then ranked according to a detectability rating (ie, the number of observers who correctly located the lesion divided by the total number of observers), and this was correlated with breast density, lesion size, and various descriptors of lesion shape and texture. RESULTS: Negative and positive correlations between lesion detection and density (r = -0.64, P = .007) and size (r = 0.65, P = .005), respectively, were demonstrated. In terms of lesion size and shape, there were significant correlations between the probability of detection and area (r = 0.43, P = .04), perimeter (r = 0.66, P = .0004), lesion elongation (r = 0.49, P = .02), and lesion nonspiculation (r = 0.78, P < .0001). CONCLUSIONS: The results of this study have identified specific lesion characteristics associated with shape that may contribute to reduced cancer detection. Mammographic sensitivity may be adversely affected without appropriate attention to spiculation.

Breast screen new South wales generally demonstrates good radiologic viewing conditions.

This study measured reading workstation monitors and the viewing environment currently available within BreastScreen New South Wales (BSNSW) centres to determine levels of adherence to national and international guidelines. Thirteen workstations from four BSNSW service centres were assessed using the American Association of Physicists in Medicine Task Group 18 Quality Control test pattern. Reading workstation monitor performance and ambient light levels when interpreting screening mammographic images were assessed using spectroradiometer CS-2000 and chroma meter CL-200. Overall, radiologic monitors within BSNSW were operating at good acceptable levels. Some non-adherence to published guidelines included the percentage difference in maximum luminance between pairs of primary monitors at individual workstations (61.5 % or 30.8 % of workstations depending on specific guidelines), maximum luminance (23.1 % of workstations), luminance non-uniformity (11.5 % of workstations) and minimum luminance (3.8 % of workstations). A number of ambient light measurements did not comply with the only available evidence-based guideline relevant to the methodology used in this study. Larger ambient light variations across sites are shown when monitors were switched off, suggesting that differences in ambient lighting between sites can be masked when a standard mammogram is displayed for photometric measurements. Overall, BSNSW demonstrated good adherence to available guidelines, although some non-compliance has been shown. Recently updated United Kingdom and Australian guidelines should help reduce confusion generated by the plethora and sometimes dated nature of currently available recommendations.

Mammographic screening for breast cancer: A review.

In 2011, BreastScreen Australia celebrated 20 years of mammographic screening for breast cancer in Australia. There has been a reduction in mortality from breast cancer over the last two decades, coincident with mammographic screening. However, there are concerns that mammographic screening may result in overdiagnosis of breast cancer and that the reduction in mortality from breast cancer is the result of better treatment rather than screening. This article reviews the evidence on which mammographic screening for breast cancer is based, considers the issue of overdiagnosis of breast cancer by screening mammography, and assesses the role of screening mammography in the reduction in breast cancer mortality seen over the last two decades.

Malignancy detection in digital mammograms: important reader characteristics and required case numbers.

RATIONALE AND OBJECTIVES: To determine the relationship between heightened levels of reader performance and reader practice in terms of number of cases read and previous experience. MATERIALS AND METHODS: A test set of mammograms was developed comprising 50 cases. These cases consisted of 15 abnormals (biopsy proven) and 35 normals (confirmed at subsequent rescreen). Sixty-nine breast image readers reviewed these cases independently and their performance was measured by recording their individual receiver operating characteristic score (area under the curve), sensitivity, and specificity. These measures of performance were then compared to a range of factors relating to the reader such as years of certification and reporting, number of cases read per year, previous experiences, and satisfaction levels. Correlation analyses using Spearman methods were performed along with the Mann-Whitney test to detect differences in performance between specific reader groups. RESULTS: Improved reader performance was found for years certified (P = .004), years of experience (P = .0001), and hours reading per week (P = .003) shown by positive statistical significant relationships with Az values (area under receiver operating characteristic curve). Statistical comparisons of Az values scored for individuals who read varying number of cases per year showed that those individuals whose annual mammographic case load was 5000 or more (P = .03) or between 2000 and 4999 (P = .05), had statistically significantly higher scores than those who read less than 1000 cases per year. CONCLUSION: The results of this study have shown variations in reader performance relating to parameters of reader practice and experience. Levels of variance are shown and potential acceptance levels for diagnostic efficacy are proposed which may inform policy makers, judicial systems and public debate.