Experiences with a self-test for Dutch breast screening radiologists: lessons learnt.

PURPOSE: To evaluate a self-test for Dutch breast screening radiologists introduced as part of the national quality assurance programme. METHODS AND MATERIALS: A total of 144 radiologists were invited to complete a test-set of 60 screening mammograms (20 malignancies). Participants assigned findings such as location, lesion type and BI-RADS. We determined areas under the receiver operating characteristics (ROC) curves (AUC), case and lesion sensitivity and specificity, agreement (kappa) and correlation between reader characteristics and case sensitivity (Spearman correlation coefficients). RESULTS: A total of 110 radiologists completed the test (76%). Participants read a median number of 10,000 screening mammograms/year. Median AUC value was 0.93, case and lesion sensitivity was 91% and case specificity 94%. We found substantial agreement for recall (κ = 0.77) and laterality (κ = 0.80), moderate agreement for lesion type (κ = 0.57) and BI-RADS (κ = 0.45) and no correlation between case sensitivity and reader characteristics. CONCLUSION: Areas under the ROC curve, case sensitivity and lesion sensitivity were satisfactory and recall agreement was substantial. However, agreement in lesion type and BI-RADS could be improved; further education might be aimed at reducing interobserver variation in interpretation and description of abnormalities. We offered individual feedback on interpretive performance and overall feedback at group level. Future research will determine whether performance has improved. KEY POINTS: • We introduced and evaluated a self-test for Dutch breast screening radiologists. • ROC curves, case and lesion sensitivity and recall agreement were all satisfactory. • Agreement in BI-RADS interpretation and description of abnormalities could be improved. • These are areas that should be targeted with further education and training. • We offered individual feedback on interpretative performance and overall group feedback.

Breast cancer risk prediction model: a nomogram based on common mammographic screening findings.

OBJECTIVES: To develop a prediction model for breast cancer based on common mammographic findings on screening mammograms aiming to reduce reader variability in assigning BI-RADS. METHODS: We retrospectively reviewed 352 positive screening mammograms of women participating in the Dutch screening programme (Nijmegen region, 2006-2008). The following mammographic findings were assessed by consensus reading of three expert radiologists: masses and mass density, calcifications, architectural distortion, focal asymmetry and mammographic density, and BI-RADS. Data on age, diagnostic workup and final diagnosis were collected from patient records. Multivariate logistic regression analyses were used to build a breast cancer prediction model, presented as a nomogram. RESULTS: Breast cancer was diagnosed in 108 cases (31 %). The highest positive predictive value (PPV) was found for spiculated masses (96 %) and the lowest for well-defined masses (10 %). Characteristics included in the nomogram are age, mass, calcifications, architectural distortion and focal asymmetry. CONCLUSION: With our nomogram we developed a tool assisting screening radiologists in determining the chance of malignancy based on mammographic findings. We propose cutoff values for assigning BI-RADS in the Dutch programme based on our nomogram, which will need to be validated in future research. These values can easily be adapted for use in other screening programmes. KEY POINTS: • There is substantial reader variability in assigning BI-RADS in mammographic screening. • There are no strict guidelines linking mammographic findings to BI-RADS categories. • We developed a model (nomogram) predicting the presence of breast cancer. • Our nomogram is based on common findings on positive screening mammograms. • The nomogram aims to assist screening radiologists in assigning BI-RADS categories.

The Breast Imaging Reporting and Data System (BI-RADS) in the Dutch breast cancer screening programme: its role as an assessment and stratification tool.

OBJECTIVES: To assess the suitability of the Breast Imaging Reporting and Data System (BI-RADS) as a quality assessment tool in the Dutch breast cancer screening programme. METHODS: The data of 93,793 screened women in the Amsterdam screening region (November 2005-July 2006) were reviewed. BI-RADS categories, work-up, age, final diagnosis and final TNM classification were available from the screening registry. Interval cancers were obtained through linkage with the cancer registry. BI-RADS was introduced as a pilot in the Amsterdam region before the nationwide introduction of digital mammography (2009-2010). RESULTS: A total of 1,559 women were referred to hospital (referral rate 1.7 %). Breast cancer was diagnosed in 485 women (detection rate 0.52 %); 253 interval cancers were reported, yielding a programme sensitivity of 66 % and specificity of 99 %. BI-RADS 0 had a lower positive predictive value (PPV, 14.1 %) than BI-RADS 4 (39.1 %) and BI-RADS 5 (92.9 %; P < 0.0001). The number of invasive procedures and tumour size also differed significantly between BI-RADS categories (P < 0.0001). CONCLUSION: The significant differences in PPV, invasive procedures and tumour size match with stratification into BI-RADS categories. It revealed inter-observer variability between screening radiologists and can thus be used as a quality assessment tool in screening and as a stratification tool in diagnostic work-up. KEY POINTS: • The BI-RADS atlas is widely used in breast cancer screening programmes. • There were significant differences in results amongst different BI-RADS categories. • Those differences represented the radiologists' degree of suspicion for malignancy, thus enabling stratification of referrals. • BI-RADS can be used as a quality assessment tool in screening. • Training should create more uniformity in applying the BI-RADS lexicon.

A dedicated BI-RADS training programme: effect on the inter-observer variation among screening radiologists.

INTRODUCTION: The Breast Imaging Reporting and Data System (BI-RADS) was introduced in the Dutch breast cancer screening programme to improve communication between medical specialists. Following introduction, a substantial variation in the use of the BI-RADS lexicon for final assessment categories was noted among screening radiologists. We set up a dedicated training programme to reduce this variation. This study evaluates whether this programme was effective. MATERIALS AND METHODS: Two comparable test sets were read before and after completion of the training programme. Each set contained 30 screening mammograms of referred women selected from screening practice. The sets were read by 25 experienced and 30 new screening radiologists. Cohen's kappa (κ) was used to calculate the inter-observer agreement. The BI-RADS 2003 version was implemented in the screening programme as the BI-RADS 2008 version requires the availability of diagnostic work-up, and this is unavailable. RESULTS: The inter-observer agreement of all participating radiologists (n=55) with the expert panel increased from a pre-training κ-value of 0.44 to a post-training κ-value of 0.48 (p=0.14). The inter-observer agreement of the new screening radiologists (n=30) with the expert panel increased from κ=0.41 to κ=0.50 (p=0.01), whereas there was no difference in agreement among the 25 experienced radiologists (from κ=0.48 to κ=0.46, p=0.60). CONCLUSION: Our training programme in the BI-RADS lexicon resulted in a significant improvement of agreement among new screening radiologists. Overall, the agreement among radiologists was moderate (guidelines Landis and Koch). This is in line with results found in the literature.