Service and clinical impacts of reader bias in breast cancer screening: a retrospective study.

OBJECTIVES: To determine factors influencing reader agreement in breast screening and investigate the relationship between agreement level and patient outcomes. METHODS: Reader pair agreement for 83 265 sets of mammograms from the Scottish Breast Screening service (2015-2020) was evaluated using Cohen's kappa statistic. Each mammography examination was read by two readers, per routine screening practice, with the second initially blinded but able to choose to view the first reader's opinion. If the two readers disagreed, a third reader arbitrated. Variation in reader agreement was examined by: whether the reader acted as the first or second reader, reader experience, and recall, cancer detection and arbitration recall rate. RESULTS: Readers' opinions varied by whether they acted as the first or second reader. Furthermore, reader 2 was more likely to agree with reader 1 if reader 1 was more experienced than they were, and less likely to agree if they themselves were more experienced than reader 1 (P < .001). Agreement was not significantly associated with cancer detection rate, overall recall rate or arbitration recall rates (P > .05). Lower agreement between readers led to a higher arbiter workload (P < .001). CONCLUSIONS: In mammography screening, the second reader's opinion is influenced by the first reader's opinion, with the degree of influence dependent on the readers' relative experience levels. ADVANCES IN KNOWLEDGE: While less-experienced readers relied on their more experienced reading partner, no adverse impact on service outcomes was observed. Allowing access to the first reader's opinion may benefit newly qualified readers, but reduces independent evaluation, which may lower cancer detection rates.

Impact of Different Mammography Systems on Artificial Intelligence Performance in Breast Cancer Screening.

Artificial intelligence (AI) tools may assist breast screening mammography programs, but limited evidence supports their generalizability to new settings. This retrospective study used a 3-year dataset (April 1, 2016-March 31, 2019) from a U.K. regional screening program. The performance of a commercially available breast screening AI algorithm was assessed with a prespecified and site-specific decision threshold to evaluate whether its performance was transferable to a new clinical site. The dataset consisted of women (aged approximately 50-70 years) who attended routine screening, excluding self-referrals, those with complex physical requirements, those who had undergone a previous mastectomy, and those who underwent screening that had technical recalls or did not have the four standard image views. In total, 55 916 screening attendees (mean age, 60 years ± 6 [SD]) met the inclusion criteria. The prespecified threshold resulted in high recall rates (48.3%, 21 929 of 45 444), which reduced to 13.0% (5896 of 45 444) following threshold calibration, closer to the observed service level (5.0%, 2774 of 55 916). Recall rates also increased approximately threefold following a software upgrade on the mammography equipment, requiring per-software version thresholds. Using software-specific thresholds, the AI algorithm would have recalled 277 of 303 (91.4%) screen-detected cancers and 47 of 138 (34.1%) interval cancers. AI performance and thresholds should be validated for new clinical settings before deployment, while quality assurance systems should monitor AI performance for consistency. Keywords: Breast, Screening, Mammography, Computer Applications-Detection/Diagnosis, Neoplasms-Primary, Technology Assessment Supplemental material is available for this article. © RSNA, 2023.

AI in breast screening mammography: breast screening readers' perspectives.

OBJECTIVES: This study surveyed the views of breast screening readers in the UK on how to incorporate Artificial Intelligence (AI) technology into breast screening mammography. METHODS: An online questionnaire was circulated to the UK breast screening readers. Questions included their degree of approval of four AI implementation scenarios: AI as triage, AI as a companion reader/reader aid, AI replacing one of the initial two readers, and AI replacing all readers. They were also asked to rank five AI representation options (discrete opinion; mammographic scoring; percentage score with 100% indicating malignancy; region of suspicion; heat map) and indicate which evidence they considered necessary to support the implementation of AI into their practice among six options offered. RESULTS: The survey had 87 nationally accredited respondents across the UK; 73 completed the survey in full. Respondents approved of AI replacing one of the initial two human readers and objected to AI replacing all human readers. Participants were divided on AI as triage and AI as a reader companion. A region of suspicion superimposed on the image was the preferred AI representation option. Most screen readers considered national guidelines (77%), studies using a nationally representative dataset (65%) and independent prospective studies (60%) as essential evidence. Participants' free-text comments highlighted concerns and the need for additional validation. CONCLUSIONS: Overall, screen readers supported the introduction of AI as a partial replacement of human readers and preferred a graphical indication of the suspected tumour area, with further evidence and national guidelines considered crucial prior to implementation.