Use of Artificial Intelligence for Digital Breast Tomosynthesis Screening: A Preliminary Real-world Experience.

OBJECTIVE: The purpose of this study is to assess the "real-world" impact of an artificial intelligence (AI) tool designed to detect breast cancer in digital breast tomosynthesis (DBT) screening exams following 12 months of utilization in a subspecialized academic breast center. METHODS: Following IRB approval, mammography audit reports, as specified in the BI-RADS atlas, were retrospectively generated for five radiologists reading at three locations during a 12-month time frame. One location had the AI tool (iCAD ProFound AI v2.0), and the other two locations did not. The co-primary endpoints were cancer detection rate (CDR) and abnormal interpretation rate (AIR). Secondary endpoints included positive predictive values (PPVs) for cancer among screenings with abnormal interpretations (PPV1) and for biopsies performed (PPV3). Odds ratios (OR) with two-sided 95% confidence intervals (CIs) summarized the impact of AI across radiologists using generalized estimating equations. RESULTS: Nonsignificant differences were observed in CDR, AIR, and PPVs. The CDR was 7.3 with AI and 5.9 without AI (OR 1.3, 95% CI: 0.9-1.7). The AIR was 11.7% with AI and 11.8% without AI (OR 1.0, 95% CI: 0.8-1.3). The PPV1 was 6.2% with AI and 5.0% without AI (OR 1.3, 95% CI: 0.97-1.7). The PPV3 was 33.3% with AI and 32.0% without AI (OR 1.1, 95% CI: 0.8-1.5). CONCLUSION: Although we are unable to show statistically significant changes in CDR and AIR outcomes in the two groups, the results are consistent with prior reader studies. There is a nonsignificant trend toward improvement in CDR with AI, without significant increases in AIR.

Improving Accuracy and Efficiency with Concurrent Use of Artificial Intelligence for Digital Breast Tomosynthesis.

PURPOSE: To evaluate the use of artificial intelligence (AI) to shorten digital breast tomosynthesis (DBT) reading time while maintaining or improving accuracy. MATERIALS AND METHODS: A deep learning AI system was developed to identify suspicious soft-tissue and calcified lesions in DBT images. A reader study compared the performance of 24 radiologists (13 of whom were breast subspecialists) reading 260 DBT examinations (including 65 cancer cases) both with and without AI. Readings occurred in two sessions separated by at least 4 weeks. Area under the receiver operating characteristic curve (AUC), reading time, sensitivity, specificity, and recall rate were evaluated with statistical methods for multireader, multicase studies. RESULTS: Radiologist performance for the detection of malignant lesions, measured by mean AUC, increased 0.057 with the use of AI (95% confidence interval [CI]: 0.028, 0.087; P < .01), from 0.795 without AI to 0.852 with AI. Reading time decreased 52.7% (95% CI: 41.8%, 61.5%; P < .01), from 64.1 seconds without to 30.4 seconds with AI. Sensitivity increased from 77.0% without AI to 85.0% with AI (8.0%; 95% CI: 2.6%, 13.4%; P < .01), specificity increased from 62.7% without to 69.6% with AI (6.9%; 95% CI: 3.0%, 10.8%; noninferiority P < .01), and recall rate for noncancers decreased from 38.0% without to 30.9% with AI (7.2%; 95% CI: 3.1%, 11.2%; noninferiority P < .01). CONCLUSION: The concurrent use of an accurate DBT AI system was found to improve cancer detection efficacy in a reader study that demonstrated increases in AUC, sensitivity, and specificity and a reduction in recall rate and reading time.© RSNA, 2019See also the commentary by Hsu and Hoyt in this issue.

Quality assurance and training procedures for computer-aided detection and diagnosis systems in clinical use.

Computer-aided detection/diagnosis (CAD) is increasingly used for decision support by clinicians for detection and interpretation of diseases. However, there are no quality assurance (QA) requirements for CAD in clinical use at present. QA of CAD is important so that end users can be made aware of changes in CAD performance both due to intentional or unintentional causes. In addition, end-user training is critical to prevent improper use of CAD, which could potentially result in lower overall clinical performance. Research on QA of CAD and user training are limited to date. The purpose of this paper is to bring attention to these issues, inform the readers of the opinions of the members of the American Association of Physicists in Medicine (AAPM) CAD subcommittee, and thus stimulate further discussion in the CAD community on these topics. The recommendations in this paper are intended to be work items for AAPM task groups that will be formed to address QA and user training issues on CAD in the future. The work items may serve as a framework for the discussion and eventual design of detailed QA and training procedures for physicists and users of CAD. Some of the recommendations are considered by the subcommittee to be reasonably easy and practical and can be implemented immediately by the end users; others are considered to be "best practice" approaches, which may require significant effort, additional tools, and proper training to implement. The eventual standardization of the requirements of QA procedures for CAD will have to be determined through consensus from members of the CAD community, and user training may require support of professional societies. It is expected that high-quality CAD and proper use of CAD could allow these systems to achieve their true potential, thus benefiting both the patients and the clinicians, and may bring about more widespread clinical use of CAD for many other diseases and applications. It is hoped that the awareness of the need for appropriate CAD QA and user training will stimulate new ideas and approaches for implementing such procedures efficiently and effectively as well as funding opportunities to fulfill such critical efforts.

Computer-aided detection system applied to full-field digital mammograms.

BACKGROUND: although mammography remains the mainstay for breast cancer screening, it is an imperfect examination with a sensitivity of 75-92% for breast cancer. Computer-aided detection (CAD) has been developed to improve mammographic detection of breast cancer. PURPOSE: to retrospectively estimate CAD sensitivity and false-positive rate with full-field digital mammograms (FFDMs). MATERIAL AND METHODS: CAD was used to evaluate 151 cases of ductal carcinoma in situ (DCIS) (n=48) and invasive breast cancer (n=103) detected with FFDM. Retrospectively, CAD sensitivity was estimated based on breast density, mammographic presentation, histopathology type, and lesion size. CAD false-positive rate was estimated with screening FFDMs from 200 women. RESULTS: CAD detected 93% (141/151) of cancer cases: 97% (28/29) in fatty breasts, 94% (81/86) in breasts containing scattered fibroglandular densities, 90% (28/31) in heterogeneously dense breasts, and 80% (4/5) in extremely dense breasts. CAD detected 98% (54/55) of cancers manifesting as calcifications, 89% (74/83) as masses, and 100% (13/13) as mixed masses and calcifications. CAD detected 92% (73/79) of invasive ductal carcinomas, 89% (8/9) of invasive lobular carcinomas, 93% (14/15) of other invasive carcinomas, and 96% (46/48) of DCIS. CAD sensitivity for cancers 1-10 mm was 87% (47/54); 11-20 mm, 99% (70/71); 21-30 mm, 86% (12/14); and larger than 30 mm, 100% (12/12). The CAD false-positive rate was 2.5 marks per case. CONCLUSION: CAD with FFDM showed a high sensitivity in identifying cancers manifesting as calcifications or masses. CAD sensitivity was maintained in small lesions (1-20 mm) and invasive lobular carcinomas, which have lower mammographic sensitivity.

Effect of computer-aided detection for CT colonography in a multireader, multicase trial.

PURPOSE: To assess the effect of using computer-aided detection (CAD) in second-read mode on readers' accuracy in interpreting computed tomographic (CT) colonographic images. MATERIALS AND METHODS: The contributing institutions performed the examinations under approval of their local institutional review board, with waiver of informed consent, for this HIPAA-compliant study. A cohort of 100 colonoscopy-proved cases was used: In 52 patients with findings positive for polyps, 74 polyps of 6 mm or larger were observed in 65 colonic segments; in 48 patients with findings negative for polyps, no polyps were found. Nineteen blinded readers interpreted each case at two different times, with and without the assistance of a commercial CAD system. The effect of CAD was assessed in segment-level and patient-level receiver operating characteristic (ROC) curve analyses. RESULTS: Thirteen (68%) of 19 readers demonstrated higher accuracy with CAD, as measured with the segment-level area under the ROC curve (AUC). The readers' average segment-level AUC with CAD (0.758) was significantly greater (P = .015) than the average AUC in the unassisted read (0.737). Readers' per-segment, per-patient, and per-polyp sensitivity for all polyps of 6 mm or larger was higher (P < .011, .007, .005, respectively) for readings with CAD compared with unassisted readings (0.517 versus 0.465, 0.521 versus 0.466, and 0.477 versus 0.422, respectively). Sensitivity for patients with at least one large polyp of 10 mm or larger was also higher (P < .047) with CAD than without (0.777 versus 0.743). Average reader sensitivity also improved with CAD by more than 0.08 for small adenomas. Use of CAD reduced specificity of readers by 0.025 (P = .05). CONCLUSION: Use of CAD resulted in a significant improvement in overall reader performance. CAD improves reader sensitivity when measured per segment, per patient, and per polyp for small polyps and adenomas and also reduces specificity by a small amount.

Detection of breast cancer with full-field digital mammography and computer-aided detection.

OBJECTIVE: The purpose of this study was to evaluate computer-aided detection (CAD) performance with full-field digital mammography (FFDM). MATERIALS AND METHODS: CAD (Second Look, version 7.2) was used to evaluate 123 cases of breast cancer detected with FFDM (Senographe DS). Retrospectively, CAD sensitivity was assessed using breast density, mammographic presentation, histopathology results, and lesion size. To determine the case-based false-positive rate, patients with four standard views per case were included in the study group. Eighteen unilateral mammography examinations with nonstandard views were excluded, resulting in a sample of 105 bilateral cases. RESULTS: CAD detected 115 (94%) of 123 cancer cases: six of six (100%) in fatty breasts, 63 of 66 (95%) in breasts containing scattered fibroglandular densities, 43 of 46 (93%) in heterogeneously dense breasts, and three of five (60%) in extremely dense breasts. CAD detected 93% (41/44) of cancers manifesting as calcifications, 92% (57/62) as masses, and 100% (17/17) as mixed masses and calcifications. CAD detected 94% of the invasive ductal carcinomas (n = 63), 100% of the invasive lobular carcinomas (n = 7), 91% of the other invasive carcinomas (n = 11), and 93% of the ductal carcinomas in situ (n = 42). CAD sensitivity for cancers 1-10 mm (n = 55) was 89%; 11-20 mm (n = 37), 97%; 21-30 mm (n = 16), 100%; and larger than 30 mm (n = 15), 93%. The CAD false-positive rate was 2.3 marks per four-image case. CONCLUSION: CAD with FFDM showed a high sensitivity in identifying cancers manifesting as calcifications and masses. Sensitivity was maintained in cancers with lower mammographic sensitivity, including invasive lobular carcinomas and small neoplasms (1-20 mm). CAD with FFDM should be effective in assisting radiologists with earlier detection of breast cancer. Future studies are needed to assess CAD accuracy in larger populations.

Computer-aided detection and diagnosis of breast cancer.

The use of computer-aided detection (CAD) with film or digital mammography is now widely regarded as the standard of practice in mammography and has been shown to increase the rate of breast cancer detection. There are inherent limitations in 2D mammography, and new technologies involving 2D and 3D imaging with X-rays, ultrasound, and MRI are in use or under investigation. CAD can aid in the reduction of oversight error for these modalities and has the potential to assist the physician in unifying the interpretation across alternative modalities. We believe the result will be improved sensitivity and specificity due to both improved detection and diagnosis.

Evaluation of breast cancer with a computer-aided detection system by mammographic appearance and histopathology.

BACKGROUND: The objective of this study was to evaluate the performance of a computer-aided detection (CAD) system for the detection of breast cancer, based on mammographic appearance and histopathology. METHODS: From 1000 consecutive screening mammograms from women with biopsy-proven breast carcinoma, 273 mammograms were selected randomly for retrospective evaluation by CAD. The sensitivity of the CAD system for breast cancer was assessed from the proportion of masses and microcalcifications detected. The corresponding tumor histopathologies also were evaluated. Normal mammograms (n = 155 patients) were used to determine the false-positive rate of the system. RESULTS: Of the 273 breast carcinomas, 149 appeared mammographically as masses, and 88 appeared as microcalcifications, including 36 carcinomas that presented as mixed lesions. The CAD system marked 125 of 149 masses correctly (84%), marked 86 of 88 microcalcifications correctly (98%), and marked 32 of 36 of mixed lesions correctly (89%.). The system showed a high sensitivity for the detection of ductal carcinoma in situ (95%; 73 of 77 lesions), invasive lobular carcinoma (95%; 18 of 19 lesions), invasive ductal carcinoma (85%; 125 of 147 lesions), and invasive mammary carcinoma (90%; 27 of 30 lesions). The highest CAD system sensitivity was for all invasive carcinomas that presented as microcalcifications (100%). On normal mammograms, there was an average of 1.3 false-positive CAD marks per image. CONCLUSIONS: The CAD system correctly marked a large majority of biopsy-proven breast cancers, with a greater sensitivity for lesions with microcalcifications and without significant impact of performance based on tumor histopathology. CAD was highly effective in detecting invasive lobular carcinoma (sensitivity, 95%) and ductal carcinoma in situ (sensitivity, 95%). CAD represents a useful tool for the detection of breast cancer.

A computer-aided detection system for the evaluation of breast cancer by mammographic appearance and lesion size.

OBJECTIVE: The purpose of our study was to evaluate the performance of a computer-aided detection (CAD) system in the detection of breast cancer based on mammographic appearance and lesion size. CONCLUSION: The CAD system correctly marked most biopsy-proven breast cancers, with a greater sensitivity for microcalcification than for mass lesions but with no significant difference in performance based on cancer size. CAD was highly effective in detecting even the smallest lesions, with a sensitivity of 92% for lesions of 5 mm or less. CAD is a useful tool for the detection of breast cancer.

Impact of breast density on computer-aided detection for breast cancer.

OBJECTIVE: Our aim was to determine whether breast density affects the performance of a computer-aided detection (CAD) system for the detection of breast cancer. MATERIALS AND METHODS: Nine hundred six sequential mammographically detected breast cancers and 147 normal screening mammograms from 18 facilities were classified by mammographic density. BI-RADS 1 and 2 density cases were classified as nondense breasts; BI-RADS 3 and 4 density cases were classified as dense breasts. Cancers were classified as either masses or microcalcifications. All mammograms from the cancer and normal cases were evaluated by the CAD system. The sensitivity and false-positive rates from CAD in dense and nondense breasts were evaluated and compared. RESULTS: Overall, 809 (89%) of 906 cancer cases were detected by CAD; 455/505 (90%) cancers in nondense breasts and 354/401 (88%) cancers in dense breasts were detected. CAD sensitivity was not affected by breast density (p=0.38). Across both breast density categories, 280/296 (95%) microcalcification cases and 529/610 (87%) mass cases were detected. One hundred fourteen (93%) of the 122 microcalcifications in nondense breasts and 166 (95%) of 174 microcalcifications in dense breasts were detected, showing that CAD sensitivity to microcalcifications is not dependent on breast density (p=0.46). Three hundred forty-one (89%) of 383 masses in nondense breasts, and 188 (83%) of 227 masses in dense breasts were detected-that is, CAD sensitivity to masses is affected by breast density (p=0.03). There were more false-positive marks on dense versus nondense mammograms (p=0.04). CONCLUSION: Breast density does not impact overall CAD detection of breast cancer. There is no statistically significant difference in breast cancer detection in dense and nondense breasts. However, the detection of breast cancer manifesting as masses is impacted by breast density. The false-positive rate is lower in nondense versus dense breasts. CAD may be particularly advantageous in patients with dense breasts, in which mammography is most challenging.

Determining efficacy of mammographic CAD systems.

Computer-aided detection (CAD) system sensitivity estimates without a radiologist in the loop are straightforward to measure but are extremely data dependent. The only relevant performance metric is improvement in CAD-assisted radiologist sensitivity. Unfortunately, this is difficult to accurately assess. Without a large study measuring the improvement in CAD-assisted radiologist sensitivity over the same cases, it is not possible to make valid comparisons between systems. As multiple CAD systems become commercially available, comparison issues need to be explored and resolved. Data from clinical trials of 2 systems are examined. Statistical hypothesis tests are applied to these data. Additionally, sensitivities of 2 systems are compared from an experiment testing over the same 120 cases. Even with large databases, there is not sufficient evidence to conclude performance differences exist between the 2 systems. It is prohibitively expensive to show conclusive sensitivity differences between commercially available mammographic CAD systems.