Impact of Original and Artificially Improved Artificial Intelligence-based Computer-aided Diagnosis on Breast US Interpretation.

OBJECTIVE: For breast US interpretation, to assess impact of computer-aided diagnosis (CADx) in original mode or with improved sensitivity or specificity. METHODS: In this IRB approved protocol, orthogonal-paired US images of 319 lesions identified on screening, including 88 (27.6%) cancers (median 7 mm, range 1-34 mm), were reviewed by 9 breast imaging radiologists. Each observer provided BI-RADS assessments (2, 3, 4A, 4B, 4C, 5) before and after CADx in a mode-balanced design: mode 1, original CADx (outputs benign, probably benign, suspicious, or malignant); mode 2, artificially-high-sensitivity CADx (benign or malignant); and mode 3, artificially-high-specificity CADx (benign or malignant). Area under the receiver operating characteristic curve (AUC) was estimated under each modality and for standalone CADx outputs. Multi-reader analysis accounted for inter-reader variability and correlation between same-lesion assessments. RESULTS: AUC of standalone CADx was 0.77 (95% CI: 0.72-0.83). For mode 1, average reader AUC was 0.82 (range 0.76-0.84) without CADx and not significantly changed with CADx. In high-sensitivity mode, all observers' AUCs increased: average AUC 0.83 (range 0.78-0.86) before CADx increased to 0.88 (range 0.84-0.90), P < 0.001. In high-specificity mode, all observers' AUCs increased: average AUC 0.82 (range 0.76-0.84) before CADx increased to 0.89 (range 0.87-0.92), P < 0.0001. Radiologists responded more frequently to malignant CADx cues in high-specificity mode (42.7% vs 23.2% mode 1, and 27.0% mode 2, P = 0.008). CONCLUSION: Original CADx did not substantially impact radiologists' interpretations. Radiologists showed improved performance and were more responsive when CADx produced fewer false-positive malignant cues.

Contrast Enhanced Digital Mammography (CEDM) Helps to Safely Reduce Benign Breast Biopsies for Low to Moderately Suspicious Soft Tissue Lesions.

RATIONALE AND OBJECTIVES: To preliminarily asses if Contrast Enhanced Digital Mammography (CEDM) can accurately reduce biopsy rates for soft tissue BI-RADS 4A or 4B lesions. MATERIALS AND METHODS: Eight radiologists retrospectively and independently reviewed 60 lesions in 54 consenting patients who underwent CEDM under Health Insurance Portability and Accountability Act compliant institutional review board-approved protocols. Readers provided Breast Imaging Reporting & Data System ratings sequentially for digital mammography/digital breast tomosynthesis (DM/DBT), then with ultrasound, then with CEDM for each lesion. Area under the curve (AUC), true positive rates and false positive rates, positive predictive values and negative predictive values were calculated. Statistical analysis accounting for correlation between lesion-examinations and between-reader variability was performed using OR/DBM (for SAS v.3.0), generalized linear mixed model for binary data (proc glimmix, SAS v.9.4, SAS Institute, Cary North Carolina), and bootstrap. RESULTS: The cohort included 49 benign, two high-risk and nine cancerous lesions in 54 women aged 34-74 (average 50) years. Reader-averaged AUC for CEDM was significantly higher than DM/DBT alone (0.85 versus 0.66, p < 0.001) or with US (0.85 versus 0.75, p = 0.001). CEDM increased true positive rates from 0.74 under DB/DBT, and 0.89 with US, to 0.90 with CEDM, (p = 0.019 DM/DBT versus CEDM, p = 0.78 DM/DBT + US versus CEDM) and decreased false positive rates from 0.47 using DM/DBT and 0.61 with US to 0.39 with CEDM (p = 0.017 DM/DBT versus CEDM, p = 0.001 DM/DBT+ US versus CEDM). For an expected cancer rate of 10%, CEDM positive predictive values was 20.5% (95% CI: 16%-27%) and negative predictive values 98.3% (95% CI: 96%-100%). CONCLUSION: Addition of CEDM for evaluation of low-moderate suspicion soft tissue breast lesions can substantially reduce biopsy of benign lesions without compromising cancer detection.

Ultrasound Guided Core Biopsy versus Fine Needle Aspiration for Evaluation of Axillary Lymphadenopathy in Patients with Breast Cancer.

Rationale and Objectives. To compare the sensitivities of ultrasound guided core biopsy and fine needle aspiration (FNA) for detection of axillary lymph node metastases in patients with a current diagnosis of ipsilateral breast cancer. Materials and Methods. From December 2008 to December 2010, 105 patients with breast cancer and abnormal appearing lymph nodes in the ipsilateral axilla consented to undergo FNA of an axillary node immediately followed by core biopsy of the same node, both with ultrasound guidance. Experienced pathologists evaluated the aspirate cytology without knowledge of the core histology. Cytology and core biopsy results were compared to sentinel node excision or axillary dissection pathology. Sensitivities were compared using McNemar's test. Results. Of 70 patients with axillary node metastases, FNA was positive in 55/70 (78.6%) and core was positive in 61/70 (87.1%) (P = 0.18). The FNA and core results were discordant in 14/70 (20%) patients. Ten cases were FNA negative/core positive. Four cases were FNA positive/core negative. Conclusion. Core biopsy detected six (8.6%) more cases of metastatic lymphadenopathy than FNA but the difference in sensitivities was not statistically significant. Core biopsy should be considered if the node is clearly imaged and readily accessible. FNA is a good alternative when a smaller needle is desired due to node location or other patient factors. This trial is registered with NCT01920139.

Time to diagnosis and performance levels during repeat interpretations of digital breast tomosynthesis: preliminary observations.

RATIONALE AND OBJECTIVES: To compare time to interpretation and diagnostic performance levels during repeat readings of full-field digital mammography (FFDM) and digital breast tomosynthesis (DBT) in a retrospective study. MATERIALS AND METHODS: Three experienced radiologists twice interpreted 125 selected examinations, 35 with verified cancers and 90 negative for cancer during a period of 22 months using FFDM alone followed by a combined FFDM + DBT mode. Changes in time to "review and rate" these examinations as well as in diagnostic performance levels where assessed. A fixed-effect analysis accounting for cross-correlation due to the review of the same examinations by the same readers was performed. RESULTS: The total (combined) time to review and rate an examination increased on average by 33% between the first and second readings of the same examinations (P < .001). Radiologists reduced their time to review FFDM before making the DBT available for viewing. However, they spent more time reviewing the combined FFDM + DBT mode. The recall rates for examinations depicting cancer remained largely unchanged. Among the groups of examinations with concordant and discordant recall recommendations during the two readings only the group examinations that were "newly recalled" during repeat reading, took significantly longer (P < .01). CONCLUSION: DBT-based breast imaging may ultimately result in a substantial increase in performance; however, without efficiency improvements DBT may take longer to interpret. Addition of "false-positive recalls" was most strongly associated with increase in interpretation time while elimination of "false-positive recalls" did not require longer interpretation time.

Digital breast tomosynthesis: observer performance study.

OBJECTIVE: The purpose of this study was to compare in a retrospective observer study the diagnostic performance of full-field digital mammography (FFDM) with that of digital breast tomosynthesis. MATERIALS AND METHODS: Eight experienced radiologists interpreted images from 125 selected examinations, 35 with verified findings of cancer and 90 with no finding of cancer. The four display conditions included FFDM alone, 11 low-dose projections, reconstructed digital breast tomosynthesis images, and a combined display mode of FFDM and digital breast tomosynthesis images. Observers rated examinations using the screening BI-RADS rating scale and the free-response receiver operating characteristic paradigm. Observer performance levels were measured as the proportion of examinations prompting recall of patients for further diagnostic evaluation. The results were presented in terms of true-positive fraction and false-positive fraction. Performance levels were compared among the acquisitions and reading modes. Time to view and interpret an examination also was evaluated. RESULTS: Use of the combination of digital breast tomosynthesis and FFDM was associated with 30% reduction in recall rate for cancer-free examinations that would have led to recall if FFDM had been used alone (p < 0.0001 for the participating radiologists, p = 0.047 in the context of a generalized population of radiologists). Use of digital breast tomosynthesis alone also tended to reduce recall rates, an average of 10%, although the observed decrease was not statistically significant (p = 0.09 for the participating radiologists). There was no convincing evidence that use of digital breast tomosynthesis alone or in combination with FFDM results in a substantial improvement in sensitivity. CONCLUSION: Use of digital breast tomosynthesis for breast imaging may result in a substantial decrease in recall rate.

Detection of breast abnormalities using a prototype resonance electrical impedance spectroscopy system: a preliminary study.

Electrical impedance spectroscopy has been investigated with but limited success as an adjunct procedure to mammography and as a possible pre-screening tool to stratify risk for having or developing breast cancer in younger women. In this study, the authors explored a new resonance frequency based [resonance electrical impedance spectroscopy (REIS)] approach to identify breasts that may have highly suspicious abnormalities that had been recommended for biopsies. The authors assembled a prototype REIS system generating multifrequency electrical sweeps ranging from 100 to 4100 kHz every 12 s. Using only two probes, one in contact with the nipple and the other with the outer breast skin surface 60 mm away, a paired transmission signal detection system is generated. The authors recruited 150 women between 30 and 50 years old to participate in this study. REIS measurements were performed on both breasts. Of these women 58 had been scheduled for a breast biopsy and 13 had been recalled for additional imaging procedures due to suspicious findings. The remaining 79 women had negative screening examinations. Eight REIS output signals at and around the resonance frequency were computed for each breast and the subtracted signals between the left and right breasts were used in a simple jackknifing method to select an optimal feature set to be inputted into a multi-feature based artificial neural network (ANN) that aims to predict whether a woman's breast had been determined as abnormal (warranting a biopsy) or not. The classification performance was evaluated using a leave-one-case-out method and receiver operating characteristics (ROC) analysis. The study shows that REIS examination is easy to perform, short in duration, and acceptable to all participants in terms of comfort level and there is no indication of sensation of an electrical current during the measurements. Six REIS difference features were selected as input signals to the ANN. The area under the ROC curve (A(z)) was 0.707 +/- 0.033 for classifying between biopsy cases and non-biopsy (including recalled and screening negative) and the performance (A(z)) increased to 0.746 +/- 0.033 after excluding recalled but negative cases. At 95% specificity, the sensitivity levels were approximately 20.5% and 30.4% in the two data sets tested. The results suggest that differences in REIS signals between two breasts measured in and around the tissue resonance frequency can be used to identify at least some of the women with suspicious abnormalities warranting biopsy with high specificity.

Digital breast tomosynthesis: a pilot observer study.

OBJECTIVE: The objective of our study was to assess ergonomic and diagnostic performance-related issues associated with the interpretation of digital breast tomosynthesis-generated examinations. MATERIALS AND METHODS: Thirty selected cases were read under three different display conditions by nine experienced radiologists in a fully crossed, mode-balanced observer performance study. The reading modes included full-field digital mammography (FFDM) alone, the 11 low-dose projections acquired for the reconstruction of tomosynthesis images, and the reconstructed digital breast tomosynthesis examination. Observers rated cases under the free-response receiver operating characteristic, as well as a screening paradigm, and provided subjective assessments of the relative diagnostic value of the two digital breast tomosynthesis-based image sets as compared with FFDM. The time to review and diagnose each case was also evaluated. RESULTS: Observer performance measures were not statistically significant (p > 0.05) primarily because of the small sample size in this pilot study, suggesting that showing significant improvements in diagnosis, if any, will require a larger study. Several radiologists did perceive the digital breast tomosynthesis image set and the projection series to be better than FFDM (p < 0.05) for diagnosing this specific case set. The time to review, interpret, and rate the examinations was significantly different for the techniques in question (p < 0.05). CONCLUSION: Tomosynthesis-based breast imaging may have great potential, but much work is needed before its optimal role in the clinical environment is known.

Interactive computer-aided diagnosis of breast masses: computerized selection of visually similar image sets from a reference library.

RATIONALE AND OBJECTIVES: The clinical utility of interactive computer-aided diagnosis (ICAD) systems depends on clinical relevance and visual similarity between the queried breast lesions and the ICAD-selected reference regions. The objective of this study is to develop and test a new ICAD scheme that aims improve visual similarity of ICAD-selected reference regions. MATERIALS AND METHODS: A large and diverse reference library involving 3,000 regions of interests was established. For each queried breast mass lesion by the observer, the ICAD scheme segments the lesion, classifies its boundary spiculation level, and computes 14 image features representing the segmented lesion and its surrounding tissue background. A conditioned k-nearest neighbor algorithm is applied to select a set of the 25 most "similar" lesions from the reference library. After computing the mutual information between the queried lesion and each of these initially selected 25 lesions, the scheme displays the six reference lesions with the highest mutual information scores. To evaluate the automated selection process of the six "visually similar" lesions to the queried lesion, we conducted a two-alternative forced-choice observer preference study using 85 queried mass lesions. Two sets of reference lesions selected by one new automated ICAD scheme and the other previously reported scheme using a subjective rating method were randomly displayed on the left and right side of the queried lesion. Nine observers were asked to decide for each of the 85 queried lesions which one of the two reference sets was "more visually similar" to the queried lesion. RESULTS: In classification of mass boundary spiculation levels, the overall agreement rate between the automated scheme and an observer is 58.8% (Kappa = 0.31). In observer preference study, the nine observers preferred on average the reference lesion sets selected by the automated scheme as being more visually similar than the set selected by the subjective rating approach in 53.2% of the queried lesions. The results were not significantly different for the two methods (P = .128). CONCLUSIONS: This study suggests that using the new automated ICAD scheme, the interobserver variability related issues can thus be avoided. Furthermore, the new scheme maintains the similar performance level as the previous scheme using the subjective rating method that can select reference sets that are significantly more visually similar (P < .05) than when using traditional ICAD schemes in which the mass boundary spiculation levels are not accurately detected and quantified.

Multiview-based computer-aided detection scheme for breast masses.

In this study, we developed and tested a new multiview-based computer-aided detection (CAD) scheme that aims to maintain the same case-based sensitivity level as a single-image-based scheme while substantially increasing the number of masses being detected on both ipsilateral views. An image database of 450 four-view examinations (1800 images) was assembled. In this database, 250 cases depicted malignant masses, of which 236 masses were visible on both views and 14 masses were visible only on one view. First, we detected suspected mass regions depicted on each image in the database using a single-image-based CAD. For each identified region (with detection score > or = 0.55), we then identified a matching strip of interest on the ipsilateral view based on the projected distance to the nipple along the centerline. By lowering CAD operating threshold inside the matching strip, we searched for a region located inside the strip and paired it with the original region. A multifeature-based artificial neural network scored the likelihood of the paired "matched" regions representing true-positive masses. All single (unmatched) regions except for those either with very high detection scores (> or = 0.85) or those located near the chest wall that cannot be matched on the other view were discarded. The original single-image-based CAD scheme detected 186 masses (74.4% case-based sensitivity) and 593 false-positive regions. Of the 186 identified masses, 91 were detected on two views (48.9%) and 95 were detected only on one view (51.1%). Of the false-positive detections, 54 were paired on the ipsilateral view inside the corresponding matching strips and the remaining 485 were not, which represented 539 case-based false-positive detections (0.3 per image). Applying the multiview-based CAD scheme, the same case-based sensitivity was maintained while cueing 169 of 186 masses (90.9%) on both views and at the same time reducing the case-based false-positive detection rate by 23.7% (from 539 to 411). The study demonstrated that the new multiview-based CAD scheme could substantially increase the number of masses being cued on two ipsilateral views while reducing the case-based false-positive detection rate.

Head-mounted versus remote eye tracking of radiologists searching for breast cancer: a comparison.

PURPOSE: We compared performance and visual search parameters of radiologists detecting masses on mammograms by using both a head-mounted (HDMT) and a remote (REM) eye tracker. MATERIALS AND METHODS: Five experienced radiologists read twice a case set of 20 one-view (medial-lateral oblique) mammograms, of which 12 contained a malignant mass and eight were lesion-free. For each observer, one trial used an HDMT eye-tracking system and the other used an REM system. Trials were separated on average by 2 months. Time to hit the location of the mass, dwell, and number of fixations in the location of the mass were measured. The same parameters were measured on a per-trial basis to determine whether there were memory effects from the previous trial. RESULTS: Dwell times in the location of true-positive, false-positive, and false-negative results were significantly shorter (P < .05) using the HDMT (median, 0.395 seconds) than REM (median, 0.482 seconds) systems, but the number of fixations in the location of the response was smaller using the REM system (median, 4.33 versus 5.0 for the HDMT). The observed differences did not seem to be caused by a memory effect. In addition, the relative lack of head mobility using the REM system caused observers to report neck strain. CONCLUSION: Overall, radiologists' visual search behavior was very similar using both types of eye-tracking device. However, because the REM system did not contain a magnetic head tracker, radiologists were allowed very limited head movements when using it, which made them uncomfortable during the experiment.

Performance and reproducibility of a computerized mass detection scheme for digitized mammography using rotated and resampled images: an assessment.

OBJECTIVE: Our objective was to compare the performance and reproducibility of a computer-aided detection (CAD) scheme that uses multiple rotated and resampled images with an in-house-developed CAD scheme (single-image-based) and a commercial CAD product in detecting masses depicted on digitized mammograms. MATERIALS AND METHODS: Ninety-two film mammograms (acquired from 23 patients) were selected. Forty-four mass regions associated with malignancy were visually identified. A commercial CAD system was used to scan and process each image four times, for a total of 368 digitized images depicting 176 mass regions. Images were processed using two CAD schemes developed in our laboratory. One uses the detection results generated from a single image, and the other averages five detection scores generated after processing the originally digitized image and four slightly rotated and resampled images. A region-based analysis was used to compare reproducibility and performance levels among the two in-house schemes and the commercial system. RESULTS: The commercial system detected a total of 98 mass regions (55.7% sensitivity) and 136 false-positive regions (an average of 0.37 per image). Among the detected mass regions, 76 represented 19 regions that were detected on all four scans and 22 represented 10 regions that were not fully reproducible. Eighty-eight false-positive detections represented 22 reproducible detections on all four scans. Our single-image-based scheme identified 87 mass regions and 160 false-positive regions. Seventeen mass regions and 28 false-positive regions were detected on all four scans. The multiple-image-based scheme identified 98 mass regions and 132 false-positive regions. Twenty-three mass regions were detected on all four scans. One hundred twelve of the 132 false-positive regions represented 28 reproducible detections. CONCLUSION: Averaging detection scores from multiple rotated and resampled images generated from a single digitization of a film can reduce variations in detection scores. Our multiple-image-based scheme improved both performance and reproducibility over the single-image-based scheme. The multiple-image-based scheme yielded an overall performance comparable to that of the commercial system but with improved reproducibility.

Trends in recall, biopsy, and positive biopsy rates for screening mammography in an academic practice.

PURPOSE: To retrospectively evaluate whether recall, biopsy, and positive biopsy rates for a group of radiologists who met requirements of Mammography Quality Standards Act of 1992 (MQSA) demonstrated any change over time during a 27-month period (nine consecutive calendar quarters). MATERIALS AND METHODS: Institutional review board approved study protocol, and informed consent was waived. All screening mammograms that had been interpreted by MQSA-qualified radiologists between January 1, 2001, and March 31, 2003, were reviewed. Group recall rates, biopsy rates, and detected cancer rates for nine calendar quarters were computed and attributed to performance date of original screening mammogram. Type of biopsy performed was classified as follows: stereotactic vacuum-assisted biopsy, ultrasonography (US)-guided core biopsy, US-guided fine-needle aspiration biopsy, surgical excision, and multiple biopsies. chi(2) Test for trend (two sided) and linear regression were used to assess trends over time for recall and biopsy rates, biopsy rates according to type of biopsy performed, and percentage of biopsy results positive for cancer. RESULTS: Group recall rate did not show a statistically significant trend during period studied (P = .59). Biopsy rates increased significantly from 13.02 to 20.12 per 1000 screening examinations (P < .001). A corresponding substantial decrease was seen in percentage of biopsies in which malignancy was found, although this trend was not statistically significant (P = .24). A significant increase (from 4.72 to 9.88 per 1000 screening examinations) was found in rate of stereotactic vacuum-assisted 11-gauge core biopsies performed (P < .001). CONCLUSION: Observed increase in biopsy rates reinforces the need to carefully select patients for biopsy to achieve efficient, efficacious, and cost-effective programs for early detection of breast cancers.

Computer-aided detection schemes: the effect of limiting the number of cued regions in each case.

OBJECTIVE: We assessed performance changes of a mammographic computer-aided detection scheme when we restricted the maximum number of regions that could be identified (cued) as showing positive findings in each case. MATERIALS AND METHODS: A computer-aided detection scheme was applied to 500 cases (or 2,000 images), including 300 cases in which mammograms showed verified malignant masses. We evaluated the overall case-based performance of the scheme using a free-response receiver operating characteristic approach, and we measured detection sensitivity at a fixed false-positive detection rate of 0.4 per image after gradually reducing the maximum number of cued regions allowed for each case from seven to one. RESULTS: The original computer-aided detection scheme achieved a maximum case-based sensitivity of 97% at 3.3 false-positive detected regions per image. For a detection decision score set at 0.565, the scheme had a 79% (237/300) case-based sensitivity, with 0.4 false-positive detected regions per image. After limiting the number of maximum allowed cued regions per case, the false-positive rates decreased faster than the true-positive rates. At a maximum of two cued regions per case, the false-positive rate decreased from 0.4 to 0.21 per image, whereas detection sensitivity decreased from 237 to 220 masses. To maintain sensitivity at 79%, we reduced the detection decision score to as low as 0.36, which resulted in a reduction of false-positive detected regions from 0.4 to 0.3 per image and a reduction in region-based sensitivity from 66.1% to 61.4%. CONCLUSION: Limiting the maximum number of cued regions per case can improve the overall case-based performance of computer-aided detection schemes in mammography.