Computer aid for decision to biopsy breast masses on mammography: validation on new cases.

RATIONALE AND OBJECTIVES: The purpose of this study was to validate the performance of a previously developed computer aid for breast mass classification for mammography on a new, independent database of cases not used for algorithm development. MATERIALS AND METHODS: A computer aid (classifier) based on the likelihood ratio (LRb) was previously developed on a database of 670 mass cases. The 670 cases (245 malignant) from one medical institution were described using 16 features from the American College of Radiology Breast Imaging-Reporting and Data System lexicon and patient history findings. A separate database of 151 (43 malignant) validation cases were collected that were previously unseen by the classifier. These new validation cases were evaluated by the classifier without retraining. Performance evaluation methods included Receiver Operating Characteristic (ROC), round-robin, and leave-one-out bootstrap sampling. RESULTS: The performance of the classifier on the training data yielded an average ROC area of 0.90 +/- 0.02 and partial ROC area (0.90AUC) of 0.60 +/- 0.06. The exact nonparametric performance on the validation set of 151 cases yielded a ROC area of 0.88 and 0.90AUC of 0.57. Using a 100% sensitivity cutoff threshold established on the training data (100% negative predictive value), the classifier correctly identified 100% of the malignant masses in the validation test set, while potentially obviating 26% of the biopsies performed on benign masses. CONCLUSION: The LRb classifier performed consistently on new data that was not used for classifier development. The LRb classifier shows promise as a potential aid in reducing the number of biopsies performed on benign masses.

Tolerance to missing data using a likelihood ratio based classifier for computer-aided classification of breast cancer.

While mammography is a highly sensitive method for detecting breast tumours, its ability to differentiate between malignant and benign lesions is low, which may result in as many as 70% of unnecessary biopsies. The purpose of this study was to develop a highly specific computer-aided diagnosis algorithm to improve classification of mammographic masses. A classifier based on the likelihood ratio was developed to accommodate cases with missing data. Data for development included 671 biopsy cases (245 malignant), with biopsy-proved outcome. Sixteen features based on the BI-RADS lexicon and patient history had been recorded for the cases, with 1.3 +/- 1.1 missing feature values per case. Classifier evaluation methods included receiver operating characteristic and leave-one-out bootstrap sampling. The classifier achieved 32% specificity at 100% sensitivity on the 671 cases with 16 features that had missing values. Utilizing just the seven features present for all cases resulted in decreased performance at 100% sensitivity with average 19% specificity. No cases and no feature data were omitted during classifier development, showing that it is more beneficial to utilize cases with missing values than to discard incomplete cases that cannot be handled by many algorithms. Classification of mammographic masses was commendable at high sensitivity levels, indicating that benign cases could be potentially spared from biopsy.

Application of likelihood ratio to classification of mammographic masses; performance comparison to case-based reasoning.

The likelihood ratio (LR) is an optimal approach for deciding which of two alternate hypotheses best describes a given situation. We adopted this formalism for predicting whether biopsy results of mammographic masses will be benign or malignant, aiming to reduce the number of biopsies performed on benign lesions. We compared the performance of this LR-based algorithm (LRb) to a case-based reasoning (CBR) classifier, which provides a solution to a new problem using past similiar cases. Each classifier used mammographers' BI-RADS descriptions of mammographic masses as input. The database consisted of 646 biopsy-proven mammography cases. Performance was evaluated using Receiver Operating Characteristic (ROC) analysis, Round Robin sampling, and bootstrap. The ROC areas (AUC) for the LRb and CBR were 0.91+/- 0.01 and 0.92 +/- 0.01, respectively. The partial ROC area index (0.90AUC) was the same for both classifiers, 0.59 +/- 0.05. At a sensitivity of 98%, the CBR would spare 204 (49%) of benign lesions from biopsy; the LRb would spare 209 (51%) benign lesions. The performance of the two classifiers was very similar, with no statistical differences in AUC or 0.90AUC. Although the CBR and LRb originate from different fields of study, their implementations differ only in the estimation of the probability density functions (PDFs) of the feature distributions. The CBR performs this estimation implicitly, while using various similarity metrics. On the other hand, the estimation of the PDFs is specified explicitly in the LRb implementation. This difference in the estimation of the PDFs results in the very small difference in performance, and at 98% sensitivity, both classifiers would spare about half of the benign mammographic masses from biopsy. The CBR and LRb are equivalent methods in implementation and performance.

Development and evaluation of a case-based reasoning classifier for prediction of breast biopsy outcome with BI-RADS lexicon.

Approximately 70-85% of breast biopsies are performed on benign lesions. To reduce this high number of biopsies performed on benign lesions, a case-based reasoning (CBR) classifier was developed to predict biopsy results from BI-RADS findings. We used 1433 (931 benign) biopsy-proven mammographic cases. CBR similarity was defined using either the Hamming or Euclidean distance measure over case features. Ten features represented each case: calcification distribution, calcification morphology, calcification number, mass margin, mass shape, mass density, mass size, associated findings, special cases, and age. Performance was evaluated using Round Robin sampling, Receiver Operating Characteristic (ROC) analysis, and bootstrap. To determine the most influential features for the CBR, an exhaustive feature search was performed over all possible feature combinations (1022) and similarity thresholds. Influential features were defined as the most frequently occurring features in the feature subsets with the highest partial ROC areas (0.90AUC). For CBR with Hamming distance, the most influential features were found to be mass margin, calcification morphology, age, calcification distribution, calcification number, and mass shape, resulting in an 0.90AUC of 0.33. At 95% sensitivity, the Hamming CBR would spare from biopsy 34% of the benign lesions. At 98% sensitivity, the Hamming CBR would spare 27% benign lesions. For the CBR with Euclidean distance, the most influential feature subset consisted of mass margin, calcification morphology, age, mass density, and associated findings, resulting in 0.90AUC of 0.37. At 95% sensitivity, the Euclidean CBR would spare from biopsy 41% benign lesions. At 98% sensitivity, the Euclidean CBR would spare 27% benign lesions. The profile of cases spared by both distance measures at 98% sensitivity indicates that the CBR is a potentially useful diagnostic tool for the classification of mammographic lesions, by recommending short-term follow-up for likely benign lesions that is in agreement with final biopsy results and mammographer's intuition.