Integrative Data Augmentation with U-Net Segmentation Masks Improves Detection of Lymph Node Metastases in Breast Cancer Patients.

Deep learning models have potential to improve performance of automated computer-assisted diagnosis tools in digital histopathology and reduce subjectivity. The main objective of this study was to further improve diagnostic potential of convolutional neural networks (CNNs) in detection of lymph node metastasis in breast cancer patients by integrative augmentation of input images with multiple segmentation channels. For this retrospective study, we used the PatchCamelyon dataset, consisting of 327,680 histopathology images of lymph node sections from breast cancer. Images had labels for the presence or absence of metastatic tissue. In addition, we used four separate histopathology datasets with annotations for nucleus, mitosis, tubule, and epithelium to train four instances of U-net. Then our baseline model was trained with and without additional segmentation channels and their performances were compared. Integrated gradient was used to visualize model attribution. The model trained with concatenation/integration of original input plus four additional segmentation channels, which we refer to as ConcatNet, was superior (AUC 0.924) compared to baseline with or without augmentations (AUC 0.854; 0.884). Baseline model trained with one additional segmentation channel showed intermediate performance (AUC 0.870-0.895). ConcatNet had sensitivity of 82.0% and specificity of 87.8%, which was an improvement in performance over the baseline (sensitivity of 74.6%; specificity of 80.4%). Integrated gradients showed that models trained with additional segmentation channels had improved focus on particular areas of the image containing aberrant cells. Augmenting images with additional segmentation channels improved baseline model performance as well as its ability to focus on discrete areas of the image.

Identification of intrinsic imaging phenotypes for breast cancer tumors: preliminary associations with gene expression profiles.

PURPOSE: To present a method for identifying intrinsic imaging phenotypes in breast cancer tumors and to investigate their association with prognostic gene expression profiles. MATERIALS AND METHODS: The authors retrospectively analyzed dynamic contrast material-enhanced (DCE) magnetic resonance (MR) images of the breast in 56 women (mean age, 55.6 years; age range, 37-74 years) diagnosed with estrogen receptor-positive breast cancer between 2005 and 2010. The study was approved by the institutional review board and compliant with HIPAA. The requirement to obtain informed consent was waived. Primary tumors were assayed with a validated gene expression assay that provides a score for the likelihood of recurrence. A multiparametric imaging phenotype vector was extracted for each tumor by using quantitative morphologic, kinetic, and spatial heterogeneity features. Multivariate linear regression was performed to test associations between DCE MR imaging features and recurrence likelihood. To identify intrinsic imaging phenotypes, hierarchical clustering was performed on the extracted feature vectors. Multivariate logistic regression was used to classify tumors at high versus low or medium risk of recurrence. To determine the additional value of intrinsic phenotypes, the phenotype category was tested as an additional variable. Receiver operating characteristic analysis and the area under the receiver operating characteristic curve (Az) were used to assess classification performance. RESULTS: There was a moderate correlation (r = 0.71, R(2) = 0.50, P < .001) between DCE MR imaging features and the recurrence score. DCE MR imaging features were predictive of recurrence risk as determined by the surrogate assay, with an Az of 0.77 (P < .01). Four dominant imaging phenotypes were detected, with two including only low- and medium-risk tumors. When the phenotype category was used as an additional variable, the Az increased to 0.82 (P < .01). CONCLUSION: Intrinsic imaging phenotypes exist for breast cancer tumors and correlate with recurrence likelihood as determined with gene expression profiling. These imaging biomarkers could ultimately help guide treatment decisions.

Fourier Lucas-Kanade algorithm.

In this paper, we propose a framework for both gradient descent image and object alignment in the Fourier domain. Our method centers upon the classical Lucas & Kanade (LK) algorithm where we represent the source and template/model in the complex 2D Fourier domain rather than in the spatial 2D domain. We refer to our approach as the Fourier LK (FLK) algorithm. The FLK formulation is advantageous when one preprocesses the source image and template/model with a bank of filters (e.g., oriented edges, Gabor, etc.) as 1) it can handle substantial illumination variations, 2) the inefficient preprocessing filter bank step can be subsumed within the FLK algorithm as a sparse diagonal weighting matrix, 3) unlike traditional LK, the computational cost is invariant to the number of filters and as a result is far more efficient, and 4) this approach can be extended to the Inverse Compositional (IC) form of the LK algorithm where nearly all steps (including Fourier transform and filter bank preprocessing) can be precomputed, leading to an extremely efficient and robust approach to gradient descent image matching. Further, these computational savings translate to nonrigid object alignment tasks that are considered extensions of the LK algorithm, such as those found in Active Appearance Models (AAMs).

Reinterpreting the application of gabor filters as a manipulation of the margin in linear support vector machines.

Linear filters are ubiquitously used as a preprocessing step for many classification tasks in computer vision. In particular, applying Gabor filters followed by a classification stage, such as a support vector machine (SVM), is now common practice in computer vision applications like face identity and expression recognition. A fundamental problem occurs, however, with respect to the high dimensionality of the concatenated Gabor filter responses in terms of memory requirements and computational efficiency during training and testing. In this paper, we demonstrate how the preprocessing step of applying a bank of linear filters can be reinterpreted as manipulating the type of margin being maximized within the linear SVM. This new interpretation leads to sizable memory and computational advantages with respect to existing approaches. The reinterpreted formulation turns out to be independent of the number of filters, thereby allowing the examination of the feature spaces derived from arbitrarily large number of linear filters, a hitherto untestable prospect. Further, this new interpretation of filter banks gives new insights, other than the often cited biological motivations, into why the preprocessing of images with filter banks, like Gabor filters, improves classification performance.

The Painful Face - Pain Expression Recognition Using Active Appearance Models.

Pain is typically assessed by patient self-report. Self-reported pain, however, is difficult to interpret and may be impaired or in some circumstances (i.e., young children and the severely ill) not even possible. To circumvent these problems behavioral scientists have identified reliable and valid facial indicators of pain. Hitherto, these methods have required manual measurement by highly skilled human observers. In this paper we explore an approach for automatically recognizing acute pain without the need for human observers. Specifically, our study was restricted to automatically detecting pain in adult patients with rotator cuff injuries. The system employed video input of the patients as they moved their affected and unaffected shoulder. Two types of ground truth were considered. Sequence-level ground truth consisted of Likert-type ratings by skilled observers. Frame-level ground truth was calculated from presence/absence and intensity of facial actions previously associated with pain. Active appearance models (AAM) were used to decouple shape and appearance in the digitized face images. Support vector machines (SVM) were compared for several representations from the AAM and of ground truth of varying granularity. We explored two questions pertinent to the construction, design and development of automatic pain detection systems. First, at what level (i.e., sequence- or frame-level) should datasets be labeled in order to obtain satisfactory automatic pain detection performance? Second, how important is it, at both levels of labeling, that we non-rigidly register the face?