Predicting Patterns of Distant Metastasis in Breast Cancer Patients following Local Regional Therapy Using Machine Learning.

Up to 30% of breast cancer (BC) patients will develop distant metastases (DM), for which there is no cure. Here, statistical and machine learning (ML) models were developed to estimate the risk of site-specific DM following local-regional therapy. This retrospective study cohort included 175 patients diagnosed with invasive BC who later developed DM. Clinicopathological information was collected for analysis. Outcome variables were the first site of metastasis (brain, bone or visceral) and the time interval (months) to developing DM. Multivariate statistical analysis and ML-based multivariable gradient boosting machines identified factors associated with these outcomes. Machine learning models predicted the site of DM, demonstrating an area under the curve of 0.74, 0.75, and 0.73 for brain, bone and visceral sites, respectively. Overall, most patients (57%) developed bone metastases, with increased odds associated with estrogen receptor (ER) positivity. Human epidermal growth factor receptor-2 (HER2) positivity and non-anthracycline chemotherapy regimens were associated with a decreased risk of bone DM, while brain metastasis was associated with ER-negativity. Furthermore, non-anthracycline chemotherapy alone was a significant predictor of visceral metastasis. Here, clinicopathologic and treatment variables used in ML prediction models predict the first site of metastasis in BC. Further validation may guide focused patient-specific surveillance practices.

A hierarchical self-attention-guided deep learning framework to predict breast cancer response to chemotherapy using pre-treatment tumor biopsies.

BACKGROUND: Pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) has demonstrated a strong correlation to improved survival in breast cancer (BC) patients. However, pCR rates to NAC are less than 30%, depending on the BC subtype. Early prediction of NAC response would facilitate therapeutic modifications for individual patients, potentially improving overall treatment outcomes and patient survival. PURPOSE: This study, for the first time, proposes a hierarchical self-attention-guided deep learning framework to predict NAC response in breast cancer patients using digital histopathological images of pre-treatment biopsy specimens. METHODS: Digitized hematoxylin and eosin-stained slides of BC core needle biopsies were obtained from 207 patients treated with NAC, followed by surgery. The response to NAC for each patient was determined using the standard clinical and pathological criteria after surgery. The digital pathology images were processed through the proposed hierarchical framework consisting of patch-level and tumor-level processing modules followed by a patient-level response prediction component. A combination of convolutional layers and transformer self-attention blocks were utilized in the patch-level processing architecture to generate optimized feature maps. The feature maps were analyzed through two vision transformer architectures adapted for the tumor-level processing and the patient-level response prediction components. The feature map sequences for these transformer architectures were defined based on the patch positions within the tumor beds and the bed positions within the biopsy slide, respectively. A five-fold cross-validation at the patient level was applied on the training set (144 patients with 9430 annotated tumor beds and 1,559,784 patches) to train the models and optimize the hyperparameters. An unseen independent test set (63 patients with 3574 annotated tumor beds and 173,637 patches) was used to evaluate the framework. RESULTS: The obtained results on the test set showed an AUC of 0.89 and an F1-score of 90% for predicting pCR to NAC a priori by the proposed hierarchical framework. Similar frameworks with the patch-level, patch-level + tumor-level, and patch-level + patient-level processing components resulted in AUCs of 0.79, 0.81, and 0.84 and F1-scores of 86%, 87%, and 89%, respectively. CONCLUSIONS: The results demonstrate a high potential of the proposed hierarchical deep-learning methodology for analyzing digital pathology images of pre-treatment tumor biopsies to predict the pathological response of breast cancer to NAC.

A Cascaded Deep Learning Framework for Segmentation of Nuclei in Digital Histology Images.

Accurate segmentation of nuclei is an essential step in analysis of digital histology images for diagnostic and prognostic applications. Despite recent advances in automated frameworks for nuclei segmentation, this task is still challenging. Specifically, detecting small nuclei in large-scale histology images and delineating the border of touching nuclei accurately is a complicated task even for advanced deep neural networks. In this study, a cascaded deep learning framework is proposed to segment nuclei accurately in digitized microscopy images of histology slides. A U-Net based model with customized pixel-wised weighted loss function is adapted in the proposed framework, followed by a U-Net based model with VGG16 backbone and a soft Dice loss function. The model was pretrained on the Post-NAT-BRCA public dataset before training and independent evaluation on the MoNuSeg dataset. The cascaded model could outperform the other state-of-the-art models with an AJI of 0.72 and a F1-score of 0.83 on the MoNuSeg test set.

Quantitative digital histopathology and machine learning to predict pathological complete response to chemotherapy in breast cancer patients using pre-treatment tumor biopsies.

Complete pathological response (pCR) to neoadjuvant chemotherapy (NAC) is a prognostic factor for breast cancer (BC) patients and is correlated with improved survival. However, pCR rates are variable to standard NAC, depending on BC subtype. This study investigates quantitative digital histopathology coupled with machine learning (ML) to predict NAC response a priori. Clinicopathologic data and digitized slides of BC core needle biopsies were collected from 149 patients treated with NAC. The nuclei within the tumor regions were segmented on the histology images of biopsy samples using a weighted U-Net model. Five pathomic feature subsets were extracted from segmented digitized samples, including the morphological, intensity-based, texture, graph-based and wavelet features. Seven ML experiments were conducted with different feature sets to develop a prediction model of therapy response using a gradient boosting machine with decision trees. The models were trained and optimized using a five-fold cross validation on the training data and evaluated using an unseen independent test set. The prediction model developed with the best clinical features (tumor size, tumor grade, age, and ER, PR, HER2 status) demonstrated an area under the ROC curve (AUC) of 0.73. Various pathomic feature subsets resulted in models with AUCs in the range of 0.67 and 0.87, with the best results associated with the graph-based and wavelet features. The selected features among all subsets of the pathomic and clinicopathologic features included four wavelet and three graph-based features and no clinical features. The predictive model developed with these features outperformed the other models, with an AUC of 0.90, a sensitivity of 85% and a specificity of 82% on the independent test set. The results demonstrated the potential of quantitative digital histopathology features integrated with ML methods in predicting BC response to NAC. This study is a step forward towards precision oncology for BC patients to potentially guide future therapies.

A review and comparison of breast tumor cell nuclei segmentation performances using deep convolutional neural networks.

Breast cancer is currently the second most common cause of cancer-related death in women. Presently, the clinical benchmark in cancer diagnosis is tissue biopsy examination. However, the manual process of histopathological analysis is laborious, time-consuming, and limited by the quality of the specimen and the experience of the pathologist. This study's objective was to determine if deep convolutional neural networks can be trained, with transfer learning, on a set of histopathological images independent of breast tissue to segment tumor nuclei of the breast. Various deep convolutional neural networks were evaluated for the study, including U-Net, Mask R-CNN, and a novel network (GB U-Net). The networks were trained on a set of Hematoxylin and Eosin (H&E)-stained images of eight diverse types of tissues. GB U-Net demonstrated superior performance in segmenting sites of invasive diseases (AJI = 0.53, mAP = 0.39 & AJI = 0.54, mAP = 0.38), validated on two hold-out datasets exclusively containing breast tissue images of approximately 7,582 annotated cells. The results of the networks, trained on images independent of breast tissue, demonstrated that tumor nuclei of the breast could be accurately segmented.

Machine Learning Frameworks to Predict Neoadjuvant Chemotherapy Response in Breast Cancer Using Clinical and Pathological Features.

PURPOSE: Neoadjuvant chemotherapy (NAC) is used to treat locally advanced breast cancer (LABC) and high-risk early breast cancer (BC). Pathological complete response (pCR) has prognostic value depending on BC subtype. Rates of pCR, however, can be variable. Predictive modeling is desirable to help identify patients early who may have suboptimal NAC response. Here, we test and compare the predictive performances of machine learning (ML) prediction models to a standard statistical model, using clinical and pathological data. METHODS: Clinical and pathological variables were collected in 431 patients, including tumor size, patient demographics, histological characteristics, molecular status, and staging information. A standard multivariable logistic regression (MLR) was developed and compared with five ML models: k-nearest neighbor classifier, random forest (RF) classifier, naive Bayes algorithm, support vector machine, and multilayer perceptron model. Model performances were measured using a receiver operating characteristic (ROC) analysis and statistically compared. RESULTS: MLR predictors of NAC response included: estrogen receptor (ER) status, human epidermal growth factor-2 (HER2) status, tumor size, and Nottingham grade. The strongest MLR predictors of pCR included HER2+ versus HER2- BC (odds ratio [OR], 0.13; 95% CI, 0.07 to 0.23; P < .001) and Nottingham grade G3 versus G1-2 (G1-2: OR, 0.36; 95% CI, 0.20 to 0.65; P < .001). The area under the curve (AUC) for the MLR was AUC = 0.64. Among the various ML models, an RF classifier performed best, with an AUC = 0.88, sensitivity of 70.7%, and specificity of 84.6%, and included the following variables: menopausal status, ER status, HER2 status, Nottingham grade, tumor size, nodal status, and presence of inflammatory BC. CONCLUSION: Modeling performances varied between standard versus ML classification methods. RF ML classifiers demonstrated the best predictive performance among all models.

An Attention-Guided Deep Neural Network for Annotating Abnormalities in Chest X-ray Images: Visualization of Network Decision Basis.

Despite the potential of deep convolutional neural networks for classification of thorax diseases from chest X-ray images, this task is still challenging as it is categorized as a weakly supervised learning problem, and deep neural networks in general suffer from a lack of interpretability. In this paper, a deep convolutional neural network framework with recurrent attention mechanism was investigated to annotate abnormalities in chest X-ray images. A modified MobileNet architecture was adapted in the framework for classification and the prediction difference analysis method was utilized to visualize the basis of network's decision on each image. A long short-term memory network was utilized as the attention model to focus on relevant regions of each image for classification. The framework was evaluated on NIH chest X-ray dataset. The attention-guided model versus the model with no attention mechanism could annotate the images in an independent test set with an F1-score of 0.58 versus 0.46, and an AUC of 0.94 versus 0.73. The obtained results implied that the proposed attention-guided model could outperform the other methods investigated previously for annotating the same dataset.

Multi-scale segmentation in GBM treatment using diffusion tensor imaging.

Glioblastoma (GBM) is the commonest primary malignant brain tumor in adults, and despite advances in multi-modality therapy, the outlook for patients has changed little in the last 10 years. Local recurrence is the predominant pattern of treatment failure, hence improved local therapies (surgery and radiotherapy) are needed to improve patient outcomes. Currently segmentation of GBM for surgery or radiotherapy (RT) planning is labor intensive, especially for high-dimensional MR imaging methods that may provide more sensitive indicators of tumor phenotype. Automating processing and segmentation of these images will aid treatment planning. Diffusion tensor magnetic resonance imaging is a recently developed technique (DTI) that is exquisitely sensitive to the ordered diffusion of water in white matter tracts. Our group has shown that decomposition of the tensor information into the isotropic component (p - shown to represent tumor invasion) and the anisotropic component (q - shown to represent the tumor bulk) can provide valuable prognostic information regarding tumor infiltration and patient survival. However, tensor decomposition of DTI data is not commonly used for neurosurgery or radiotherapy treatment planning due to difficulties in segmenting the resultant image maps. For this reason, automated techniques for segmentation of tensor decomposition maps would have significant clinical utility. In this paper, we modified a well-established convolutional neural network architecture (CNN) for medical image segmentation and used it as an automatic multi-sequence GBM segmentation based on both DTI image maps (p and q maps) and conventional MRI sequences (T2-FLAIR and T1 weighted post contrast (T1c)). In this proof-of-concept work, we have used multiple MRI sequences, each with individually defined ground truths for better understanding of the contribution of each image sequence to the segmentation performance. The high accuracy and efficiency of our proposed model demonstrates the potential of utilizing diffusion MR images for target definition in precision radiation treatment planning and surgery in routine clinical practice.

Quantitative Thermal Imaging Biomarkers to Detect Acute Skin Toxicity From Breast Radiation Therapy Using Supervised Machine Learning.

PURPOSE: Radiation-induced dermatitis is a common side effect of breast radiation therapy (RT). Current methods to evaluate breast skin toxicity include clinical examination, visual inspection, and patient-reported symptoms. Physiological changes associated with radiation-induced dermatitis, such as inflammation, may also increase body-surface temperature, which can be detected by thermal imaging. Quantitative thermal imaging markers were identified and used in supervised machine learning to develop a predictive model for radiation dermatitis. METHODS AND MATERIALS: Ninety patients treated for adjuvant whole-breast RT (4250 cGy/fx = 16) were recruited for the study. Thermal images of the treated breast were taken at 4 intervals: before RT, then weekly at fx = 5, fx = 10, and fx = 15. Parametric thermograms were analyzed and yielded 26 thermal-based features that included surface temperature (°C) and texture parameters obtained from (1) gray-level co-occurrence matrix, (2) gray-level run-length matrix, and (3) neighborhood gray-tone difference matrix. Skin toxicity was evaluated at the end of RT using the Common Terminology Criteria for Adverse Events (CTCAE) guidelines (Ver.5). Binary group classes were labeled according to a CTCAE cut-off score of ≥2, and thermal features obtained at fx = 5 were used for supervised machine learning to predict skin toxicity. The data set was partitioned for model training, independent testing, and validation. Fifteen patients (∼17% of the whole data set) were randomly selected as an unseen test data set, and 75 patients (∼83% of the whole data set) were used for training and validation of the model. A random forest classifier with leave-1-patient-out cross-validation was employed for modeling single and hybrid parameters. The model performance was reported using receiver operating characteristic analysis on patients from an independent test set. RESULTS: Thirty-seven patients presented with adverse skin effects, denoted by a CTCAE score ≥2, and had significantly higher local increases in skin temperature, reaching 36.06°C at fx = 10 (P = .029). However, machine-learning models demonstrated early thermal signals associated with skin toxicity after the fifth RT fraction. The cross-validated model showed high prediction accuracy on the independent test data (test accuracy = 0.87) at fx = 5 for predicting skin toxicity at the end of RT. CONCLUSIONS: Early thermal markers after 5 fractions of RT are predictive of radiation-induced skin toxicity in breast RT.

Personalized Breast Cancer Treatments Using Artificial Intelligence in Radiomics and Pathomics.

Progress in computing power and advances in medical imaging over recent decades have culminated in new opportunities for artificial intelligence (AI), computer vision, and using radiomics to facilitate clinical decision-making. These opportunities are growing in medical specialties, such as radiology, pathology, and oncology. As medical imaging and pathology are becoming increasingly digitized, it is recently recognized that harnessing data from digital images can yield parameters that reflect the underlying biology and physiology of various malignancies. This greater understanding of the behaviour of cancer can potentially improve on therapeutic strategies. In addition, the use of AI is particularly appealing in oncology to facilitate the detection of malignancies, to predict the likelihood of tumor response to treatments, and to prognosticate the patients' risk of cancer-related mortality. AI will be critical for identifying candidate biomarkers from digital imaging and developing robust and reliable predictive models. These models will be used to personalize oncologic treatment strategies, and identify confounding variables that are related to the complex biology of tumors and diversity of patient-related factors (ie, mining "big data"). This commentary describes the growing body of work focussed on AI for precision oncology. Advances in AI-driven computer vision and machine learning are opening new pathways that can potentially impact patient outcomes through response-guided adaptive treatments and targeted therapies based on radiomic and pathomic analysis.