Deep Learning of Rhabdomyosarcoma Pathology Images for Classification and Survival Outcome Prediction.

Rhabdomyosarcoma (RMS), the most common malignant soft tissue tumor in children, has several histologic subtypes that influence treatment and predict patient outcomes. Assistance with histologic classification for pathologists as well as discovery of optimized predictive biomarkers is needed. A convolutional neural network for RMS histology subtype classification was developed using digitized pathology images from 80 patients collected at time of diagnosis. A subsequent embryonal rhabdomyosarcoma (eRMS) prognostic model was also developed in a cohort of 60 eRMS patients. The RMS classification model reached a performance of an area under the receiver operating curve of 0.94 for alveolar rhabdomyosarcoma and an area under the receiver operating curve of 0.92 for eRMS at slide level in the test data set (n = 192). The eRMS prognosis model separated the patients into predicted high- and low-risk groups with significantly different event-free survival outcome (likelihood ratio test; P = 0.02) in the test data set (n = 136). The predicted risk group is significantly associated with patient event-free survival outcome after adjusting for patient age and sex (predicted high- versus low-risk group hazard ratio, 4.64; 95% CI, 1.05-20.57; P = 0.04). This is the first comprehensive study to develop computational algorithms for subtype classification and prognosis prediction for RMS histopathology images. Such models can aid pathology evaluation and provide additional parameters for risk stratification.

Correlation of histopathology and multi-modal magnetic resonance imaging in childhood osteosarcoma: Predicting tumor response to chemotherapy.

BACKGROUND: Osteosarcoma, which is the most common malignant pediatric bone cancer, remains dependent on an imprecise systemic treatment largely unchanged in 30 years. In this study, we correlated histopathology with magnetic resonance imaging (MRI), used the correlation to extract MRI-specific features representative of tumor necrosis, and subsequently developed a novel classification model for predicting tumor response to neoadjuvant chemotherapy in pediatric patients with osteosarcoma using multi-modal MRI. The model could ultimately serve as a testable biomarker for a high-risk malignancy without successful precision treatments. METHODS: Patients with newly diagnosed high-grade appendicular osteosarcoma were enrolled in a single-center observational study, wherein patients underwent pre-surgical evaluation using both conventional MRI (post-contrast T1-weighted with fat saturation, pre-contrast T1-weighted, and short inversion-time inversion recovery (STIR)) and advanced MRI (diffusion weighted (DW) and dynamic contrast enhanced (DCE)). A classification model was established based on a direct correlation between histopathology and MRI, which was achieved through histologic-MR image co-registration and subsequent extraction of MR image features for identifying histologic tumor necrosis. By operating on the MR image features, tumor necrosis was estimated from different combinations of MR images using a multi-feature fuzzy clustering technique together with a weighted majority ruling. Tumor necrosis calculated from MR images, for either an MRI plane of interest or whole tumor volume, was compared to pathologist-estimated necrosis and necrosis quantified from digitized histologic section images using a previously described deep learning classification method. RESULTS: 15 patients were enrolled, of whom two withdrew, one became ineligible, and two were subjected to inadequate pre-surgical imaging. MRI sequences of n = 10 patients were subsequently used for classification model development. Different MR image features, depending on the modality of MRI, were shown to be significant in distinguishing necrosis from viable tumor. The scales at which MR image features optimally signified tumor necrosis were different as well depending on the MR image type. Conventional MRI was shown capable of differentiating necrosis from viable tumor with an accuracy averaging above 90%. Conventional MRI was equally effective as DWI in distinguishing necrotic from viable tumor regions. The accuracy of tumor necrosis prediction by conventional MRI improved to above 95% when DCE-MRI was added into consideration. Volume-based tumor necrosis estimations tended to be lower than those evaluated on an MRI plane of interest. CONCLUSIONS: The study has shown a proof-of-principle model for interpreting chemotherapeutic response using multi-modal MRI for patients with high-grade osteosarcoma. The model will continue to be evaluated as MR image features indicative of tumor response are now computable for the disease prior to surgery.

A Role for Artificial Intelligence in the Classification of Craniofacial Anomalies.

ABSTRACT: Development of an objective algorithm to diagnose and assess craniofacial conditions has the potential to facilitate early diagnosis, especially for care providers with limited craniofacial expertise. Deep learning, a branch of artificial intelligence, can automatically analyze and categorize disease without human assistance. Convolutional neural networks (CNN) have excelled in utilizing medical images to automatically classify disease. In this study, the authors developed CNN models to detect and classify non-syndromic craniosynostosis (CS) using 2D images. The authors created an annotated data set of labeled CS (normal, metopic, sagittal, and unicoronal) conditions using standard clinical photography from the image repository at our center. The authors extended this dataset set by adding photographic images of children with craniofacial conditions from the internet. A total of 1076 images were used in this study. The authors developed a CNN model using a pre-trained ResNet-50 model to classify the data as metopic, sagittal, and unicoronal. The testing accuracy for the CS ResNet50 model achieved an overall testing accuracy of 90.6%. The sensitivity and precision were: 100% and 100% for metopic, 93.3% and 100% for sagittal, and 66.7% and 100% for unicoronal, respectively. The CNN model performed with promising accuracy. These results support the idea that deep learning has a role in diagnosis of craniofacial conditions. Using standard 2D clinical photography, such systems can provide automated screening and detection of these conditions. In the future, ML may be applied to prediction and assessment of surgical outcomes, or as an open-source remote diagnostic resource.

Viable and necrotic tumor assessment from whole slide images of osteosarcoma using machine-learning and deep-learning models.

Pathological estimation of tumor necrosis after chemotherapy is essential for patients with osteosarcoma. This study reports the first fully automated tool to assess viable and necrotic tumor in osteosarcoma, employing advances in histopathology digitization and automated learning. We selected 40 digitized whole slide images representing the heterogeneity of osteosarcoma and chemotherapy response. With the goal of labeling the diverse regions of the digitized tissue into viable tumor, necrotic tumor, and non-tumor, we trained 13 machine-learning models and selected the top performing one (a Support Vector Machine) based on reported accuracy. We also developed a deep-learning architecture and trained it on the same data set. We computed the receiver-operator characteristic for discrimination of non-tumor from tumor followed by conditional discrimination of necrotic from viable tumor and found our models performing exceptionally well. We then used the trained models to identify regions of interest on image-tiles generated from test whole slide images. The classification output is visualized as a tumor-prediction map, displaying the extent of viable and necrotic tumor in the slide image. Thus, we lay the foundation for a complete tumor assessment pipeline from original histology images to tumor-prediction map generation. The proposed pipeline can also be adopted for other types of tumor.

Convolutional Neural Network for Histopathological Analysis of Osteosarcoma.

Pathologists often deal with high complexity and sometimes disagreement over osteosarcoma tumor classification due to cellular heterogeneity in the dataset. Segmentation and classification of histology tissue in H&E stained tumor image datasets is a challenging task because of intra-class variations, inter-class similarity, crowded context, and noisy data. In recent years, deep learning approaches have led to encouraging results in breast cancer and prostate cancer analysis. In this article, we propose convolutional neural network (CNN) as a tool to improve efficiency and accuracy of osteosarcoma tumor classification into tumor classes (viable tumor, necrosis) versus nontumor. The proposed CNN architecture contains eight learned layers: three sets of stacked two convolutional layers interspersed with max pooling layers for feature extraction and two fully connected layers with data augmentation strategies to boost performance. The use of a neural network results in higher accuracy of average 92% for the classification. We compare the proposed architecture with three existing and proven CNN architectures for image classification: AlexNet, LeNet, and VGGNet. We also provide a pipeline to calculate percentage necrosis in a given whole slide image. We conclude that the use of neural networks can assure both high accuracy and efficiency in osteosarcoma classification.

COMPUTER AIDED IMAGE SEGMENTATION AND CLASSIFICATION FOR VIABLE AND NON-VIABLE TUMOR IDENTIFICATION IN OSTEOSARCOMA.

Osteosarcoma is one of the most common types of bone cancer in children. To gauge the extent of cancer treatment response in the patient after surgical resection, the H&E stained image slides are manually evaluated by pathologists to estimate the percentage of necrosis, a time consuming process prone to observer bias and inaccuracy. Digital image analysis is a potential method to automate this process, thus saving time and providing a more accurate evaluation. The slides are scanned in Aperio Scanscope, converted to digital Whole Slide Images (WSIs) and stored in SVS format. These are high resolution images, of the order of 109 pixels, allowing up to 40X magnification factor. This paper proposes an image segmentation and analysis technique for segmenting tumor and non-tumor regions in histopathological WSIs of osteosarcoma datasets. Our approach is a combination of pixel-based and object-based methods which utilize tumor properties such as nuclei cluster, density, and circularity to classify tumor regions as viable and non-viable. A K-Means clustering technique is used for tumor isolation using color normalization, followed by multi-threshold Otsu segmentation technique to further classify tumor region as viable and non-viable. Then a Flood-fill algorithm is applied to cluster similar pixels into cellular objects and compute cluster data for further analysis of regions under study. To the best of our knowledge this is the first comprehensive solution that is able to produce such a classification for Osteosarcoma cancer. The results are very conclusive in identifying viable and non-viable tumor regions. In our experiments, the accuracy of the discussed approach is 100% in viable tumor and coagulative necrosis identification while it is around 90% for fibrosis and acellular/hypocellular tumor osteoid, for all the sampled datasets used. We expect the developed software to lead to a significant increase in accuracy and decrease in inter-observer variability in assessment of necrosis by the pathologists and a reduction in the time spent by the pathologists in such assessments.

Minireview: Multiomic candidate biomarkers for clinical manifestations of sickle cell severity: Early steps to precision medicine.

In this review, we provide a description of those candidate biomarkers which have been demonstrated by multiple-omics approaches to vary in correlation with specific clinical manifestations of sickle cell severity. We believe that future clinical analyses of severity phenotype will require a multiomic analysis, or an omics stack approach, which includes integrated interactomics. It will also require the analysis of big data sets. These candidate biomarkers, whether they are individual or panels of functionally linked markers, will require future validation in large prospective and retrospective clinical studies. Once validated, the hope is that informative biomarkers will be used for the identification of individuals most likely to experience severe complications, and thereby be applied for the design of patient-specific therapeutic approaches and response to treatment. This would be the beginning of precision medicine for sickle cell disease.

The proteomics and interactomics of human erythrocytes.

In this minireview, we focus on advances in our knowledge of the human erythrocyte proteome and interactome that have occurred since our seminal review on the topic published in 2007. As will be explained, the number of unique proteins has grown from 751 in 2007 to 2289 as of today. We describe how proteomics and interactomics tools have been used to probe critical protein changes in disorders impacting the blood. The primary example used is the work done on sickle cell disease where biomarkers of severity have been identified, protein changes in the erythrocyte membranes identified, pharmacoproteomic impact of hydroxyurea studied and interactomics used to identify erythrocyte protein changes that are predicted to have the greatest impact on protein interaction networks.

Stabbing balls and simplifying proteins.

We address the problem of stabbing a sequence of indexed balls B = {B1,B2, . . . , Bn} in R(3), where Bi (1

The human red blood cell proteome and interactome.

The red blood cell or erythrocyte is easily purified, readily available, and has a relatively simple structure. Therefore, it has become a very well studied cell in terms of protein composition and function. RBC proteomic studies performed over the last five years, by several laboratories, have identified 751 proteins within the human erythrocyte. As RBCs contain few internal structures, the proteome will contain far fewer proteins than nucleated cells. In this minireview, we summarize the current knowledge of the RBC proteome, discuss alterations in this partial proteome in varied human disease states, and demonstrate how in silico studies of the RBC interactome can lead to considerable insight into disease diagnosis, severity, and drug or gene therapy response. To make these latter points we focus on what is known concerning changes in the RBC proteome in Sickle Cell Disease.