Accurate tumor segmentation and treatment outcome prediction with DeepTOP.

BACKGROUND: Accurate outcome prediction prior to treatment can facilitate trial design and clinical decision making to achieve better treatment outcome. METHOD: We developed the DeepTOP tool with deep learning approach for region-of-interest segmentation and clinical outcome prediction using magnetic resonance imaging (MRI). DeepTOP was constructed with an automatic pipeline from tumor segmentation to outcome prediction. In DeepTOP, the segmentation model used U-Net with a codec structure, and the prediction model was built with a three-layer convolutional neural network. In addition, the weight distribution algorithm was developed and applied in the prediction model to optimize the performance of DeepTOP. RESULTS: A total of 1889 MRI slices from 99 patients in the phase III multicenter randomized clinical trial (NCT01211210) on neoadjuvant treatment for rectal cancer was used to train and validate DeepTOP. We systematically optimized and validated DeepTOP with multiple devised pipelines in the clinical trial, demonstrating a better performance than other competitive algorithms in accurate tumor segmentation (Dice coefficient: 0.79; IoU: 0.75; slice-specific sensitivity: 0.98) and predicting pathological complete response to chemo/radiotherapy (accuracy: 0.789; specificity: 0.725; and sensitivity: 0.812). DeepTOP is a deep learning tool that could avoid manual labeling and feature extraction and realize automatic tumor segmentation and treatment outcome prediction by using the original MRI images. CONCLUSION: DeepTOP is open to provide a tractable framework for the development of other segmentation and predicting tools in clinical settings. DeepTOP-based tumor assessment can provide a reference for clinical decision making and facilitate imaging marker-driven trial design.

Automatic treatment outcome prediction with DeepInteg based on multimodal radiological images in rectal cancer.

Neoadjuvant systemic treatment before surgery is a prevalent regimen in the patients with advanced-stage or high-risk tumor, which has shaped the treatment strategies and cancer survival in the past decades. However, some patients present with poor response to the neoadjuvant treatment. Therefore, it is of great significance to develop tools to help distinguish the patients that could achieve pathological complete response before surgery to avoid inappropriate treatment. Here, this study demonstrated a multi-task deep learning tool called DeepInteg. In the DeepInteg framework, the segmentation module was constructed based on the CE-Net with a context extractor to achieve end-to-end delineation of region of interest (ROI) from radiological images, then the features of segmented Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) images of each case were fused and input to the classification module based on a convolution neural network for treatment outcome prediction. The dataset with 1700 MRI and CT slices collected from the prospectively randomized clinical trial (NCT01211210) on systemic treatment for rectal cancer was used to develop and systematically optimize DeepInteg. As a result, DeepInteg achieved automatic segmentation of tumoral ROI with Dices of 0.766 and 0.719 and mIoUs of 0.788 and 0.756 in CT and MRI images, respectively. In addition, DeepInteg achieved AUC of 0.833, accuracy of 0.826 and specificity of 0.856 in the prediction for pathological complete response after treatment, which showed better performance compared with the model based on CT or MRI alone. This study provide a robust framework to develop disease-specific tools for automatic delineation of ROI and clinical outcome prediction. The well-trained DeepInteg could be readily applied in clinic to predict pathological complete response after neoadjuvant therapy in rectal cancer patients.