Multi-tissue segmentation model of whole slide image of pancreatic cancer based on multi task and attention mechanism / 生物医学工程学杂志

Accurate segmentation of whole slide images is of great significance for the diagnosis of pancreatic cancer. However, developing an automatic model is challenging due to the complex content, limited samples, and high sample heterogeneity of pathological images. This paper presented a multi-tissue segmentation model for whole slide images of pancreatic cancer. We introduced an attention mechanism in building blocks, and designed a multi-task learning framework as well as proper auxiliary tasks to enhance model performance. The model was trained and tested with the pancreatic cancer pathological image dataset from Shanghai Changhai Hospital. And the data of TCGA, as an external independent validation cohort, was used for external validation. The F1 scores of the model exceeded 0.97 and 0.92 in the internal dataset and external dataset, respectively. Moreover, the generalization performance was also better than the baseline method significantly. These results demonstrate that the proposed model can accurately segment eight kinds of tissue regions in whole slide images of pancreatic cancer, which can provide reliable basis for clinical diagnosis.

Hepatocellular carcinoma segmentation and pathological differentiation degree prediction method based on multi-task learning / 生物医学工程学杂志

Hepatocellular carcinoma (HCC) is the most common liver malignancy, where HCC segmentation and prediction of the degree of pathological differentiation are two important tasks in surgical treatment and prognosis evaluation. Existing methods usually solve these two problems independently without considering the correlation of the two tasks. In this paper, we propose a multi-task learning model that aims to accomplish the segmentation task and classification task simultaneously. The model consists of a segmentation subnet and a classification subnet. A multi-scale feature fusion method is proposed in the classification subnet to improve the classification accuracy, and a boundary-aware attention is designed in the segmentation subnet to solve the problem of tumor over-segmentation. A dynamic weighted average multi-task loss is used to make the model achieve optimal performance in both tasks simultaneously. The experimental results of this method on 295 HCC patients are superior to other multi-task learning methods, with a Dice similarity coefficient (Dice) of (83.9 ± 0.88)% on the segmentation task, while the average recall is (86.08 ± 0.83)% and an F1 score is (80.05 ± 1.7)% on the classification task. The results show that the multi-task learning method proposed in this paper can perform the classification task and segmentation task well at the same time, which can provide theoretical reference for clinical diagnosis and treatment of HCC patients.

Non-local attention and multi-task learning based lung segmentation in chest X-ray / 生物医学工程学杂志

Precise segmentation of lung field is a crucial step in chest radiographic computer-aided diagnosis system. With the development of deep learning, fully convolutional network based models for lung field segmentation have achieved great effect but are poor at accurate identification of the boundary and preserving lung field consistency. To solve this problem, this paper proposed a lung segmentation algorithm based on non-local attention and multi-task learning. Firstly, an encoder-decoder convolutional network based on residual connection was used to extract multi-scale context and predict the boundary of lung. Secondly, a non-local attention mechanism to capture the long-range dependencies between pixels in the boundary regions and global context was proposed to enrich feature of inconsistent region. Thirdly, a multi-task learning to predict lung field based on the enriched feature was conducted. Finally, experiments to evaluate this algorithm were performed on JSRT and Montgomery dataset. The maximum improvement of Dice coefficient and accuracy were 1.99% and 2.27%, respectively, comparing with other representative algorithms. Results show that by enhancing the attention of boundary, this algorithm can improve the accuracy and reduce false segmentation.

Application of a multi-task learning-based light-weight convolution neural network for the automatic segmentation of organs at risk in thorax / 中华放射肿瘤学杂志

Objective:To evaluate the application of a multi-task learning-based light-weight convolution neural network (MTLW-CNN) for the automatic segmentation of organs at risk (OARs) in thorax.Methods:MTLW-CNN consisted of several layers for sharing features and 3 branches for segmenting 3 OARs. 497 cases with thoracic tumors were collected. Among them, the computed tomography (CT) images encompassing lung, heart and spinal cord were included in this study. The corresponding contours delineated by experienced radiation oncologists were ground truth. All cases were randomly categorized into the training and validation set ( n=300) and test set ( n=197). By applying MTLW-CNN on the test set, the Dice similarity coefficients (DSCs) of 3 OARs, training and testing time and space complexity (S) were calculated and compared with those of Unet and DeepLabv3+ . To evaluate the effect of multi-task learning on the generalization performance of the model, 3 single-task light-weight CNNs (STLW-CNNs) were built. Their structures were totally the same as the corresponding branches in MTLW-CNN. After using the same data and algorithm to train STLW-CNN, the DSCs were statistically compared with MTLW-CNN on the testing set. Results:For MTLW-CNN, the averages (μ) of lung, heart and spinal cord DSCs were 0.954, 0.921 and 0.904, respectively. The differences of μ between MTLW-CNN and other two models (Unet and DeepLabv3+ ) were less than 0.020. The training and testing time of MTLW-CNN were 1/3 to 1/30 of that of Unet and DeepLabv3+ . S of MTLW-CNN was 1/42 of that of Unet and 1/1 220 of that of DeepLabv3+ . The differences of μ and standard deviation (σ) of lung and heart between MTLW-CNN and STLW-CNN were approximately 0.005 and 0.002. The difference of μ of spinal cord was 0.001, but σof STLW-CNN was 0.014 higher than that of MTLW-CNN.Conclusions:MTLW-CNN spends less time and space on high-precision automatic segmentation of thoracic OARs. It can improve the application efficiency and generalization performance of the models.

Multi-task learning-based three-dimensional dose distribution prediction for multiple organs in a single model / 中华放射肿瘤学杂志

Objective To establish a three-dimensional (3D) dose prediction model,which can predict multiple organs simultaneously in a single model and automatically learn the effect of the geometric anatomical structure on dose distribution.Methods Clinical radiotherapy plans of patients diagnosed with the same type of tumors were collected and retrospectively analyzed.For every plan,each organs at risk (OAR) voxel was regarded as the study sample and its deposited dose was considered as the dosimetric feature.A regularized multi-task learning method than could learn the relationship among different tasks was employed to establish the relationship matrix among tasks and the correlation between geometric structure and dose distribution among organs.In this experiment,the spinal cord,brainstem and bilateral parotids involved in the intensity-modulated radiotherapy (IMRT) plan of 15 nasopharyngeal cancer patients were utilized to establish the multi-organ prediction model.The relative percentage error between the predicted dose of voxel and the clinical planning dose was calculated to assess the feasibility of the model.Results Ten cases receiving IMRT plans were utilized as the training data,and the remaining five cases were used as the test data.The test results demonstrated a higher prediction accuracy and less data demand.And the average voxel dose errors among the spinal cord,brainstem and the left and right parotids were (2.01±0.02)％,(2.65± 0.02) ％,(2.45± 0.02) ％ and (2.55± 0.02) ％,respectively.Conclusion The proposed model can accurately predict the dose of multiple organs in a single model and avoid the establishment of multiple single-organ prediction models,laying a solid foundation for patient-specific plan quality control and knowledge-based treatment planning.