Semisupervised 3D segmentation of pancreatic tumors in positron emission tomography/computed tomography images using a mutual information minimization and cross-fusion strategy.

Background: Accurate segmentation of pancreatic cancer tumors using positron emission tomography/computed tomography (PET/CT) multimodal images is crucial for clinical diagnosis and prognosis evaluation. However, deep learning methods for automated medical image segmentation require a substantial amount of manually labeled data, making it time-consuming and labor-intensive. Moreover, addition or simple stitching of multimodal images leads to redundant information, failing to fully exploit the complementary information of multimodal images. Therefore, we developed a semisupervised multimodal network that leverages limited labeled samples and introduces a cross-fusion and mutual information minimization (MIM) strategy for PET/CT 3D segmentation of pancreatic tumors. Methods: Our approach combined a cross multimodal fusion (CMF) module with a cross-attention mechanism. The complementary multimodal features were fused to form a multifeature set to enhance the effectiveness of feature extraction while preserving specific features of each modal image. In addition, we designed an MIM module to mitigate redundant high-level modal information and compute the latent loss of PET and CT. Finally, our method employed the uncertainty-aware mean teacher semi-supervised framework to segment regions of interest from PET/CT images using a small amount of labeled data and a large amount of unlabeled data. Results: We evaluated our combined MIM and CMF semisupervised segmentation network (MIM-CMFNet) on a private dataset of pancreatic cancer, yielding an average Dice coefficient of 73.14%, an average Jaccard index score of 60.56%, and an average 95% Hausdorff distance (95HD) of 6.30 mm. In addition, to verify the broad applicability of our method, we used a public dataset of head and neck cancer, yielding an average Dice coefficient of 68.71%, an average Jaccard index score of 57.72%, and an average 95HD of 7.88 mm. Conclusions: The experimental results demonstrate the superiority of our MIM-CMFNet over existing semisupervised techniques. Our approach can achieve a performance similar to that of fully supervised segmentation methods while significantly reducing the data annotation cost by 80%, suggesting it is highly practicable for clinical application.

MFCNet: A multi-modal fusion and calibration networks for 3D pancreas tumor segmentation on PET-CT images.

In clinical diagnosis, positron emission tomography and computed tomography (PET-CT) images containing complementary information are fused. Tumor segmentation based on multi-modal PET-CT images is an important part of clinical diagnosis and treatment. However, the existing current PET-CT tumor segmentation methods mainly focus on positron emission tomography (PET) and computed tomography (CT) feature fusion, which weakens the specificity of the modality. In addition, the information interaction between different modal images is usually completed by simple addition or concatenation operations, but this has the disadvantage of introducing irrelevant information during the multi-modal semantic feature fusion, so effective features cannot be highlighted. To overcome this problem, this paper propose a novel Multi-modal Fusion and Calibration Networks (MFCNet) for tumor segmentation based on three-dimensional PET-CT images. First, a Multi-modal Fusion Down-sampling Block (MFDB) with a residual structure is developed. The proposed MFDB can fuse complementary features of multi-modal images while retaining the unique features of different modal images. Second, a Multi-modal Mutual Calibration Block (MMCB) based on the inception structure is designed. The MMCB can guide the network to focus on a tumor region by combining different branch decoding features using the attention mechanism and extracting multi-scale pathological features using a convolution kernel of different sizes. The proposed MFCNet is verified on both the public dataset (Head and Neck cancer) and the in-house dataset (pancreas cancer). The experimental results indicate that on the public and in-house datasets, the average Dice values of the proposed multi-modal segmentation network are 74.14% and 76.20%, while the average Hausdorff distances are 6.41 and 6.84, respectively. In addition, the experimental results show that the proposed MFCNet outperforms the state-of-the-art methods on the two datasets.

A multidomain fusion model of radiomics and deep learning to discriminate between PDAC and AIP based on 18F-FDG PET/CT images.

PURPOSE: To explore a multidomain fusion model of radiomics and deep learning features based on 18F-fluorodeoxyglucose positron emission tomography/computed tomography (18F-FDG PET/CT) images to distinguish pancreatic ductal adenocarcinoma (PDAC) and autoimmune pancreatitis (AIP), which could effectively improve the accuracy of diseases diagnosis. MATERIALS AND METHODS: This retrospective study included 48 patients with AIP (mean age, 65 ± 12.0 years; range, 37-90 years) and 64 patients with PDAC patients (mean age, 66 ± 11.3 years; range, 32-88 years). Three different methods were discussed to identify PDAC and AIP based on 18F-FDG PET/CT images, including the radiomics model (RAD_model), the deep learning model (DL_model), and the multidomain fusion model (MF_model). We also compared the classification results of PET/CT, PET, and CT images in these three models. In addition, we explored the attributes of deep learning abstract features by analyzing the correlation between radiomics and deep learning features. Five-fold cross-validation was used to calculate receiver operating characteristic (ROC), area under the roc curve (AUC), accuracy (Acc), sensitivity (Sen), and specificity (Spe) to quantitatively evaluate the performance of different classification models. RESULTS: The experimental results showed that the multidomain fusion model had the best comprehensive performance compared with radiomics and deep learning models, and the AUC, accuracy, sensitivity, specificity were 96.4% (95% CI 95.4-97.3%), 90.1% (95% CI 88.7-91.5%), 87.5% (95% CI 84.3-90.6%), and 93.0% (95% CI 90.3-95.6%), respectively. And our study proved that the multimodal features of PET/CT were superior to using either PET or CT features alone. First-order features of radiomics provided valuable complementary information for the deep learning model. CONCLUSION: The preliminary results of this paper demonstrated that our proposed multidomain fusion model fully exploits the value of radiomics and deep learning features based on 18F-FDG PET/CT images, which provided competitive accuracy for the discrimination of PDAC and AIP.

Prognostic Evaluation Based on Dual-Time 18F-FDG PET/CT Radiomics Features in Patients with Locally Advanced Pancreatic Cancer Treated by Stereotactic Body Radiation Therapy.

Background: 18F-FDG PET/CT is widely used in the prognosis evaluation of tumor patients. The radiomics features can provide additional information for clinical prognostic assessment. Purpose: Purpose is to explore the prognostic value of radiomics features from dual-time 18F-FDG PET/CT images for locally advanced pancreatic cancer (LAPC) patients treated with stereotactic body radiation therapy (SBRT). Materials and Methods: This retrospective study included 70 LAPC patients who received early and delayed 18F-FDG PET/CT scans before SBRT treatment. A total of 1188 quantitative imaging features were extracted from dual-time PET/CT images. To avoid overfitting, the univariate analysis and elastic net were used to obtain a sparse set of image features that were applied to develop a radiomics score (Rad-score). Then, the Harrell consistency index (C-index) was used to evaluate the prognosis model. Results: The Rad-score from dual-time images contains six features, including intensity histogram, morphological, and texture features. In the validation cohort, the univariate analysis showed that the Rad-score was the independent prognostic factor (p < 0.001, hazard ratio [HR]: 3.2). And in the multivariate analysis, the Rad-score was the only prognostic factor (p < 0.01, HR: 4.1) that was significantly associated with the overall survival (OS) of patients. In addition, according to cross-validation, the C-index of the prognosis model based on the Rad-score from dual-time images is better than the early and delayed images (0.720 vs. 0.683 vs. 0.583). Conclusion: The Rad-score based on dual-time 18F-FDG PET/CT images is a promising noninvasive method with better prognostic value.

Measurement of Human Semicircular Canal Spatial Attitude.

Located deep in the temporal bone, the semicircular canal is a subtle structure that requires a spatial coordinate system for measurement and observation. In this study, 55 semicircular canal and eyeball models were obtained by segmentation of MRI data. The spatial coordinate system was established by taking the top of the common crus and the bottom of the eyeball as the horizontal plane. First, the plane equation was established according to the centerline of the semicircular canals. Then, according to the parameters of the plane equation, the plane normal vectors were obtained. Finally, the average unit normal vector of each semicircular canal plane was obtained by calculating the average value of the vectors. The standard normal vectors of the and left posterior semicircular canal, superior semicircular canal and lateral semicircular canal were [-0.651, 0.702, 0.287], [0.749, 0.577, 0.324], [-0.017, -0.299, 0.954], [0.660, 0.702, 0.266], [-0.739, 0.588, 0.329], [0.025, -0.279, 0.960]. The different angles for the different ways of calculating the standard normal vectors of the right and left posterior semicircular canal, superior semicircular canal and lateral semicircular canal were 0.011, 0.028, 0.008, 0.011, 0.024, and 0.006 degrees. The technology for measuring the semicircular canal spatial attitudes in this study are reliable, and the measurement results can guide vestibular function examinations and help with guiding the diagnosis and treatment of BPPV.

Deep attention branch networks for skin lesion classification.

BACKGROUND AND OBJECTIVE: The skin lesion usually covers a small region of the dermoscopy image, and the lesions of different categories might own high similarities. Therefore, it is essential to design an elaborate network for accurate skin lesion classification, which can focus on semantically meaningful lesion parts. Although the Class Activation Mapping (CAM) shows good localization capability of highlighting the discriminative parts, it cannot be obtained in the forward propagation process. METHODS: We propose a Deep Attention Branch Network (DABN) model, which introduces the attention branches to expand the conventional Deep Convolutional Neural Networks (DCNN). The attention branch is designed to obtain the CAM in the training stage, which is then utilized as an attention map to make the network focus on discriminative parts of skin lesions. DABN is applicable to multiple DCNN structures and can be trained in an end-to-end manner. Moreover, a novel Entropy-guided Loss Weighting (ELW) strategy is designed to counter class imbalance influence in the skin lesion datasets. RESULTS: The proposed method achieves an Average Precision (AP) of 0.719 on the ISIC-2016 dataset and an average area under the ROC curve (AUC) of 0.922 on the ISIC-2017 dataset. Compared with other state-of-the-art methods, our method obtains better performance without external data and ensemble learning. Moreover, extensive experiments demonstrate that it can be applied to multi-class classification tasks and improves mean sensitivity by more than 2.6% in different DCNN structures. CONCLUSIONS: The proposed method can adaptively focus on the discriminative regions of dermoscopy images and allows for effective training when facing class imbalance, leading to the performance improvement of skin lesion classification, which could also be applied to other clinical applications.

Radiomics model of dual-time 2-[18F]FDG PET/CT imaging to distinguish between pancreatic ductal adenocarcinoma and autoimmune pancreatitis.

OBJECTIVES: Pancreatic ductal adenocarcinoma (PDAC) and autoimmune pancreatitis (AIP) are diseases with a highly analogous visual presentation that are difficult to distinguish by imaging. The purpose of this research was to create a radiomics-based prediction model using dual-time PET/CT imaging for the noninvasive classification of PDAC and AIP lesions. METHODS: This retrospective study was performed on 112 patients (48 patients with AIP and 64 patients with PDAC). All cases were confirmed by imaging and clinical follow-up, and/or pathology. A total of 502 radiomics features were extracted from the dual-time PET/CT images to develop a radiomics decision model. An additional 12 maximum intensity projection (MIP) features were also calculated to further improve the radiomics model. The optimal radiomics feature set was selected by support vector machine recursive feature elimination (SVM-RFE), and the final classifier was built using a linear SVM. The performance of the proposed dual-time model was evaluated using nested cross-validation for accuracy, sensitivity, specificity, and area under the curve (AUC). RESULTS: The final prediction model was developed from a combination of the SVM-RFE and linear SVM with the required quantitative features. The multimodal and multidimensional features performed well for classification (average AUC: 0.9668, accuracy: 89.91%, sensitivity: 85.31%, specificity: 96.04%). CONCLUSIONS: The radiomics model based on 2-[18F]fluoro-2-deoxy-D-glucose (2-[18F]FDG) PET/CT dual-time images provided promising performance for discriminating between patients with benign AIP and malignant PDAC lesions, which shows its potential for use as a diagnostic tool for clinical decision-making. KEY POINTS: • The clinical symptoms and imaging visual presentations of PDAC and AIP are highly similar, and accurate differentiation of PDAC and AIP lesions is difficult. • Radiomics features provided a potential noninvasive method for differentiation of AIP from PDAC. • The diagnostic performance of the proposed radiomics model indicates its potential to assist doctors in making treatment decisions.

Virtual simulation of otolith movement for the diagnosis and treatment of benign paroxysmal positional vertigo.

Benign paroxysmal positional vertigo (BPPV) is a clinical condition. The existing diagnostic methods cannot determine the specific location of otolith on the short or long brachial sides. Thus, visual and quantitative evaluation of the existing clinical standard diagnostic modality Dix-Hallpike test is needed to improve medical efficiency. Our goal was to develop a real-time virtual simulation system to assess a BPPV treatment manipulation. In this study, we used the proposed simulation system to observe otolith movement during a posterior semicircular canal BPPV diagnostic test, and to analyze the diagnostic mechanisms and strategies. Through visual cluster analysis of otolith position and analysis of otolith movement time in the standard Dix-Hallpike test, we can find that the positions of otoliths are relatively scattered, especially on the z-axis (z 1 = 10.67 ± 3.98), and the fall time of otoliths at different positions has relatively large changes (t 1 = 22.21 ± 1.40). But in the modified experiment z 2 = 4.93 ± 0.32 and t 2 = 26.21 ± 0.28. The experimental results show that the simulation system could track the state and the movement of otolith in real-time, which is of great significance for understanding the diagnostic mechanisms of BPPV evaluations and improving the diagnostic method.

Detection and analysis of MEG signals in occipital region with double-channel OPM sensors.

BACKGROUND: Magnetoencephalography (MEG) has high temporal and spatial resolution and good spatial accuracy in determining the locations of source activity among most non-invasive imaging. The recently developed technology of optically-pumped magnetometer (OPM) has sensitivity comparable to that of the superconducting quantum interference device (SQUID) used in commercial MEG system. NEW METHOD: Double-channel OPM-MEG system detects human photic blocking of alpha rhythm at the occipital region of skull in the magnetically shielded environment via a wearable whole-cortex 3D-printed helmet. RESULTS: The alpha rhythm can be detected by the OPM-MEG system, the MEG signals are undoubtedly caused by photic blocking and similar with the results measured by SQUID magnetometer. COMPARISON WITH EXISTING METHODS: Due to the dependency of current commercial whole-cortex SQUID-MEG system on the liquid helium, the separation from the liquid helium space to the human head is usually at least a few centimeters. The wearable OPM-MEG system, however, can significantly improve the detection efficiency since its sensors can be mounted close to scalp, normally less than 1 cm. CONCLUSIONS: OPM-MEG system successfully detects alpha rhythm blocked by light stimulation and works well in the home-made magnetically shielded environment. OPM-MEG system shows a substitute for the traditional MEG system.

Blood species identification based on deep learning analysis of Raman spectra.

Blood analysis is an indispensable means of detection in criminal investigation, customs security and quarantine, anti-poaching of wildlife, and other incidents. Detecting the species of blood is one of the most important analyses. In order to classify species by analyzing Raman spectra of blood, a recognition method based on deep learning principle is proposed in this paper. This method can realize multi-identification blood species, by constructing a one-dimensional convolution neural network and establishing a Raman spectra database containing 20 kinds of blood. The network model is obtained through training, and then is employed to predict the testing set data. The average accuracy of blind detection is more than 97%. In this paper, we try to increase the diversity of data to improve the robustness of the model, optimize the network and adjust the hyperparameters to improve the recognition ability of the model. The evaluation results show that the deep learning model has high recognition performance to distinguish the species of blood.

[Differentiation of autoimmune pancreatitis and pancreatic ductal adenocarcinoma based on multi-modality texture features in 18F-FDG PET/CT].

Autoimmune pancreatitis (AIP) is a unique subtype of chronic pancreatitis, which shares many clinical presentations with pancreatic ductal adenocarcinoma (PDA). The misdiagnosis of AIP often leads to unnecessary pancreatic resection. 18F-FDG positron emission tomography/ computed tomography (PET/CT) could provide comprehensive information on the morphology, density, and functional metabolism of the pancreas at the same time. It has been proved to be a promising modality for noninvasive differentiation between AIP and PDA. However, there is a lack of clinical analysis of PET/CT image texture features. Difficulty still remains in differentiating AIP and PDA based on commonly used diagnostic methods. Therefore, this paper studied the differentiation of AIP and PDA based on multi-modality texture features. We utilized multiple feature extraction algorithms to extract the texture features from CT and PET images at first. Then, the Fisher criterion and sequence forward floating selection algorithm (SFFS) combined with support vector machine (SVM) was employed to select the optimal multi-modality feature subset. Finally, the SVM classifier was used to differentiate AIP from PDA. The results prove that texture analysis of lesions helps to achieve accurate differentiation of AIP and PDA.

Radiomics analysis for the differentiation of autoimmune pancreatitis and pancreatic ductal adenocarcinoma in 18 F-FDG PET/CT.

PURPOSE: To perform a radiomics analysis with comparisons of multidomain features and a variety of feature selection strategies and classifiers, with the goal of evaluating the value of quantified radiomics method for noninvasively differentiating autoimmune pancreatitis (AIP) from pancreatic ductal adenocarcinoma (PDAC) in 18 F-fluorodeoxglucose positron emission tomography/computed tomography (18 F FDG PET/CT) images. METHODS: We extracted 251 expert-designed features from 2D and 3D PET/CT images of 111 patients, and recombined these features into five feature sets according to their modalities and dimensions. Among the five feature sets, the optimal one was found leveraging four feature selection strategies and four machine learning classifiers based on the area under the receiver operating characteristic curve (AUC). The feature selection strategies include spearman's rank correlation coefficient, minimum redundancy maximum relevance, support vector machine recursive feature elimination (SVM-RFE), and no feature selection, while the classifiers are random forest, adaptive boosting, support vector machine (SVM) with the Gaussian radial basis function, and SVM with the linear kernel function respectively. Based on the optimal feature set, these feature selection strategies and classifiers were comparatively studied to achieve the best differentiation. Finally, the quantified radiomics prediction model was developed based on the best combination of the feature selection strategy and classifier, and it was compared with two clinical factors based prediction models, and human doctors using nested cross-validation in terms of AUC, accuracy, sensitivity, and specificity. RESULTS: Comparison experiments demonstrated that CT features and three-dimensional (3D) features performed better than positron emission tomography (PET) features and three-dimensional (2D) features respectively, and multidomain features were superior to single domain features. In addition, the combination of SVM-RFE feature selection strategy and Linear SVM classifier had the highest diagnostic performance (i.e., AUC = 0.93 ± 0.01, ACC = 0.85 ± 0.02, SEN = 0.86 ± 0.03, SPE = 0.84 ± 0.03). The quantified radiomics model developed is significantly superior to both human doctors and clinical factors based prediction models in terms of accuracy and specificity. CONCLUSIONS: Our preliminary results confirmed that the quantified radiomics method could aid the noninvasive differentiation of AIP and PDAC in 18 F FDG PET/CT images and the integration of multidomain features is beneficial for the differentiation.

Low-dose CT reconstruction via L1 dictionary learning regularization using iteratively reweighted least-squares.

BACKGROUND: In order to reduce the radiation dose of CT (computed tomography), compressed sensing theory has been a hot topic since it provides the possibility of a high quality recovery from the sparse sampling data. Recently, the algorithm based on DL (dictionary learning) was developed to deal with the sparse CT reconstruction problem. However, the existing DL algorithm focuses on the minimization problem with the L2-norm regularization term, which leads to reconstruction quality deteriorating while the sampling rate declines further. Therefore, it is essential to improve the DL method to meet the demand of more dose reduction. METHODS: In this paper, we replaced the L2-norm regularization term with the L1-norm one. It is expected that the proposed L1-DL method could alleviate the over-smoothing effect of the L2-minimization and reserve more image details. The proposed algorithm solves the L1-minimization problem by a weighting strategy, solving the new weighted L2-minimization problem based on IRLS (iteratively reweighted least squares). RESULTS: Through the numerical simulation, the proposed algorithm is compared with the existing DL method (adaptive dictionary based statistical iterative reconstruction, ADSIR) and other two typical compressed sensing algorithms. It is revealed that the proposed algorithm is more accurate than the other algorithms especially when further reducing the sampling rate or increasing the noise. CONCLUSION: The proposed L1-DL algorithm can utilize more prior information of image sparsity than ADSIR. By transforming the L2-norm regularization term of ADSIR with the L1-norm one and solving the L1-minimization problem by IRLS strategy, L1-DL could reconstruct the image more exactly.

Spherical operator classification for coronary artery extraction.

Computed tomography angiography (CTA) is a major noninvasive technology for imaging coronary artery disease, and effective and accurate vessel tracking method can help radiologists diagnose the disease more accurately. In this paper, a novel 3D vessel tracking method based on CTA data is presented. The method is comprised of preprocessing, a novel spherical operator, and hierarchical clustering, where the spherical operator consists of rays that are casted different directions in a spherical coordinate system. The vascular boundary is extracted by the spherical operator, and the tracking direction is also obtained by the hierarchical clustering. The method is evaluated with the Rotterdam Coronary Artery Algorithm Evaluation Framework. Results indicate that our method outperforms current state-of-the-art methods in terms of the overlap ratio on the vessel tracking of coronary arteries in CTA data.

Retinal image graph-cut segmentation algorithm using multiscale Hessian-enhancement-based nonlocal mean filter.

We propose a new method to enhance and extract the retinal vessels. First, we employ a multiscale Hessian-based filter to compute the maximum response of vessel likeness function for each pixel. By this step, blood vessels of different widths are significantly enhanced. Then, we adopt a nonlocal mean filter to suppress the noise of enhanced image and maintain the vessel information at the same time. After that, a radial gradient symmetry transformation is adopted to suppress the nonvessel structures. Finally, an accurate graph-cut segmentation step is performed using the result of previous symmetry transformation as an initial. We test the proposed approach on the publicly available databases: DRIVE. The experimental results show that our method is quite effective.