ChatCAD+: Towards a Universal and Reliable Interactive CAD using LLMs.

The integration of Computer-Aided Diagnosis (CAD) with Large Language Models (LLMs) presents a promising frontier in clinical applications, notably in automating diagnostic processes akin to those performed by radiologists and providing consultations similar to a virtual family doctor. Despite the promising potential of this integration, current works face at least two limitations: (1) From the perspective of a radiologist, existing studies typically have a restricted scope of applicable imaging domains, failing to meet the diagnostic needs of different patients. Also, the insufficient diagnostic capability of LLMs further undermine the quality and reliability of the generated medical reports. (2) Current LLMs lack the requisite depth in medical expertise, rendering them less effective as virtual family doctors due to the potential unreliability of the advice provided during patient consultations. To address these limitations, we introduce ChatCAD+, to be universal and reliable. Specifically, it is featured by two main modules: (1) Reliable Report Generation and (2) Reliable Interaction. The Reliable Report Generation module is capable of interpreting medical images from diverse domains and generate high-quality medical reports via our proposed hierarchical in-context learning. Concurrently, the interaction module leverages up-to-date information from reputable medical websites to provide reliable medical advice. Together, these designed modules synergize to closely align with the expertise of human medical professionals, offering enhanced consistency and reliability for interpretation and advice. The source code is available at GitHub.

Spatial and Modal Optimal Transport for Fast Cross-Modal MRI Reconstruction.

Multi-modal magnetic resonance imaging (MRI) plays a crucial role in comprehensive disease diagnosis in clinical medicine. However, acquiring certain modalities, such as T2-weighted images (T2WIs), is time-consuming and prone to be with motion artifacts. It negatively impacts subsequent multi-modal image analysis. To address this issue, we propose an end-to-end deep learning framework that utilizes T1-weighted images (T1WIs) as auxiliary modalities to expedite T2WIs' acquisitions. While image pre-processing is capable of mitigating misalignment, improper parameter selection leads to adverse pre-processing effects, requiring iterative experimentation and adjustment. To overcome this shortage, we employ Optimal Transport (OT) to synthesize T2WIs by aligning T1WIs and performing cross-modal synthesis, effectively mitigating spatial misalignment effects. Furthermore, we adopt an alternating iteration framework between the reconstruction task and the cross-modal synthesis task to optimize the final results. Then, we prove that the reconstructed T2WIs and the synthetic T2WIs become closer on the T2 image manifold with iterations increasing, and further illustrate that the improved reconstruction result enhances the synthesis process, whereas the enhanced synthesis result improves the reconstruction process. Finally, experimental results from FastMRI and internal datasets confirm the effectiveness of our method, demonstrating significant improvements in image reconstruction quality even at low sampling rates.

Attention-Based MultiOffset Deep Learning Reconstruction of Chemical Exchange Saturation Transfer (AMO-CEST) MRI.

One challenge of chemical exchange saturation transfer (CEST) magnetic resonance imaging (MRI) is the long scan time due to multiple acquisitions of images at different saturation frequency offsets. k-space under-sampling strategy is commonly used to accelerate MRI acquisition, while this could introduce artifacts and reduce signal-to-noise ratio (SNR). To accelerate CEST-MRI acquisition while maintaining suitable image quality, we proposed an attention-based multioffset deep learning reconstruction network (AMO-CEST) with a multiple radial k-space sampling strategy for CEST-MRI. The AMO-CEST also contains dilated convolution to enlarge the receptive field and data consistency module to preserve the sampled k-space data. We evaluated the proposed method on a mouse brain dataset containing 5760 CEST images acquired at a pre-clinical 3T MRI scanner. Quantitative results demonstrated that AMO-CEST showed obvious improvement over zero-filling method with a PSNR enhancement of 11 dB, a SSIM enhancement of 0.15, and a NMSE decrease of 4.37×10-2 in three acquisition orientations. Compared with other deep learning-based models, AMO-CEST showed visual and quantitative improvements in images from three different orientations. We also extracted molecular contrast maps, including the amide proton transfer (APT) and the relayed nuclear Overhauser enhancement (rNOE). The results demonstrated that the CEST contrast maps derived from the CEST images of AMO-CEST were comparable to those derived from the original high-resolution CEST images. The proposed AMO-CEST can efficiently reconstruct high-quality CEST images from under-sampled k-space data and thus has the potential to accelerate CEST-MRI acquisition.

Real-time diagnosis of intracerebral hemorrhage by generating dual-energy CT from single-energy CT.

Real-time diagnosis of intracerebral hemorrhage after thrombectomy is crucial for follow-up treatment. However, this is difficult to achieve with standard single-energy CT (SECT) due to similar CT values of blood and contrast agents under a single energy spectrum. In contrast, dual-energy CT (DECT) scanners employ two different energy spectra, which allows for real-time differentiation between hemorrhage and contrast extravasation based on energy-related attenuation characteristics. Unfortunately, DECT scanners are not as widely used as SECT scanners due to their high costs. To address this dilemma, in this paper, we generate pseudo DECT images from a SECT image for real-time diagnosis of hemorrhage. More specifically, we propose a SECT-to-DECT Transformer-based Generative Adversarial Network (SDTGAN), which is a 3D transformer-based multi-task learning framework equipped with a shared attention mechanism. In this way, SDTGAN can be guided to focus more on high-density areas (crucial for hemorrhage diagnosis) during the generation. Meanwhile, the introduced multi-task learning strategy and the shared attention mechanism also enable SDTGAN to model dependencies between interconnected generation tasks, improving generation performance while significantly reducing model parameters and computational complexity. In the experiments, we approximate real SECT images using mixed 120kV images from DECT data to address the issue of not being able to obtain the true paired DECT and SECT data. Extensive experiments demonstrate that SDTGAN can generate DECT images better than state-of-the-art methods. The code of our implementation is available at https://github.com/jiang-cw/SDTGAN.

A domain knowledge-based interpretable deep learning system for improving clinical breast ultrasound diagnosis.

BACKGROUND: Though deep learning has consistently demonstrated advantages in the automatic interpretation of breast ultrasound images, its black-box nature hinders potential interactions with radiologists, posing obstacles for clinical deployment. METHODS: We proposed a domain knowledge-based interpretable deep learning system for improving breast cancer risk prediction via paired multimodal ultrasound images. The deep learning system was developed on 4320 multimodal breast ultrasound images of 1440 biopsy-confirmed lesions from 1348 prospectively enrolled patients across two hospitals between August 2019 and December 2022. The lesions were allocated to 70% training cohort, 10% validation cohort, and 20% test cohort based on case recruitment date. RESULTS: Here, we show that the interpretable deep learning system can predict breast cancer risk as accurately as experienced radiologists, with an area under the receiver operating characteristic curve of 0.902 (95% confidence interval = 0.882 - 0.921), sensitivity of 75.2%, and specificity of 91.8% on the test cohort. With the aid of the deep learning system, particularly its inherent explainable features, junior radiologists tend to achieve better clinical outcomes, while senior radiologists experience increased confidence levels. Multimodal ultrasound images augmented with domain knowledge-based reasoning cues enable an effective human-machine collaboration at a high level of prediction performance. CONCLUSIONS: Such a clinically applicable deep learning system may be incorporated into future breast cancer screening and support assisted or second-read workflows.

Breast cancer is one of the most common cancers, and finding it early can greatly improve patients' chances of survival and recovery. We create a tool based on artificial intelligence (AI)­whereby computer software learns to perform tasks that normally require human thinking­called MUP-Net. MUP-Net can analyze medical images to predict a patient's risk of having breast cancer. To make this AI tool usable in clinical practice, we enabled doctors to see the reasoning behind the AI's predictions by visualizing the key image features it analyzed. We showed that our AI tool not only makes doctors more confident in their diagnosis but also helps them make better decisions, especially for less experienced doctors. With further testing, our AI tool may help clinicians to diagnose breast cancer more accurately and quickly, potentially improving patient outcomes.

Adaptive and Iterative Learning with Multi-perspective Regularizations for Metal Artifact Reduction.

Metal artifact reduction (MAR) is important for clinical diagnosis with CT images. The existing state-of-the-art deep learning methods usually suppress metal artifacts in sinogram or image domains or both. However, their performance is limited by the inherent characteristics of the two domains, i.e., the errors introduced by local manipulations in the sinogram domain would propagate throughout the whole image during backprojection and lead to serious secondary artifacts, while it is difficult to distinguish artifacts from actual image features in the image domain. To alleviate these limitations, this study analyzes the desirable properties of wavelet transform in-depth and proposes to perform MAR in the wavelet domain. First, wavelet transform yields components that possess spatial correspondence with the image, thereby preventing the spread of local errors to avoid secondary artifacts. Second, using wavelet transform could facilitate identification of artifacts from image since metal artifacts are mainly high-frequency signals. Taking these advantages of the wavelet transform, this paper decomposes an image into multiple wavelet components and introduces multi-perspective regularizations into the proposed MAR model. To improve the transparency and validity of the model, all the modules in the proposed MAR model are designed to reflect their mathematical meanings. In addition, an adaptive wavelet module is also utilized to enhance the flexibility of the model. To optimize the model, an iterative algorithm is developed. The evaluation on both synthetic and real clinical datasets consistently confirms the superior performance of the proposed method over the competing methods.

Unified Model for Children's Brain Image Segmentation with Co-Registration Framework Guided by Longitudinal MRI.

Accurate segmentation of brain structures is crucial for analyzing longitudinal changes in children's brains. However, existing methods are mostly based on models established at a single time-point due to difficulty in obtaining annotated data and dynamic variation of tissue intensity. The main problem with such approaches is that, when conducting longitudinal analysis, images from different time points are segmented by different models, leading to significant variation in estimating development trends. In this paper, we propose a novel unified model with co-registration framework to segment children's brain images covering neonates to preschoolers, which is formulated as two stages. First, to overcome the shortage of annotated data, we propose building gold-standard segmentation with co-registration framework guided by longitudinal data. Second, we construct a unified segmentation model tailored to brain images at 0-6 years old through the introduction of a convolutional network (named SE-VB-Net), which combines our previously proposed VB-Net with Squeeze-and-Excitation (SE) block. Moreover, different from existing methods that only require both T1- and T2-weighted MR images as inputs, our designed model also allows a single T1-weighted MR image as input. The proposed method is evaluated on the main dataset (320 longitudinal subjects with average 2 time-points) and two external datasets (10 cases with 6-month-old and 40 cases with 20-45 weeks, respectively). Results demonstrate that our proposed method achieves a high performance (>92%), even over a single time-point. This means that it is suitable for brain image analysis with large appearance variation, and largely broadens the application scenarios.

Constructing hierarchical attentive functional brain networks for early AD diagnosis.

Analyzing functional brain networks (FBN) with deep learning has demonstrated great potential for brain disorder diagnosis. The conventional construction of FBN is typically conducted at a single scale with a predefined brain region atlas. However, numerous studies have identified that the structure and function of the brain are hierarchically organized in nature. This urges the need of representing FBN in a hierarchical manner for more effective analysis of the complementary diagnostic insights at different scales. To this end, this paper proposes to build hierarchical FBNs adaptively within the Transformer framework. Specifically, a sparse attention-based node-merging module is designed to work alongside the conventional network feature extraction modules in each layer. The proposed module generates coarser nodes for further FBN construction and analysis by combining fine-grained nodes. By stacking multiple such layers, a hierarchical representation of FBN can be adaptively learned in an end-to-end manner. The hierarchical structure can not only integrate the complementary information from multiscale FBN for joint analysis, but also reduce the model complexity due to decreasing node sizes. Moreover, this paper argues that the nodes defined by the existing atlases are not necessarily the optimal starting level to build FBN hierarchy and exploring finer nodes may further enrich the FBN representation. In this regard, each predefined node in an atlas is split into multiple sub-nodes, overcoming the scale limitation of the existing atlases. Extensive experiments conducted on various data sets consistently demonstrate the superior performance of the proposed method over the competing methods.

Dysfunctions of multiscale dynamic brain functional networks in subjective cognitive decline.

Subjective cognitive decline is potentially the earliest symptom of Alzheimer's disease, whose objective neurological basis remains elusive. To explore the potential biomarkers for subjective cognitive decline, we developed a novel deep learning method based on multiscale dynamical brain functional networks to identify subjective cognitive declines. We retrospectively constructed an internal data set (with 112 subjective cognitive decline and 64 healthy control subjects) to develop and internally validate the deep learning model. Conventional deep learning methods based on static and dynamic brain functional networks are compared. After the model is established, we prospectively collect an external data set (26 subjective cognitive decline and 12 healthy control subjects) for testing. Meanwhile, our method provides monitoring of the transitions between normal and abnormal (subjective cognitive decline-related) dynamical functional network states. The features of abnormal dynamical functional network states are quantified by network and variability metrics and associated with individual cognitions. Our method achieves an area under the receiver operating characteristic curve of 0.807 ± 0.046 in the internal validation data set and of 0.707 (P = 0.007) in the external testing data set, which shows improvements compared to conventional methods. The method further suggests that, at the local level, the abnormal dynamical functional network states are characterized by decreased connectivity strength and increased connectivity variability at different spatial scales. At the network level, the abnormal states are featured by scale-specifically altered modularity and all-scale decreased efficiency. Low tendencies to stay in abnormal states and high state transition variabilities are significantly associated with high general, language and executive functions. Overall, our work supports the deficits in multiscale brain dynamical functional networks detected by the deep learning method as reliable and meaningful neural alternation underpinning subjective cognitive decline.

Multi-modal Modality-masked Diffusion Network for Brain MRI Synthesis with Random Modality Missing.

Synthesis of unavailable imaging modalities from available ones can generate modality-specific complementary information and enable multi-modality based medical images diagnosis or treatment. Existing generative methods for medical image synthesis are usually based on cross-modal translation between acquired and missing modalities. These methods are usually dedicated to specific missing modality and perform synthesis in one shot, which cannot deal with varying number of missing modalities flexibly and construct the mapping across modalities effectively. To address the above issues, in this paper, we propose a unified Multi-modal Modality-masked Diffusion Network (M2DN), tackling multi-modal synthesis from the perspective of "progressive whole-modality inpainting", instead of "cross-modal translation". Specifically, our M2DN considers the missing modalities as random noise and takes all the modalities as a unity in each reverse diffusion step. The proposed joint synthesis scheme performs synthesis for the missing modalities and self-reconstruction for the available ones, which not only enables synthesis for arbitrary missing scenarios, but also facilitates the construction of common latent space and enhances the model representation ability. Besides, we introduce a modality-mask scheme to encode availability status of each incoming modality explicitly in a binary mask, which is adopted as condition for the diffusion model to further enhance the synthesis performance of our M2DN for arbitrary missing scenarios. We carry out experiments on two public brain MRI datasets for synthesis and downstream segmentation tasks. Experimental results demonstrate that our M2DN outperforms the state-of-the-art models significantly and shows great generalizability for arbitrary missing modalities.

ScribFormer: Transformer Makes CNN Work Better for Scribble-based Medical Image Segmentation.

Most recent scribble-supervised segmentation methods commonly adopt a CNN framework with an encoder-decoder architecture. Despite its multiple benefits, this framework generally can only capture small-range feature dependency for the convolutional layer with the local receptive field, which makes it difficult to learn global shape information from the limited information provided by scribble annotations. To address this issue, this paper proposes a new CNN-Transformer hybrid solution for scribble-supervised medical image segmentation called ScribFormer. The proposed ScribFormer model has a triple-branch structure, i.e., the hybrid of a CNN branch, a Transformer branch, and an attention-guided class activation map (ACAM) branch. Specifically, the CNN branch collaborates with the Transformer branch to fuse the local features learned from CNN with the global representations obtained from Transformer, which can effectively overcome limitations of existing scribble-supervised segmentation methods. Furthermore, the ACAM branch assists in unifying the shallow convolution features and the deep convolution features to improve model's performance further. Extensive experiments on two public datasets and one private dataset show that our ScribFormer has superior performance over the state-of-the-art scribble-supervised segmentation methods, and achieves even better results than the fully-supervised segmentation methods. The code is released at https://github.com/HUANGLIZI/ScribFormer.

Multimodal Brain Tumor Segmentation Boosted by Monomodal Normal Brain Images.

Many deep learning based methods have been proposed for brain tumor segmentation. Most studies focus on deep network internal structure to improve the segmentation accuracy, while valuable external information, such as normal brain appearance, is often ignored. Inspired by the fact that radiologists often screen lesion regions with normal appearance as reference in mind, in this paper, we propose a novel deep framework for brain tumor segmentation, where normal brain images are adopted as reference to compare with tumor brain images in a learned feature space. In this way, features at tumor regions, i.e., tumor-related features, can be highlighted and enhanced for accurate tumor segmentation. It is known that routine tumor brain images are multimodal, while normal brain images are often monomodal. This causes the feature comparison a big issue, i.e., multimodal vs. monomodal. To this end, we present a new feature alignment module (FAM) to make the feature distribution of monomodal normal brain images consistent/inconsistent with multimodal tumor brain images at normal/tumor regions, making the feature comparison effective. Both public (BraTS2022) and in-house tumor brain image datasets are used to evaluate our framework. Experimental results demonstrate that for both datasets, our framework can effectively improve the segmentation accuracy and outperforms the state-of-the-art segmentation methods. Codes are available at https://github.com/hb-liu/Normal-Brain-Boost-Tumor-Segmentation.

A Unified Multi-Modality Fusion Framework for Deep Spatio-Temporal-Spectral Feature Learning in Resting-State fMRI Denoising.

Resting-state functional magnetic resonance imaging (rs-fMRI) is a commonly used functional neuroimaging technique to investigate the functional brain networks. However, rs-fMRI data are often contaminated with noise and artifacts that adversely affect the results of rs-fMRI studies. Several machine/deep learning methods have achieved impressive performance to automatically regress the noise-related components decomposed from rs-fMRI data, which are expressed as the pairs of a spatial map and its associated time series. However, most of the previous methods individually analyze each modality of the noise-related components and simply aggregate the decision-level information (or knowledge) extracted from each modality to make a final decision. Moreover, these approaches consider only the limited modalities making it difficult to explore class-discriminative spectral information of noise-related components. To overcome these limitations, we propose a unified deep attentive spatio-spectral-temporal feature fusion framework. We first adopt a learnable wavelet transform module at the input-level of the framework to elaborately explore the spectral information in subsequent processes. We then construct a feature-level multi-modality fusion module to efficiently exchange the information from multi-modality inputs in the feature space. Finally, we design confidence-based voting strategies for decision-level fusion at the end of the framework to make a robust final decision. In our experiments, the proposed method achieved remarkable performance for noise-related component detection on various rs-fMRI datasets.

A deep learning system for predicting time to progression of diabetic retinopathy.

Diabetic retinopathy (DR) is the leading cause of preventable blindness worldwide. The risk of DR progression is highly variable among different individuals, making it difficult to predict risk and personalize screening intervals. We developed and validated a deep learning system (DeepDR Plus) to predict time to DR progression within 5 years solely from fundus images. First, we used 717,308 fundus images from 179,327 participants with diabetes to pretrain the system. Subsequently, we trained and validated the system with a multiethnic dataset comprising 118,868 images from 29,868 participants with diabetes. For predicting time to DR progression, the system achieved concordance indexes of 0.754-0.846 and integrated Brier scores of 0.153-0.241 for all times up to 5 years. Furthermore, we validated the system in real-world cohorts of participants with diabetes. The integration with clinical workflow could potentially extend the mean screening interval from 12 months to 31.97 months, and the percentage of participants recommended to be screened at 1-5 years was 30.62%, 20.00%, 19.63%, 11.85% and 17.89%, respectively, while delayed detection of progression to vision-threatening DR was 0.18%. Altogether, the DeepDR Plus system could predict individualized risk and time to DR progression over 5 years, potentially allowing personalized screening intervals.

Modality-Specific Information Disentanglement From Multi-Parametric MRI for Breast Tumor Segmentation and Computer-Aided Diagnosis.

Breast cancer is becoming a significant global health challenge, with millions of fatalities annually. Magnetic Resonance Imaging (MRI) can provide various sequences for characterizing tumor morphology and internal patterns, and becomes an effective tool for detection and diagnosis of breast tumors. However, previous deep-learning based tumor segmentation methods from multi-parametric MRI still have limitations in exploring inter-modality information and focusing task-informative modality/modalities. To address these shortcomings, we propose a Modality-Specific Information Disentanglement (MoSID) framework to extract both inter- and intra-modality attention maps as prior knowledge for guiding tumor segmentation. Specifically, by disentangling modality-specific information, the MoSID framework provides complementary clues for the segmentation task, by generating modality-specific attention maps to guide modality selection and inter-modality evaluation. Our experiments on two 3D breast datasets and one 2D prostate dataset demonstrate that the MoSID framework outperforms other state-of-the-art multi-modality segmentation methods, even in the cases of missing modalities. Based on the segmented lesions, we further train a classifier to predict the patients' response to radiotherapy. The prediction accuracy is comparable to the case of using manually-segmented tumors for treatment outcome prediction, indicating the robustness and effectiveness of the proposed segmentation method. The code is available at https://github.com/Qianqian-Chen/MoSID.

One model, two brains: Automatic fetal brain extraction from MR images of twins.

Fetal brain extraction from magnetic resonance (MR) images is of great importance for both clinical applications and neuroscience studies. However, it is a challenging task, especially when dealing with twins, which are commonly existing in pregnancy. Currently, there is no brain extraction method dedicated to twins, raising significant demand to develop an effective twin fetal brain extraction method. To this end, we propose the first twin fetal brain extraction framework, which possesses three novel features. First, to narrow down the region of interest and preserve structural information between the two brains in twin fetal MR images, we take advantage of an advanced object detector to locate all the brains in twin fetal MR images at once. Second, we propose a Twin Fetal Brain Extraction Network (TFBE-Net) to further suppress insignificant features for segmenting brain regions. Finally, we propose a Two-step Training Strategy (TTS) to learn correlation features of the single fetal brain for further improving the performance of TFBE-Net. We validate the proposed framework on a twin fetal brain dataset. The experiments show that our framework achieves promising performance on both quantitative and qualitative evaluations, and outperforms state-of-the-art methods for fetal brain extraction.

Beam-wise dose composition learning for head and neck cancer dose prediction in radiotherapy.

Automatic and accurate dose distribution prediction plays an important role in radiotherapy plan. Although previous methods can provide promising performance, most methods did not consider beam-shaped radiation of treatment delivery in clinical practice. This leads to inaccurate prediction, especially on beam paths. To solve this problem, we propose a beam-wise dose composition learning (BDCL) method for dose prediction in the context of head and neck (H&N) radiotherapy plan. Specifically, a global dose network is first utilized to predict coarse dose values in the whole-image space. Then, we propose to generate individual beam masks to decompose the coarse dose distribution into multiple field doses, called beam voters, which are further refined by a subsequent beam dose network and reassembled to form the final dose distribution. In particular, we design an overlap consistency module to keep the similarity of high-level features in overlapping regions between different beam voters. To make the predicted dose distribution more consistent with the real radiotherapy plan, we also propose a dose-volume histogram (DVH) calibration process to facilitate feature learning in some clinically concerned regions. We further apply an edge enhancement procedure to enhance the learning of the extracted feature from the dose falloff regions. Experimental results on a public H&N cancer dataset from the AAPM OpenKBP challenge show that our method achieves superior performance over other state-of-the-art approaches by significant margins. Source code is released at https://github.com/TL9792/BDCLDosePrediction.

DSMT-Net: Dual Self-Supervised Multi-Operator Transformation for Multi-Source Endoscopic Ultrasound Diagnosis.

Pancreatic cancer has the worst prognosis of all cancers. The clinical application of endoscopic ultrasound (EUS) for the assessment of pancreatic cancer risk and of deep learning for the classification of EUS images have been hindered by inter-grader variability and labeling capability. One of the key reasons for these difficulties is that EUS images are obtained from multiple sources with varying resolutions, effective regions, and interference signals, making the distribution of the data highly variable and negatively impacting the performance of deep learning models. Additionally, manual labeling of images is time-consuming and requires significant effort, leading to the desire to effectively utilize a large amount of unlabeled data for network training. To address these challenges, this study proposes the Dual Self-supervised Multi-Operator Transformation Network (DSMT-Net) for multi-source EUS diagnosis. The DSMT-Net includes a multi-operator transformation approach to standardize the extraction of regions of interest in EUS images and eliminate irrelevant pixels. Furthermore, a transformer-based dual self-supervised network is designed to integrate unlabeled EUS images for pre-training the representation model, which can be transferred to supervised tasks such as classification, detection, and segmentation. A large-scale EUS-based pancreas image dataset (LEPset) has been collected, including 3,500 pathologically proven labeled EUS images (from pancreatic and non-pancreatic cancers) and 8,000 unlabeled EUS images for model development. The self-supervised method has also been applied to breast cancer diagnosis and was compared to state-of-the-art deep learning models on both datasets. The results demonstrate that the DSMT-Net significantly improves the accuracy of pancreatic and breast cancer diagnosis.

A Novel Approach Analysing the Dynamic Brain Functional Connectivity for Improved MCI Detection.

OBJECTIVE: Dynamic functional connectivity (dFC) of the brain has been explored for the detection of mild cognitive impairment (MCI), preventing potential development of Alzheimer's disease. Deep learning is widely used method for dFC analysis but is unfortunately computationally expensive and unexplainable. Root mean square value (RMS) of the pairwise Pearson's correlation of the dFC is also proposed but is insufficient for accurate MCI detection. The present study aims at exploring the feasibility of several novel features for dFC analysis, and thus, reliable MCI detection. METHODS: A public resting-state functional magnetic resonance imaging dataset containing healthy controls (HC), early MCI (eMCI), and late MCI (lMCI) patients was used. In addition to RMS, nine features were extracted from the pairwise Pearson's correlation of the dFC, inducing amplitude-, spectral-, entropy-, and autocorrelation-related features, and time reversibility. A Student's t-test and a least absolute shrinkage and selection operator (LASSO) regression were employed for feature dimension reduction. A SVM was then adopted for two classification objectives: HC vs. lMCI and HC vs. eMCI. Accuracy, sensitivity, specificity, F1-score, and area under the receiver operating characteristic curve were calculated as performance metrics. RESULTS: 6109 out of 66700 features are significantly different between HC and lMCI and 5905 between HC and eMCI. Besides, the proposed features produce excellent classification results for both tasks, outperforming most of the existing methods. SIGNIFICANCE: This study proposes a novel and general framework for dFC analysis, providing a promising tool for the detection of many neurological brain diseases using different brain signals.

An Anatomy- and Topology-Preserving Framework for Coronary Artery Segmentation.

Coronary artery segmentation is critical for coronary artery disease diagnosis but challenging due to its tortuous course with numerous small branches and inter-subject variations. Most existing studies ignore important anatomical information and vascular topologies, leading to less desirable segmentation performance that usually cannot satisfy clinical demands. To deal with these challenges, in this paper we propose an anatomy- and topology-preserving two-stage framework for coronary artery segmentation. The proposed framework consists of an anatomical dependency encoding (ADE) module and a hierarchical topology learning (HTL) module for coarse-to-fine segmentation, respectively. Specifically, the ADE module segments four heart chambers and aorta, and thus five distance field maps are obtained to encode distance between chamber surfaces and coarsely segmented coronary artery. Meanwhile, ADE also performs coronary artery detection to crop region-of-interest and eliminate foreground-background imbalance. The follow-up HTL module performs fine segmentation by exploiting three hierarchical vascular topologies, i.e., key points, centerlines, and neighbor connectivity using a multi-task learning scheme. In addition, we adopt a bottom-up attention interaction (BAI) module to integrate the feature representations extracted across hierarchical topologies. Extensive experiments on public and in-house datasets show that the proposed framework achieves state-of-the-art performance for coronary artery segmentation.