Pact-Net: Parallel CNNs and Transformers for medical image segmentation.

BACKGROUND AND OBJECTIVE: The image segmentation of diseases can help clinical diagnosis and treatment in medical image analysis. Because medical images usually have low contrast and large changes in the size and shape of some structures, this will lead to over-segmentation and under-segmentation. These problems are particularly evident in the segmentation of skin damage. The blurring of the boundary in skin images and the specificity of patients will further increase the difficulty of skin lesion segmentation. Currently, most researchers use deep learning networks to solve these skin segmentation problems. However, traditional convolution methods often fail to obtain satisfactory segmentation performance due to their shortcomings in obtaining global features. Recently, Transformers with good global information extraction ability has achieved satisfactory results in computer vision, which brings new solutions to optimize the model of medical image segmentation further. METHODS: To extract more features related to medical image segmentation and effectively use features to further optimize the skin image segmentation model, we designed a network that combines CNNs and Transformers to improve local and global features, called Parallel CNNs and Transformers for Medical Image Segmentation (Pact-Net). Specifically, due to the advantages of Transformers in extracting global information, we create a novel fusion module CSMF, which uses channel and spatial attention mechanism and multi-scale mechanism to effectively fuse the global information extracted by Transformers into the local features extracted by CNNs. Therefore, our Pact-Net dual-branch runs in parallel to effectively capture global and local information. RESULTS: Our Pact-Net exceeds the models submitted on the three datasets ISIC 2016, ISIC 2017 and ISIC 2018, and the indicators required for the datasets reach 86.95%, 79.31% and 84.14%, respectively. We also conducted medical image segmentation experiments on cell and polyp datasets to evaluate the robustness, learning and generalization ability of the network. The ablation study of each part of Pact-Net proves the validity of each component, and the comparison with state-of-the-art methods on different indicators proves the predominance of the network. CONCLUSIONS: This paper uses the advantages of CNNs and Transformers in extracting local and global features, and further integrates features for skin lesion segmentation. Compared with the state-of-the-art methods, Pact-Net can achieve the most advanced segmentation ability on the skin lesion segmentation dataset, which can help doctors diagnose and treat diseases.

Retinal blood vessel segmentation by using the MS-LSDNet network and geometric skeleton reconnection method.

Automatic retinal blood vessel segmentation is a key link in the diagnosis of ophthalmic diseases. Recent deep learning methods have achieved high accuracy in vessel segmentation but still face challenges in maintaining vascular structural connectivity. Therefore, this paper proposes a novel retinal blood vessel segmentation strategy that includes three stages: vessel structure detection, vessel branch extraction and broken vessel segment reconnection. First, we propose a multiscale linear structure detection network (MS-LSDNet), which improves the detection ability of fine blood vessels by learning the types of rich hierarchical features. In addition, to maintain the connectivity of the vascular structure in the process of binarization of the vascular probability map, an adaptive hysteresis threshold method for vascular extraction is proposed. Finally, we propose a vascular tree structure reconstruction algorithm based on a geometric skeleton to connect the broken vessel segments. Experimental results on three publicly available datasets show that compared with current state-of-the-art algorithms, our strategy effectively maintains the connectivity of retinal vascular tree structure.

X-ray coronary centerline extraction based on C-UNet and a multifactor reconnection algorithm.

BACKGROUND AND OBJECTIVE: Accurate extraction of the coronary artery centerline is crucial in the processes of coronary artery reconstruction, coronary artery stenosis or lesion detection, and surgical navigation. Furthermore, in clinical medicine, the complex background of angiography, low signal-to-noise ratio, and complex vascular structure make coronary artery centerline extraction challenging. In this study, a direct centerline extraction method is proposed that automatically and accurately extracts vascular centerlines from X-ray coronary angiography images based on deep learning and conventional methods. METHODS: In this study, a coronary artery centerline extraction method is proposed that comprises two parts: the preliminary centerline extraction network based on U-Net with a residual network, called C-UNet, and the multifactor centerline reconnection algorithm based on the geometric characteristics of blood vessels. RESULTS: The qualitative and quantitative results demonstrate the effectiveness of the presented method. In this study, three widely used evaluation indices were adopted to evaluate the performance of the method: precision, recall, and F1_Score. The experimental results show that this method can accurately extract coronary artery centerlines. CONCLUSIONS: The proposed centerline extraction method accurately extracts centerlines from X-ray coronary angiography images and improves both the accuracy and continuity of centerline extraction.

Automated coronary artery tree segmentation in coronary CTA using a multiobjective clustering and toroidal model-guided tracking method.

BACKGROUND AND OBJECTIVE: Accurate coronary artery tree segmentation can now be developed to assist radiologists in detecting coronary artery disease. In clinical medicine, the noise, low contrast, and uneven intensity of medical images along with complex shapes and vessel bifurcation structures make coronary artery segmentation challenging. In this work, we propose a multiobjective clustering and toroidal model-guided tracking method that can accurately extract coronary arteries from computed tomography angiography (CTA) imagery. METHODS: Utilizing integrated noise reduction, candidate region detection, geometric feature extraction, and coronary artery tracking techniques, a new segmentation framework for 3D coronary artery trees is presented. The candidate regions are extracted using a multiobjective clustering method, and the coronary arteries are tracked by a toroidal model-guided tracking method. RESULTS: The qualitative and quantitative results demonstrate the effectiveness of the presented framework, which achieves better performance than the compared segmentation methods in three widely used evaluation indices: the Dice similarity coefficient (DSC), Jaccard index and Recall across the CTA data. The proposed method can accurately identify the coronary artery tree with a mean DSC of 84%, a Jaccard index of 74%, and a Recall of 93%. CONCLUSIONS: The proposed segmentation framework effectively segments the coronary tree from the CTA volume, which improves the accuracy of 3D vascular tree segmentation.

Single-Image Super-Resolution Based on Rational Fractal Interpolation.

This paper presents a novel single-image super-resolution (SR) procedure, which upscales a given low-resolution (LR) input image to a high-resolution image while preserving the textural and structural information. First, we construct a new type of bivariate rational fractal interpolation model and investigate its analytical properties. This model has different forms of expression with various values of the scaling factors and shape parameters; thus, it can be employed to better describe image features than current interpolation schemes. Furthermore, this model combines the advantages of rational interpolation and fractal interpolation, and its effectiveness is validated through theoretical analysis. Second, we develop a single-image SR algorithm based on the proposed model. The LR input image is divided into texture and non-texture regions, and then, the image is interpolated according to the characteristics of the local structure. Specifically, in the texture region, the scaling factor calculation is the critical step. We present a method to accurately calculate scaling factors based on local fractal analysis. Extensive experiments and comparisons with the other state-of-the-art methods show that our algorithm achieves competitive performance, with finer details and sharper edges.