ScribFormer: Transformer Makes CNN Work Better for Scribble-Based Medical Image Segmentation.

Most recent scribble-supervised segmentation methods commonly adopt a CNN framework with an encoder-decoder architecture. Despite its multiple benefits, this framework generally can only capture small-range feature dependency for the convolutional layer with the local receptive field, which makes it difficult to learn global shape information from the limited information provided by scribble annotations. To address this issue, this paper proposes a new CNN-Transformer hybrid solution for scribble-supervised medical image segmentation called ScribFormer. The proposed ScribFormer model has a triple-branch structure, i.e., the hybrid of a CNN branch, a Transformer branch, and an attention-guided class activation map (ACAM) branch. Specifically, the CNN branch collaborates with the Transformer branch to fuse the local features learned from CNN with the global representations obtained from Transformer, which can effectively overcome limitations of existing scribble-supervised segmentation methods. Furthermore, the ACAM branch assists in unifying the shallow convolution features and the deep convolution features to improve model's performance further. Extensive experiments on two public datasets and one private dataset show that our ScribFormer has superior performance over the state-of-the-art scribble-supervised segmentation methods, and achieves even better results than the fully-supervised segmentation methods. The code is released at https://github.com/HUANGLIZI/ScribFormer.

A direct slicing technique for the 3D printing of implicitly represented medical models.

In conventional medical image printing methods, volumetric medical data needs to be conversed into STereo Lithography (STL) format, the most commonly used format for representing geometric models for 3D printing. However, this STL conversion process is not only time consuming, but more importantly, it often leads to the loss of accuracy. It has become a critical factor hindering the printing efficiency and precision of organ models. By examining the key characteristics of discrete medical volume data, this paper proposes a direct slicing technique for printing implicitly represented 3D medical models. The proposed method mainly consists of three algorithms: (1) A layer-based contour extraction algorithm for discrete volume data; (2) An inner shell construction algorithm based on discrete point differential indentation; (3) An infill generation algorithm based on the constructed virtual contour and scan lines. The proposed method has been applied to the slicing of several organ models for experiments, and the ratios of time cost and memory cost between the conventional method and the proposed method are about 4-100 and 1.1 to 1.4 respectively, which demonstrate that the proposed method has a great improvement in both time and space performance when compared with the conventional STL-based method. Our technique extends the direct input format of geometric models for additive manufacturing. That is, discrete volume data can be used as a direct input for additive manufacturing without conversion to STL format.

High-quality vascular modeling and modification with implicit extrusion surfaces for blood flow computations.

BACKGROUND AND OBJECTIVE: High-quality vascular modeling is crucial for blood flow simulations, i.e., computational fluid dynamics (CFD). As without an accurate geometric representation of the smooth vascular surface, it is impossible to make meaningful blood flow simulations. The purpose of this work is to develop high-quality vascular modeling and modification method for blood flow computations. METHODS: We develop a new technique for the accurate geometric modeling and modification of vasculatures using implicit extrusion surfaces (IES). In the proposed method, the skeleton of the vascular structure is subdivided into short curve segments, each of which is then represented implicitly locally as the intersection of two mutually orthogonal implicit surfaces defined by distance functions. A set of contour points is extracted and fitted with an implicit curve for accurately specifying the vessel cross-section profile, which is then extruded locally along the skeleton to fill the gaps between two vascular tube cross sections. We also present a new implicit geometric editing technique to modify the constructed vascular model with pathology for virtual stenting. RESULTS: Experimental results and validations show that accurate vascular models with highly smooth surfaces can be generated by the proposed method. In addition, we conduct some blood flow simulations to indicate the effectiveness of proposed method for hemodynamic simulations. CONCLUSIONS: The proposed technique can achieve precise geometric models of vasculatures with any required degree of smoothness for reliable blood flow simulations.

Face Frontalization Using an Appearance-Flow-Based Convolutional Neural Network.

Facial pose variation is one of the major factors making face recognition (FR) a challenging task. One popular solution is to convert non-frontal faces to frontal ones on which FR is performed. Rotating faces causes facial pixel value changes. Therefore, existing CNN-based methods learn to synthesize frontal faces in color space. However, this learning problem in a color space is highly non-linear, causing the synthetic frontal faces to lose fine facial textures. In this paper, we take the view that the nonfrontal-frontal pixel changes are essentially caused by geometric transformations (rotation, translation, and so on) in space. Therefore, we aim to learn the nonfrontal-frontal facial conversion in the spatial domain rather than the color domain to ease the learning task. To this end, we propose an appearance-flow-based face frontalization convolutional neural network (A3F-CNN). Specifically, A3F-CNN learns to establish the dense correspondence between the non-frontal and frontal faces. Once the correspondence is built, frontal faces are synthesized by explicitly "moving" pixels from the non-frontal one. In this way, the synthetic frontal faces can preserve fine facial textures. To improve the convergence of training, an appearance-flow-guided learning strategy is proposed. In addition, generative adversarial network loss is applied to achieve a more photorealistic face, and a face mirroring method is introduced to handle the self-occlusion problem. Extensive experiments are conducted on face synthesis and pose invariant FR. Results show that our method can synthesize more photorealistic faces than the existing methods in both the controlled and uncontrolled lighting environments. Moreover, we achieve a very competitive FR performance on the Multi-PIE, LFW and IJB-A databases.

Research on Techniques of Multifeatures Extraction for Tongue Image and Its Application in Retrieval.

Tongue diagnosis is one of the important methods in the Chinese traditional medicine. Doctors can judge the disease's situation by observing patient's tongue color and texture. This paper presents a novel approach to extract color and texture features of tongue images. First, we use improved GLA (Generalized Lloyd Algorithm) to extract the main color of tongue image. Considering that the color feature cannot fully express tongue image information, the paper analyzes tongue edge's texture features and proposes an algorithm to extract them. Then, we integrate the two features in retrieval by different weight. Experimental results show that the proposed method can improve the detection rate of lesion in tongue image relative to single feature retrieval.

3D vasculature segmentation using localized hybrid level-set method.

BACKGROUND: Intensity inhomogeneity occurs in many medical images, especially in vessel images. Overcoming the difficulty due to image inhomogeneity is crucial for the segmentation of vessel image. METHODS: This paper proposes a localized hybrid level-set method for the segmentation of 3D vessel image. The proposed method integrates both local region information and boundary information for vessel segmentation, which is essential for the accurate extraction of tiny vessel structures. The local intensity information is firstly embedded into a region-based contour model, and then incorporated into the level-set formulation of the geodesic active contour model. Compared with the preset global threshold based method, the use of automatically calculated local thresholds enables the extraction of the local image information, which is essential for the segmentation of vessel images. RESULTS: Experiments carried out on the segmentation of 3D vessel images demonstrate the strengths of using locally specified dynamic thresholds in our level-set method. Furthermore, both qualitative comparison and quantitative validations have been performed to evaluate the effectiveness of our proposed model. CONCLUSIONS: Experimental results and validations demonstrate that our proposed model can achieve more promising segmentation results than the original hybrid method does.

GPU accelerating technique for rendering implicitly represented vasculatures.

With the flooding datasets of medical Computed Tomography (CT) and Magnetic Resonance Imaging (MRI), implicit modeling techniques are increasingly applied to reconstruct the human organs, especially the vasculature. However, displaying implicitly represented geometric objects arises heavy computational burden. In this study, a Graphics Processing Unit (GPU) accelerating technique was developed for high performance rendering of implicitly represented objects, especially the vasculatures. The experimental results suggested that the rendering performance was greatly enhanced via exploiting the advantages of modern GPUs.

Propagation of Hermite-Gaussian and Laguerre-Gaussian beams beyond the paraxial approximation.

On the basis of the vectorial Rayleigh-Sommerfeld formulas and by means of the relation between Hermite and Laguerre polynomials, the analytical expressions for the propagation of the Hermite-Gaussian (HG) and Laguerre-Gaussian (LG) beams beyond the paraxial approximation are derived, with the corresponding far-field propagation expressions and that for the Gaussian beams being given as special cases of the results. Some detailed comparisons of our results with the expansion series and paraxial expressions are made, which show the advantages of our results over the expansion series. With the results obtained, some typical intensity patterns of nonparaxial HG and LG beams are shown.