Smoke-Aware Global-Interactive Non-Local Network for Smoke Semantic Segmentation.

Compared with other objects, smoke semantic segmentation (SSS) is more difficult and challenging due to some special characteristics of smoke, such as non-rigid, translucency, variable mode and so on. To achieve accurate positioning of smoke in real complex scenes and promote the development of intelligent fire detection, we propose a Smoke-Aware Global-Interactive Non-local Network (SAGINN) for SSS, which harness the power of both convolution and transformer to capture local and global information simultaneously. Non-local is a powerful means for modeling long-range context dependencies, however, friendliness to single-scale low-resolution features limits its potential to produce high-quality representations. Therefore, we propose a Global-Interactive Non-local (GINL) module, leveraging global interaction between multi-scale key information to improve the robustness of feature representations. To solve the interference of smoke-like objects, a Pyramid High-level Semantic Aggregation (PHSA) module is designed, where the learned high-level category semantics from classification aids model by providing additional guidance to correct the wrong information in segmentation representations at the image level and alleviate the inter-class similarity problem. Besides, we further propose a novel loss function, termed Smoke-aware loss (SAL), by assigning different weights to different objects contingent on their importance. We evaluate our SAGINN on extensive synthetic and real data to verify its generalization ability. Experimental results show that SAGINN achieves 83% average mIoU on the three testing datasets (83.33%, 82.72% and 82.94%) of SYN70K with an accuracy improvement of about 0.5%, 0.002 mMse and 0.805 Fß on SMOKE5K, which can obtain more accurate location and finer boundaries of smoke, achieving satisfactory results on smoke-like objects.

A Gated Recurrent Network With Dual Classification Assistance for Smoke Semantic Segmentation.

Smoke has semi-transparency property leading to highly complicated mixture of background and smoke. Sparse or small smoke is visually inconspicuous, and its boundary is often ambiguous. These reasons result in a very challenging task of separating smoke from a single image. To solve these problems, we propose a Classification-assisted Gated Recurrent Network (CGRNet) for smoke semantic segmentation. To discriminate smoke and smoke-like objects, we present a smoke segmentation strategy with dual classification assistance. Our classification module outputs two prediction probabilities for smoke. The first assistance is to use one probability to explicitly regulate the segmentation module for accuracy improvement by supervising a cross-entropy classification loss. The second one is to multiply the segmentation result by another probability for further refinement. This dual classification assistance greatly improves performance at image level. In the segmentation module, we design an Attention Convolutional GRU module (Att-ConvGRU) to learn the long-range context dependence of features. To perceive small or inconspicuous smoke, we design a Multi-scale Context Contrasted Local Feature structure (MCCL) and a Dense Pyramid Pooling Module (DPPM) for improving the representation ability of our network. Extensive experiments validate that our method significantly outperforms existing state-of-art algorithms on smoke datasets, and also obtain satisfactory results on challenging images with inconspicuous smoke and smoke-like objects.

Differential Diagnosis of Atypical Hepatocellular Carcinoma in Contrast-Enhanced Ultrasound Using Spatio-Temporal Diagnostic Semantics.

Atypical Hepatocellular Carcinoma (HCC) is very hard to distinguish from Focal Nodular Hyperplasia (FNH) in routine imaging. However little attention was paid to this problem. This paper proposes a novel liver tumor Computer-Aided Diagnostic (CAD) approach extracting spatio-temporal semantics for atypical HCC. With respect to useful diagnostic semantics, our model automatically calculates three types of semantic feature with equally down-sampled frames based on Contrast-Enhanced Ultrasound (CEUS). Thereafter, a Support Vector Machine (SVM) classifier is trained to make the final diagnosis. Compared with traditional methods for diagnosing HCC, the proposed model has the advantage of less computational complexity and being able to handle the atypical HCC cases. The experimental results show that our method obtained a pretty considerable performance and outperformed two traditional methods. According to the results, the average accuracy reaches 94.40%, recall rate 94.76%, F1-score value 94.62%, specificity 93.62% and sensitivity 94.76%, indicating good merit for automatically diagnosing atypical HCC cases.

Segmentation of breast ultrasound image with semantic classification of superpixels.

Breast cancer is a great threat to females. Ultrasound imaging has been applied extensively in diagnosis of breast cancer. Due to the poor image quality, segmentation of breast ultrasound (BUS) image remains a very challenging task. Besides, BUS image segmentation is a crucial step for further analysis. In this paper, we proposed a novel method to segment the breast tumor via semantic classification and merging patches. The proposed method firstly selects two diagonal points to crop a region of interest (ROI) on the original image. Then, histogram equalization, bilateral filter and pyramid mean shift filter are adopted to enhance the image. The cropped image is divided into many superpixels using simple linear iterative clustering (SLIC). Furthermore, some features are extracted from the superpixels and a bag-of-words model can be created. The initial classification can be obtained by a back propagation neural network (BPNN). To refine preliminary result, k-nearest neighbor (KNN) is used for reclassification and the final result is achieved. To verify the proposed method, we collected a BUS dataset containing 320 cases. The segmentation results of our method have been compared with the corresponding results obtained by five existing approaches. The experimental results show that our method achieved competitive results compared to conventional methods in terms of TP and FP, and produced good approximations to the hand-labelled tumor contours with comprehensive consideration of all metrics (the F1-score = 89.87% ± 4.05%, and the average radial error = 9.95% ± 4.42%).

A Wave-shaped Deep Neural Network for Smoke Density Estimation.

Smoke density estimation from a single image is a totally new but highly ill-posed problem. To solve the problem, we stack several convolutional encoder-decoder structures together to propose a wave-shaped neural network, termed W-Net. Stacking encoder-decoders directly increases the network depth, leading to the enlargement of receptive fields for encoding more semantic information. To maximize the degrees of feature re-usage, we copy and resize the outputs of encoding layers to corresponding decoding layers, and then concatenate them to implement short-cut connections for improving spatial accuracy. The crests and troughs of W-Net are special structures containing abundant localization and semantic information, so we also use short-cut connections between these structures and decoding layers. Estimated smoke density is useful in many applications, such as smoke segmentation, smoke detection, disaster simulation. Experimental results show that our method outperforms existing methods on both smoke density estimation and segmentation. It also achieves satisfying results in visual detection of auto exhausts.

A variational surface deformation and subdivision-based modeling framework for noisy and small n-furcated tube-like structures.

It is challenging to construct an accurate and smooth mesh for noisy and small n-furcated tube-like structures, such as arteries, veins, and pathological vessels, due to tiny vessel size, noise, n -furcations, and irregular shapes of pathological vessels. We propose a framework by dividing the modeling process into mesh construction and mesh refinement. In the first step, we focus on mesh topological correctness, and just create an initial rough mesh for the n-furcated tube-like structures. In the second step, we propose a variational surface deformation method to push the initial mesh to structure boundaries for positional accuracy improvement. By iteratively solving Euler-Lagrange equations derived from the minimization of the shell and distance energies, the initial mesh can be gradually pushed to the boundaries. A mesh dilation method is proposed to prevent the extremely deviated initial mesh moving toward wrong boundaries. We combine deformation and subdivision to propose a coarse-to-fine modeling framework for the improvement of efficiency and accuracy. Experiments show our method can construct an accurate and smooth mesh for noisy and small n-furcated tube-like structures, and it is useful in hemodynamics, quantitative measurement, and analysis of vessels.

Modeling n-furcated liver vessels from a 3-D segmented volume using hole-making and subdivision methods.

It is difficult to build an accurate and smooth liver vessel model due to the tiny size, noise, and n-furcations of vessels. To overcome these problems, we propose an n-furcation vessel tree modeling method. In this method, given a segmented volume and a point indicating the root of the vessels, centerlines and cross-sectional contours of the vessels are extracted and organized as a tree first. Then, the tree is broken up into separate branches in descending order of length, and polygonal meshes of all the branches are separately constructed from the cross-sectional contours. Finally, all the meshes are combined sequentially using our hole-making approach. Holes are made on a coarse mesh, and a final fine mesh is generated using a subdivision method. The hole-making approach with the subdivision method provides good efficiency in mesh construction as well as great flexibilities in mesh editing. Experiments show that our method can automatically construct smooth mesh models for n-furcated vessels with mean absolute error of 0.92 voxel and mean relative error of 0.17. It is promising to be used in diagnosis, analysis, and surgery simulation of liver diseases, and is able to model tubular structures with tree topology.

[Fast extraction of iso-surface in surface rendering of medical images].

In this paper a new direct volume rendering method is presented for fast extraction of iso-surface by adopting the idea from the Shear-Warp algorithm. By creating the sorted volumetric data from the original volume data and specifying a value range of data which determines the part of the sorted volumetric data traversed, the amount of volume data traversed would be reduced obviously and the extraction operation of iso-surface would be very fast. In addition, we can adjust the value range to obtain the different rendering speed and image quality according to the purpose in application. Moreover, the proposed algorithm will not output any intermediate data after the sorted volumetric data being produced. Therefore, it is possible to realize the rapid 3D surface reconstruction for medical images on the personal computer without the support of any hardware accelerator.