Multiple mask and boundary scoring R-CNN with cGAN data augmentation for bladder tumor segmentation in WLC videos.

Automatic diagnosis systems capable of handling multiple pathologies are essential in clinical practice. This study focuses on enhancing precise lesion localization, classification and delineation in transurethral resection of bladder tumor (TURBT) to reduce cancer recurrence. Despite deep learning models success, medical applications face challenges like small and limited datasets and poor image characterization, including the absence lack of color/texture modeling. To address these issues, three solutions are proposed: (1) an improved texture-constrained version of the pix2pixHD cGAN for data augmentation, addressing the tradeoff of generating high-quality images with enough stochasticity using the Fréchet Inception Distance (FID) measure. (2) Introducing the Multiple Mask and Boundary Scoring R-CNN (MM&BS R-CNN), a new mask sub-net scheme where multiple masks are generated from the different levels of the mask sub-net pipeline, improving segmentation accuracy by including a new scoring module to refine object boundaries. (3) A novel accelerated training strategy based on the SGD optimizer with the second momentum. Experimental results show significant mAP improvements: the data generation scheme improves by more than 12 %; MM&BS R-CNN proposed architecture is responsible for an improvement of about 1.25 %, and the training algorithm based on the second-order momentum increases mAP by 2-3 %. The simultaneous use of all three proposals improved the state-of-the-art mAP by 17.44 %.

Detection of bladder cancer with feature fusion, transfer learning and CapsNets.

This paper confronts two approaches to classify bladder lesions shown in white light cystoscopy images when using small datasets: the classical one, where handcrafted-based features feed pattern recognition systems and the modern deep learning-based (DL) approach. In between, there are alternative DL models that had not received wide attention from the scientific community, even though they can be more appropriate for small datasets such as the human brain motivated capsule neural networks (CapsNets). However, CapsNets have not yet matured hence presenting lower performances than the most classic DL models. These models require higher computational resources, more computational skills from the physician and are more prone to overfitting, making them sometimes prohibitive in the routine of clinical practice. This paper shows that carefully handcrafted features used with more robust models can reach similar performances to the conventional DL-based models and deep CapsNets, making them more useful for clinical applications. Concerning feature extraction, it is proposed a new feature fusion approach for Ta and T1 bladder tumor detection by using decision fusion from multiple classifiers in a scheme known as stacking of classifiers. Three Neural Networks perform classification on three different feature sets, namely: Covariance of Color Histogram of Oriented Gradients, proposed in the ambit of this paper; Local Binary Patterns and Wavelet Coefficients taken from lower scales. Data diversity is ensured by a fourth Neural Network, which is used for decision fusion by combining the outputs of the ensemble elements to produce the classifier output. Both Feed Forward Neural Networks and Radial Basis Functions are used in the experiments. Contrarily, DL-based models extract automatically the best features at the cost of requiring huge amounts of training data, which in turn can be alleviated by using the Transfer Learning (TL) strategy. In this paper VGG16 and ResNet-34 pretrained in ImageNet were used for TL, slightly outperforming the proposed ensemble. CapsNets may overcome CNNs given their ability to deal with objects rotational invariance and spatial relationships. Therefore, they can be trained from scratch in applications using small amounts of data, which was beneficial for the current case, improving accuracy from 94.6% to 96.9%.

Multi-pathology detection and lesion localization in WCE videos by using the instance segmentation approach.

The majority of current systems for automatic diagnosis considers the detection of a unique and previously known pathology. Considering specifically the diagnosis of lesions in the small bowel using endoscopic capsule images, very few consider the possible existence of more than one pathology and when they do, they are mainly detection based systems therefore unable to localize the suspected lesions. Such systems do not fully satisfy the medical community, that in fact needs a system that detects any pathology and eventually more than one, when they coexist. In addition, besides the diagnostic capability of these systems, localizing the lesions in the image has been of great interest to the medical community, mainly for training medical personnel purposes. So, nowadays, the inclusion of the lesion location in automatic diagnostic systems is practically mandatory. Multi-pathology detection can be seen as a multi-object detection task and as each frame can contain different instances of the same lesion, instance segmentation seems to be appropriate for the purpose. Consequently, we argue that a multi-pathology system benefits from using the instance segmentation approach, since classification and segmentation modules are both required complementing each other in lesion detection and localization. According to our best knowledge such a system does not yet exist for the detection of WCE pathologies. This paper proposes a multi-pathology system that can be applied to WCE images, which uses the Mask Improved RCNN (MI-RCNN), a new mask subnet scheme which has shown to significantly improve mask predictions of the high performing state-of-the-art Mask-RCNN and PANet systems. A novel training strategy based on the second momentum is also proposed for the first time for training Mask-RCNN and PANet based systems. These approaches were tested using the public database KID, and the included pathologies were bleeding, angioectasias, polyps and inflammatory lesions. Experimental results show significant improvements for the proposed versions, reaching increases of almost 7% over the PANet model when the new proposed training approach was employed.

Automatic detection of small bowel tumors in wireless capsule endoscopy images using ensemble learning.

PURPOSE: Wireless Capsule Endoscopy (WCE) is a minimally invasive diagnosis tool for lesion detection in the gastrointestinal tract, reaching places where conventional endoscopy is unable to. However, the significant amount of acquired data leads to difficulties in the diagnosis by the physicians; which can be eased with computer assistance. This paper addresses a method for the automatic detection of tumors in WCE by using a two-step based procedure: region of interest selection and classification. METHODS: The first step aims to separate abnormal from normal tissue by using automatic segmentation based on a Gaussian Mixture Model (GMM). A modified version of the Anderson method for convergence acceleration of the expectation-maximization (EM) algorithm is proposed. The proposed features for both segmentation and classification are based on the CIELab color space, as a way of bypassing lightness variations, where the L component is discarded. Tissue variability among subjects, light inhomogeneities and even intensity differences among different devices can be overcome by using simultaneously features from both regions. In the second step, an ensemble system with partition of the training data with a new training scheme is proposed. At this stage, the gating network is trained after the experts have been trained decoupling the joint maximization of both modules. The partition module is also used at the test step, leading the incoming data to the most likely expert allowing incremental adaptation by preserving data diversity. RESULTS: This algorithm outperforms others based on texture features selected from Wavelets and Curvelets transforms, classified by a regular support vector machine (SVM) in more than 5%. CONCLUSIONS: This work shows that simpler features can outperform more elaborate ones if appropriately designed. In the current case, luminance was discarded to cope with saturated tissue, facilitating the color perception. Ensemble systems remain an open research field. In the current case, changes in both topology and training strategy have led to significant performance improvements. A system with this level of performance can be used in current clinical practice.

Automatic T1 bladder tumor detection by using wavelet analysis in cystoscopy images.

Correct classification of cystoscopy images depends on the interpreter's experience. Bladder cancer is a common lesion that can only be confirmed by biopsying the tissue, therefore, the automatic identification of tumors plays a significant role in early stage diagnosis and its accuracy. To our best knowledge, the use of white light cystoscopy images for bladder tumor diagnosis has not been reported so far. In this paper, a texture analysis based approach is proposed for bladder tumor diagnosis presuming that tumors change in tissue texture. As is well accepted by the scientific community, texture information is more present in the medium to high frequency range which can be selected by using a discrete wavelet transform (DWT). Tumor enhancement can be improved by using automatic segmentation, since a mixing with normal tissue is avoided under ideal conditions. The segmentation module proposed in this paper takes advantage of the wavelet decomposition tree to discard poor texture information in such a way that both steps of the proposed algorithm segmentation and classification share the same focus on texture. Multilayer perceptron and a support vector machine with a stratified ten-fold cross-validation procedure were used for classification purposes by using the hue-saturation-value (HSV), red-green-blue, and CIELab color spaces. Performances of 91% in sensitivity and 92.9% in specificity were obtained regarding HSV color by using both preprocessing and classification steps based on the DWT. The proposed method can achieve good performance on identifying bladder tumor frames. These promising results open the path towards a deeper study regarding the applicability of this algorithm in computer aided diagnosis.

Using cystoscopy to segment bladder tumors with a multivariate approach in different color spaces.

Nowadays the diagnosis of bladder lesions relies upon cystoscopy examination and depends on the interpreter's experience. State of the art of bladder tumor identification are based on 3D reconstruction, using CT images (Virtual Cystoscopy) or images where the structures are exalted with the use of pigmentation, but none uses white light cystoscopy images. An initial attempt to automatically identify tumoral tissue was already developed by the authors and this paper will develop this idea. Traditional cystoscopy images processing has a huge potential to improve early tumor detection and allows a more effective treatment. In this paper is described a multivariate approach to do segmentation of bladder cystoscopy images, that will be used to automatically detect and improve physician diagnose. Each region can be assumed as a normal distribution with specific parameters, leading to the assumption that the distribution of intensities is a Gaussian Mixture Model (GMM). Region of high grade and low grade tumors, usually appears with higher intensity than normal regions. This paper proposes a Maximum a Posteriori (MAP) approach based on pixel intensities read simultaneously in different color channels from RGB, HSV and CIELab color spaces. The Expectation-Maximization (EM) algorithm is used to estimate the best multivariate GMM parameters. Experimental results show that the proposed method does bladder tumor segmentation into two classes in a more efficient way in RGB even in cases where the tumor shape is not well defined. Results also show that the elimination of component L from CIELab color space does not allow definition of the tumor shape.