Fused Projection-Based Point Cloud Segmentation.

Semantic segmentation is used to enable a computer to understand its surrounding environment. In image processing, images are partitioned into segments for this purpose. State-of-the-art methods make use of Convolutional Neural Networks to segment a 2D image. Compared to that, 3D approaches suffer from computational cost and are not applicable without any further steps. In this work, we focus on semantic segmentation based on 3D point clouds. We use the idea to project the 3D data into a 2D image to accelerate the segmentation process. Afterward, the processed image gets re-projected to receive the desired result. We investigate different projection views and compare them to clarify their strengths and weaknesses. To compensate for projection errors and the loss of geometrical information, we evolve the approach and show how to fuse different views. We have decided to fuse the bird's-eye and the spherical projection as each of them achieves reasonable results, and the two perspectives complement each other best. For training and evaluation, we use the real-world datasets SemanticKITTI. Further, we use the ParisLille and synthetic data generated by the simulation framework Carla to analyze the approaches in more detail and clarify their strengths and weaknesses. Although these methods achieve reasonable and competitive results, they lack flexibility. They depend on the sensor used and the setup in which the sensor is used.

A novel endoimaging system for endoscopic 3D reconstruction in bladder cancer patients.

INTRODUCTION: The methods employed to document cystoscopic findings in bladder cancer patients lack accuracy and are subject to observer variability. We propose a novel endoimaging system and an online documentation platform to provide post-procedural 3D bladder reconstructions for improved diagnosis, management and follow-up. MATERIAL AND METHODS: The RaVeNNA4pi consortium is comprised of five industrial partners, two university hospitals and two technical institutes. These are grouped into hardware, software and clinical partners according to their professional expertise. The envisaged endoimaging system consists of an innovative cystoscope that generates 3D bladder reconstructions allowing users to remotely access a cloud-based centralized database to visualize individualized 3D bladder models from previous cystoscopies archived in DICOM format. RESULTS: Preliminary investigations successfully tracked the endoscope's rotational and translational movements. The structure-from-motion pipeline was tested in a bladder phantom and satisfactorily demonstrated 3D reconstructions of the processing sequence. AI-based semantic image segmentation achieved a 0.67 dice-score-coefficient over all classes. An online-platform allows physicians and patients to digitally visualize endoscopic findings by navigating a 3D bladder model. CONCLUSIONS: Our work demonstrates the current developments of a novel endoimaging system equipped with the potential to generate 3D bladder reconstructions from cystoscopy videos and AI-assisted automated detection of bladder tumors.

Application of artificial neural networks for automated analysis of cystoscopic images: a review of the current status and future prospects.

BACKGROUND: Optimal detection and surveillance of bladder cancer (BCa) rely primarily on the cystoscopic visualization of bladder lesions. AI-assisted cystoscopy may improve image recognition and accelerate data acquisition. OBJECTIVE: To provide a comprehensive review of machine learning (ML), deep learning (DL) and convolutional neural network (CNN) applications in cystoscopic image recognition. EVIDENCE ACQUISITION: A detailed search of original articles was performed using the PubMed-MEDLINE database to identify recent English literature relevant to ML, DL and CNN applications in cystoscopic image recognition. EVIDENCE SYNTHESIS: In total, two articles and one conference abstract were identified addressing the application of AI methods in cystoscopic image recognition. These investigations showed accuracies exceeding 90% for tumor detection; however, future work is necessary to incorporate these methods into AI-aided cystoscopy and compared to other tumor visualization tools. Furthermore, we present results from the RaVeNNA-4pi consortium initiative which has extracted 4200 frames from 62 videos, analyzed them with the U-Net network and achieved an average dice score of 0.67. Improvements in its precision can be achieved by augmenting the video/frame database. CONCLUSION: AI-aided cystoscopy has the potential to outperform urologists at recognizing and classifying bladder lesions. To ensure their real-life implementation, however, these algorithms require external validation to generalize their results across other data sets.

Current European Trends in Endoscopic Imaging and Transurethral Resection of Bladder Tumors.

Purpose: The aim of this survey was to obtain an overview of current European standards in the endoscopic visualization and management of bladder tumors. Methods: An online survey was launched in July 2018 for a duration of 4 months. It was distributed to all members of the European Association of Urology (EAU) and included 23 questions divided into 3 thematic sections: general information, white light cystoscopy (WLC) and imaging, and transurethral resection of bladder tumor (TURBT) techniques. Results: Responses of 222 participants were included for analysis. The majority of physicians were between 30 and 40 years of age (48.2%, n = 107) and performed over 50 TURBT per year (52.2%, n = 115). Overall, 52.3% (n = 116) reported WLC findings in written form only, 23.8% (n = 53) added endoscopic footage, and 79.2% (n = 176) considered preliminary WLC/TURBT reports before performing a subsequent bladder intervention. About half of the participants (50.5%, n = 104) used additional tumor visualization methods (aTVMs), but aTVMs were utilized by a greater proportion of physicians from Western countries (58.1%, n = 90) compared with developing countries (20.0%, n = 7). Photodynamic diagnosis was the predominant aTVM technique employed (43.8%, n = 60). Bipolar current was the most common technique for TURBT (46.6%, n = 149). Most urologists in this study occasionally utilized techniques like resections in fractions (80%, n = 161) or en bloc resection (87.2%, n = 182). A repeated TURBT was performed when no muscle was found in the specimen (70.6%, n = 149) and/or if the tumor was stage pT1 (72.0%, n = 152) or high grade (63.0%, n = 133). Conclusion: Implementation of resection techniques or repeated TURBT within EAU guidelines is promising, but it can be further challenged. For example, WLC/TURBT reporting should be improved since urologists consistently consider previous documentation. Given the moderate application rate of aTVMs, an attempt to increase its utilization would lead to a better assessment of its potential benefit.

Image-based 3D surface approximation of the bladder using structure-from-motion for enhanced cystoscopy based on phantom data.

Bladder cancer is likely to recur after resection. For this reason, bladder cancer survivors often undergo follow-up cystoscopy for years after treatment to look for bladder cancer recurrence. 3D modeling of the bladder could provide more reliable cystoscopic documentation by giving an overall picture of the organ and tumor positions. However, 3D reconstruction of the urinary bladder based on endoscopic images is challenging. This is due to the small field of view of the endoscope, considerable image distortion, and occlusion by urea, blood or particles. In this paper, we will demonstrate a method for the conversion of uncalibrated, monocular, endoscopic videos of the bladder into a 3D model using structure-from-motion (SfM). First of all, frames are extracted from video sequences. Distortions are then corrected in a calibration procedure. Finally, the 3D reconstruction algorithm generates a sparse surface approximation of the bladder lining based on the corrected frames. This method was tested using an endoscopic video of a phantom that mimics the rich structure of the bladder. The reconstructed 3D model covered a large part of the object, with an average reprojection error of 1.15 pixels and a relative accuracy of 99.4%.