Deep Learning for nasopharyngeal Carcinoma Identification Using Both White Light and Narrow-Band Imaging Endoscopy.

OBJECTIVES/HYPOTHESIS: To develop a deep-learning-based automatic diagnosis system for identifying nasopharyngeal carcinoma (NPC) from noncancer (inflammation and hyperplasia), using both white light imaging (WLI) and narrow-band imaging (NBI) nasopharyngoscopy images. STUDY DESIGN: Retrospective study. METHODS: A total of 4,783 nasopharyngoscopy images (2,898 WLI and 1,885 NBI) of 671 patients were collected and a novel deep convolutional neural network (DCNN) framework was developed named Siamese deep convolutional neural network (S-DCNN), which can simultaneously utilize WLI and NBI images to improve the classification performance. To verify the effectiveness of combining the above-mentioned two modal images for prediction, we compared the proposed S-DCNN with two baseline models, namely DCNN-1 (only considering WLI images) and DCNN-2 (only considering NBI images). RESULTS: In the threefold cross-validation, an overall accuracy and area under the curve of the three DCNNs achieved 94.9% (95% confidence interval [CI] 93.3%-96.5%) and 0.986 (95% CI 0.982-0.992), 87.0% (95% CI 84.2%-89.7%) and 0.930 (95% CI 0.906-0.961), and 92.8% (95% CI 90.4%-95.3%) and 0.971 (95% CI 0.953-0.992), respectively. The accuracy of S-DCNN is significantly improved compared with DCNN-1 (P-value <.001) and DCNN-2 (P-value = .008). CONCLUSION: Using the deep-learning technology to automatically diagnose NPC under nasopharyngoscopy can provide valuable reference for NPC screening. Superior performance can be obtained by simultaneously utilizing the multimodal features of NBI image and WLI image of the same patient. LEVEL OF EVIDENCE: 3 Laryngoscope, 132:999-1007, 2022.

Deep reconstruction-recoding network for unsupervised domain adaptation and multi-center generalization in colonoscopy polyp detection.

BACKGROUND AND OBJECTIVE: Currently, the best performing methods in colonoscopy polyp detection are primarily based on deep neural networks (DNNs), which are usually trained on large amounts of labeled data. However, different hospitals use different endoscope models and set different imaging parameters, which causes the collected endoscopic images and videos to vary greatly in style. There may be variations in the color space, brightness, contrast, and resolution, and there are also differences between white light endoscopy (WLE) and narrow band image endoscopy (NBIE). We call these variations the domain shift. The DNN performance may decrease when the training data and the testing data come from different hospitals or different endoscope models. Additionally, it is quite difficult to collect enough new labeled data and retrain a new DNN model before deploying that DNN to a new hospital or endoscope model. METHODS: To solve this problem, we propose a domain adaptation model called Deep Reconstruction-Recoding Network (DRRN), which jointly learns a shared encoding representation for two tasks: i) a supervised object detection network for labeled source data, and ii) an unsupervised reconstruction-recoding network for unlabeled target data. Through the DRRN, the object detection network's encoder not only learns the features from the labeled source domain, but also encodes useful information from the unlabeled target domain. Therefore, the distribution difference of the two domains' feature spaces can be reduced. RESULTS: We evaluate the performance of the DRRN on a series of cross-domain datasets. Compared with training the polyp detection network using only source data, the performance of the DRRN on the target domain is improved. Through feature statistics and visualization, it is demonstrated that the DRRN can learn the common distribution and feature invariance of the two domains. The distribution difference between the feature spaces of the two domains can be reduced. CONCLUSION: The DRRN can improve cross-domain polyp detection. With the DRRN, the generalization performance of the DNN-based polyp detection model can be improved without additional labeled data. This improvement allows the polyp detection model to be easily transferred to datasets from different hospitals or different endoscope models.

A real-time system using deep learning to detect and track ureteral orifices during urinary endoscopy.

BACKGROUND AND OBJECTIVE: To automatically identify and locate various types and states of the ureteral orifice (UO) in real endoscopy scenarios, we developed and verified a real-time computer-aided UO detection and tracking system using an improved real-time deep convolutional neural network and a robust tracking algorithm. METHODS: The single-shot multibox detector (SSD) was refined to perform the detection task. We trained both the SSD and Refined-SSD using 447 resectoscopy images with UO and tested them on 818 ureteroscopy images. We also evaluated the detection performance on endoscopy video frames, which comprised 892 resectoscopy frames and 1366 ureteroscopy frames. UOs could not be identified with certainty because sometimes they appeared on the screen in a closed state of peristaltic contraction. To mitigate this problem and mimic the inspection behavior of urologists, we integrated the SSD and Refined-SSD with five different tracking algorithms. RESULTS: When tested on 818 ureteroscopy images, our proposed UO detection network, Refined-SSD, achieved an accuracy of 0.902. In the video sequence analysis, our detection model yielded test sensitivities of 0.840 and 0.922 on resectoscopy and ureteroscopy video frames, respectively. In addition, by testing Refined-SSD on 1366 ureteroscopy video frames, the sensitivity achieved a value of 0.922, and a lowest false positive per image of 0.049 was obtained. For UO tracking performance, our proposed UO detection and tracking system (Refined-SSD integrated with CSRT) performed the best overall. At an overlap threshold of 0.5, the success rate of our proposed UO detection and tracking system was greater than 0.95 on 17 resectoscopy video clips and achieved nearly 0.95 on 40 ureteroscopy video clips. CONCLUSIONS: We developed a deep learning system that could be used for detecting and tracking UOs in endoscopy scenarios in real time. This system can simultaneously maintain high accuracy. This approach has great potential to serve as an excellent learning and feedback system for trainees and new urologists in clinical settings.