LNDb challenge on automatic lung cancer patient management.

Lung cancer is the deadliest type of cancer worldwide and late detection is the major factor for the low survival rate of patients. Low dose computed tomography has been suggested as a potential screening tool but manual screening is costly and time-consuming. This has fuelled the development of automatic methods for the detection, segmentation and characterisation of pulmonary nodules. In spite of promising results, the application of automatic methods to clinical routine is not straightforward and only a limited number of studies have addressed the problem in a holistic way. With the goal of advancing the state of the art, the Lung Nodule Database (LNDb) Challenge on automatic lung cancer patient management was organized. The LNDb Challenge addressed lung nodule detection, segmentation and characterization as well as prediction of patient follow-up according to the 2017 Fleischner society pulmonary nodule guidelines. 294 CT scans were thus collected retrospectively at the Centro Hospitalar e Universitrio de So Joo in Porto, Portugal and each CT was annotated by at least one radiologist. Annotations comprised nodule centroids, segmentations and subjective characterization. 58 CTs and the corresponding annotations were withheld as a separate test set. A total of 947 users registered for the challenge and 11 successful submissions for at least one of the sub-challenges were received. For patient follow-up prediction, a maximum quadratic weighted Cohen's kappa of 0.580 was obtained. In terms of nodule detection, a sensitivity below 0.4 (and 0.7) at 1 false positive per scan was obtained for nodules identified by at least one (and two) radiologist(s). For nodule segmentation, a maximum Jaccard score of 0.567 was obtained, surpassing the interobserver variability. In terms of nodule texture characterization, a maximum quadratic weighted Cohen's kappa of 0.733 was obtained, with part solid nodules being particularly challenging to classify correctly. Detailed analysis of the proposed methods and the differences in performance allow to identify the major challenges remaining and future directions - data collection, augmentation/generation and evaluation of under-represented classes, the incorporation of scan-level information for better decision-making and the development of tools and challenges with clinical-oriented goals. The LNDb Challenge and associated data remain publicly available so that future methods can be tested and benchmarked, promoting the development of new algorithms in lung cancer medical image analysis and patient follow-up recommendation.

Unboxing AI - Radiological Insights Into a Deep Neural Network for Lung Nodule Characterization.

RATIONALE AND OBJECTIVES: To explain predictions of a deep residual convolutional network for characterization of lung nodule by analyzing heat maps. MATERIALS AND METHODS: A 20-layer deep residual CNN was trained on 1245 Chest CTs from National Lung Screening Trial (NLST) trial to predict the malignancy risk of a nodule. We used occlusion to systematically block regions of a nodule and map drops in malignancy risk score to generate clinical attribution heatmaps on 103 nodules from Lung Image Database Consortium image collection and Image Database Resource Initiative (LIDC-IDRI) dataset, which were analyzed by a thoracic radiologist. The features were described as heat inside nodule -bright areas inside nodule, peripheral heat continuous/interrupted bright areas along nodule contours, heat in adjacent plane -brightness in scan planes juxtaposed with the nodule, satellite heat - a smaller bright spot in proximity to nodule in the same scan plane, heat map larger than nodule bright areas corresponding to the shape of the nodule seen outside the nodule margins and heat in calcification. RESULTS: These six features were assigned binary values. This feature vector was fedinto a standard J48 decision tree with 10-fold cross-validation, which gave an 85 % weighted classification accuracy with a 77.8% True Positive (TP) rate, 8% False Positive (FP) rate for benign cases and 91.8% TP and 22.2% FP rates for malignant cases. Heat Inside nodule was more frequently observed in nodules classified as malignant whereas peripheral heat, heat in adjacent plane, and satellite heat were more commonly seen in nodules classified as benign. CONCLUSION: We discuss the potential ability of a radiologist to visually parse the deep learning algorithm generated "heat map" to identify features aiding classification.

Longitudinal multiple sclerosis lesion segmentation: Resource and challenge.

In conjunction with the ISBI 2015 conference, we organized a longitudinal lesion segmentation challenge providing training and test data to registered participants. The training data consisted of five subjects with a mean of 4.4 time-points, and test data of fourteen subjects with a mean of 4.4 time-points. All 82 data sets had the white matter lesions associated with multiple sclerosis delineated by two human expert raters. Eleven teams submitted results using state-of-the-art lesion segmentation algorithms to the challenge, with ten teams presenting their results at the conference. We present a quantitative evaluation comparing the consistency of the two raters as well as exploring the performance of the eleven submitted results in addition to three other lesion segmentation algorithms. The challenge presented three unique opportunities: (1) the sharing of a rich data set; (2) collaboration and comparison of the various avenues of research being pursued in the community; and (3) a review and refinement of the evaluation metrics currently in use. We report on the performance of the challenge participants, as well as the construction and evaluation of a consensus delineation. The image data and manual delineations will continue to be available for download, through an evaluation website2 as a resource for future researchers in the area. This data resource provides a platform to compare existing methods in a fair and consistent manner to each other and multiple manual raters.