Automated localization and segmentation of cervical lymph nodes on contrast-enhanced CT using a 3D foveal fully convolutional neural network.

BACKGROUND: In the management of cancer patients, determination of TNM status is essential for treatment decision-making and therefore closely linked to clinical outcome and survival. Here, we developed a tool for automatic three-dimensional (3D) localization and segmentation of cervical lymph nodes (LNs) on contrast-enhanced computed tomography (CECT) examinations. METHODS: In this IRB-approved retrospective single-center study, 187 CECT examinations of the head and neck region from patients with various primary diseases were collected from our local database, and 3656 LNs (19.5 ± 14.9 LNs/CECT, mean ± standard deviation) with a short-axis diameter (SAD) ≥ 5 mm were segmented manually by expert physicians. With these data, we trained an independent fully convolutional neural network based on 3D foveal patches. Testing was performed on 30 independent CECTs with 925 segmented LNs with an SAD ≥ 5 mm. RESULTS: In total, 4,581 LNs were segmented in 217 CECTs. The model achieved an average localization rate (LR), i.e., percentage of localized LNs/CECT, of 78.0% in the validation dataset. In the test dataset, average LR was 81.1% with a mean Dice coefficient of 0.71. For enlarged LNs with a SAD ≥ 10 mm, LR was 96.2%. In the test dataset, the false-positive rate was 2.4 LNs/CECT. CONCLUSIONS: Our trained AI model demonstrated a good overall performance in the consistent automatic localization and 3D segmentation of physiological and metastatic cervical LNs with a SAD ≥ 5 mm on CECTs. This could aid clinical localization and automatic 3D segmentation, which can benefit clinical care and radiomics research. RELEVANCE STATEMENT: Our AI model is a time-saving tool for 3D segmentation of cervical lymph nodes on contrast-enhanced CT scans and serves as a solid base for N staging in clinical practice and further radiomics research. KEY POINTS: • Determination of N status in TNM staging is essential for therapy planning in oncology. • Segmenting cervical lymph nodes manually is highly time-consuming in clinical practice. • Our model provides a robust, automated 3D segmentation of cervical lymph nodes. • It achieves a high accuracy for localization especially of enlarged lymph nodes. • These segmentations should assist clinical care and radiomics research.

Cervical spine fracture detection in computed tomography using convolutional neural networks.

Objective.In the context of primary in-hospital trauma management timely reading of computed tomography (CT) images is critical. However, assessment of the spine is time consuming, fractures can be very subtle, and the potential for under-diagnosis or delayed diagnosis is relevant. Artificial intelligence is increasingly employed to assist radiologists with the detection of spinal fractures and prioritization of cases. Currently, algorithms focusing on the cervical spine are commercially available. A common approach is the vertebra-wise classification. Instead of a classification task, we formulate fracture detection as a segmentation task aiming to find and display all individual fracture locations presented in the image.Approach.Based on 195 CT examinations, 454 cervical spine fractures were identified and annotated by radiologists at a tertiary trauma center. We trained for the detection a U-Net via four-fold-cross validation to segment spine fractures and the spine via a multi-task loss. We further compared advantages of two image reformation approaches-straightened curved planar reformatted (CPR) around the spine and spinal canal aligned volumes of interest (VOI)-to achieve a unified vertebral alignment in comparison to processing the Cartesian data directly.Main results.Of the three data versions (Cartesian, reformatted, VOI) the VOI approach showed the best detection rate and a reduced computation time. The proposed algorithm was able to detect 87.2% of cervical spine fractures at an average number of false positives of 3.5 per case. Evaluation of the method on a public spine dataset resulted in 0.9 false positive detections per cervical spine case.Significance.The display of individual fracture locations as provided with high sensitivity by the proposed voxel classification based fracture detection has the potential to support the trauma CT reading workflow by reducing missed findings.

Additional value of chest CT AI-based quantification of lung involvement in predicting death and ICU admission for COVID-19 patients.

Objectives: We evaluated the contribution of lung lesion quantification on chest CT using a clinical Artificial Intelligence (AI) software in predicting death and intensive care units (ICU) admission for COVID-19 patients. Methods: For 349 patients with positive COVID-19-PCR test that underwent a chest CT scan at admittance or during hospitalization, we applied the AI for lung and lung lesion segmentation to obtain lesion volume (LV), and LV/Total Lung Volume (TLV) ratio. ROC analysis was used to extract the best CT criterion in predicting death and ICU admission. Two prognostic models using multivariate logistic regressions were constructed to predict each outcome and were compared using AUC values. The first model ("Clinical") was based on patients' characteristics and clinical symptoms only. The second model ("Clinical+LV/TLV") included also the best CT criterion. Results: LV/TLV ratio demonstrated best performance for both outcomes; AUC of 67.8% (95% CI: 59.5 - 76.1) and 81.1% (95% CI: 75.7 - 86.5) respectively. Regarding death prediction, AUC values were 76.2% (95% CI: 69.9 - 82.6) and 79.9% (95%IC: 74.4 - 85.5) for the "Clinical" and the "Clinical+LV/TLV" models respectively, showing significant performance increase (+ 3.7%; p-value<0.001) when adding LV/TLV ratio. Similarly, for ICU admission prediction, AUC values were 74.9% (IC 95%: 69.2 - 80.6) and 84.8% (IC 95%: 80.4 - 89.2) respectively corresponding to significant performance increase (+ 10%: p-value<0.001). Conclusions: Using a clinical AI software to quantify the COVID-19 lung involvement on chest CT, combined with clinical variables, allows better prediction of death and ICU admission.

Automated mapping and N-Staging of thoracic lymph nodes in contrast-enhanced CT scans of the chest using a fully convolutional neural network.

PURPOSE: To develop a deep-learning (DL)-based approach for thoracic lymph node (LN) mapping based on their anatomical location. METHOD: The training-and validation-dataset included 89 contrast-enhanced computed tomography (CT) scans of the chest. 4201 LNs were semi-automatically segmented and then assigned to LN levels according to their anatomical location. The LN level classification task was addressed by a multi-class segmentation procedure using a fully convolutional neural network. Mapping was performed by firstly determining potential level affiliation for each voxel and then performing majority voting over all voxels belonging to each LN. Mean classification accuracies on the validation data were calculated separately for each level and overall Top-1, Top-2 and Top-3 scores were determined, where a Top-X score describes how often the annotated class was within the top-X predictions. To demonstrate the clinical applicability of our model, we tested its N-staging capabilities in a simulated clinical use case scenario assuming a patient diseased with lung cancer. RESULTS: The artificial intelligence(AI)-based assignment revealed mean classification accuracies of 86.36 % (Top-1), 94.48 % (Top-2) and 96.10 % (Top-3). Best accuracies were achieved for LNs in the subcarinal level 7 (98.31 %) and axillary region (98.74 %). The highest misclassification rates were observed among LNs in adjacent levels. The proof-of-principle application in a simulated clinical use case scenario for automated tumor N-staging showed a mean classification accuracy of up to 96.14 % (Top-1). CONCLUSIONS: The proposed AI approach for automatic classification of LN levels in chest CT as well as the proof-of-principle-experiment for automatic N-staging, revealed promising results, warranting large-scale validation for clinical application.

Automated detection and segmentation of thoracic lymph nodes from CT using 3D foveal fully convolutional neural networks.

BACKGROUND: In oncology, the correct determination of nodal metastatic disease is essential for patient management, as patient treatment and prognosis are closely linked to the stage of the disease. The aim of the study was to develop a tool for automatic 3D detection and segmentation of lymph nodes (LNs) in computed tomography (CT) scans of the thorax using a fully convolutional neural network based on 3D foveal patches. METHODS: The training dataset was collected from the Computed Tomography Lymph Nodes Collection of the Cancer Imaging Archive, containing 89 contrast-enhanced CT scans of the thorax. A total number of 4275 LNs was segmented semi-automatically by a radiologist, assessing the entire 3D volume of the LNs. Using this data, a fully convolutional neuronal network based on 3D foveal patches was trained with fourfold cross-validation. Testing was performed on an unseen dataset containing 15 contrast-enhanced CT scans of patients who were referred upon suspicion or for staging of bronchial carcinoma. RESULTS: The algorithm achieved a good overall performance with a total detection rate of 76.9% for enlarged LNs during fourfold cross-validation in the training dataset with 10.3 false-positives per volume and of 69.9% in the unseen testing dataset. In the training dataset a better detection rate was observed for enlarged LNs compared to smaller LNs, the detection rate for LNs with a short-axis diameter (SAD) ≥ 20 mm and SAD 5-10 mm being 91.6% and 62.2% (p < 0.001), respectively. Best detection rates were obtained for LNs located in Level 4R (83.6%) and Level 7 (80.4%). CONCLUSIONS: The proposed 3D deep learning approach achieves an overall good performance in the automatic detection and segmentation of thoracic LNs and shows reasonable generalizability, yielding the potential to facilitate detection during routine clinical work and to enable radiomics research without observer-bias.

Towards radiologist-level cancer risk assessment in CT lung screening using deep learning.

PURPOSE: Lung cancer is the leading cause of cancer mortality in the US, responsible for more deaths than breast, prostate, colon and pancreas cancer combined and large population studies have indicated that low-dose computed tomography (CT) screening of the chest can significantly reduce this death rate. Recently, the usefulness of Deep Learning (DL) models for lung cancer risk assessment has been demonstrated. However, in many cases model performances are evaluated on small/medium size test sets, thus not providing strong model generalization and stability guarantees which are necessary for clinical adoption. In this work, our goal is to contribute towards clinical adoption by investigating a deep learning framework on larger and heterogeneous datasets while also comparing to state-of-the-art models. METHODS: Three low-dose CT lung cancer screening datasets were used: National Lung Screening Trial (NLST, n = 3410), Lahey Hospital and Medical Center (LHMC, n = 3154) data, Kaggle competition data (from both stages, n = 1397 + 505) and the University of Chicago data (UCM, a subset of NLST, annotated by radiologists, n = 132). At the first stage, our framework employs a nodule detector; while in the second stage, we use both the image context around the nodules and nodule features as inputs to a neural network that estimates the malignancy risk for the entire CT scan. We trained our algorithm on a part of the NLST dataset, and validated it on the other datasets. Special care was taken to ensure there was no patient overlap between the train and validation sets. RESULTS AND CONCLUSIONS: The proposed deep learning model is shown to: (a) generalize well across all three data sets, achieving AUC between 86% to 94%, with our external test-set (LHMC) being at least twice as large compared to other works; (b) have better performance than the widely accepted PanCan Risk Model, achieving 6 and 9% better AUC score in our two test sets; (c) have improved performance compared to the state-of-the-art represented by the winners of the Kaggle Data Science Bowl 2017 competition on lung cancer screening; (d) have comparable performance to radiologists in estimating cancer risk at a patient level.

Automatic detection of osteoporotic vertebral fractures in routine thoracic and abdominal MDCT.

OBJECTIVES: To develop a prototype algorithm for automatic spine segmentation in MDCT images and use it to automatically detect osteoporotic vertebral fractures. METHODS: Cross-sectional routine thoracic and abdominal MDCT images of 71 patients including 8 males and 9 females with 25 osteoporotic vertebral fractures and longitudinal MDCT images of 9 patients with 18 incidental fractures in the follow-up MDCT were retrospectively selected. The spine segmentation algorithm localised and identified the vertebrae T5-L5. Each vertebra was automatically segmented by using corresponding vertebra surface shape models that were adapted to the original images. Anterior, middle, and posterior height of each vertebra was automatically determined; the anterior-posterior ratio (APR) and middle-posterior ratio (MPR) were computed. As the gold standard, radiologists graded vertebral fractures from T5 to L5 according to the Genant classification in consensus. RESULTS: Using ROC analysis to differentiate vertebrae without versus with prevalent fracture, AUC values of 0.84 and 0.83 were obtained for APR and MPR, respectively (p < 0.001). Longitudinal changes in APR and MPR were significantly different between vertebrae without versus with incidental fracture (ΔAPR: -8.5 % ± 8.6 % versus -1.6 % ± 4.2 %, p = 0.002; ΔMPR: -11.4 % ± 7.7 % versus -1.2 % ± 1.6 %, p < 0.001). CONCLUSIONS: This prototype algorithm may support radiologists in reporting currently underdiagnosed osteoporotic vertebral fractures so that appropriate therapy can be initiated. KEY POINTS: • This spine segmentation algorithm automatically localised, identified, and segmented the vertebrae in MDCT images. • Osteoporotic vertebral fractures could be automatically detected using this prototype algorithm. • The prototype algorithm helps radiologists to report underdiagnosed osteoporotic vertebral fractures.

A radial structure tensor and its use for shape-encoding medical visualization of tubular and nodular structures.

The concept of curvature and shape-based rendering is beneficial for medical visualization of CT and MRI image volumes. Color-coding of local shape properties derived from the analysis of the local Hessian can implicitly highlight tubular structures such as vessels and airways, and guide the attention to potentially malignant nodular structures such as tumors, enlarged lymph nodes, or aneurysms. For some clinical applications, however, the evaluation of the Hessian matrix does not yield satisfactory renderings, in particular for hollow structures such as airways, and densely embedded low contrast structures such as lymph nodes. Therefore, as a complement to Hessian-based shape-encoding rendering, this paper introduces a combination of an efficient sparse radial gradient sampling scheme in conjunction with a novel representation, the radial structure tensor (RST). As an extension of the well-known general structure tensor, which has only positive definite eigenvalues, the radial structure tensor correlates position and direction of the gradient vectors in a local neighborhood, and thus yields positive and negative eigenvalues which can be used to discriminate between different shapes. As Hessian-based rendering, also RST-based rendering is ideally suited for GPU implementation. Feedback from clinicians indicates that shape-encoding rendering can be an effective image navigation tool to aid diagnostic workflow and quality assurance.

Model based 3D segmentation and OCT image undistortion of percutaneous implants.

Optical Coherence Tomography (OCT) is a noninvasive imaging technique which is used here for in vivo biocompatibility studies of percutaneous implants. A prerequisite for a morphometric analysis of the OCT images is the correction of optical distortions caused by the index of refraction in the tissue. We propose a fully automatic approach for 3D segmentation of percutaneous implants using Markov random fields. Refraction correction is done by using the subcutaneous implant base as a prior for model based estimation of the refractive index using a generalized Hough transform. Experiments show the competitiveness of our algorithm towards manual segmentations done by experts.

Four-dimensional computed tomography pulmonary ventilation images vary with deformable image registration algorithms and metrics.

PURPOSE: A novel pulmonary ventilation imaging technique based on four-dimensional (4D) CT has advantages over existing techniques and could be used for functional avoidance in radiotherapy. There are various deformable image registration (DIR) algorithms and two classes of ventilation metric that can be used for 4D-CT ventilation imaging, each yielding different images. The purpose of this study was to quantify the variability of the 4D-CT ventilation to DIR algorithms and metrics. METHODS: 4D-CT ventilation images were created for 12 patients using different combinations of two DIR algorithms, volumetric (DIR(vol)) and surface-based (DIR(sur)), yielding two displacement vector fields (DVFs) per patient (DVF(voI) and DVF(sur)), and two metrics, Hounsfield unit (HU) change (V(HU)) and Jacobian determinant of deformation (V(Jac)), yielding four ventilation image sets (V(HU)(vol), V(HU)(sur), V(Jac)(voI), and V(Jac)(sur). First DVF(vol) and DVF(sur) were compared visually and quantitatively to the length of 3D displacement vector difference. Second, four ventilation images were compared based on voxel-based Spearman's rank correlation coefficients and coefficients of variation as a measure of spatial heterogeneity. V(HU)(vol) was chosen as the reference for the comparison. RESULTS: The mean length of 3D vector difference between DVF(vol) and DVF(sur) was 2.0 +/- 1.1 mm on average, which was smaller than the voxel dimension of the image set and the variations. Visually, the reference V(HU)(vol) demonstrated similar regional distributions with V(HU)(sur); the reference, however, was markedly different from V(Jac)(vol) and V((Jac)(sur). The correlation coefficients of V(HU)(vol) with V(HU)(sur), V(Jac)(vol) and V(Jac)(sur) were 0.77 +/- 0.06, 0.25 +/- 0.06 and 0.15 +/- 0.07, respectively, indicating that the metric introduced larger variations in the ventilation images than the DIR algorithm. The spatial heterogeneities for V(HU)(vol), 'V(HU)(sur), V(Jac)(vol), and V(Jac)(sur) were 1.8 +/- 1.6, 1.8 +/- 1.5 (p = 0. 85), 0.6 +/- 0.2 (p = 0.02), and 0.7 +/- 0.2 (p = 0.03), respectively, also demonstrating that the metric introduced larger variations. CONCLUSIONS: 4D-CT pulmonary ventilation images vary widely with DIR algorithms and metrics. Careful physiologic validation to determine the appropriate DIR algorithm and metric is needed prior to its applications.

Investigation of four-dimensional computed tomography-based pulmonary ventilation imaging in patients with emphysematous lung regions.

A pulmonary ventilation imaging technique based on four-dimensional (4D) computed tomography (CT) has advantages over existing techniques. However, physiologically accurate 4D-CT ventilation imaging has not been achieved in patients. The purpose of this study was to evaluate 4D-CT ventilation imaging by correlating ventilation with emphysema. Emphysematous lung regions are less ventilated and can be used as surrogates for low ventilation. We tested the hypothesis: 4D-CT ventilation in emphysematous lung regions is significantly lower than in non-emphysematous regions. Four-dimensional CT ventilation images were created for 12 patients with emphysematous lung regions as observed on CT, using a total of four combinations of two deformable image registration (DIR) algorithms: surface-based (DIR(sur)) and volumetric (DIR(vol)), and two metrics: Hounsfield unit (HU) change (V(HU)) and Jacobian determinant of deformation (V(Jac)), yielding four ventilation image sets per patient. Emphysematous lung regions were detected by density masking. We tested our hypothesis using the one-tailed t-test. Visually, different DIR algorithms and metrics yielded spatially variant 4D-CT ventilation images. The mean ventilation values in emphysematous lung regions were consistently lower than in non-emphysematous regions for all the combinations of DIR algorithms and metrics. V(HU) resulted in statistically significant differences for both DIR(sur) (0.14 ± 0.14 versus 0.29 ± 0.16, p = 0.01) and DIR(vol) (0.13 ± 0.13 versus 0.27 ± 0.15, p < 0.01). However, V(Jac) resulted in non-significant differences for both DIR(sur) (0.15 ± 0.07 versus 0.17 ± 0.08, p = 0.20) and DIR(vol) (0.17 ± 0.08 versus 0.19 ± 0.09, p = 0.30). This study demonstrated the strong correlation between the HU-based 4D-CT ventilation and emphysema, which indicates the potential for HU-based 4D-CT ventilation imaging to achieve high physiologic accuracy. A further study is needed to confirm these results.

Prediction framework for statistical respiratory motion modeling.

Breathing motion complicates many image-guided interventions working on the thorax or upper abdomen. However, prior knowledge provided by a statistical breathing model, can reduce the uncertainties of organ location. In this paper, a prediction framework for statistical motion modeling is presented and different representations of the dynamic data for motion model building of the lungs are investigated. Evaluation carried out on 4D-CT data sets of 10 patients showed that a displacement vector-based representation can reduce most of the respiratory motion with a prediction error of about 2 mm, when assuming the diaphragm motion to be known.

3D reconstruction of the human rib cage from 2D projection images using a statistical shape model.

PURPOSE: This paper describes an approach for the three-dimensional (3D) shape and pose reconstruction of the human rib cage from few segmented two-dimensional (2D) projection images. Our work is aimed at supporting temporal subtraction techniques of subsequently acquired radiographs by establishing a method for the assessment of pose differences in sequences of chest radiographs of the same patient. METHODS: The reconstruction method is based on a 3D statistical shape model (SSM) of the rib cage, which is adapted to binary 2D projection images of an individual rib cage. To drive the adaptation we minimize a distance measure that quantifies the dissimilarities between 2D projections of the 3D SSM and the projection images of the individual rib cage. We propose different silhouette-based distance measures and evaluate their suitability for the adaptation of the SSM to the projection images. RESULTS: An evaluation was performed on 29 sets of biplanar binary images (posterior-anterior and lateral). Depending on the chosen distance measure, our experiments on the combined reconstruction of shape and pose of the rib cages yield reconstruction errors from 2.2 to 4.7 mm average mean 3D surface distance. Given a geometry of an individual rib cage, the rotational errors for the pose reconstruction range from 0.1 degrees to 0.9 degrees. CONCLUSIONS: The results show that our method is suitable for the estimation of pose differences of the human rib cage in binary projection images. Thus, it is able to provide crucial 3D information for registration during the generation of 2D subtraction images.

Automated model-based vertebra detection, identification, and segmentation in CT images.

For many orthopaedic, neurological, and oncological applications, an exact segmentation of the vertebral column including an identification of each vertebra is essential. However, although bony structures show high contrast in CT images, the segmentation and labelling of individual vertebrae is challenging. In this paper, we present a comprehensive solution for automatically detecting, identifying, and segmenting vertebrae in CT images. A framework has been designed that takes an arbitrary CT image, e.g., head-neck, thorax, lumbar, or whole spine, as input and provides a segmentation in form of labelled triangulated vertebra surface models. In order to obtain a robust processing chain, profound prior knowledge is applied through the use of various kinds of models covering shape, gradient, and appearance information. The framework has been tested on 64 CT images even including pathologies. In 56 cases, it was successfully applied resulting in a final mean point-to-surface segmentation error of 1.12+/-1.04mm. One key issue is a reliable identification of vertebrae. For a single vertebra, we achieve an identification success of more than 70%. Increasing the number of available vertebrae leads to an increase in the identification rate reaching 100% if 16 or more vertebrae are shown in the image.

Evaluation of 4D-CT lung registration.

Non-rigid registration accuracy assessment is typically performed by evaluating the target registration error at manually placed landmarks. For 4D-CT lung data, we compare two sets of landmark distributions: a smaller set primarily defined on vessel bifurcations as commonly described in the literature and a larger set being well-distributed throughout the lung volume. For six different registration schemes (three in-house schemes and three schemes frequently used by the community) the landmark error is evaluated and found to depend significantly on the distribution of the landmarks. In particular, lung regions near to the pleura show a target registration error three times larger than near-mediastinal regions. While the inter-method variability on the landmark positions is rather small, the methods show discriminating differences with respect to consistency and local volume change. In conclusion, both a well-distributed set of landmarks and a deformation vector field analysis are necessary for reliable non-rigid registration accuracy assessment.

Spine segmentation using articulated shape models.

Including prior shape in the form of anatomical models is a well-known approach for improving segmentation results in medical images. Currently, most approaches are focused on the modeling and segmentation of individual objects. In case of object constellations, a simultaneous segmentation of the ensemble that uses not only prior knowledge of individual shapes but also additional information about spatial relations between the objects is often beneficial. In this paper, we present a two-scale framework for the modeling and segmentation of the spine as an example for object constellations. The global spine shape is expressed as a consecution of local vertebra coordinate systems while individual vertebrae are modeled as triangulated surface meshes. Adaptation is performed by attracting the model to image features but restricting the attraction to a former learned shape. With the developed approach, we obtained a segmentation accuracy of 1.0 mm in average for ten thoracic CT images improving former results.

Automated model-based rib cage segmentation and labeling in CT images.

We present a new model-based approach for an automated labeling and segmentation of the rib cage in chest CT scans. A mean rib cage model including a complete vertebral column is created out of 29 data sets. We developed a ray search based procedure for rib cage detection and initial model pose. After positioning the model, it was adapted to 18 unseen CT data. In 16 out of 18 data sets, detection, labeling, and segmentation succeeded with a mean segmentation error of less than 1.3 mm between true and detected object surface. In one case the rib cage detection failed, in another case the automated labeling.