Comparing algorithms for automated vessel segmentation in computed tomography scans of the lung: the VESSEL12 study.

The VESSEL12 (VESsel SEgmentation in the Lung) challenge objectively compares the performance of different algorithms to identify vessels in thoracic computed tomography (CT) scans. Vessel segmentation is fundamental in computer aided processing of data generated by 3D imaging modalities. As manual vessel segmentation is prohibitively time consuming, any real world application requires some form of automation. Several approaches exist for automated vessel segmentation, but judging their relative merits is difficult due to a lack of standardized evaluation. We present an annotated reference dataset containing 20 CT scans and propose nine categories to perform a comprehensive evaluation of vessel segmentation algorithms from both academia and industry. Twenty algorithms participated in the VESSEL12 challenge, held at International Symposium on Biomedical Imaging (ISBI) 2012. All results have been published at the VESSEL12 website http://vessel12.grand-challenge.org. The challenge remains ongoing and open to new participants. Our three contributions are: (1) an annotated reference dataset available online for evaluation of new algorithms; (2) a quantitative scoring system for objective comparison of algorithms; and (3) performance analysis of the strengths and weaknesses of the various vessel segmentation methods in the presence of various lung diseases.

Evaluation of prostate segmentation algorithms for MRI: the PROMISE12 challenge.

Prostate MRI image segmentation has been an area of intense research due to the increased use of MRI as a modality for the clinical workup of prostate cancer. Segmentation is useful for various tasks, e.g. to accurately localize prostate boundaries for radiotherapy or to initialize multi-modal registration algorithms. In the past, it has been difficult for research groups to evaluate prostate segmentation algorithms on multi-center, multi-vendor and multi-protocol data. Especially because we are dealing with MR images, image appearance, resolution and the presence of artifacts are affected by differences in scanners and/or protocols, which in turn can have a large influence on algorithm accuracy. The Prostate MR Image Segmentation (PROMISE12) challenge was setup to allow a fair and meaningful comparison of segmentation methods on the basis of performance and robustness. In this work we will discuss the initial results of the online PROMISE12 challenge, and the results obtained in the live challenge workshop hosted by the MICCAI2012 conference. In the challenge, 100 prostate MR cases from 4 different centers were included, with differences in scanner manufacturer, field strength and protocol. A total of 11 teams from academic research groups and industry participated. Algorithms showed a wide variety in methods and implementation, including active appearance models, atlas registration and level sets. Evaluation was performed using boundary and volume based metrics which were combined into a single score relating the metrics to human expert performance. The winners of the challenge where the algorithms by teams Imorphics and ScrAutoProstate, with scores of 85.72 and 84.29 overall. Both algorithms where significantly better than all other algorithms in the challenge (p<0.05) and had an efficient implementation with a run time of 8min and 3s per case respectively. Overall, active appearance model based approaches seemed to outperform other approaches like multi-atlas registration, both on accuracy and computation time. Although average algorithm performance was good to excellent and the Imorphics algorithm outperformed the second observer on average, we showed that algorithm combination might lead to further improvement, indicating that optimal performance for prostate segmentation is not yet obtained. All results are available online at http://promise12.grand-challenge.org/.

Automatic segmentation of the pulmonary lobes from chest CT scans based on fissures, vessels, and bronchi.

Segmentation of the pulmonary lobes is relevant in clinical practice and particularly challenging for cases with severe diseases or incomplete fissures. In this work, an automated segmentation approach is presented that performs a marker-based watershed transformation on computed tomography (CT) scans to subdivide the lungs into lobes. A cost image for the watershed transformation is computed by combining information from fissures, bronchi, and pulmonary vessels. The lobar markers are calculated by an analysis of the automatically labeled bronchial tree. By integration of information from several anatomical structures the segmentation is made robust against incomplete fissures. For evaluation the method was compared to a recently published method on 20 CT scans with no or mild disease. The average distances to the reference segmentation were 0.69, 0.67, and 1.21 mm for the left major, right major, and right minor fissure, respectively. In addition the results were submitted to LOLA11, an international lung lobe segmentation challenge with publically available data including cases with severe diseases. The average distances to the reference for the 55 CT scans provided by LOLA11 were 0.98, 3.97, and 3.09 mm for the left major, right major, and right minor fissure. Moreover, an analysis of the relation between segmentation quality and fissure completeness showed that the method is robust against incomplete fissures.