Toward a standard for the evaluation of PET-Auto-Segmentation methods following the recommendations of AAPM task group No. 211: Requirements and implementation.

PURPOSE: The aim of this paper is to define the requirements and describe the design and implementation of a standard benchmark tool for evaluation and validation of PET-auto-segmentation (PET-AS) algorithms. This work follows the recommendations of Task Group 211 (TG211) appointed by the American Association of Physicists in Medicine (AAPM). METHODS: The recommendations published in the AAPM TG211 report were used to derive a set of required features and to guide the design and structure of a benchmarking software tool. These items included the selection of appropriate representative data and reference contours obtained from established approaches and the description of available metrics. The benchmark was designed in a way that it could be extendable by inclusion of bespoke segmentation methods, while maintaining its main purpose of being a standard testing platform for newly developed PET-AS methods. An example of implementation of the proposed framework, named PETASset, was built. In this work, a selection of PET-AS methods representing common approaches to PET image segmentation was evaluated within PETASset for the purpose of testing and demonstrating the capabilities of the software as a benchmark platform. RESULTS: A selection of clinical, physical, and simulated phantom data, including "best estimates" reference contours from macroscopic specimens, simulation template, and CT scans was built into the PETASset application database. Specific metrics such as Dice Similarity Coefficient (DSC), Positive Predictive Value (PPV), and Sensitivity (S), were included to allow the user to compare the results of any given PET-AS algorithm to the reference contours. In addition, a tool to generate structured reports on the evaluation of the performance of PET-AS algorithms against the reference contours was built. The variation of the metric agreement values with the reference contours across the PET-AS methods evaluated for demonstration were between 0.51 and 0.83, 0.44 and 0.86, and 0.61 and 1.00 for DSC, PPV, and the S metric, respectively. Examples of agreement limits were provided to show how the software could be used to evaluate a new algorithm against the existing state-of-the art. CONCLUSIONS: PETASset provides a platform that allows standardizing the evaluation and comparison of different PET-AS methods on a wide range of PET datasets. The developed platform will be available to users willing to evaluate their PET-AS methods and contribute with more evaluation datasets.

Technical Note: A new zeolite PET phantom to test segmentation algorithms on heterogeneous activity distributions featured with ground-truth contours.

PURPOSE: Design, realization, scan, and characterization of a phantom for PET Automatic Segmentation (PET-AS) assessment are presented. Radioactive zeolites immersed in a radioactive heterogeneous background simulate realistic wall-less lesions with known irregular shape and known homogeneous or heterogeneous internal activity. METHOD: Three different zeolite families were evaluated in terms of radioactive uptake homogeneity, necessary to define activity and contour ground truth. Heterogeneous lesions were simulated by the perfect matching of two portions of a broken zeolite, soaked in two different 18 F-FDG radioactive solutions. Heterogeneous backgrounds were obtained with tissue paper balls and sponge pieces immersed into radioactive solutions. RESULTS: Natural clinoptilolite proved to be the most suitable zeolite for the construction of artificial objects mimicking homogeneous and heterogeneous uptakes in 18 F-FDG PET lesions. Heterogeneous backgrounds showed a coefficient of variation equal to 269% and 443% of a uniform radioactive solution. Assembled phantom included eight lesions with volumes ranging from 1.86 to 7.24 ml and lesion to background contrasts ranging from 4.8:1 to 21.7:1. CONCLUSIONS: A novel phantom for the evaluation of PET-AS algorithms was developed. It is provided with both reference contours and activity ground truth, and it covers a wide range of volumes and lesion to background contrasts. The dataset is open to the community of PET-AS developers and utilizers.

Background based Gaussian mixture model lesion segmentation in PET.

PURPOSE: Quantitative (18)F-fluorodeoxyglucose positron emission tomography is limited by the uncertainty in lesion delineation due to poor SNR, low resolution, and partial volume effects, subsequently impacting oncological assessment, treatment planning, and follow-up. The present work develops and validates a segmentation algorithm based on statistical clustering. The introduction of constraints based on background features and contiguity priors is expected to improve robustness vs clinical image characteristics such as lesion dimension, noise, and contrast level. METHODS: An eight-class Gaussian mixture model (GMM) clustering algorithm was modified by constraining the mean and variance parameters of four background classes according to the previous analysis of a lesion-free background volume of interest (background modeling). Hence, expectation maximization operated only on the four classes dedicated to lesion detection. To favor the segmentation of connected objects, a further variant was introduced by inserting priors relevant to the classification of neighbors. The algorithm was applied to simulated datasets and acquired phantom data. Feasibility and robustness toward initialization were assessed on a clinical dataset manually contoured by two expert clinicians. Comparisons were performed with respect to a standard eight-class GMM algorithm and to four different state-of-the-art methods in terms of volume error (VE), Dice index, classification error (CE), and Hausdorff distance (HD). RESULTS: The proposed GMM segmentation with background modeling outperformed standard GMM and all the other tested methods. Medians of accuracy indexes were VE <3%, Dice >0.88, CE <0.25, and HD <1.2 in simulations; VE <23%, Dice >0.74, CE <0.43, and HD <1.77 in phantom data. Robustness toward image statistic changes (±15%) was shown by the low index changes: <26% for VE, <17% for Dice, and <15% for CE. Finally, robustness toward the user-dependent volume initialization was demonstrated. The inclusion of the spatial prior improved segmentation accuracy only for lesions surrounded by heterogeneous background: in the relevant simulation subset, the median VE significantly decreased from 13% to 7%. Results on clinical data were found in accordance with simulations, with absolute VE <7%, Dice >0.85, CE <0.30, and HD <0.81. CONCLUSIONS: The sole introduction of constraints based on background modeling outperformed standard GMM and the other tested algorithms. Insertion of a spatial prior improved the accuracy for realistic cases of objects in heterogeneous backgrounds. Moreover, robustness against initialization supports the applicability in a clinical setting. In conclusion, application-driven constraints can generally improve the capabilities of GMM and statistical clustering algorithms.

Optimization of rapid acquisition with relaxation enhancement (RARE) pulse sequence parameters for ¹9F-MRI studies.

PURPOSE: To optimize signal-to-noise ratio (SNR) in fast spin echo (rapid acquisition with relaxation enhancement [RARE]) sequences and to improve sensitivity in ¹9F magnetic resonance imaging (MRI) on a 7T preclinical MRI system, based on a previous experimental evaluation of T1 and T2 actual relaxation times. MATERIALS AND METHODS: Relative SNR changes were theoretically calculated at given relaxation times (T1, T2) and mapped in RARE parameter space (TR, number of echoes, flip back pulse), at fixed acquisition times. T1 and T2 of KPF6 phantom samples (solution, agar mixtures, ex vivo perfused brain) were measured and experimental SNR values were compared with simulations, at optimal and suboptimal RARE parameter values. RESULTS: The optimized setting largely depended on T1, T2 times and the use of flip back pulse improved SNR up to 30% in case of low T1/T2 ratios. Relaxation times in different conditions showed negligible changes in T1 (below 14%) and more evident changes in T2 (-95% from water solution to ex vivo brain). Experimental data confirmed theoretical forecasts, within an error margin always below 4.1% at SNR losses of ~20% and below 8.8% at SNR losses of ~40%. The optimized settings permitted a detection threshold at a concentration of 0.5 mM, corresponding to 6.22 × 10¹6 fluorine atoms per voxel. CONCLUSION: Optimal settings according to measured relaxation times can significantly improve the sensitivity threshold in ¹9F MRI studies. They were provided in a wide range of (T1, T2) values and experimentally validated showing good agreement.

The use of zeolites to generate PET phantoms for the validation of quantification strategies in oncology.

PURPOSE: In recent years, segmentation algorithms and activity quantification methods have been proposed for oncological (18)F-fluorodeoxyglucose (FDG) PET. A full assessment of these algorithms, necessary for a clinical transfer, requires a validation on data sets provided with a reliable ground truth as to the imaged activity distribution, which must be as realistic as possible. The aim of this work is to propose a strategy to simulate lesions of uniform uptake and irregular shape in an anthropomorphic phantom, with the possibility to easily obtain a ground truth as to lesion activity and borders. METHODS: Lesions were simulated with samples of clinoptilolite, a family of natural zeolites of irregular shape, able to absorb aqueous solutions of (18)F-FDG, available in a wide size range, and nontoxic. Zeolites were soaked in solutions of (18)F-FDG for increasing times up to 120 min and their absorptive properties were characterized as function of soaking duration, solution concentration, and zeolite dry weight. Saturated zeolites were wrapped in Parafilm, positioned inside an Alderson thorax-abdomen phantom and imaged with a PET-CT scanner. The ground truth for the activity distribution of each zeolite was obtained by segmenting high-resolution finely aligned CT images, on the basis of independently obtained volume measurements. The fine alignment between CT and PET was validated by comparing the CT-derived ground truth to a set of zeolites' PET threshold segmentations in terms of Dice index and volume error. RESULTS: The soaking time necessary to achieve saturation increases with zeolite dry weight, with a maximum of about 90 min for the largest sample. At saturation, a linear dependence of the uptake normalized to the solution concentration on zeolite dry weight (R(2) = 0.988), as well as a uniform distribution of the activity over the entire zeolite volume from PET imaging were demonstrated. These findings indicate that the (18)F-FDG solution is able to saturate the zeolite pores and that the concentration does not influence the distribution uniformity of both solution and solute, at least at the trace concentrations used for zeolite activation. An additional proof of uniformity of zeolite saturation was obtained observing a correspondence between uptake and adsorbed volume of solution, corresponding to about 27.8% of zeolite volume. As to the ground truth for zeolites positioned inside the phantom, the segmentation of finely aligned CT images provided reliable borders, as demonstrated by a mean absolute volume error of 2.8% with respect to the PET threshold segmentation corresponding to the maximum Dice. CONCLUSIONS: The proposed methodology allowed obtaining an experimental phantom data set that can be used as a feasible tool to test and validate quantification and segmentation algorithms for PET in oncology. The phantom is currently under consideration for being included in a benchmark designed by AAPM TG211, which will be available to the community to evaluate PET automatic segmentation methods.