Systematic analysis of bias and variability of texture measurements in computed tomography.

Texture is a key radiomics measurement for quantification of disease and disease progression. The sensitivity of the measurements to image acquisition, however, is uncertain. We assessed bias and variability of computed tomography (CT) texture feature measurements across many clinical image acquisition settings and reconstruction algorithms. Diverse, anatomically informed textures (texture A, B, and C) were simulated across 1188 clinically relevant CT imaging conditions representing four in-plane pixel sizes (0.4, 0.5, 0.7, and 0.9 mm), three slice thicknesses (0.625, 1.25, and 2.5 mm), three dose levels ( CTDI vol 1.90, 3.75, and 7.50 mGy), and 33 reconstruction kernels. Imaging conditions corresponded to noise and resolution properties representative of five commercial scanners (GE LightSpeed VCT, GE Discovery 750 HD, GE Revolution, Siemens Definition Flash, and Siemens Force) in filtered backprojection and iterative reconstruction. About 21 texture features were calculated and compared between the ground-truth phantom (i.e., preimaging) and its corresponding images. Each feature was measured with four unique volumes of interest (VOIs) sizes (244, 579, 1000, and 1953 mm 3 . To characterize the bias, the percentage relative difference [PRD(%)] in each feature was calculated between the imaged scenario and the ground truth for all VOI sizes. Feature variability was assessed in terms of (1)  σ PRD ( % ) indicating the variability between the ground truth and simulated image scenario based on the PRD(%), (2)  COV f indicating the simulation-based variability, and (3)  COV T indicating the natural variability present in the ground-truth phantom. The PRD ranged widely from - 97 % to 1220%, with an underlying variability ( σ ) of up to 241%. Features such as gray-level nonuniformity, texture entropy, sum average, and homogeneity exhibited low susceptibility to reconstruction kernel effects ( PRD < 3 % ) with relatively small σ PRD ( % ) ( ≤ 5 % ) across imaging conditions. The dynamic range of results indicates that image acquisition and reconstruction conditions of in-plane pixel sizes, slice thicknesses, dose levels, and reconstruction kernels can lead to significant bias and variability in feature measurements.

Systematic analysis of bias and variability of morphologic features for lung lesions in computed tomography.

We propose to characterize the bias and variability of quantitative morphology features of lung lesions across a range of computed tomography (CT) imaging conditions. A total of 15 lung lesions were simulated (five in each of three spiculation classes: low, medium, and high). For each lesion, a series of simulated CT images representing different imaging conditions were synthesized by applying three-dimensional blur and adding correlated noise based on the measured noise and resolution properties of five commercial multislice CT systems, representing three dose levels ( CTDI vol of 1.90, 3.75, 7.50 mGy), three slice thicknesses (0.625, 1.25, 2.5 mm), and 33 clinical reconstruction kernels from five clinical scanners. The images were segmented using three segmentation algorithms and each algorithm was evaluated by computing a Sørensen-Dice coefficient between the ground truth and the segmentation. A series of 21 shape-based morphology features were extracted from both "ground truth" (i.e., preblur without noise) and "image rendered" lesions (i.e., postblur and with noise). For each morphology feature, the bias was quantified by comparing the percentage relative error in the morphology metric between the imaged lesions and the ground-truth lesions. The variability was characterized by calculating the average coefficient of variation averaged across repeats and imaging conditions. The active contour segmentation had the highest average Dice coefficient of 0.80 followed by 0.63 for threshold, and 0.39 for fuzzy c-means. The bias of the features was segmentation algorithm and feature-dependent, with sharper kernels being less biased and smoother kernels being more biased in general. The feature variability from simulated images ranged from 0.30% to 10% for repeats of the same condition and from 0.74% to 25.3% for different lesions in the same spiculation class. In conclusion, the bias of morphology features is dependent on the acquisition protocol in combination with the segmentation algorithm used and the variability is primarily dependent on the segmentation algorithm.

Validation of lesion simulations in clinical CT data for anonymized chest and abdominal CT databases.

PURPOSE: To make available to the medical imaging community a computed tomography (CT) image database composed of hybrid datasets (patient CT images with digitally inserted anthropomorphic lesions) where lesion ground truth is known a priori. It is envisioned that such a dataset could be a resource for the assessment of CT image quality, machine learning, and imaging technologies [e.g., computer aided detection (CAD) and segmentation algorithms]. ACQUISITION AND VALIDATION METHODS: This HIPPA compliant, IRB waiver of approval study consisted of utilizing 120 chest and 100 abdominal clinically acquired adult CT exams. One image series per patient exam was utilized based on coverage of the anatomical region of interest (either the thorax or abdomen). All image series were de-identified. Simulated lesions were derived from a library of anatomically informed digital lesions (93 lung and 50 liver lesions) where six and four digital lesions with nominal diameters ranging from 4 to 20 mm were inserted into lung and liver image series, respectively. Locations for lesion insertion were randomly chosen. A previously validated lesion simulation and virtual insertion technique were utilized. The resulting hybrid images were reviewed by three experienced radiologists to assure similarity with routine clinical imaging in a diverse adult population. DATA FORMAT AND USAGE NOTES: The database is composed of four datasets that contain 100 patient cases each, for a total of 400 image series accompanied by Matlab.mat tables that provide descriptive information about the virtually inserted lesions (i.e., size, shape, opacity, and insertion location in physical (world) coordinates and voxel indices). All image and metadata are stored in DICOM format on the Quantitative Imaging Data Warehouse (https://qidw.rsna.org/#collection/57d463471cac0a4ec8ff8f46/folder/5b23dceb1cac0a4ec800a770?dialog=login), in two sets: (a) QIBA CT Hybrid Dataset I which contains Lung I and Liver I datasets, and (b) QIBA CT Hybrid Dataset II which contains Lung II and Liver II datasets. The QIDW is supported by the Radiological Society of North America (RSNA). Registration is required upon initial log in. POTENTIAL APPLICATIONS: By simulating lesion opacity (full solid, part solid and ground glass), size, and texture, the relationship between lesion morphology and segmentation or CAD algorithm performance can be investigated without the need for repetitive patient exams. This database can also serve as a reference standard for device and reader performance studies.

Evaluation of Simulated Lesions as Surrogates to Clinical Lesions for Thoracic CT Volumetry: The Results of an International Challenge.

RATIONALE AND OBJECTIVES: To evaluate a new approach to establish compliance of segmentation tools with the computed tomography volumetry profile of the Quantitative Imaging Biomarker Alliance (QIBA); and determine the statistical exchangeability between real and simulated lesions through an international challenge. MATERIALS AND METHODS: The study used an anthropomorphic phantom with 16 embedded physical lesions and 30 patient cases from the Reference Image Database to Evaluate Therapy Response with pathologically confirmed malignancies. Hybrid datasets were generated by virtually inserting simulated lesions corresponding to physical lesions into the phantom datasets using one projection-domain-based method (Method 1), two image-domain insertion methods (Methods 2 and 3), and simulated lesions corresponding to real lesions into the Reference Image Database to Evaluate Therapy Response dataset (using Method 2). The volumes of the real and simulated lesions were compared based on bias (measured mean volume differences between physical and virtually inserted lesions in phantoms as quantified by segmentation algorithms), repeatability, reproducibility, equivalence (phantom phase), and overall QIBA compliance (phantom and clinical phase). RESULTS: For phantom phase, three of eight groups were fully QIBA compliant, and one was marginally compliant. For compliant groups, the estimated biases were -1.8 ± 1.4%, -2.5 ± 1.1%, -3 ± 1%, -1.8 ± 1.5% (±95% confidence interval). No virtual insertion method showed statistical equivalence to physical insertion in bias equivalence testing using Schuirmann's two one-sided test (±5% equivalence margin). Differences in repeatability and reproducibility across physical and simulated lesions were largely comparable (0.1%-16% and 7%-18% differences, respectively). For clinical phase, 7 of 16 groups were QIBA compliant. CONCLUSION: Hybrid datasets yielded conclusions similar to real computed tomography datasets where phantom QIBA compliant was also compliant for hybrid datasets. Some groups deemed compliant for simulated methods, not for physical lesion measurements. The magnitude of this difference was small (<5.4%). While technical performance is not equivalent, they correlate, such that, volumetrically simulated lesions could potentially serve as practical proxies.

3D task-transfer function representation of the signal transfer properties of low-contrast lesions in FBP- and iterative-reconstructed CT.

PURPOSE: The purpose of this study was to investigate how accurately the task-transfer function (TTF) models the signal transfer properties of low-contrast features in a non-linear commercial CT system. METHODS: A cylindrical phantom containing 24 anthropomorphic "physical" lesions was 3D printed. Lesions had two sizes (523, 2145 mm3 ), and two nominal radio-densities (80 and 100 HU at 120 kV). CT images were acquired on a commercial CT system (Siemens Flash scanner) at four dose levels (CTDIvol , 32 cm phantom:1.5, 3.0, 6.0, 22.0 mGy) and reconstructed using FBP and IR kernels (B31f, B45f, I31f\2, I44f\2). Low-contrast rod inserts (in-plane) and a slanted edge (z-direction) were used to estimate 3D-TTFs. CAD versions of lesions were blurred by the 3D-TTFs, virtually superimposed into corresponding phantom images, and compared to the physical lesions in terms of (a) a 4AFC visual assessment, (b) edge gradient, (c) size, and (d) shape similarity. Assessments 2 and 3 were based on an equivalence criterion D ¯ ≥ COV ¯ to determine if the natural variability COV ¯ in the physical lesions was greater or equal to the difference D ¯ between physical and simulated. Shape similarity was quantified via Sorensen-Dice coefficient (SDC). Comparisons were done for each lesion and for all imaging conditions. RESULTS: The readers detected simulated lesions at a rate of 37.9 ± 3.1% (25% implies random guessing). Lesion edge blur and volume differences D ¯ were on average less than physical lesions' natural variability COV ¯ . The SDC (average ± SD) was 0.80 ± 0.13 (max of 1 possible). CONCLUSIONS: The visual appearance, edge blur, size, and shape of simulated lesions were similar to the physical lesions, which suggests 3D-TTF models the low-contrast signal transfer properties of this non-linear CT system reasonably well.

Estimability index for volume quantification of homogeneous spherical lesions in computed tomography.

Volume of lung nodules is an important biomarker, quantifiable from computed tomography (CT) images. The usefulness of volume quantification, however, depends on the precision of quantification. Experimental assessment of precision is time consuming. A mathematical estimability model was used to assess the quantification precision of CT nodule volumetry in terms of an index ([Formula: see text]), incorporating image noise and resolution, nodule properties, and segmentation software. The noise and resolution were characterized in terms of noise power spectrum and task transfer function. The nodule properties and segmentation algorithm were modeled in terms of a task function and a template function, respectively. The [Formula: see text] values were benchmarked against experimentally acquired precision values from an anthropomorphic chest phantom across 54 acquisition protocols, 2 nodule sizes, and 2 volume segmentation softwares. [Formula: see text] exhibited correlation with experimental precision across nodule sizes and acquisition protocols but dependence on segmentation software. Compared to the assessment of empirical precision, which required [Formula: see text] to perform the segmentation, the [Formula: see text] method required [Formula: see text] from data collection to mathematical computation. A mathematical modeling of volume quantification provides efficient prediction of quantitative performance. It establishes a method to verify quantitative compliance and to optimize clinical protocols for chest CT volumetry.

Interchangeability between real and three-dimensional simulated lung tumors in computed tomography: an interalgorithm volumetry study.

Using hybrid datasets consisting of patient-derived computed tomography (CT) images with digitally inserted computational tumors, we establish volumetric interchangeability between real and computational lung tumors in CT. Pathologically-confirmed malignancies from 30 thoracic patient cases from the RIDER database were modeled. Tumors were either isolated or attached to lung structures. Patient images were acquired on one of two CT scanner models (Lightspeed 16 or VCT; GE Healthcare) using standard chest protocol. Real tumors were segmented and used to inform the size and shape of simulated tumors. Simulated tumors developed in Duke Lesion Tool (Duke University) were inserted using a validated image-domain insertion program. Four readers performed volume measurements using three commercial segmentation tools. We compared the volume estimation performance of segmentation tools between real tumors in actual patient CT images and corresponding simulated tumors virtually inserted into the same patient images (i.e., hybrid datasets). Comparisons involved (1) direct assessment of measured volumes and the standard deviation between simulated and real tumors across readers and tools, respectively, (2) multivariate analysis, involving segmentation tools, readers, tumor shape, and attachment, and (3) effect of local tumor environment on volume measurement. Volume comparison showed consistent trends (9% volumetric difference) between real and simulated tumors across all segmentation tools, readers, shapes, and attachments. Across all cases, readers, and segmentation tools, an intraclass correlation coefficient = 0.99 indicates that simulated tumors correlated strongly with real tumors ( p = 0.95 ). In addition, the impact of the local tumor environment on tumor volume measurement was found to have a segmentation tool-related influence. Strong agreement between simulated tumors modeled in this study compared to their real counterparts suggests a high degree of similarity. This indicates that, volumetrically, simulated tumors embedded into patient CT data can serve as reasonable surrogates to real patient data.

Techniques for virtual lung nodule insertion: volumetric and morphometric comparison of projection-based and image-based methods for quantitative CT.

Virtual nodule insertion paves the way towards the development of standardized databases of hybrid CT images with known lesions. The purpose of this study was to assess three methods (an established and two newly developed techniques) for inserting virtual lung nodules into CT images. Assessment was done by comparing virtual nodule volume and shape to the CT-derived volume and shape of synthetic nodules. 24 synthetic nodules (three sizes, four morphologies, two repeats) were physically inserted into the lung cavity of an anthropomorphic chest phantom (KYOTO KAGAKU). The phantom was imaged with and without nodules on a commercial CT scanner (SOMATOM Definition Flash, Siemens) using a standard thoracic CT protocol at two dose levels (1.4 and 22 mGy CTDIvol). Raw projection data were saved and reconstructed with filtered back-projection and sinogram affirmed iterative reconstruction (SAFIRE, strength 5) at 0.6 mm slice thickness. Corresponding 3D idealized, virtual nodule models were co-registered with the CT images to determine each nodule's location and orientation. Virtual nodules were voxelized, partial volume corrected, and inserted into nodule-free CT data (accounting for system imaging physics) using two methods: projection-based Technique A, and image-based Technique B. Also a third Technique C based on cropping a region of interest from the acquired image of the real nodule and blending it into the nodule-free image was tested. Nodule volumes were measured using a commercial segmentation tool (iNtuition, TeraRecon, Inc.) and deformation was assessed using the Hausdorff distance. Nodule volumes and deformations were compared between the idealized, CT-derived and virtual nodules using a linear mixed effects regression model which utilized the mean, standard deviation, and coefficient of variation ([Formula: see text], [Formula: see text] and [Formula: see text] of the regional Hausdorff distance. Overall, there was a close concordance between the volumes of the CT-derived and virtual nodules. Percent differences between them were less than 3% for all insertion techniques and were not statistically significant in most cases. Correlation coefficient values were greater than 0.97. The deformation according to the Hausdorff distance was also similar between the CT-derived and virtual nodules with minimal statistical significance in the ([Formula: see text]) for Techniques A, B, and C. This study shows that both projection-based and image-based nodule insertion techniques yield realistic nodule renderings with statistical similarity to the synthetic nodules with respect to nodule volume and deformation. These techniques could be used to create a database of hybrid CT images containing nodules of known size, location and morphology.

Volumetric quantification of lung nodules in CT with iterative reconstruction (ASiR and MBIR).

PURPOSE: Volume quantifications of lung nodules with multidetector computed tomography (CT) images provide useful information for monitoring nodule developments. The accuracy and precision of the volume quantification, however, can be impacted by imaging and reconstruction parameters. This study aimed to investigate the impact of iterative reconstruction algorithms on the accuracy and precision of volume quantification with dose and slice thickness as additional variables. METHODS: Repeated CT images were acquired from an anthropomorphic chest phantom with synthetic nodules (9.5 and 4.8 mm) at six dose levels, and reconstructed with three reconstruction algorithms [filtered backprojection (FBP), adaptive statistical iterative reconstruction (ASiR), and model based iterative reconstruction (MBIR)] into three slice thicknesses. The nodule volumes were measured with two clinical software (A: Lung VCAR, B: iNtuition), and analyzed for accuracy and precision. RESULTS: Precision was found to be generally comparable between FBP and iterative reconstruction with no statistically significant difference noted for different dose levels, slice thickness, and segmentation software. Accuracy was found to be more variable. For large nodules, the accuracy was significantly different between ASiR and FBP for all slice thicknesses with both software, and significantly different between MBIR and FBP for 0.625 mm slice thickness with Software A and for all slice thicknesses with Software B. For small nodules, the accuracy was more similar between FBP and iterative reconstruction, with the exception of ASIR vs FBP at 1.25 mm with Software A and MBIR vs FBP at 0.625 mm with Software A. CONCLUSIONS: The systematic difference between the accuracy of FBP and iterative reconstructions highlights the importance of extending current segmentation software to accommodate the image characteristics of iterative reconstructions. In addition, a calibration process may help reduce the dependency of accuracy on reconstruction algorithms, such that volumes quantified from scans of different reconstruction algorithms can be compared. The little difference found between the precision of FBP and iterative reconstructions could be a result of both iterative reconstruction's diminished noise reduction at the edge of the nodules as well as the loss of resolution at high noise levels with iterative reconstruction. The findings do not rule out potential advantage of IR that might be evident in a study that uses a larger number of nodules or repeated scans.