Reference standard for the evaluation of automatic segmentation algorithms: Quantification of inter observer variability of manual delineation of prostate contour on MRI.

PURPOSE: The purpose of this study was to investigate the relationship between inter-reader variability in manual prostate contour segmentation on magnetic resonance imaging (MRI) examinations and determine the optimal number of readers required to establish a reliable reference standard. MATERIALS AND METHODS: Seven radiologists with various experiences independently performed manual segmentation of the prostate contour (whole-gland [WG] and transition zone [TZ]) on 40 prostate MRI examinations obtained in 40 patients. Inter-reader variability in prostate contour delineations was estimated using standard metrics (Dice similarity coefficient [DSC], Hausdorff distance and volume-based metrics). The impact of the number of readers (from two to seven) on segmentation variability was assessed using pairwise metrics (consistency) and metrics with respect to a reference segmentation (conformity), obtained either with majority voting or simultaneous truth and performance level estimation (STAPLE) algorithm. RESULTS: The average segmentation DSC for two readers in pairwise comparison was 0.919 for WG and 0.876 for TZ. Variability decreased with the number of readers: the interquartile ranges of the DSC were 0.076 (WG) / 0.021 (TZ) for configurations with two readers, 0.005 (WG) / 0.012 (TZ) for configurations with three readers, and 0.002 (WG) / 0.0037 (TZ) for configurations with six readers. The interquartile range decreased slightly faster between two and three readers than between three and six readers. When using consensus methods, variability often reached its minimum with three readers (with STAPLE, DSC = 0.96 [range: 0.945-0.971] for WG and DSC = 0.94 [range: 0.912-0.957] for TZ, and interquartile range was minimal for configurations with three readers. CONCLUSION: The number of readers affects the inter-reader variability, in terms of inter-reader consistency and conformity to a reference. Variability is minimal for three readers, or three readers represent a tipping point in the variability evolution, with both pairwise-based metrics or metrics with respect to a reference. Accordingly, three readers may represent an optimal number to determine references for artificial intelligence applications.

Prostate volume prediction on MRI: tools, accuracy and variability.

OBJECTIVE: A reliable estimation of prostate volume (PV) is essential to prostate cancer management. The objective of our multi-rater study was to compare intra- and inter-rater variability of PV from manual planimetry and ellipsoid formulas. METHODS: Forty treatment-naive patients who underwent prostate MRI were selected from a local database. PV and corresponding PSA density (PSAd) were estimated on 3D T2-weighted MRI (3 T) by 7 independent radiologists using the traditional ellipsoid formula (TEF), the newer biproximate ellipsoid formula (BPEF), and the manual planimetry method (MPM) used as ground truth. Intra- and inter-rater variability was calculated using the mixed model-based intraclass correlation coefficient (ICC). RESULTS: Mean volumes were 67.00 (± 36.61), 66.07 (± 35.03), and 64.77 (± 38.27) cm3 with the TEF, BPEF, and MPM methods, respectively. Both TEF and BPEF overestimated PV relative to MPM, with the former presenting significant differences (+ 1.91 cm3, IQ = [- 0.33 cm3, 5.07 cm3], p val = 0.03). Both intra- (ICC > 0.90) and inter-rater (ICC > 0.90) reproducibility were excellent. MPM had the highest inter-rater reproducibility (ICC = 0.999). Inter-rater PV variation led to discrepancies in classification according to the clinical criterion of PSAd > 0.15 ng/mL for 2 patients (5%), 7 patients (17.5%), and 9 patients (22.5%) when using MPM, TEF, and BPEF, respectively. CONCLUSION: PV measurements using ellipsoid formulas and MPM are highly reproducible. MPM is a robust method for PV assessment and PSAd calculation, with the lowest variability. TEF showed a high degree of concordance with MPM but a slight overestimation of PV. Precise anatomic landmarks as defined with the BPEF led to a more accurate PV estimation, but also to a higher variability. KEY POINTS: • Manual planimetry used for prostate volume estimation is robust and reproducible, with the lowest variability between readers. • Ellipsoid formulas are accurate and reproducible but with higher variability between readers. • The traditional ellipsoid formula tends to overestimate prostate volume.

Challenge of prostate MRI segmentation on T2-weighted images: inter-observer variability and impact of prostate morphology.

BACKGROUND: Accurate prostate zonal segmentation on magnetic resonance images (MRI) is a critical prerequisite for automated prostate cancer detection. We aimed to assess the variability of manual prostate zonal segmentation by radiologists on T2-weighted (T2W) images, and to study factors that may influence it. METHODS: Seven radiologists of varying levels of experience segmented the whole prostate gland (WG) and the transition zone (TZ) on 40 axial T2W prostate MRI images (3D T2W images for all patients, and both 3D and 2D images for a subgroup of 12 patients). Segmentation variabilities were evaluated based on: anatomical and morphological variation of the prostate (volume, retro-urethral lobe, intensity contrast between zones, presence of a PI-RADS ≥ 3 lesion), variation in image acquisition (3D vs 2D T2W images), and reader's experience. Several metrics including Dice Score (DSC) and Hausdorff Distance were used to evaluate differences, with both a pairwise and a consensus (STAPLE reference) comparison. RESULTS: DSC was 0.92 (± 0.02) and 0.94 (± 0.03) for WG, 0.88 (± 0.05) and 0.91 (± 0.05) for TZ respectively with pairwise comparison and consensus reference. Variability was significantly (p < 0.05) lower for the mid-gland (DSC 0.95 (± 0.02)), higher for the apex (0.90 (± 0.06)) and the base (0.87 (± 0.06)), and higher for smaller prostates (p < 0.001) and when contrast between zones was low (p < 0.05). Impact of the other studied factors was non-significant. CONCLUSIONS: Variability is higher in the extreme parts of the gland, is influenced by changes in prostate morphology (volume, zone intensity ratio), and is relatively unaffected by the radiologist's level of expertise.

Quantitative and Qualitative Approach for Shear Wave Elastography in Superficial Lymph Nodes.

The diagnostic contribution of 2-D shear-wave elastography (SWE) in management of superficial lymph nodes (LNs) of any origin was evaluated in 222 patients referred for needle core biopsy. Each patient underwent conventional B-mode/Doppler ultrasound examinations (conventional ultrasound) and SWE. Quantitative SWE parameters and qualitative SWE map features were extracted. Carcinomas were found to be significantly stiffer than benign LNs (29.5 ± 32.3 kPa vs. 6.7 ± 12.3 kPa). Lymphomas exhibited intermediate stiffness (11.4 ± 5.2 kPa). Qualitative SWE analysis provided color patterns specific to histopathology (stiff rim, nodular and undetermined patterns related to malignancy and blue pattern to benignity). Adding SWE to conventional ultrasound improved the sensitivity of LN diagnosis (from 81.1% to 92.0%) but decreased its specificity (from 73.2% to 67.6%) because of the high prevalence of lymphomas compared with carcinomas. Inter-observer agreement for quantitative SWE was good (intra-class correlation coefficient = 0.82) as was inter-observer diagnostic agreement for qualitative SWE (κ = 0.65). LN location and histology type were found to influence the reported diagnostic performance of SWE.

MRI for prostate cancer: can computed high b-value DWI replace native acquisitions?

OBJECTIVE: To compare computed high b-value diffusion-weighted images (c-DWI) derived from low b-value DWI images and acquired high b-value DWI (a-DWI), in overall image quality and prostate cancer detection rate. MATERIALS AND METHODS: A total of 124 consecutive men with suspected prostate cancer (PCa) underwent diagnosis prostate MRI on a 3.0 T MR system using a 32-channel phased-array torso coil. Among them, 63 underwent prostate biopsy. MRI protocol included 3DT2w images, high resolution Fov Optimized and Constrained Undistorted Single-Shot (FOCUS™) DWI images with b-values of 100, 400, 800, and 2000 s/mm2 and dynamic contrast enhanced images. C-DWI images (2000 and 2500 s/mm2) were derived from the three lower acquired b-value DWI images using a mono-exponential diffusion decay. C-DWI and acquired high b-value DWI (a-DWI) (2000 s/mm2) were compared for image quality (background signal suppression, anatomic clarity, ghosting, distortion) and tumor conspicuity by four radiologists. RESULTS: C-DWIs demonstrated higher rating than a-DWIs for overall image quality despite worsened ghosting. In patients with a biopsy, similar detection rate was observed while conspicuity was better with c-DWI (p < 0.001). Non-acquisition of high b-value a-DWI reduced total acquisition time by 220 s per patient. CONCLUSION: C-DWI provides a substantial reduction in acquisition time while maintaining comparable prostate cancer detection rate and improving global image quality. KEY POINTS: • Computed DWI improves global quality of prostate MRI. • Computed DWI improves analysis of DWI images with decrease acquisition time. • Computed DWI provides greater background suppression of parenchyma and improves conspicuity of suspicious lesion.

18F-Fluorocholine PET/CT and Parathyroid 4D Computed Tomography for Primary Hyperparathyroidism: The Challenge of Reoperative Patients.

BACKGROUND: To evaluate FCH-PET/CT and parathyroid 4D-CT so as to guide surgery in patients with primary hyperparathyroidism (pHPT) and prior neck surgery. METHODS: Medical records of all patients referred for a FCH-PET/CT in our institution were systematically reviewed. Only patients with pHPT, a history of neck surgery (for pHPT or another reason) and an indication of reoperation were included. All patients had parathyroid ultrasound (US) and Tc-99m-sestaMIBI scintigraphy, and furthermore, some patients had 4D-CT. Gold standard was defined by pathological findings and/or US-guided fine-needle aspiration with PTH level measurement in the washing liquid. RESULTS: Twenty-nine patients were included in this retrospective study. FCH-PET/CT identified 34 abnormal foci including 19 ectopic localizations. 4D-CT, performed in 20 patients, detected 11 abnormal glands at first reading and 6 more under FCH-PET/CT guidance. US and Tc-99m-sestaMIBI found concordant foci in 8/29 patients. Gold standard was obtained for 32 abnormal FCH-PET/CT foci in 27 patients. On a per-lesion analysis, sensitivity, specificity, positive and negative predictive values were, respectively, 96%, 13%, 77% and 50% for FCH-PET/CT, 75%, 40%, 80% and 33% for 4D-CT. On a per-patient analysis, sensitivity was 85% for FCH-PET/CT and 63% for 4D-CT. FCH-PET/CT results made it possible to successfully remove an abnormal gland in 21 patients, including 12 with a negative or discordant US/Tc-99m-sestaMIBI scintigraphy result, with a global cure rate of 73%. CONCLUSION: FCH-PET/CT is a promising tool in the challenging population of reoperative patients with pHPT. Parathyroid 4D-CT appears as a confirmatory imaging modality.

Multiparametric MRI for Suspected Recurrent Prostate Cancer after HIFU:Is DCE still needed?

PURPOSE: To assess the added value of the dynamic contrast-enhanced sequence (DCE) to combination T2-weighted imaging (T2w) + diffusion-weighted imaging (DWI) in detecting prostate cancer (PCa) recurrence after HIFU (high-intensity focused ultrasound). METHODS: Forty-five males with clinical and biological suspected PCa recurrence were retrospectively selected. All underwent multi-parametric MRI (mpMRI) before biopsies. Two readers independently assigned a Likert score of cancer likelihood on T2w + DWI + DCE and T2w + DWI images. Prostatic biopsies were taken as the gold standard. RESULTS: Recurrent PCa was identified at biopsy for 37 patients (82%). Areas under the receiver-operating curve of T2w + DWI and T2w + DWI + DCE imaging were not significantly different for both readers. Using a Likert score ≥ 3 for the PCa diagnosis threshold, sensitivity at the lobe level for the (1) senior and (2) junior reader for T2w +DWI +DCE sensitivity was (1) 0.97 and (2) 0.94 vs. (1) 0.94 and (2) 0.97 for T2w + DWI. CONCLUSION: Accuracy of mpMRI was not significantly improved by adding DCE to T2w + DWI. Sensitivity was high for T2w + DWI + DCE and T2w + DWI with no significant difference for either the junior or senior reader. KEY POINTS: • MpMRI has the capability to detect PCa recurrence in post-HIFU monitoring. • The sensitivity of T2w and DWI for detecting PCa recurrence was not improved by DCE. • Readers with different degrees of experience did not improve their performance with DCE.