Part II: Effect of different evaluation methods to the application of a computer-aided prostate MRI detection/diagnosis (CADe/CADx) device on reader performance.

INTRODUCTION: The construction and results of a multiple-reader multiple-case prostate MRI study are described and reported to illustrate recommendations for how to standardize artificial intelligence (AI) prostate studies per the review constituting Part I1. METHODS: Our previously reported approach was applied to review and report an IRB approved, HIPAA compliant multiple-reader multiple-case clinical study of 150 bi-parametric prostate MRI studies across 9 readers, measuring physician performance both with and without the use of the recently FDA cleared CADe/CADx software ProstatID. RESULTS: Unassisted reader AUC values ranged from 0.418 - 0.759, with AI assisted AUC values ranging from 0.507 - 0.787. This represented a statistically significant AUC improvement of 0.045 (α = 0.05). A free-response ROC (FROC) analysis similarly demonstrated a statistically significant increase in θ from 0.405 to 0.453 (α = 0.05). The standalone performance of ProstatID performed across all prostate tissues demonstrated an AUC of 0.929, while the standalone lesion level performance of ProstatID at all biopsied locations achieved an AUC of 0.710. CONCLUSION: This study applies and illustrates suggested reporting and standardization methods for prostate AI studies that will make it easier to understand, evaluate and compare between AI studies. Providing radiologists with the ProstatID CADe/CADx software significantly increased diagnostic performance as assessed by both ROC and free-response ROC metrics. Such algorithms have the potential to improve radiologist performance in the detection and localization of clinically significant prostate cancer.

Part I: prostate cancer detection, artificial intelligence for prostate cancer and how we measure diagnostic performance: a comprehensive review.

MRI has firmly established itself as a mainstay for the detection, staging and surveillance of prostate cancer. Despite its success, prostate MRI continues to suffer from poor inter-reader variability and a low positive predictive value. The recent emergence of Artificial Intelligence (AI) to potentially improve diagnostic performance shows great potential. Understanding and interpreting the AI landscape as well as ever-increasing research literature, however, is difficult. This is in part due to widely varying study design and reporting techniques. This paper aims to address this need by first outlining the different types of AI used for the detection and diagnosis of prostate cancer, next deciphering how data collection methods, statistical analysis metrics (such as ROC and FROC analysis) and end points/outcomes (lesion detection vs. case diagnosis) affect the performance and limit the ability to compare between studies. Finally, this work explores the need for appropriately enriched investigational datasets and proper ground truth, and provides guidance on how to best conduct AI prostate MRI studies. Published in parallel, a clinical study applying this suggested study design was applied to review and report a multiple-reader multiple-case clinical study of 150 bi-parametric prostate MRI studies across nine readers, measuring physician performance both with and without the use of a recently FDA cleared Artificial Intelligence software.1.

Prototype Description and Ex Vivo Evaluation of a System for Combined Endorectal Magnetic Resonance Imaging and In-Bore Biopsy of the Prostate.

ABSTRACT: We describe early ex vivo proof-of-concept testing of a novel system composed of a disposable endorectal coil and converging multichannel needle guide with a reusable clamp stand, embedded electronics, and baseplate to allow for endorectal magnetic resonance (MR) imaging and in-bore MRI-targeted biopsy of the prostate as a single integrated procedure. Using prostate phantoms imaged with standard T 2 -weighted sequences in a Siemens 3T Prisma MR scanner, we measured the signal-to-noise ratio in successive 1-cm distances from the novel coil and from a commercially available inflatable balloon coil and measured the lateral and longitudinal deviation of the tip of a deployed MR compatible needle from the intended target point. Signal-to-noise ratio obtained with the novel system was significantly better than the inflatable balloon coil at each of five 1-cm intervals, with a mean improvement of 78% ( P < 0.05). In a representative sampling of 15 guidance channels, the mean lateral deviation for MR imaging-guided needle positioning was 1.7 mm and the mean longitudinal deviation was 2.0 mm. Our ex vivo results suggest that our novel system provides significantly improved signal-to-noise ratio when compared with an inflatable balloon coil and is capable of accurate MRI-guided needle deployment.

Improving Prostate Cancer Detection With MRI: A Multi-Reader, Multi-Case Study Using Computer-Aided Detection (CAD).

RATIONALE AND OBJECTIVES: To evaluate whether addition of a computer-aided diagnostic (CAD) generated MRI series improves detection of clinically significant prostate cancer. MATERIALS AND METHODS: Nine radiologists retrospectively interpreted 150 prostate MRI examinations without and then with an additional random forest-based CAD model-generated MRI series. Characteristics of biopsy negative versus positive (Gleason ≥ 7 adenocarcinoma) groups were compared using the Wilcoxon test for continuous and Pearson's chi-squared test for categorical variables. The diagnostic performance of readers was compared without versus with CAD using MRMC methods to estimate the area under the receiver operator characteristic curve (AUC). Inter-reader agreement was assessed using weighted inter-rater agreement statistics. Analyses were repeated in peripheral and transition zone subgroups. RESULTS: Among 150 men with median age 67 ± 7.4 years, those with clinically significant prostate cancer were older (68 ± 7.6 years vs. 66 ± 7.0 years; p < .02), had smaller prostate volume (43.9 mL vs. 60.6 mL; p < .001), and no difference in prostate specific antigen (PSA) levels (7.8 ng/mL vs. 6.9 ng/mL; p = .08), but higher PSA density (0.17 ng/mL/cc vs. 0.10 ng/mL/cc; p < .001). Inter-rater agreement (IRA) for PI-RADS scores was moderate without CAD and significantly improved to substantial with CAD (IRA = 0.47 vs. 0.65; p < .001). CAD also significantly improved average reader AUC (AUC = 0.72, vs. AUC = 0.67; p = .02). CONCLUSION: Addition of a random forest method-based, CAD-generated MRI image series improved inter-reader agreement and diagnostic performance for detection of clinically significant prostate cancer, particularly in the transition zone.