Mixed Supervision of Histopathology Improves Prostate Cancer Classification from MRI.

Non-invasive prostate cancer classification from MRI has the potential to revolutionize patient care by providing early detection of clinically significant disease, but has thus far shown limited positive predictive value. To address this, we present a image-based deep learning method to predict clinically significant prostate cancer from screening MRI in patients that subsequently underwent biopsy with results ranging from benign pathology to the highest grade tumors. Specifically, we demonstrate that mixed supervision via diverse histopathological ground truth improves classification performance despite the cost of reduced concordance with image-based segmentation. Where prior approaches have utilized pathology results as ground truth derived from targeted biopsies and whole-mount prostatectomy to strongly supervise the localization of clinically significant cancer, our approach also utilizes weak supervision signals extracted from nontargeted systematic biopsies with regional localization to improve overall performance. Our key innovation is performing regression by distribution rather than simply by value, enabling use of additional pathology findings traditionally ignored by deep learning strategies. We evaluated our model on a dataset of 973 (testing n = 198) multi-parametric prostate MRI exams collected at UCSF from 2016-2019 followed by MRI/ultrasound fusion (targeted) biopsy and systematic (nontargeted) biopsy of the prostate gland, demonstrating that deep networks trained with mixed supervision of histopathology can feasibly exceed the performance of the Prostate Imaging-Reporting and Data System (PI-RADS) clinical standard for prostate MRI interpretation (71.6% vs 66.7% balanced accuracy and 0.724 vs 0.716 AUC).

Synthetic PET via Domain Translation of 3-D MRI.

Historically, patient datasets have been used to develop and validate various reconstruction algorithms for PET/MRI and PET/CT. To enable such algorithm development, without the need for acquiring hundreds of patient exams, in this article we demonstrate a deep learning technique to generate synthetic but realistic whole-body PET sinograms from abundantly available whole-body MRI. Specifically, we use a dataset of 56 18F-FDG-PET/MRI exams to train a 3-D residual UNet to predict physiologic PET uptake from whole-body T1-weighted MRI. In training, we implemented a balanced loss function to generate realistic uptake across a large dynamic range and computed losses along tomographic lines of response to mimic the PET acquisition. The predicted PET images are forward projected to produce synthetic PET (sPET) time-of-flight (ToF) sinograms that can be used with vendor-provided PET reconstruction algorithms, including using CT-based attenuation correction (CTAC) and MR-based attenuation correction (MRAC). The resulting synthetic data recapitulates physiologic 18F-FDG uptake, e.g., high uptake localized to the brain and bladder, as well as uptake in liver, kidneys, heart, and muscle. To simulate abnormalities with high uptake, we also insert synthetic lesions. We demonstrate that this sPET data can be used interchangeably with real PET data for the PET quantification task of comparing CTAC and MRAC methods, achieving ≤ 7.6% error in mean-SUV compared to using real data. These results together show that the proposed sPET data pipeline can be reasonably used for development, evaluation, and validation of PET/MRI reconstruction methods.

Presurgical 68Ga-PSMA-11 Positron Emission Tomography for Biochemical Recurrence Risk Assessment: A Follow-up Analysis of a Multicenter Prospective Phase 3 Imaging Trial.

BACKGROUND: In the initial staging of patients with high-risk prostate cancer (PCa), prostate-specific membrane antigen positron emission tomography (PSMA-PET) has been established as a front-line imaging modality. The increasing number of PSMA-PET scans performed in the primary staging setting might be associated with decreases in biochemical recurrence (BCR)-free survival (BCR-FS). OBJECTIVE: To assess the added prognostic value of presurgical PSMA-PET for BCR-FS compared with the presurgical Cancer of the Prostate Risk Assessment (CAPRA) and postsurgical CAPRA-Surgery (CAPRA-S) scores in patients with intermediate- to high-risk PCa treated with radical prostatectomy (RP) and pelvic lymph node dissection. DESIGN, SETTING, AND PARTICIPANTS: This is a follow-up study of the surgical cohort evaluated in the multicenter prospective phase 3 imaging trial (n = 277; NCT03368547, NCT02611882, and NCT02919111). OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: Each 68Ga-PSMA-11-PET scan was read by three blinded independent readers. PSMA-PET prostate uptake (low vs high), PSMA-PET extraprostatic disease (N1/M1), and CAPRA and CAPRA-S scores were used to assess the risk of BCR. Patients were followed after RP by local investigators using electronic medical records. BCR was defined by a prostate-specific antigen (PSA) level increasing to ≥0.2 ng/ml after RP or initiation of PCa-specific secondary treatment (>6 mo after surgery). Univariate and multivariable Cox models, and c-statistic index were performed to assess the prognostic value of PSMA-PET and for a comparison with the CAPRA and CAPRA-S scores. RESULTS AND LIMITATIONS: From December 2015 to December 2019, 277 patients underwent surgery after PSMA-PET. Clinical follow-up was obtained in 240/277 (87%) patients. The median follow-up after surgery was 32.4 (interquartile range 23.3-42.9) mo. Of 240 BCR events, 91 (38%) were observed. PSMA-PET N1/M1 was found in 41/240 (17%) patients. PSMA-PET prostate uptake, PSMA-PET N1/M1, and CAPRA and CAPRA-S scores were significant univariate predictors of BCR. The addition of PSMA-PET N1/M1 status to the presurgical CAPRA score improved the risk assessment for BCR significantly in comparison with the presurgical CAPRA score alone (c-statistic 0.70 [0.64-0.75] vs 0.63 [0.57-0.69]; p < 0.001). The C-index of the postsurgical model utilizing the postsurgical CAPRA-S score alone was not significantly different from the presurgical model combining the presurgical CAPRA score and PSMA-PET N1/M1 status (p = 0.19). CONCLUSIONS: Presurgical PSMA-PET was a strong prognostic biomarker improving BCR-FS risk assessment. Its implementation in the presurgical risk assessment with the CAPRA score improved the performance and reduced the difference with the reference standard (postsurgical CAPRA-S score). PATIENT SUMMARY: The use prostate-specific membrane antigen positron emission tomography improved the assessment of biochemical recurrence risk in patients with intermediate- and high-risk prostate cancer who were treated with radical prostatectomy and pelvic lymph node dissection.

Synthesizing Complex-Valued Multicoil MRI Data from Magnitude-Only Images.

Despite the proliferation of deep learning techniques for accelerated MRI acquisition and enhanced image reconstruction, the construction of large and diverse MRI datasets continues to pose a barrier to effective clinical translation of these technologies. One major challenge is in collecting the MRI raw data (required for image reconstruction) from clinical scanning, as only magnitude images are typically saved and used for clinical assessment and diagnosis. The image phase and multi-channel RF coil information are not retained when magnitude-only images are saved in clinical imaging archives. Additionally, preprocessing used for data in clinical imaging can lead to biased results. While several groups have begun concerted efforts to collect large amounts of MRI raw data, current databases are limited in the diversity of anatomy, pathology, annotations, and acquisition types they contain. To address this, we present a method for synthesizing realistic MR data from magnitude-only data, allowing for the use of diverse data from clinical imaging archives in advanced MRI reconstruction development. Our method uses a conditional GAN-based framework to generate synthetic phase images from input magnitude images. We then applied ESPIRiT to derive RF coil sensitivity maps from fully sampled real data to generate multi-coil data. The synthetic data generation method was evaluated by comparing image reconstruction results from training Variational Networks either with real data or synthetic data. We demonstrate that the Variational Network trained on synthetic MRI data from our method, consisting of GAN-derived synthetic phase and multi-coil information, outperformed Variational Networks trained on data with synthetic phase generated using current state-of-the-art methods. Additionally, we demonstrate that the Variational Networks trained with synthetic k-space data from our method perform comparably to image reconstruction networks trained on undersampled real k-space data.

Federated Learning with Research Prototypes: Application to Multi-Center MRI-based Detection of Prostate Cancer with Diverse Histopathology.

RATIONALE AND OBJECTIVES: Early prostate cancer detection and staging from MRI is extremely challenging for both radiologists and deep learning algorithms, but the potential to learn from large and diverse datasets remains a promising avenue to increase their performance within and across institutions. To enable this for prototype-stage algorithms, where the majority of existing research remains, we introduce a flexible federated learning framework for cross-site training, validation, and evaluation of custom deep learning prostate cancer detection algorithms. MATERIALS AND METHODS: We introduce an abstraction of prostate cancer groundtruth that represents diverse annotation and histopathology data. We maximize use of this groundtruth if and when they are available using UCNet, a custom 3D UNet that enables simultaneous supervision of pixel-wise, region-wise, and gland-wise classification. We leverage these modules to perform cross-site federated training using 1400+ heterogeneous multi-parameteric prostate MRI exams from two University hospitals. RESULTS: We observe a positive result, with significant improvements in cross-site generalization performance with negligible intra-site performance degradation for both lesion segmentation and per-lesion binary classification of clinically-significant prostate cancer. Cross-site lesion segmentation performance intersection-over-union (IoU) improved by 100%, while cross-site lesion classification performance overall accuracy improved by 9.5-14.8%, depending on the optimal checkpoint selected by each site. CONCLUSION: Federated learning can improve the generalization performance of prostate cancer detection models across institutions while protecting patient health information and institution-specific code and data. However, even more data and participating institutions are likely required to improve the absolute performance of prostate cancer classification models. To enable adoption of federated learning with limited re-engineering of federated components, we open-source our FLtools system at https://federated.ucsf.edu, including examples that can be easily adapted to other medical imaging deep learning projects.

Generation of synthetic megavoltage CT for MRI-only radiotherapy treatment planning using a 3D deep convolutional neural network.

BACKGROUND: Megavoltage computed tomography (MVCT) has been implemented on many radiotherapy treatment machines for on-board anatomical visualization, localization, and adaptive dose calculation. Implementing an MR-only workflow by synthesizing MVCT from magnetic resonance imaging (MRI) would offer numerous advantages for treatment planning and online adaptation. PURPOSE: In this work, we sought to synthesize MVCT (sMVCT) datasets from MRI using deep learning to demonstrate the feasibility of MRI-MVCT only treatment planning. METHODS: MVCTs and T1-weighted MRIs for 120 patients treated for head-and-neck cancer were retrospectively acquired and co-registered. A deep neural network based on a fully-convolutional 3D U-Net architecture was implemented to map MRI intensity to MVCT HU. Input to the model were volumetric patches generated from paired MRI and MVCT datasets. The U-Net was initialized with random parameters and trained on a mean absolute error (MAE) objective function. Model accuracy was evaluated on 18 withheld test exams. sMVCTs were compared to respective MVCTs. Intensity-modulated volumetric radiotherapy (IMRT) plans were generated on MVCTs of four different disease sites and compared to plans calculated onto corresponding sMVCTs using the gamma metric and dose-volume-histograms (DVHs). RESULTS: MAE values between sMVCT and MVCT datasets were 93.3 ± 27.5, 78.2 ± 27.5, and 138.0 ± 43.4 HU for whole body, soft tissue, and bone volumes, respectively. Overall, there was good agreement between sMVCT and MVCT, with bone and air posing the greatest challenges. The retrospective dataset introduced additional deviations due to sinus filling or tumor growth/shrinkage between scans, differences in external contours due to variability in patient positioning, or when immobilization devices were absent from diagnostic MRIs. Dose distributions of IMRT plans evaluated for four test cases showed close agreement between sMVCT and MVCT images when evaluated using DVHs and gamma dose metrics, which averaged to 98.9 ± 1.0% and 96.8 ± 2.6% analyzed at 3%/3 mm and 2%/2 mm, respectively. CONCLUSIONS: MVCT datasets can be generated from T1-weighted MRI using a 3D deep convolutional neural network with dose calculation on a sample sMVCT in close agreement with the MVCT. These results demonstrate the feasibility of using MRI-derived sMVCT in an MR-only treatment planning workflow.

A Path to Qualification of PET/MRI Scanners for Multicenter Brain Imaging Studies: Evaluation of MRI-Based Attenuation Correction Methods Using a Patient Phantom.

PET/MRI scanners cannot be qualified in the manner adopted for hybrid PET/CT devices. The main hurdle with qualification in PET/MRI is that attenuation correction (AC) cannot be adequately measured in conventional PET phantoms because of the difficulty in converting the MR images of the physical structures (e.g., plastic) into electron density maps. Over the last decade, a plethora of novel MRI-based algorithms has been developed to more accurately derive the attenuation properties of the human head, including the skull. Although promising, none of these techniques has yet emerged as an optimal and universally adopted strategy for AC in PET/MRI. In this work, we propose a path for PET/MRI qualification for multicenter brain imaging studies. Specifically, our solution is to separate the head AC from the other factors that affect PET data quantification and use a patient as a phantom to assess the former. The emission data collected on the integrated PET/MRI scanner to be qualified should be reconstructed using both MRI- and CT-based AC methods, and whole-brain qualitative and quantitative (both voxelwise and regional) analyses should be performed. The MRI-based approach will be considered satisfactory if the PET quantification bias is within the acceptance criteria specified here. We have implemented this approach successfully across 2 PET/MRI scanner manufacturers at 2 sites.

Harmonization of PET image reconstruction parameters in simultaneous PET/MRI.

OBJECTIVE: Simultaneous PET/MRIs vary in their quantitative PET performance due to inherent differences in the physical systems and differences in the image reconstruction implementation. This variability in quantitative accuracy confounds the ability to meaningfully combine and compare data across scanners. In this work, we define image reconstruction parameters that lead to comparable contrast recovery curves across simultaneous PET/MRI systems. METHOD: The NEMA NU-2 image quality phantom was imaged on one GE Signa and on one Siemens mMR PET/MRI scanner. The phantom was imaged at 9.7:1 contrast with standard spheres (diameter 10, 13, 17, 22, 28, 37 mm) and with custom spheres (diameter: 8.5, 11.5, 15, 25, 32.5, 44 mm) using a standardized methodology. Analysis was performed on a 30 min listmode data acquisition and on 6 realizations of 5 min from the listmode data. Images were reconstructed with the manufacturer provided iterative image reconstruction algorithms with and without point spread function (PSF) modeling. For both scanners, a post-reconstruction Gaussian filter of 3-7 mm in steps of 1 mm was applied. Attenuation correction was provided from a scaled computed tomography (CT) image of the phantom registered to the MR-based attenuation images and verified to align on the non-attenuation corrected PET images. For each of these image reconstruction parameter sets, contrast recovery coefficients (CRCs) were determined for the SUVmean, SUVmax and SUVpeak for each sphere. A hybrid metric combining the root-mean-squared discrepancy (RMSD) and the absolute CRC values was used to simultaneously optimize for best match in CRC between the two scanners while simultaneously weighting toward higher resolution reconstructions. The image reconstruction parameter set was identified as the best candidate reconstruction for each vendor for harmonized PET image reconstruction. RESULTS: The range of clinically relevant image reconstruction parameters demonstrated widely different quantitative performance across cameras. The best match of CRC curves was obtained at the lowest RMSD values with: for CRCmean, 2 iterations-7 mm filter on the GE Signa and 4 iterations-6 mm filter on the Siemens mMR, for CRCmax, 4 iterations-6 mm filter on the GE Signa, 4 iterations-5 mm filter on the Siemens mMR and for CRCpeak, 4 iterations-7 mm filter with PSF on the GE Signa and 4 iterations-7 mm filter on the Siemens mMR. Over all reconstructions, the RMSD between CRCs was 1.8%, 3.6% and 2.9% for CRC mean, max and peak, respectively. The solution of 2 iterations-3 mm on the GE Signa and 4 iterations-3 mm on Siemens mMR, both with PSF, led to simultaneous harmonization and with high CRC and low RMSD for CRC mean, max and peak with RMSD values of 2.8%, 5.8% and 3.2%, respectively. CONCLUSIONS: For two commercially available PET/MRI scanners, user-selectable parameters that control iterative updates, image smoothing and PSF modeling provide a range of contrast recovery curves that allow harmonization in harmonization strategies of optimal match in CRC or high CRC values. This work demonstrates that nearly identical CRC curves can be obtained on different commercially available scanners by selecting appropriate image reconstruction parameters.

Nonlinear electrocardiographic imaging using polynomial approximation networks.

Electrocardiography is a valuable tool to aid in medical understanding and treatment of heart-related ailments, specifically atrial fibrillation (AF) and other irregular cardiac behavior. Although signs of AF will manifest in conventional electrocardiogram (ECG) recordings, interpretation and localization of AF sources require significant clinical expertise. In this vein, electrocardiographic imaging has emerged as an important medical imaging modality that provides reconstructions of the heart's electrical activity from non-invasive multi-lead body-surface ECG and anatomical x-ray computed tomography images. In this paper, we present a nonlinear inversion model for computing this mapping to improve upon the reconstruction performance of current methods. While contemporary techniques typically determine an inverse solution by discretizing and inverting an underdetermined linear system of partial differential equations governing the relationship between voltage potentials of the heart and torso, the presented technique re-casts this problem as a task in function approximation and provides a direct parameterization of the inverse operator using a polynomial neural network. That is, the outlined nonlinear inversion technique is a generalization of contemporary reconstruction techniques which allows geometrical and material parameterizations of the forward-model to be optimized using real experimental data collected from patients suffering from AF, as to better represent the inverse operator with respect to reconstruction metrics applicable to electrophysiology. The accuracy of our model is evaluated against a dataset of real-patient recordings to demonstrate its validity, and mathematical analysis is provided to support the polynomial expansion used in our inversion model.

Noise reduction in intracranial pressure signal using causal shape manifolds.

We present the Iterative/Causal Subspace Tracking framework (I/CST) for reducing noise in continuously monitored quasi-periodic biosignals. Signal reconstruction of the basic segments of the noisy signal (e.g. beats) is achieved by projection to a reduced space on which probabilistic tracking is performed. The attractiveness of the presented method lies in the fact that the subspace, or manifold, is learned by incorporating temporal, morphological, and signal elevation constraints, so that segment samples with similar shapes, and that are close in time and elevation, are also close in the subspace representation. Evaluation of the algorithm's effectiveness on the intracranial pressure (ICP) signal serves as a practical illustration of how it can operate in clinical conditions on routinely acquired biosignals. The reconstruction accuracy of the system is evaluated on an idealized 20-min ICP recording established from the average ICP of patients monitored for various ICP related conditions. The reconstruction accuracy of the ground truth signal is tested in presence of varying levels of additive white Gaussian noise (AWGN) and Poisson noise processes, and measures significant increases of 758% and 396% in the average signal-to-noise ratio (SNR).

Plasmonic field confinement for separate absorption-multiplication in InGaAs nanopillar avalanche photodiodes.

Avalanche photodiodes (APDs) are essential components in quantum key distribution systems and active imaging systems requiring both ultrafast response time to measure photon time of flight and high gain to detect low photon flux. The internal gain of an APD can improve system signal-to-noise ratio (SNR). Excess noise is typically kept low through the selection of material with intrinsically low excess noise, using separate-absorption-multiplication (SAM) heterostructures, or taking advantage of the dead-space effect using thin multiplication regions. In this work we demonstrate the first measurement of excess noise and gain-bandwidth product in III-V nanopillars exhibiting substantially lower excess noise factors compared to bulk and gain-bandwidth products greater than 200 GHz. The nanopillar optical antenna avalanche detector (NOAAD) architecture is utilized for spatially separating the absorption region from the avalanche region via the NOA resulting in single carrier injection without the use of a traditional SAM heterostructure.