Virtual and Augmented Reality in Interventional Radiology: Current Applications, Challenges, and Future Directions.

Virtual reality (VR) and augmented Reality (AR) are emerging technologies with the potential to revolutionize Interventional radiology (IR). These innovations offer advantages in patient care, interventional planning, and educational training by improving the visualization and navigation of medical images. Despite progress, several challenges hinder their widespread adoption, including limitations in navigation systems, cost, clinical acceptance, and technical constraints of AR/VR equipment. However, ongoing research holds promise with recent advancements such as shape-sensing needles and improved organ deformation modeling. The development of deep learning techniques, particularly for medical imaging segmentation, presents a promising avenue to address existing accuracy and precision issues. Future applications of AR/VR in IR include simulation-based training, preprocedural planning, intraprocedural guidance, and increased patient engagement. As these technologies advance, they are expected to facilitate telemedicine, enhance operational efficiency, and improve patient outcomes, marking a new frontier in interventional radiology.

Automatically tracking brain metastases after stereotactic radiosurgery.

Background and purpose: Patients with brain metastases (BMs) are surviving longer and returning for multiple courses of stereotactic radiosurgery. BMs are monitored after radiation with follow-up magnetic resonance (MR) imaging every 2-3 months. This study investigated whether it is possible to automatically track BMs on longitudinal imaging and quantify the tumor response after radiotherapy. Methods: The METRO process (MEtastasis Tracking with Repeated Observations was developed to automatically process patient data and track BMs. A longitudinal intrapatient registration method for T1 MR post-Gd was conceived and validated on 20 patients. Detections and volumetric measurements of BMs were obtained from a deep learning model. BM tracking was validated on 32 separate patients by comparing results with manual measurements of BM response and radiologists' assessments of new BMs. Linear regression and residual analysis were used to assess accuracy in determining tumor response and size change. Results: A total of 123 irradiated BMs and 38 new BMs were successfully tracked. 66 irradiated BMs were visible on follow-up imaging 3-9 months after radiotherapy. Comparing their longest diameter changes measured manually vs. METRO, the Pearson correlation coefficient was 0.88 (p < 0.001); the mean residual error was -8 ± 17%. The mean registration error was 1.5 ± 0.2 mm. Conclusions: Automatic, longitudinal tracking of BMs using deep learning methods is feasible. In particular, the software system METRO fulfills a need to automatically track and quantify volumetric changes of BMs prior to, and in response to, radiation therapy.

Automated full body tumor segmentation in DOTATATE PET/CT for neuroendocrine cancer patients.

PURPOSE: Neuroendocrine tumors (NETs) are a rare form of cancer that can occur anywhere in the body and commonly metastasizes. The large variance in location and aggressiveness of the tumors makes it a difficult cancer to treat. Assessments of the whole-body tumor burden in a patient image allow for better tracking of disease progression and inform better treatment decisions. Currently, radiologists rely on qualitative assessments of this metric since manual segmentation is unfeasible within a typical busy clinical workflow. METHODS: We address these challenges by extending the application of the nnU-net pipeline to produce automatic NET segmentation models. We utilize the ideal imaging type of 68Ga-DOTATATE PET/CT to produce segmentation masks from which to calculate total tumor burden metrics. We provide a human-level baseline for the task and perform ablation experiments of model inputs, architectures, and loss functions. RESULTS: Our dataset is comprised of 915 PET/CT scans and is divided into a held-out test set (87 cases) and 5 training subsets to perform cross-validation. The proposed models achieve test Dice scores of 0.644, on par with our inter-annotator Dice score on a subset 6 patients of 0.682. If we apply our modified Dice score to the predictions, the test performance reaches a score of 0.80. CONCLUSION: In this paper, we demonstrate the ability to automatically generate accurate NET segmentation masks given PET images through supervised learning. We publish the model for extended use and to support the treatment planning of this rare cancer.

Fast, light, and scalable: harnessing data-mined line annotations for automated tumor segmentation on brain MRI.

OBJECTIVES: While fully supervised learning can yield high-performing segmentation models, the effort required to manually segment large training sets limits practical utility. We investigate whether data mined line annotations can facilitate brain MRI tumor segmentation model development without requiring manually segmented training data. METHODS: In this retrospective study, a tumor detection model trained using clinical line annotations mined from PACS was leveraged with unsupervised segmentation to generate pseudo-masks of enhancing tumors on T1-weighted post-contrast images (9911 image slices; 3449 adult patients). Baseline segmentation models were trained and employed within a semi-supervised learning (SSL) framework to refine the pseudo-masks. Following each self-refinement cycle, a new model was trained and tested on a held-out set of 319 manually segmented image slices (93 adult patients), with the SSL cycles continuing until Dice score coefficient (DSC) peaked. DSCs were compared using bootstrap resampling. Utilizing the best-performing models, two inference methods were compared: (1) conventional full-image segmentation, and (2) a hybrid method augmenting full-image segmentation with detection plus image patch segmentation. RESULTS: Baseline segmentation models achieved DSC of 0.768 (U-Net), 0.831 (Mask R-CNN), and 0.838 (HRNet), improving with self-refinement to 0.798, 0.871, and 0.873 (each p < 0.001), respectively. Hybrid inference outperformed full image segmentation alone: DSC 0.884 (Mask R-CNN) vs. 0.873 (HRNet), p < 0.001. CONCLUSIONS: Line annotations mined from PACS can be harnessed within an automated pipeline to produce accurate brain MRI tumor segmentation models without manually segmented training data, providing a mechanism to rapidly establish tumor segmentation capabilities across radiology modalities. KEY POINTS: • A brain MRI tumor detection model trained using clinical line measurement annotations mined from PACS was leveraged to automatically generate tumor segmentation pseudo-masks. • An iterative self-refinement process automatically improved pseudo-mask quality, with the best-performing segmentation pipeline achieving a Dice score of 0.884 on a held-out test set. • Tumor line measurement annotations generated in routine clinical radiology practice can be harnessed to develop high-performing segmentation models without manually segmented training data, providing a mechanism to rapidly establish tumor segmentation capabilities across radiology modalities.

Deep Learning Achieves Neuroradiologist-Level Performance in Detecting Hydrocephalus Requiring Treatment.

In large clinical centers a small subset of patients present with hydrocephalus that requires surgical treatment. We aimed to develop a screening tool to detect such cases from the head MRI with performance comparable to neuroradiologists. We leveraged 496 clinical MRI exams collected retrospectively at a single clinical site from patients referred for any reason. This diagnostic dataset was enriched to have 259 hydrocephalus cases. A 3D convolutional neural network was trained on 16 manually segmented exams (ten hydrocephalus) and subsequently used to automatically segment the remaining 480 exams and extract volumetric anatomical features. A linear classifier of these features was trained on 240 exams to detect cases of hydrocephalus that required treatment with surgical intervention. Performance was compared to four neuroradiologists on the remaining 240 exams. Performance was also evaluated on a separate screening dataset of 451 exams collected from a routine clinical population to predict the consensus reading from four neuroradiologists using images alone. The pipeline was also tested on an external dataset of 31 exams from a 2nd clinical site. The most discriminant features were the Magnetic Resonance Hydrocephalic Index (MRHI), ventricle volume, and the ratio between ventricle and brain volume. At matching sensitivity, the specificity of the machine and the neuroradiologists did not show significant differences for detection of hydrocephalus on either dataset (proportions test, p > 0.05). ROC performance compared favorably with the state-of-the-art (AUC 0.90-0.96), and replicated in the external validation. Hydrocephalus cases requiring treatment can be detected automatically from MRI in a heterogeneous patient population based on quantitative characterization of brain anatomy with performance comparable to that of neuroradiologists.

Semisupervised Training of a Brain MRI Tumor Detection Model Using Mined Annotations.

Background Artificial intelligence (AI) applications for cancer imaging conceptually begin with automated tumor detection, which can provide the foundation for downstream AI tasks. However, supervised training requires many image annotations, and performing dedicated post hoc image labeling is burdensome and costly. Purpose To investigate whether clinically generated image annotations can be data mined from the picture archiving and communication system (PACS), automatically curated, and used for semisupervised training of a brain MRI tumor detection model. Materials and Methods In this retrospective study, the cancer center PACS was mined for brain MRI scans acquired between January 2012 and December 2017 and included all annotated axial T1 postcontrast images. Line annotations were converted to boxes, excluding boxes shorter than 1 cm or longer than 7 cm. The resulting boxes were used for supervised training of object detection models using RetinaNet and Mask region-based convolutional neural network (R-CNN) architectures. The best-performing model trained from the mined data set was used to detect unannotated tumors on training images themselves (self-labeling), automatically correcting many of the missing labels. After self-labeling, new models were trained using this expanded data set. Models were scored for precision, recall, and F1 using a held-out test data set comprising 754 manually labeled images from 100 patients (403 intra-axial and 56 extra-axial enhancing tumors). Model F1 scores were compared using bootstrap resampling. Results The PACS query extracted 31 150 line annotations, yielding 11 880 boxes that met inclusion criteria. This mined data set was used to train models, yielding F1 scores of 0.886 for RetinaNet and 0.908 for Mask R-CNN. Self-labeling added 18 562 training boxes, improving model F1 scores to 0.935 (P < .001) and 0.954 (P < .001), respectively. Conclusion The application of semisupervised learning to mined image annotations significantly improved tumor detection performance, achieving an excellent F1 score of 0.954. This development pipeline can be extended for other imaging modalities, repurposing unused data silos to potentially enable automated tumor detection across radiologic modalities. © RSNA, 2022 Online supplemental material is available for this article.

Integrating Eye Tracking and Speech Recognition Accurately Annotates MR Brain Images for Deep Learning: Proof of Principle.

PURPOSE: To generate and assess an algorithm combining eye tracking and speech recognition to extract brain lesion location labels automatically for deep learning (DL). MATERIALS AND METHODS: In this retrospective study, 700 two-dimensional brain tumor MRI scans from the Brain Tumor Segmentation database were clinically interpreted. For each image, a single radiologist dictated a standard phrase describing the lesion into a microphone, simulating clinical interpretation. Eye-tracking data were recorded simultaneously. Using speech recognition, gaze points corresponding to each lesion were obtained. Lesion locations were used to train a keypoint detection convolutional neural network to find new lesions. A network was trained to localize lesions for an independent test set of 85 images. The statistical measure to evaluate our method was percent accuracy. RESULTS: Eye tracking with speech recognition was 92% accurate in labeling lesion locations from the training dataset, thereby demonstrating that fully simulated interpretation can yield reliable tumor location labels. These labels became those that were used to train the DL network. The detection network trained on these labels predicted lesion location of a separate testing set with 85% accuracy. CONCLUSION: The DL network was able to locate brain tumors on the basis of training data that were labeled automatically from simulated clinical image interpretation.© RSNA, 2020.

Diffusion and Perfusion MRI Predicts Response Preceding and Shortly After Radiosurgery to Brain Metastases: A Pilot Study.

BACKGROUND AND PURPOSE: To determine the ability of diffusion-weighted imaging (DWI) and dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) to predict long-term response of brain metastases prior to and within 72 hours of stereotactic radiosurgery (SRS). METHODS: In this prospective pilot study, multiple b-value DWI and T1-weighted DCE-MRI were performed in patients with brain metastases before and within 72 hours following SRS. Diffusion-weighted images were analyzed using the monoexponential and intravoxel incoherent motion (IVIM) models. DCE-MRI data were analyzed using the extended Tofts pharmacokinetic model. The parameters obtained with these methods were correlated with brain metastasis outcomes according to modified Response Assessment in Neuro-Oncology Brain Metastases criteria. RESULTS: We included 25 lesions from 16 patients; 16 patients underwent pre-SRS MRI and 12 of 16 patients underwent both pre- and early (within 72 hours) post-SRS MRI. The perfusion fraction (f) derived from IVIM early post-SRS was higher in lesions demonstrating progressive disease than in lesions demonstrating stable disease, partial response, or complete response (q = .041). Pre-SRS extracellular extravascular volume fraction, ve , and volume transfer coefficient, Ktrans , derived from DCE-MRI were higher in nonresponders versus responders (q = .041). CONCLUSIONS: Quantitative DWI and DCE-MRI are feasible imaging methods in the pre- and early (within 72 hours) post-SRS evaluation of brain metastases. DWI- and DCE-MRI-derived parameters demonstrated physiologic changes (tumor cellularity and vascularity) and offer potentially useful biomarkers that can predict treatment response. This allows for initiation of alternate therapies within an effective time window that may help prevent disease progression.

Computational Modeling of Interstitial Fluid Pressure and Velocity in Non-small Cell Lung Cancer Brain Metastases Treated With Stereotactic Radiosurgery.

Background: Early imaging-based treatment response assessment of brain metastases following stereotactic radiosurgery (SRS) remains challenging. The aim of this study is to determine whether early (within 12 weeks) intratumoral changes in interstitial fluid pressure (IFP) and velocity (IFV) estimated from computational fluid modeling (CFM) using dynamic contrast-enhanced (DCE) MRI can predict long-term outcomes of lung cancer brain metastases (LCBMs) treated with SRS. Methods: Pre- and post-treatment T1-weighted DCE-MRI data were obtained in 41 patients treated with SRS for intact LCBMs. The imaging response was assessed using RANO-BM criteria. For each lesion, extravasation of contrast agent measured from Extended Tofts pharmacokinetic Model (volume transfer constant, Ktrans) was incorporated into a computational fluid model to estimate tumor IFP and IFV. Estimates of mean IFP and IFV and heterogeneity (skewness and kurtosis) were calculated for each lesion from pre- and post-SRS imaging. The Wilcoxon rank-sum test was utilized to assess for significant differences in IFP, IFV, and IFP/IFV change (Δ) between response groups. Results: Fifty-three lesions from 41 patients were included. Median follow-up time after SRS was 11 months. The objective response (OR) rate (partial or complete response) was 79%, with 21% demonstrating stable disease (SD) or progressive disease (PD). There were significant response group differences for multiple posttreatment and Δ CFM parameters: post-SRS IFP skewness (mean -0.405 vs. -0.691, p = 0.022), IFP kurtosis (mean 2.88 vs. 3.51, p = 0.024), and IFV mean (5.75e-09 vs. 4.19e-09 m/s, p = 0.027); and Δ IFP kurtosis (mean -2.26 vs. -0.0156, p = 0.017) and IFV mean (1.91e-09 vs. 2.38e-10 m/s, p = 0.013). Posttreatment and Δ thresholds predicted non-OR with high sensitivity (sens): post-SRS IFP skewness (-0.432, sens 84%), kurtosis (2.89, sens 84%), and IFV mean (4.93e-09 m/s, sens 79%); and Δ IFP kurtosis (-0.469, sens 74%) and IFV mean (9.90e-10 m/s, sens 74%). Conclusions: Objective response was associated with lower post-treatment tumor heterogeneity, as represented by reductions in IFP skewness and kurtosis. These results suggest that early post-treatment assessment of IFP and IFV can be used to predict long-term response of lung cancer brain metastases to SRS, allowing a timelier treatment modification.

Advancing Semantic Interoperability of Image Annotations: Automated Conversion of Non-standard Image Annotations in a Commercial PACS to the Annotation and Image Markup.

Sharing radiologic image annotations among multiple institutions is important in many clinical scenarios; however, interoperability is prevented because different vendors' PACS store annotations in non-standardized formats that lack semantic interoperability. Our goal was to develop software to automate the conversion of image annotations in a commercial PACS to the Annotation and Image Markup (AIM) standardized format and demonstrate the utility of this conversion for automated matching of lesion measurements across time points for cancer lesion tracking. We created a software module in Java to parse the DICOM presentation state (DICOM-PS) objects (that contain the image annotations) for imaging studies exported from a commercial PACS (GE Centricity v3.x). Our software identifies line annotations encoded within the DICOM-PS objects and exports the annotations in the AIM format. A separate Python script processes the AIM annotation files to match line measurements (on lesions) across time points by tracking the 3D coordinates of annotated lesions. To validate the interoperability of our approach, we exported annotations from Centricity PACS into ePAD (http://epad.stanford.edu) (Rubin et al., Transl Oncol 7(1):23-35, 2014), a freely available AIM-compliant workstation, and the lesion measurement annotations were correctly linked by ePAD across sequential imaging studies. As quantitative imaging becomes more prevalent in radiology, interoperability of image annotations gains increasing importance. Our work demonstrates that image annotations in a vendor system lacking standard semantics can be automatically converted to a standardized metadata format such as AIM, enabling interoperability and potentially facilitating large-scale analysis of image annotations and the generation of high-quality labels for deep learning initiatives. This effort could be extended for use with other vendors' PACS.

Machine learning for semi-automated classification of glioblastoma, brain metastasis and central nervous system lymphoma using magnetic resonance advanced imaging.

BACKGROUND: Differentiating glioblastoma, brain metastasis, and central nervous system lymphoma (CNSL) on conventional magnetic resonance imaging (MRI) can present a diagnostic dilemma due to the potential for overlapping imaging features. We investigate whether machine learning evaluation of multimodal MRI can reliably differentiate these entities. METHODS: Preoperative brain MRI including diffusion weighted imaging (DWI), dynamic contrast enhanced (DCE), and dynamic susceptibility contrast (DSC) perfusion in patients with glioblastoma, lymphoma, or metastasis were retrospectively reviewed. Perfusion maps (rCBV, rCBF), permeability maps (K-trans, Kep, Vp, Ve), ADC, T1C+ and T2/FLAIR images were coregistered and two separate volumes of interest (VOIs) were obtained from the enhancing tumor and non-enhancing T2 hyperintense (NET2) regions. The tumor volumes obtained from these VOIs were utilized for supervised training of support vector classifier (SVC) and multilayer perceptron (MLP) models. Validation of the trained models was performed on unlabeled cases using the leave-one-subject-out method. Head-to-head and multiclass models were created. Accuracies of the multiclass models were compared against two human interpreters reviewing conventional and diffusion-weighted MR images. RESULTS: Twenty-six patients enrolled with histopathologically-proven glioblastoma (n=9), metastasis (n=9), and CNS lymphoma (n=8) were included. The trained multiclass ML models discriminated the three pathologic classes with a maximum accuracy of 69.2% accuracy (18 out of 26; kappa 0.540, P=0.01) using an MLP trained with the VpNET2 tumor volumes. Human readers achieved 65.4% (17 out of 26) and 80.8% (21 out of 26) accuracies, respectively. Using the MLP VpNET2 model as a computer-aided diagnosis (CADx) for cases in which the human reviewers disagreed with each other on the diagnosis resulted in correct diagnoses in 5 (19.2%) additional cases. CONCLUSIONS: Our trained multiclass MLP using VpNET2 can differentiate glioblastoma, brain metastasis, and CNS lymphoma with modest diagnostic accuracy and provides approximately 19% increase in diagnostic yield when added to routine human interpretation.

Automated deep-neural-network surveillance of cranial images for acute neurologic events.

Rapid diagnosis and treatment of acute neurological illnesses such as stroke, hemorrhage, and hydrocephalus are critical to achieving positive outcomes and preserving neurologic function-'time is brain'1-5. Although these disorders are often recognizable by their symptoms, the critical means of their diagnosis is rapid imaging6-10. Computer-aided surveillance of acute neurologic events in cranial imaging has the potential to triage radiology workflow, thus decreasing time to treatment and improving outcomes. Substantial clinical work has focused on computer-assisted diagnosis (CAD), whereas technical work in volumetric image analysis has focused primarily on segmentation. 3D convolutional neural networks (3D-CNNs) have primarily been used for supervised classification on 3D modeling and light detection and ranging (LiDAR) data11-15. Here, we demonstrate a 3D-CNN architecture that performs weakly supervised classification to screen head CT images for acute neurologic events. Features were automatically learned from a clinical radiology dataset comprising 37,236 head CTs and were annotated with a semisupervised natural-language processing (NLP) framework16. We demonstrate the effectiveness of our approach to triage radiology workflow and accelerate the time to diagnosis from minutes to seconds through a randomized, double-blinded, prospective trial in a simulated clinical environment.

Radioembolization for Unresectable Intrahepatic Cholangiocarcinoma: Review of Safety, Response Evaluation Criteria in Solid Tumors 1.1 Imaging Response and Survival.

The optimal palliative treatment for unresectable intrahepatic cholangiocarcinoma (ICC) remains controversial. While selective internal radiation therapy (SIRT) using yttrium-90 microspheres is a well-accepted treatment for hepatocellular carcinoma, data related to its use for locally advanced ICC remain relatively scarce. Twenty-nine patients (mean age 66 ± 11 years; 15 female) with unresectable biopsy-proven ICC treated with SIRT between June 2008 and April 2015 were retrospectively evaluated for post-treatment toxicity, overall survival, and imaging response using response evaluation criteria in solid tumors (RECIST) 1.1 criteria. RECIST 1.1 response was evaluable following 26 treatments [complete response (CR):0, partial response (PR):3; stable disease (SD):16, progression of disease (PD):7]. Objective response rate (CR+PR) was 12%. Disease control rate (CR+PR+SD) was 73%. Median time to progression was 5.6 [95% confidence interval (CI): 0-12.0] months. Median survival following SIRT was 9.1 (95% CI: 1.7-16.4) months. Post-treatment survival was prolonged in patients with absence of extrahepatic disease (p = 0.03) and correlated with RECIST 1.1 response (p = 0.02). Toxicities were limited to grade I severity and occurred following 27% of treatments. These findings support the safe, effective use of SIRT for unresectable ICC. Post-treatment survival is prolonged in patients with absence of extrahepatic disease at baseline. RECIST 1.1 response following SIRT for ICC is predictive of survival.

Neuroimaging in Central Nervous System Infections.

Radiologic imaging is often employed to supplement clinical evaluation in cases of suspected central nervous system (CNS) infection. While computed tomography (CT) is superior for evaluating osseous integrity, demineralization, and erosive changes and may be more readily available at many institutions, magnetic resonance imaging (MRI) has significantly greater sensitivity for evaluating the cerebral parenchyma, cord, and marrow for early changes that have not yet reached the threshold for CT detection. For these reasons, MRI is generally superior to CT for characterizing bacterial, viral, fungal, and parasitic infections of the CNS. The typical imaging features of common and uncommon CNS infectious processes are reviewed.

A Technical Guide Describing the Use of Transradial Access Technique for Endovascular Interventions.

Transradial arterial access (TRA) has been employed for transcatheter coronary procedures for more than 25 years, with numerous studies demonstrating improved patient safety as compared with transfemoral arterial access. However, TRA remains underused by the interventional radiology and vascular surgery communities. Advantages of TRA over transfemoral arterial access include easier accomplishment of postprocedure hemostasis, decreased risk of hemorrhagic complications, shorter patient recovery leading to immediate ambulation and decreased procedure-related costs, and increased patient satisfaction. In particular, TRA may be advantageous in the population of patients with obesity. The primary patient selection factor to consider before attempting TRA is whether the patient has adequate collateral perfusion to the hand; this is assessed using the Barbeau test. Limitations of TRA may include operator unfamiliarity or learning curve and unavailability of adequate length catheters. The most common complication, although still rare, is localized access site hematoma, which is often asymptomatic. Radial artery occlusion is rare and rarely symptomatic owing to collateral perfusion to the hand. Theoretical increased risk of cerebral embolism during TRA may be minimized by preferentially accessing the left wrist during below-diaphragm procedures, which limits transcatheter manipulation of the aortic arch. Transulnar artery access is under investigation for use in patients who cannot undergo TRA. Providing patients the option of TRA can lead to improved outcomes, potentially increasing safety and patient satisfaction while decreasing procedure costs.