Robust GPU-based virtual reality simulation of radio-frequency ablations for various needle geometries and locations.

PURPOSE: Radio-frequency ablations play an important role in the therapy of malignant liver lesions. The navigation of a needle to the lesion poses a challenge for both the trainees and intervening physicians. METHODS: This publication presents a new GPU-based, accurate method for the simulation of radio-frequency ablations for lesions at the needle tip in general and for an existing visuo-haptic 4D VR simulator. The method is implemented real time capable with Nvidia CUDA. RESULTS: It performs better than a literature method concerning the theoretical characteristic of monotonic convergence of the bioheat PDE and a in vitro gold standard with significant improvements ([Formula: see text]) in terms of Pearson correlations. It shows no failure modes or theoretically inconsistent individual simulation results after the initial phase of 10 s. On the Nvidia 1080 Ti GPU, it achieves a very high frame rendering performance of > 480 Hz. CONCLUSION: Our method provides a more robust and safer real-time ablation planning and intraoperative guidance technique, especially avoiding the overestimation of the ablated tissue death zone, which is risky for the patient in terms of tumor recurrence. Future in vitro measurements and optimization shall further improve the conservative estimate.

Fully automatic catheter segmentation in MRI with 3D convolutional neural networks: application to MRI-guided gynecologic brachytherapy.

External-beam radiotherapy followed by high dose rate (HDR) brachytherapy is the standard-of-care for treating gynecologic cancers. The enhanced soft-tissue contrast provided by magnetic resonance imaging (MRI) makes it a valuable imaging modality for diagnosing and treating these cancers. However, in contrast to computed tomography (CT) imaging, the appearance of the brachytherapy catheters, through which radiation sources are inserted to reach the cancerous tissue later on, is often variable across images. This paper reports, for the first time, a new deep-learning-based method for fully automatic segmentation of multiple closely spaced brachytherapy catheters in intraoperative MRI. Represented in the data are 50 gynecologic cancer patients treated by MRI-guided HDR brachytherapy. For each patient, a single intraoperative MRI was used. 826 catheters in the images were manually segmented by an expert radiation physicist who is also a trained radiation oncologist. The number of catheters in a patient ranged between 10 and 35. A deep 3D convolutional neural network (CNN) model was developed and trained. In order to make the learning process more robust, the network was trained 5 times, each time using a different combination of shown patients. Finally, each test case was processed by the five networks and the final segmentation was generated by voting on the obtained five candidate segmentations. 4-fold validation was executed and all the patients were segmented. An average distance error of 2.0 ± 3.4 mm was achieved. False positive and false negative catheters were 6.7% and 1.5% respectively. Average Dice score was equal to 0.60 ± 0.17. The algorithm is available for use in the open source software platform 3D Slicer allowing for wide scale testing and research discussion. In conclusion, to the best of our knowledge, fully automatic segmentation of multiple closely spaced catheters from intraoperative MR images was achieved for the first time in gynecological brachytherapy.

Accurate model-based segmentation of gynecologic brachytherapy catheter collections in MRI-images.

The gynecological cancer mortality rate, including cervical, ovarian, vaginal and vulvar cancers, is more than 20,000 annually in the US alone. In many countries, including the US, external-beam radiotherapy followed by high dose rate brachytherapy is the standard-of-care. The superior ability of MR to visualize soft tissue has led to an increase in its usage in planning and delivering brachytherapy treatment. A technical challenge associated with the use of MRI imaging for brachytherapy, in contrast to that of CT imaging, is the visualization of catheters that are used to place radiation sources into cancerous tissue. We describe here a precise, accurate method for achieving catheter segmentation and visualization. The algorithm, with the assistance of manually provided tip locations, performs segmentation using image-features, and is guided by a catheter-specific, estimated mechanical model. A final quality control step removes outliers or conflicting catheter trajectories. The mean Hausdorff error on a 54 patient, 760 catheter reference database was 1.49  mm; 51 of the outliers deviated more than two catheter widths (3.4  mm) from the gold standard, corresponding to catheter identification accuracy of 93% in a Syed-Neblett template. In a multi-user simulation experiment for evaluating RMS precision by simulating varying manually-provided superior tip positions, 3σ maximum errors were 2.44  mm. The average segmentation time for a single catheter was 3 s on a standard PC. The segmentation time, accuracy and precision, are promising indicators of the value of this method for clinical translation of MR-guidance in gynecologic brachytherapy and other catheter-based interventional procedures.

Evaluation of Direct Haptic 4D Volume Rendering of Partially Segmented Data for Liver Puncture Simulation.

This work presents an evaluation study using a force feedback evaluation framework for a novel direct needle force volume rendering concept in the context of liver puncture simulation. PTC/PTCD puncture interventions targeting the bile ducts have been selected to illustrate this concept. The haptic algorithms of the simulator system are based on (1) partially segmented patient image data and (2) a non-linear spring model effective at organ borders. The primary aim is to quantitatively evaluate force errors caused by our patient modeling approach, in comparison to haptic force output obtained from using gold-standard, completely manually-segmented data. The evaluation of the force algorithms compared to a force output from fully manually segmented gold-standard patient models, yields a low mean of 0.12 N root mean squared force error and up to 1.6 N for systematic maximum absolute errors. Force errors were evaluated on 31,222 preplanned test paths from 10 patients. Only twelve percent of the emitted forces along these paths were affected by errors. This is the first study evaluating haptic algorithms with deformable virtual patients in silico. We prove haptic rendering plausibility on a very high number of test paths. Important errors are below just noticeable differences for the hand-arm system.

Efficient patient modeling for visuo-haptic VR simulation using a generic patient atlas.

BACKGROUND AND OBJECTIVE: This work presents a new time-saving virtual patient modeling system by way of example for an existing visuo-haptic training and planning virtual reality (VR) system for percutaneous transhepatic cholangio-drainage (PTCD). METHODS: Our modeling process is based on a generic patient atlas to start with. It is defined by organ-specific optimized models, method modules and parameters, i.e. mainly individual segmentation masks, transfer functions to fill the gaps between the masks and intensity image data. In this contribution, we show how generic patient atlases can be generalized to new patient data. The methodology consists of patient-specific, locally-adaptive transfer functions and dedicated modeling methods such as multi-atlas segmentation, vessel filtering and spline-modeling. RESULTS: Our full image volume segmentation algorithm yields median DICE coefficients of 0.98, 0.93, 0.82, 0.74, 0.51 and 0.48 regarding soft-tissue, liver, bone, skin, blood and bile vessels for ten test patients and three selected reference patients. Compared to standard slice-wise manual contouring time saving is remarkable. CONCLUSIONS: Our segmentation process shows out efficiency and robustness for upper abdominal puncture simulation systems. This marks a significant step toward establishing patient-specific training and hands-on planning systems in a clinical environment.

Real-Time Ultrasound Simulation for Training of US-Guided Needle Insertion in Breathing Virtual Patients.

One draw-back of most existing VR ultrasound training simulators is the use of static 3D patient models neglecting physiological changes induced e.g. by respiration or heart motion. In this paper to the aim of more realistic Ultrasound simulation, breathing motion extracted from 4D CT image data is integrated into our visuo-haptic simulation framework. The simulated ultrasound images are used for the training of US-guided needle insertion procedures in liver surgery. The methodology developed enables US simulation, 3D visualization and haptic steering of the ultrasound probe and the needle in real-time in breathing virtual bodies.

A Virtual Reality System for PTCD Simulation Using Direct Visuo-Haptic Rendering of Partially Segmented Image Data.

This study presents a new visuo-haptic virtual reality (VR) training and planning system for percutaneous transhepatic cholangio-drainage (PTCD) based on partially segmented virtual patient models. We only use partially segmented image data instead of a full segmentation and circumvent the necessity of surface or volume mesh models. Haptic interaction with the virtual patient during virtual palpation, ultrasound probing and needle insertion is provided. Furthermore, the VR simulator includes X-ray and ultrasound simulation for image-guided training. The visualization techniques are GPU-accelerated by implementation in Cuda and include real-time volume deformations computed on the grid of the image data. Computation on the image grid enables straightforward integration of the deformed image data into the visualization components. To provide shorter rendering times, the performance of the volume deformation algorithm is improved by a multigrid approach. To evaluate the VR training system, a user evaluation has been performed and deformation algorithms are analyzed in terms of convergence speed with respect to a fully converged solution. The user evaluation shows positive results with increased user confidence after a training session. It is shown that using partially segmented patient data and direct volume rendering is suitable for the simulation of needle insertion procedures such as PTCD.

Direct Visuo-Haptic 4D Volume Rendering Using Respiratory Motion Models.

This article presents methods for direct visuo-haptic 4D volume rendering of virtual patient models under respiratory motion. Breathing models are computed based on patient-specific 4D CT image data sequences. Virtual patient models are visualized in real-time by ray casting based rendering of a reference CT image warped by a time-variant displacement field, which is computed using the motion models at run-time. Furthermore, haptic interaction with the animated virtual patient models is provided by using the displacements computed at high rendering rates to translate the position of the haptic device into the space of the reference CT image. This concept is applied to virtual palpation and the haptic simulation of insertion of a virtual bendable needle. To this aim, different motion models that are applicable in real-time are presented and the methods are integrated into a needle puncture training simulation framework, which can be used for simulated biopsy or vessel puncture in the liver. To confirm real-time applicability, a performance analysis of the resulting framework is given. It is shown that the presented methods achieve mean update rates around 2,000 Hz for haptic simulation and interactive frame rates for volume rendering and thus are well suited for visuo-haptic rendering of virtual patients under respiratory motion.

An Image-based Multiproxy Palpation Algorithm for Patient-Specific VR-Simulation.

Palpation is the first step for many medical interventions. To provide an immersive virtual training and planning environment, the palpation step has to be successfully modeled and simulated. Here, we present a multiproxy approach that calculates friction and surface resistance forces for multiple contact points on finger tips or virtual tools like ultrasound probes and displays the resulting force and torque on a 6DOF haptic device. No manual or time intensive segmentation of patient image data is needed to create a simulation based on CT data and thus our approach is usable for patient-specific simulation of palpation.

Ray-casting based evaluation framework for haptic force feedback during percutaneous transhepatic catheter drainage punctures.

PURPOSE: Development of new needle insertion force feedback algorithms requires comparison with a gold standard method. A new evaluation framework was formulated and tested on needle punctures for percutaneous transhepatic catheter drainage (PTCD). METHODS: Needle insertion is an established procedure for minimally invasive interventions in the liver. Up-to-date, needle insertions are precisely planned using 2D axial CT slices from 3D data sets. To provide a 3D virtual reality and haptic training and planning environment, the full segmentation of patient data is often a mandatory step. To lessen the time required for manual segmentation, we propose direct haptic volume-rendering based on CT gray values and partially segmented patient data. The core contribution is a new force output evaluation method driven by a ray-casting technique that defines paths from the skin to target structures, i.e., the right hepatic duct near the juncture with the common hepatic duct. A ray-casting method computes insertion trajectories from the skin to the duct considering no-go structures and plausibility criteria. A rating system scores each trajectory. Finally, the best insertion trajectories are selected that reach the target. Along the selected paths, force output comparison between a reference system and the new haptic force output algorithm is carried out, quantified and visualized. RESULTS: The evaluation framework is presented along with an exemplary study of the liver using the atlas data set from a reference patient. In a comparison of our reference method to a newer algorithm, force outputs are found to be similar in 99% of the paths. CONCLUSION: The proposed evaluation framework allows reliable detection of problematic PTCD trajectories and provides valuable hints to improve force feedback algorithm development.

Optimized image-based soft tissue deformation algorithms for visualization of haptic needle insertion.

Real-time surgical simulation relies on the fast computation of soft tissue deformations. In this paper, we present image-based algorithms for computing the deformations of a volumetric image during a needle insertion in real-time. The algorithms are based on diffusive and linear elastic finite difference methods as utilized in image registration. For an evaluation, the methods are compared to a finite element simulation of the pre-puncture phase of a needle insertion. Furthermore, the methods are improved and tested; contrary to our assumption, an improved diffusion based approach outperforms a linear elastic approach. The algorithms are used to perform a VR simulation of a needle insertion with visuo-haptic feedback.

Direct haptic volume rendering in lumbar puncture simulation.

The preparation phase for surgical simulations often comprises the segmentation of patient data, which is needed for realistic visual and haptic rendering. Expert segmentation of 3D patient data sets can last from several hours to days. In this paper we introduce a direct haptic volume rendering approach for lumbar punctures. Preparation time spent for segmentation is much shorter and compared to our reference system nearly identical force output at the needle tip can be observed. The number of structures to be completely segmented by an expert is reduced from 11 to 3 tissues in abdominal data sets with 300 slices.

Reanalysis precision of 3D quantitative computed tomography (QCT) of the spine.

PURPOSE: To evaluate the precision of 3D QCT of the spine. METHODS: Interoperator analysis reproducibility of two different 3D QCT analysis systems (QCTPro from Mindways Software Inc and MIAF-Spine from the Institute of Medical Physics, University of Erlangen) was evaluated in 29 postmenopausal women. For each analysis system four different trained operators analyzed all scans independently. Results of the vertebrae L1 and L2 were averaged. With QCTPro BMD of the central trabecular elliptical VOI was analyzed. With MIAF-Spine integral, trabecular and cortical BMD, BMC and volume were analyzed in the total vertebral body, the elliptical cylinder and the Osteo VOIs that were further subdivided into superior, mid and inferior subVOIs, each. RESULTS: Precision errors (%CVrms) for the central trabecular VOI that is also used in the traditional single slice QCT techniques were 1.7+/-2.2% and 0.6+/-0.6% for QCTPro and MIAF-Spine, respectively. For MIAF-Spine integral BMD precision errors were lowest in the total and mid Osteo subVOIs (0.5+/-0.5%). Trabecular BMD precision errors were lowest in the mid subVOIs (0.6+/-0.6%). For trabecular BMD there were no differences among the total vertebral body, elliptical cylinder and Osteo VOIs. Cortical BMD precision errors were lowest in the mid total vertebral body subVOI (2.1+/-1.9%) and slightly higher in the mid of the Osteo subVOI. Precision errors in the superior and inferior subVOIs were typically 50% to 100% higher compared to the mid subVOIs. DISCUSSION: Compared to QCTPro MIAF-Spine uses an automated 3D segmentation and an anatomic vertebral coordinate system to position a variety of analysis VOIs. This results in better precision than the more manually assisted analysis used by QCTPro. In-vivo precision errors will be approximately 0.5% higher compared to the analysis precision errors reported here (13). The results demonstrate that with 3D QCT in-vivo precision errors of about 1%-1.5% for trabecular and 2.5% to 3% for cortical bone can be obtained in postmenopausal women.

A hierarchical 3D segmentation method and the definition of vertebral body coordinate systems for QCT of the lumbar spine.

We have developed a new hierarchical 3D technique to segment the vertebral bodies in order to measure bone mineral density (BMD) with high trueness and precision in volumetric CT datasets. The hierarchical approach starts with a coarse separation of the individual vertebrae, applies a variety of techniques to segment the vertebral bodies with increasing detail and ends with the definition of an anatomic coordinate system for each vertebral body, relative to which up to 41 trabecular and cortical volumes of interest are positioned. In a pre-segmentation step constraints consisting of Boolean combinations of simple geometric shapes are determined that enclose each individual vertebral body. Bound by these constraints viscous deformable models are used to segment the main shape of the vertebral bodies. Volume growing and morphological operations then capture the fine details of the bone-soft tissue interface. In the volumes of interest bone mineral density and content are determined. In addition, in the segmented vertebral bodies geometric parameters such as volume or the length of the main axes of inertia can be measured. Intra- and inter-operator precision errors of the segmentation procedure were analyzed using existing clinical patient datasets. Results for segmented volume, BMD, and coordinate system position were below 2.0%, 0.6%, and 0.7%, respectively. Trueness was analyzed using phantom scans. The bias of the segmented volume was below 4%; for BMD it was below 1.5%. The long-term goal of this work is improved fracture prediction and patient monitoring in the field of osteoporosis. A true 3D segmentation also enables an accurate measurement of geometrical parameters that may augment the clinical value of a pure BMD analysis.