Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography.

BACKGROUND: Current laparoscopic images are rich in surface detail but lack information on deeper structures. This report presents a novel method for highlighting these structures during laparoscopic surgery using continuous multislice computed tomography (CT). This has resulted in a more accurate augmented reality (AR) approach, termed "live AR," which merges three-dimensional (3D) anatomy from live low-dose intraoperative CT with live images from the laparoscope. METHODS: A series of procedures with swine was conducted in a CT room with a fully equipped laparoscopic surgical suite. A 64-slice CT scanner was used to image the surgical field approximately once per second. The procedures began with a contrast-enhanced, diagnostic-quality CT scan (initial CT) of the liver followed by continuous intraoperative CT and laparoscopic imaging with an optically tracked laparoscope. Intraoperative anatomic changes included user-applied deformations and those from breathing. Through deformable image registration, an intermediate image processing step, the initial CT was warped to align spatially with the low-dose intraoperative CT scans. The registered initial CT then was rendered and merged with laparoscopic images to create live AR. RESULTS: Superior compensation for soft tissue deformations using the described method led to more accurate spatial registration between laparoscopic and rendered CT images with live AR than with conventional AR. Moreover, substitution of low-dose CT with registered initial CT helped with continuous visualization of the vasculature and offered the potential of at least an eightfold reduction in intraoperative X-ray dose. CONCLUSIONS: The authors proposed and developed live AR, a new surgical visualization approach that merges rich surface detail from a laparoscope with instantaneous 3D anatomy from continuous CT scanning of the surgical field. Through innovative use of deformable image registration, they also demonstrated the feasibility of continuous visualization of the vasculature and considerable X-ray dose reduction. This study provides motivation for further investigation and development of live AR.

Incorporation of preprocedural PET into CT-guided radiofrequency ablation of hepatic metastases: a nonrigid image registration validation study.

This study evaluates the accuracy of augmenting initial intraprocedural computed tomography (CT) during radiofrequency ablation (RFA) of hepatic metastases with preprocedural positron emission tomography (PET) through a hardware-accelerated implementation of an automatic nonrigid PET-CT registration algorithm. The feasibility of augmenting intraprocedural CT with preprocedural PET to improve localization of CT-invisible but PET-positive tumors with images from actual RFA was explored. Preprocedural PET and intraprocedural CT images from 18 cases of hepatic RFA were included. All PET images in the study originated from a hybrid PET/CT scanner, and PET-CT registration was performed in two ways: (1) direct registration of preprocedural PET with intraprocedural CT and (2) indirect registration of preprocedural CT (i.e., the CT of hybrid PET/CT scan) with intraprocedural CT. A hardware-accelerated registration took approximately 2 min. Calculated registration errors were 7.0 and 8.4 mm for the direct and indirect methods, respectively. Overall, the direct registration was found to be statistically not distinct from that performed by a group of clinical experts. The accuracy, execution speed, and compactness of our implementation of nonrigid image registration suggest that existing PET can be overlaid on intraprocedural CT, promising a novel, technically feasible, and clinically viable approach for PET augmentation of CT guidance of RFA.

Hardware implementation of hierarchical volume subdivision-based elastic registration.

Real-time, elastic and fully automated 3D image registration is critical to the efficiency and effectiveness of many image-guided diagnostic and treatment procedures relying on multimodality image fusion or serial image comparison. True, real-time performance will make many 3D image registration-based techniques clinically viable. Hierarchical volume subdivision-based image registration techniques are inherently faster than most elastic registration techniques, e.g. free-form deformation (FFD)-based techniques, and are more amenable for achieving real-time performance through hardware acceleration. Our group has previously reported an FPGA-based architecture for accelerating FFD-based image registration. In this article we show how our existing architecture can be adapted to support hierarchical volume subdivision-based image registration. A proof-of-concept implementation of the architecture achieved speedups of 100 for elastic registration against an optimized software implementation on a 3.2 GHz Pentium III Xeon workstation. Due to inherent parallel nature of the hierarchical volume subdivision-based image registration techniques further speedup can be achieved by using several computing modules in parallel.

Automated 3D elastic registration for improving tumor localization in whole-body PET-CT from combined scanner.

Combined PET/CT scanners provide the ability to produce matching metabolic (from PET) and anatomic (from CT) information in a single examination. However, misalignments continue to exist in tumor localization in PET and CT images acquired using these scanners, due to their inability to compensate for nonrigid misalignment resulting from patient breathing and involuntary movement. We demonstrate that our automatic image subdivision-based elastic registration algorithm can correct this misalignment. In a quantitative validation involving 13 expert-identified tumor nodules in six PET-CT image pairs, the algorithm demonstrated statistically significant improvement over the scanner-defined localization. The accuracy of algorithm-determined localization was evaluated to be comparable to average manually defined localization. The results indicate the potential of using our registration algorithm for applications like radiotherapy treatment planning and treatment-monitoring involving combined PET/CT scanners.