Robotic Motion Compensation for Beating Heart Intracardiac Surgery.

3D ultrasound imaging has enabled minimally invasive, beating heart intracardiac procedures. However, rapid heart motion poses a serious challenge to the surgeon that is compounded by significant time delays and noise in 3D ultrasound. This paper investigates the concept of using a one-degree-of-freedom motion compensation system to synchronize with tissue motions that may be approximated by 1D motion models. We characterize the motion of the mitral valve annulus and show that it is well approximated by a 1D model. The subsequent development of a motion compensation instrument (MCI) is described, as well as an extended Kalman filter (EKF) that compensates for system delays. The benefits and robustness of motion compensation are tested in user trials under a series of non-ideal tracking conditions. Results indicate that the MCI provides an approximately 50% increase in dexterity and 50% decrease in force when compared with a solid tool, but is sensitive to time delays. We demonstrate that the use of the EKF for delay compensation restores performance, even in situations of high heart rate variability. The resulting system is tested in an in vitro 3D ultrasound-guided servoing task, yielding accurate tracking (1.15 mm root mean square) in the presence of noisy, time-delayed 3D ultrasound measurements.

Stereoscopic vision display technology in real-time three-dimensional echocardiography-guided intracardiac beating-heart surgery.

OBJECTIVE: Stereoscopic vision display technology has been shown to be a useful tool in image-guided surgical interventions. However, the concept has not been applied to 3-dimensional echocardiography-guided cardiac procedures. We evaluated stereoscopic vision display as an aid for intracardiac navigation during 3-dimensional echocardiography-guided beating-heart surgery in a model of atrial septal defect closure. METHODS: An atrial septal defect (6 mm) was created in 6 pigs using 3-dimensional echocardiography guidance. The defect was then closed using a catheter-based patch delivery system, and the patch was attached with tissue mini-anchors. Stereoscopic vision was generated with a high-performance volume renderer with stereoscopic glasses. Three-dimensional echocardiography with stereoscopic vision display was compared with 3-dimensional echocardiography with standard display for guidance of surgical repair. Task performance measures for each anchor placement (N = 32 per group) were completion time, trajectory of the tip of the anchor deployment device, and accuracy of the anchor placement. RESULTS: The mean time of the anchor deployment for stereoscopic vision display group was shorter by 44% compared with the standard display group: 9.7 +/- 0.9 seconds versus 17.2 +/- 0.9 seconds (P < .001). Trajectory tracking of the anchor deployment device tip demonstrated greater navigational accuracy measured by trajectory deviation: 3.8 +/- 0.7 mm versus 6.1 +/- 0.3 mm, 38% improvement (P < .01). Accuracy of anchor placement was not significantly different: 2.3 +/- 0.3 mm for the stereoscopic vision display group versus 2.3 +/- 0.3 mm for the standard display group. CONCLUSION: Stereoscopic vision display combined with 3-dimensional echocardiography improved the visualization of 3-dimensional echocardiography ultrasound images, decreased the time required for surgical task completion, and increased the precision of instrument navigation, potentially improving the safety of beating-heart intracardiac surgical interventions.

GPU based real-time instrument tracking with three-dimensional ultrasound.

Real-time three-dimensional ultrasound enables new intracardiac surgical procedures, but the distorted appearance of instruments in ultrasound poses a challenge to surgeons. This paper presents a detection technique that identifies the position of the instrument within the ultrasound volume. The algorithm uses a form of the generalized Radon transform to search for long straight objects in the ultrasound image, a feature characteristic of instruments and not found in cardiac tissue. When combined with passive markers placed on the instrument shaft, the full position and orientation of the instrument is found in 3D space. This detection technique is amenable to rapid execution on the current generation of personal computer graphics processor units (GPU). Our GPU implementation detected a surgical instrument in 31 ms, sufficient for real-time tracking at the 25 volumes per second rate of the ultrasound machine. A water tank experiment found instrument orientation errors of 1.1 degrees and tip position errors of less than 1.8mm. Finally, an in vivo study demonstrated successful instrument tracking inside a beating porcine heart.

Stereo display of 3D ultrasound images for surgical robot guidance.

The recent advent of real-time 3-D ultrasound (3DUS) imaging enables a variety of surgical procedures to be performed within the beating heart. Implementation of these procedures is hampered by the difficulty of manipulating tissue guided by the distorted, low resolution 3DUS images and the dexterity constraints imposed by the confined intracardiac space. This paper investigates the use of surgical robotics in conjunction with 3DUS to overcome these limitations. In addition, it describes the development of a graphics processor based volume Tenderer for real-time stereo visualization of the ultrasound data. Stereo displayed 3DUS was compared to ID-displayed 3DUS and endoscopic guidance with a user study. Five subjects performed in vitro surgical tasks using a surgical robot. Results indicate that subjects were able to complete surgical tasks 35 % faster with stereo-displayed 3DUS images compared to conventional two dimensional display of 3DUS.

GPU based real-time instrument tracking with three dimensional ultrasound.

Real-time 3D ultrasound can enable new image-guided surgical procedures, but high data rates prohibit the use of traditional tracking techniques. We present a new method based on the modified Radon transform that identifies the axis of instrument shafts as bright patterns in planar projections. Instrument rotation and tip location are then determined using fiducial markers. These techniques are amenable to rapid execution on the current generation of personal computer graphics processor units (GPU). Our GPU implementation detected a surgical instrument in 31 ms, sufficient for real-time tracking at the 26 volumes per second rate of the ultrasound machine. A water tank experiment found instrument tip position errors of less than 0.2 mm, and an in vivo study tracked an instrument inside a beating porcine heart. The tracking results showed good correspondence to the actual movements of the instrument.

3D ultrasound in robotic surgery: performance evaluation with stereo displays.

BACKGROUND: The recent advent of real-time 3D ultrasound (3DUS) imaging enables a variety of new surgical procedures. These procedures are hampered by the difficulty of manipulating tissue guided by the distorted, low-resolution 3DUS images. To lessen the effects of these limitations, we investigated stereo displays and surgical robots for 3DUS-guided procedures. METHODS: By integrating real-time stereo rendering of 3DUS with the binocular display of a surgical robot, we compared stereo-displayed 3DUS with normally displayed 3DUS. To test the efficacy of stereo-displayed 3DUS, eight surgeons and eight non-surgeons performed in vitro tasks with the surgical robot. RESULTS: Error rates dropped by 50% with a stereo display. In addition, subjects completed tasks faster with the stereo-displayed 3DUS as compared to normal-displayed 3DUS. A 28% decrease in task time was seen across all subjects. CONCLUSIONS: The results highlight the importance of using a stereo display. By reducing errors and increasing speed, it is an important enhancement to 3DUS-guided robotics procedures.