Techniques for robot-aided intraocular surgery using monocular vision.

This paper presents techniques for robot-aided intraocular surgery using monocular vision in order to overcome erroneous stereo reconstruction in an intact eye. We propose a new retinal surface estimation method based on a structured-light approach. A handheld robot known as the Micron enables automatic scanning of a laser probe, creating projected beam patterns on the retinal surface. Geometric analysis of the patterns then allows planar reconstruction of the surface. To realize automated surgery in an intact eye, monocular hybrid visual servoing is accomplished through a scheme that incorporates surface reconstruction and partitioned visual servoing. We investigate the sensitivity of the estimation method according to relevant parameters and also evaluate its performance in both dry and wet conditions. The approach is validated through experiments for automated laser photocoagulation in a realistic eye phantom in vitro. Finally, we present the first demonstration of automated intraocular laser surgery in porcine eyes ex vivo.

Toward Monocular Camera-Guided Retinal Vein Cannulation with an Actively Stabilized Handheld Robot.

In this paper we describe work towards retinal vessel cannulation using an actively stabilized handheld robot, guided by monocular vision. We employ a previously developed monocular camera based surface reconstruction method using automated laser beam scanning over the retina. We use the reconstructed plane to find a coordinate transform between the 2D image plane coordinate system and the global 3D frame. Within a hemispherical region around the target, we use motion scaling for higher precision. The contribution of this work is the homography matrix estimation using monocular vision and application of the previously developed laser surface reconstruction to Micron guided vein cannulation. Experiments are conducted in a wet eye phantom to show the higher accuracy of the surface reconstruction as compared to standard stereo reconstruction. Further, experiments to show the increased surgical accuracy due to motion scaling are also carried out.

Comparative Evaluation of Handheld Robot-Aided Intraocular Laser Surgery.

This paper presents robot-aided intraocular laser surgery using a handheld robot known as Micron. The micromanipulator incorporated in Micron enables visual servoing of a laser probe, while maintaining a constant distance of the tool tip from the retinal surface. The comparative study was conducted with various control methods for evaluation of robot-aided intraocular laser surgery.

Hybrid position/force control of an active handheld micromanipulator for membrane peeling.

BACKGROUND: Peeling procedures in retinal surgery require micron-scale manipulation and control of sub-tactile forces. METHODS: Hybrid position/force control of an actuated handheld microsurgical instrument is presented as a means for simultaneously improving positioning accuracy and reducing forces to prevent avoidable trauma to tissue. The system response was evaluated, and membrane-peeling trials were performed by four test subjects in both artificial and animal models. RESULTS: Maximum force was reduced by 56% in both models compared with position control. No statistically significant effect on procedure duration was observed. CONCLUSIONS: A hybrid position/force control system has been implemented that successfully attenuates forces and minimizes unwanted excursions during microsurgical procedures such as membrane peeling. Results also suggest that improvements in safety using this technique may be attained without increasing the duration of the procedure.

Handheld-automated microsurgical instrumentation for intraocular laser surgery.

BACKGROUND AND OBJECTIVE: Laser photocoagulation is a mainstay or adjuvant treatment for a variety of common retinal diseases. Automated laser photocoagulation during intraocular surgery has not yet been established. The authors introduce an automated laser photocoagulation system for intraocular surgery, based on a novel handheld instrument. The goals of the system are to enhance accuracy and efficiency and improve safety. MATERIALS AND METHODS: Triple-ring patterns are introduced as a typical arrangement for the treatment of proliferative retinopathy and registered to a preoperative fundus image. In total, 32 target locations are specified along the circumferences of three rings having diameters of 1, 2, and 3 mm, with a burn spacing of 600 µm. Given the initial system calibration, the retinal surface is reconstructed using stereo vision, and the targets specified on the preoperative image are registered with the control system. During automated operation, the laser probe attached to the manipulator of the active handheld instrument is deflected as needed via visual servoing in order to correct the error between the aiming beam and a specified target, regardless of any erroneous handle motion by the surgeon. A constant distance of the laser probe from the retinal surface is maintained in order to yield consistent size of burns and ensure safety during operation. Real-time tracking of anatomical features enables compensation for any movement of the eye. A graphical overlay system within operating microscope provides the surgeon with guidance cues for automated operation. Two retinal surgeons performed automated and manual trials in an artificial model of the eye, with each trial repeated three times. For the automated trials, various targeting thresholds (50-200 µm) were used to automatically trigger laser firing. In manual operation, fixed repetition rates were used, with frequencies of 1.0-2.5 Hz. The power of the 532 nm laser was set at 3.0 W with a duration of 20 ms. After completion of each trial, the speed of operation and placement error of burns were measured. The performance of the automated laser photocoagulation was compared with manual operation, using interpolated data for equivalent firing rates from 1.0 to 1.75 Hz. RESULTS: In automated trials, average error increased from 45 ± 27 to 60 ± 37 µm as the targeting threshold varied from 50 to 200 µm, while average firing rate significantly increased from 0.69 to 1.71 Hz. The average error in the manual trials increased from 102 ± 67 to 174 ± 98 µm as firing rate increased from 1.0 to 2.5 Hz. Compared to the manual trials, the average error in the automated trials was reduced by 53.0-56.4%, resulting in statistically significant differences (P ≤ 10(-20) ) for all equivalent frequencies (1.0-1.75 Hz). The depth of the laser tip in the automated trials was consistently maintained within 18 ± 2 µm root-mean-square (RMS) of its initial position, whereas it significantly varied in the manual trials, yielding an error of 296 ± 30 µm RMS. At high firing rates in manual trials, such as at 2.5 Hz, laser photocoagulation is marginally attained, yielding failed burns of 30% over the entire pattern, whereas no failed burns are found in automated trials. Relatively regular burn sizes are attained in the automated trials by the depth servoing of the laser tip, while burn sizes in the manual trials vary considerably. Automated avoidance of blood vessels was also successfully demonstrated, utilizing the retina-tracking feature to identify avoidance zones. CONCLUSION: Automated intraocular laser surgery can improve the accuracy of photocoagulation while ensuring safety during operation. This paper provides an initial demonstration of the technique under reasonably realistic laboratory conditions; development of a clinically applicable system requires further work.

Vision-Based Control of a Handheld Surgical Micromanipulator with Virtual Fixtures.

Performing micromanipulation and delicate operations in submillimeter workspaces is difficult because of destabilizing tremor and imprecise targeting. Accurate micromanipulation is especially important for microsurgical procedures, such as vitreoretinal surgery, to maximize successful outcomes and minimize collateral damage. Robotic aid combined with filtering techniques that suppress tremor frequency bands increases performance; however, if knowledge of the operator's goals is available, virtual fixtures have been shown to further improve performance. In this paper, we derive a virtual fixture framework for active handheld micromanipulators that is based on high-bandwidth position measurements rather than forces applied to a robot handle. For applicability in surgical environments, the fixtures are generated in real-time from microscope video during the procedure. Additionally, we develop motion scaling behavior around virtual fixtures as a simple and direct extension to the proposed framework. We demonstrate that virtual fixtures significantly outperform tremor cancellation algorithms on a set of synthetic tracing tasks (p < 0.05). In more medically relevant experiments of vein tracing and membrane peeling in eye phantoms, virtual fixtures can significantly reduce both positioning error and forces applied to tissue (p < 0.05).

Micron: an Actively Stabilized Handheld Tool for Microsurgery.

We describe the design and performance of a hand-held actively stabilized tool to increase accuracy in micro-surgery or other precision manipulation. It removes involuntary motion such as tremor by actuating the tip to counteract the effect of the undesired handle motion. The key components are a three-degree-of-freedom piezoelectric manipulator that has 400 µm range of motion, 1 N force capability, and bandwidth over 100 Hz, and an optical position measurement subsystem that acquires the tool pose with 4 µm resolution at 2000 samples/s. A control system using these components attenuates hand motion by at least 15 dB (a fivefold reduction). By considering the effect of the frequency response of Micron on the human visual feedback loop, we have developed a filter that reduces unintentional motion, yet preserves intuitive eye-hand coordination. We evaluated the effectiveness of Micron by measuring the accuracy of the human/machine system in three simple manipulation tasks. Handheld testing by three eye surgeons and three non-surgeons showed a reduction in position error of between 32% and 52%, depending on the error metric.

Comparative evaluation of monocular augmented-reality display for surgical microscopes.

Medical augmented reality has undergone much development recently. However, there is a lack of studies quantitatively comparing the different display options available. This paper compares the effects of different graphical overlay systems in a simple micromanipulation task with "soft" visual servoing. We compared positioning accuracy in a real-time visually-guided task using Micron, an active handheld tremor-canceling microsurgical instrument, using three different displays: 2D screen, 3D screen, and microscope with monocular image injection. Tested with novices and an experienced vitreoretinal surgeon, display of virtual cues in the microscope via an augmented reality injection system significantly decreased 3D error (p < 0.05) compared to the 2D and 3D monitors when confounding factors such as magnification level were normalized.

Position-Based Virtual Fixtures for Membrane Peeling with a Handheld Micromanipulator.

Peeling delicate retinal membranes, which are often less than 5 µm thick, is one of the most challenging retinal surgeries. Preventing rips and tears caused by tremor and excessive force can decrease injury and reduce the need for follow up surgeries. We propose the use of a fully handheld microsurgical robot to suppress tremor while enforcing helpful constraints on the motion of the tool. Using stereo vision and tracking algorithms, the robot activates motion-scaled behavior as the tip reaches the surface, providing finer control during the critical step of engaging the membrane edge. A hard virtual fixture just below the surface limits the total downward force that can be applied. Furthermore, velocity limiting during the peeling helps the surgeon maintain a smooth, constant force while lifting and delaminating the membrane. On a phantom consisting of plastic wrap stretched across a rubber slide, we demonstrate our approach reduces maximum force by 40-70%.

Retinal vessel cannulation with an image-guided handheld robot.

Cannulation of small retinal vessels is often prohibitively difficult for surgeons, since physiological tremor often exceeds the narrow diameter of the vessel (40-120 microm). Using an active handheld micromanipulator, we introduce an image-guided robotic system that reduces tremor and provides smooth, scaled motion during the procedure. The micromanipulator assists the surgeon during the approach, puncture, and injection stages of the procedure by tracking the pipette and anatomy viewed under the microscope. In experiments performed ex vivo by an experienced retinal surgeon on 40-60 microm vessels in porcine eyes, the success rate was 29% (2/7) without the aid of the system and 63% (5/8) with the aid of the system.

Semiautomated intraocular laser surgery using handheld instruments.

BACKGROUND AND OBJECTIVE: In laser retinal photocoagulation, hundreds of dot-like burns are applied. We introduce a robot-assisted technique to enhance the accuracy and reduce the tedium of the procedure. MATERIALS AND METHODS: Laser burn locations are overlaid on preoperative retinal images using common patterns such as grids. A stereo camera/monitor setup registers and displays the planned burn locations overlaid on real-time video. Using an active handheld micromanipulator, a 7 x 7 grid of burns spaced 650 microm apart is applied to both paper slides and porcine retina in vitro using 30 milliseconds laser pulses at 532 nm. Two scenarios were tested: unaided, in which the micromanipulator is inert and the laser fires at a fixed frequency, and aided, in which the micromanipulator actively targets burn locations and the laser fires automatically upon target acquisition. Error is defined as the distance from the center of the observed burn mark to the preoperatively selected target location. RESULTS: An experienced retinal surgeon performed trials with and without robotic assistance, on both paper slides and porcine retina in vitro. In the paper slide experiments at an unaided laser repeat rate of 0.5 Hz, error was 125+/-62 microm with robotic assistance and 149+/-76 microm without (P < 0.005), and trial duration was 70+/-8 seconds with robotic assistance and 97+/-7 seconds without (P < 0.005). At a repeat rate of 1.0 Hz, error was 129+/-69 microm with robotic assistance and 166+/-91 microm without (P < 0.005), and trial duration was 26+/-4 seconds with robotic assistance and 47+/-1 seconds without (P < 0.005). At a repeat rate of 2.0 Hz on porcine retinal tissue, error was 123+/-69 microm with robotic assistance and 203+/-104 microm without (P < 0.005). CONCLUSION: Robotic assistance can increase the accuracy of laser photocoagulation while reducing the duration of the operation.

Active guidance for laser retinal surgery with a handheld instrument.

Laser photocoagulation is a standard interventional tool in vitreoretinal surgery. Commonly applied treatments such as grid photocoagulation and panretinal photocoagulation involve the application of hundreds of dot-like laser burns to the retina. In order to enhance the accuracy and reduce the tedium of this procedure, we are developing a robot-assisted technique for retinal laser photocoagulation that includes software for planning patterns of laser burns on a retinal image and uses an active handheld micromanipulator known as Micron in order to apply the pattern of burns to the retina. The paper describes the system and presents preliminary results. In a sample 7x7 pattern of lesions applied to an artificial surface, the system demonstrated a mean position error of 43+/-23 microm.

Test of tracing performance with an active handheld micromanipulator.

This paper demonstrates tremor compensation for human subjects using an active handheld micromanipulator. This instrument uses optical and inertial sensing to detect its own motion, estimates tremor using linear filtering, and a flexure-based manipulator to actuate the tip. Compensation results while tracing a line with the tool are presented for both novice users and a trained surgeon. Learning effects from repetition of the trials over a ten-day period are described.

Guidance of retrobulbar injection with real-time tomographic reflection.

OBJECTIVE: Retrobulbar and peribulbar injections are common ophthalmologic procedures used to deliver anesthetics and other medications for ophthalmic therapy and surgery. These injections, typically performed without any type of guidance, can lead to complications that are rare but visually devastating. The needle may penetrate the optic nerve, perforate the globe, or disperse toxic quantities of drugs intraocularly, causing major visual loss. Sonographic guidance may increase the accuracy of the needle placement, thereby decreasing the incidence of complications. However, difficulties arise in coordinating the relative location of the image, the needle, and the patient. Real-time tomographic reflection is a new method for in situ visualization of sonographic images, permitting direct hand-eye coordination to guide invasive instruments beneath the surface of the skin. METHOD: In this preliminary study, real-time tomographic reflection was used to visualize the eye and surrounding anatomic structures in a cadaver during a simulated retrobulbar injection. RESULT: The needle tip was easily followed as it was advanced into the retrobulbar space. CONCLUSIONS: The images presented in this preliminary study show the use of real-time tomographic reflection to visualize insertion of an invasive instrument into the human body.