Cable Tension Optimization for an Epicardial Parallel Wire Robot.

HeartPrinter is a novel under-constrained 3-cable parallel wire robot designed for minimally invasive epicardial interventions. The robot adheres to the beating heart using vacuum suction at its anchor points, with a central injector head that operates within the triangular workspace formed by the anchors, and is actuated by cables for multipoint direct gene therapy injections. Minimizing cable tensions can reduce forces on the heart at the anchor points while supporting rapid delivery of accurate injections and minimizing procedure time, risk of damage to the robot, and strain to the heart. However, cable tensions must be sufficient to hold the injector head's position as the heart moves and to prevent excessive cable slack. We pose a linear optimization problem to minimize the sum of cable tension magnitudes for HeartPrinter while ensuring the injector head is held in static equilibrium and the tensions are constrained within a feasible range. We use Karush-Kuhn-Tucker optimality conditions to derive conditional algebraic expressions for optimal cable tensions as a function of injector head position and workspace geometry, and we identify regions of injector head positions where particular combinations of cable tensions are optimally at minimum allowable tensions. The approach can rapidly solve for the minimum set of cable tensions for any robot workspace geometry and injector head position and determine whether an injection site is attainable.

Real-time vessel segmentation and reconstruction for virtual fixtures for an active handheld microneurosurgical instrument.

PURPOSE: Complications related to vascular damage such as intra-operative bleeding may be avoided during neurosurgical procedures such as petroclival meningioma surgery. To address this and improve the patient's safety, we designed a real-time blood vessel avoidance strategy that enables operation on deformable tissue during petroclival meningioma surgery using Micron, a handheld surgical robotic tool. METHODS: We integrated real-time intra-operative blood vessel segmentation of brain vasculature using deep learning, with a 3D reconstruction algorithm to obtain the vessel point cloud in real time. We then implemented a virtual-fixture-based strategy that prevented Micron's tooltip from entering a forbidden region around the vessel, thus avoiding damage to it. RESULTS: We achieved a median Dice similarity coefficient of 0.97, 0.86, 0.87 and 0.77 on datasets of phantom blood vessels, petrosal vein, internal carotid artery and superficial vessels, respectively. We conducted trials with deformable clay vessel phantoms, keeping the forbidden region 400 [Formula: see text]m outside and 400 [Formula: see text]m inside the vessel. Micron's tip entered the forbidden region with a median penetration of just 8.84 [Formula: see text]m and 9.63 [Formula: see text]m, compared to 148.74 [Formula: see text]m and 117.17 [Formula: see text]m without our strategy, for the former and latter trials, respectively. CONCLUSION: Real-time control of Micron was achieved at 33.3 fps. We achieved improvements in real-time segmentation of brain vasculature from intra-operative images and showed that our approach works even on non-stationary vessel phantoms. The results suggest that by enabling precise, real-time control, we are one step closer to using Micron in real neurosurgical procedures.

Robotic Assistance for Intraocular Microsurgery: Challenges and Perspectives.

Intraocular surgery, one of the most challenging discipline of microsurgery, requires sensory and motor skills at the limits of human physiological capabilities combined with tremendously difficult requirements for accuracy and steadiness. Nowadays, robotics combined with advanced imaging has opened conspicuous and significant directions in advancing the field of intraocular microsurgery. Having patient treatment with greater safety and efficiency as the final goal, similar to other medical applications, robotics has a real potential to fundamentally change microsurgery by combining human strengths with computer and sensor-based technology in an information-driven environment. Still in its early stages, robotic assistance for intraocular microsurgery has been accepted with precaution in the operating room and successfully tested in a limited number of clinical trials. However, owing to its demonstrated capabilities including hand tremor reduction, haptic feedback, steadiness, enhanced dexterity, micrometer-scale accuracy, and others, microsurgery robotics has evolved as a very promising trend in advancing retinal surgery. This paper will analyze the advances in retinal robotic microsurgery, its current drawbacks and limitations, as well as the possible new directions to expand retinal microsurgery to techniques currently beyond human boundaries or infeasible without robotics.

Monocular Vision-Based Retinal Membrane Peeling With a Handheld Robot.

Retinal membrane peeling requires delicate manipulation. The presence of the surgeon's physiological tremor, the high variability and often low quality of the ophthalmic image, and excessive forces make the tasks more challenging. Preventing unintended movement caused by tremor and unintentional forces can reduce membrane injury. With the use of an actively stabilized handheld robot, we employ a monocular camera-based surface reconstruction method to estimate the retinal plane and we propose the use of a virtual fixture with the application of a hard stop and motion scaling to improve control of the tool tip during delaminating in a laboratory simulation of retinal membrane peeling. A hard stop helps to limit downward force exerted on the surface. Motion scaling also improves the user's control of contact force when delaminating. We demonstrate a reduction of maximum force and maximum surface-penetration distance from the estimated retinal plane using the proposed technique.

Introducer Design Concepts for an Epicardial Parallel Wire Robot.

BACKGROUND: Cardiac gene therapies lack effective delivery methods to the myocardium. While direct injection has demonstrated success over a small region, homogenous gene expression requires many injections over a large area. To address this need, we developed a minimally invasive flexible parallel wire robot for epicardial interventions. To accurately deploy it onto the beating heart, an introducer mechanism is required. METHODS: Two mechanisms are presented. Assessment of the robot's positioning, procedure time, and pericardium insertion forces are performed on an artificial beating heart. RESULTS: Successful positioning was demonstrated. The mean procedure time was 230 ± 7 seconds for mechanism I and 259 ± 4 seconds for mechanism II. The mean pericardium insertion force was 2.2 ± 0.4 N anteriorly and 3.1 ± 0.4 N posteriorly. CONCLUSION: Introducer mechanisms demonstrate feasibility in facilitating the robot's deployment on the epicardium. Pericardium insertion forces and procedure times are consistent and reasonable.

A Soft Resistive Sensor with a Semicircular Cross-Sectional Channel for Soft Cardiac Catheter Ablation.

The field of soft robotics has attracted the interest of the medical community due to the ability of soft elastic materials to traverse the abnormal environment of the human body. However, sensing in soft robotics has been challenging due to the sensitivity of soft sensors to various loading conditions and the nonlinear signal responses that can arise under extreme loads. Ideally, soft sensors should provide a linear response under a specific loading condition and provide a different response for other loading directions. With these specifications in mind, our team created a soft elastomeric sensor designed to provide force feedback during cardiac catheter ablation surgery. Analytical and computational methods were explored to define a relationship between resistance and applied force for a semicircular, liquid metal filled channel in the soft elastomeric sensor. Pouillet's Law is utilized to calculate the resistance based on the change in cross-sectional area resulting from various applied pressures. FEA simulations were created to simulate the deformation of the sensor under various loads. To confirm the validity of these simulations, the elastomer was modeled as a neo-Hookean material and the liquid metal was modeled as an incompressible fluid with negligible shear modulus under uniaxial compression. Results show a linearly proportional relationship between the resistance of the sensor and the application of a uniaxial force. Altering the direction of applied force results in a quadratic relationship between total resistance and the magnitude of force.

Coronary vessel detection methods for organ-mounted robots.

BACKGROUND: HeartLander is a tethered robot walker that utilizes suction to adhere to the beating heart. HeartLander can be used for minimally invasive administration of cardiac medications or ablation of tissue. In order to administer injections safely, HeartLander must avoid coronary vasculature. METHODS: Doppler ultrasound signals were recorded using a custom-made cardiac phantom and used to classify different coronary vessel properties. The classification was performed by two machine learning algorithms, the support vector machines and a deep convolutional neural network. These algorithms were then validated in animal trials. RESULTS: Accuracy of identifying vessels above turbulent flow reached greater than 92% in phantom trials and greater than 98% in animal trials. CONCLUSIONS: Through the use of two machine learning algorithms, HeartLander has shown the ability to identify different sized vasculature proximally above turbulent flow. These results indicate that it is feasible to use Doppler ultrasound to identify and avoid coronary vasculature during cardiac interventions using HeartLander.

A Deep Learning Model for Automated Classification of Intraoperative Continuous EMG.

OBJECTIVE: Intraoperative neurophysiological monitoring (IONM) is the use of electrophysiological methods during certain high-risk surgeries to assess the functional integrity of nerves in real time and alert the surgeon to prevent damage. However, the efficiency of IONM in current practice is limited by latency of verbal communications, inter-rater variability, and the subjective manner in which electrophysiological signals are described. METHODS: In an attempt to address these shortcomings, we investigate automated classification of free-running electromyogram (EMG) waveforms during IONM. We propose a hybrid model with a convolutional neural network (CNN) component and a long short-term memory (LSTM) component to better capture complicated EMG patterns under conditions of both electrical noise and movement artifacts. Moreover, a preprocessing pipeline based on data normalization is used to handle classification of data from multiple subjects. To investigate model robustness, we also analyze models under different methods for processing of artifacts. RESULTS: Compared with several benchmark modeling methods, CNN-LSTM performs best in classification, achieving accuracy of 89.54% and sensitivity of 94.23% in cross-patient evaluation. CONCLUSION: The CNN-LSTM model shows promise for automated classification of continuous EMG in IONM. SIGNIFICANCE: This technique has potential to improve surgical safety by reducing cognitive load and inter-rater variability.

Soft Miniaturized Actuation and Sensing Units for Dynamic Force Control of Cardiac Ablation Catheters.

Recently, there has been active research in finding robotized solutions for the treatment of atrial fibrillation (AF) by augmenting catheter systems through the integration of force sensors at the tip. However, limited research has been aimed at providing automatic force control by also integrating actuation of the catheter tip, which can significantly enhance safety in such procedures. This article solves the demanding challenge of miniaturizing both actuation and sensing for integration into flexible catheters. Fabrication strategies are presented for a series of novel soft thick-walled cylindrical actuators, with embedded sensing using eutectic gallium-indium. The functional catheter tips have a diameter in the range of 2.6-3.6 mm and can both generate and detect forces in the range of < 0.4 N, with a bandwidth of 1-2 Hz. The deformation modeling of thick-walled cylinders with fiber reinforcement is presented in the article. An experimental setup developed for static and dynamic characterization of these units is presented. The prototyped units were validated with respect to the design specifications. The preliminary force control results indicate that these units can be used in tracking and control of contact force, which has the potential to make AF procedures much safer and more accurate.

EyeSAM: graph-based localization and mapping of retinal vasculature during intraocular microsurgery.

PURPOSE: Robot-assisted intraocular microsurgery can improve performance by aiding the surgeon in operating on delicate micron-scale anatomical structures of the eye. In order to account for the eyeball motion that is typical in intraocular surgery, there is a need for fast and accurate algorithms that map the retinal vasculature and localize the retina with respect to the microscope. METHODS: This work extends our previous work by a graph-based SLAM formulation using a sparse incremental smoothing and mapping (iSAM) algorithm. RESULTS: The resulting technique, "EyeSAM," performs SLAM for intraoperative vitreoretinal surgical use while avoiding spurious duplication of structures as with the previous simpler technique. The technique also yields reduction in average pixel error in the camera motion estimation. CONCLUSIONS: This work provides techniques to improve intraoperative tracking of retinal vasculature by handling loop closures and achieving increased robustness to quick shaky motions and drift due to uncertainties in the motion estimation.

Organ-mounted robot localization via function approximation.

BACKGROUND: Organ-mounted robots adhere to the surface of a mobile organ as a platform for minimally invasive interventions, providing passive compensation of physiological motion. This approach is beneficial during surgery on the beating heart. Accurate localization in such applications requires accounting for the heartbeat and respiratory motion. Previous work has described methods for modeling quasi-periodic motion of a point and registering to a static preoperative map. The existing techniques, while accurate, require several respiratory cycles to converge. METHODS: This paper presents a general localization technique for this application, involving function approximation using radial basis function (RBF) interpolation. RESULTS: In an experiment in the porcine model in vivo, the technique yields mean localization accuracy of 1.25 mm with a 95% confidence interval of 0.22 mm. CONCLUSIONS: The RBF approximation provides accurate estimates of robot location instantaneously.

Miniaturized Robotic End-Effector with Piezoelectric Actuation and Fiber Optic Sensing for Minimally Invasive Cardiac Procedures.

Each year 35,000 cardiac ablation procedures are performed to treat atrial fibrillation through the use of catheter systems. The success rate of this treatment is highly dependent on the force which the catheter applies on the heart wall. If the magnitude of the applied force is much higher than a certain threshold the tissue perforates, whereas if the force is lower than this threshold the lesion size may be too large and is inconsistent. Furthermore, studies have shown large variability in the applied force from trained physicians during treatment, suggesting that physicians are unable to manually regulate the levels of the force at the site of treatment. Current catheter systems do not provide the physicians with active means for contact force control and are only at most aided by visual feedback of the forces measured in situ. This paper discusses a novel design of a robotic end-effector that integrates mechanisms of sensing and actively controlling of the applied forces into a miniaturized compact form. The required specifications for design and integration were derived from the current application under investigation. An off-the-shelf miniature piezoelectric motor was chosen for actuation, and a force sensing solution was developed to meet the specifications. Experimental characterization of the actuator and the force sensor within the integrated setup show compliance with the specifications and pave the way for future experimentation where closed-loop control of the system can be implemented according to the contact force control strategies for the application.

Toward Improving Safety in Neurosurgery with an Active Handheld Instrument.

Microsurgical procedures, such as petroclival meningioma resection, require careful surgical actions in order to remove tumor tissue, while avoiding brain and vessel damaging. Such procedures are currently performed under microscope magnification. Robotic tools are emerging in order to filter surgeons' unintended movements and prevent tools from entering forbidden regions such as vascular structures. The present work investigates the use of a handheld robotic tool (Micron) to automate vessel avoidance in microsurgery. In particular, we focused on vessel segmentation, implementing a deep-learning-based segmentation strategy in microscopy images, and its integration with a feature-based passive 3D reconstruction algorithm to obtain accurate and robust vessel position. We then implemented a virtual-fixture-based strategy to control the handheld robotic tool and perform vessel avoidance. Clay vascular phantoms, lying on a background obtained from microscopy images recorded during petroclival meningioma surgery, were used for testing the segmentation and control algorithms. When testing the segmentation algorithm on 100 different phantom images, a median Dice similarity coefficient equal to 0.96 was achieved. A set of 25 Micron trials of 80 s in duration, each involving the interaction of Micron with a different vascular phantom, were recorded, with a safety distance equal to 2 mm, which was comparable to the median vessel diameter. Micron's tip entered the forbidden region 24% of the time when the control algorithm was active. However, the median penetration depth was 16.9 µm, which was two orders of magnitude lower than median vessel diameter. Results suggest the system can assist surgeons in performing safe vessel avoidance during neurosurgical procedures.

Beating-heart registration for organ-mounted robots.

BACKGROUND: Organ-mounted robots address the problem of beating-heart surgery by adhering to the heart, passively providing a platform that approaches zero relative motion. Because of the quasi-periodic deformation of the heart due to heartbeat and respiration, registration must address not only spatial registration but also temporal registration. METHODS: Motion data were collected in the porcine model in vivo (N = 6). Fourier series models of heart motion were developed. By comparing registrations generated using an iterative closest-point approach at different phases of respiration, the phase corresponding to minimum registration distance is identified. RESULTS: The spatiotemporal registration technique presented here reduces registration error by an average of 4.2 mm over the 6 trials, in comparison with a more simplistic static registration that merely averages out the physiological motion. CONCLUSIONS: An empirical metric for spatiotemporal registration of organ-mounted robots is defined and demonstrated using data from animal models in vivo.

EyeSLAM: Real-time simultaneous localization and mapping of retinal vessels during intraocular microsurgery.

BACKGROUND: Fast and accurate mapping and localization of the retinal vasculature is critical to increasing the effectiveness and clinical utility of robot-assisted intraocular microsurgery such as laser photocoagulation and retinal vessel cannulation. METHODS: The proposed EyeSLAM algorithm delivers 30 Hz real-time simultaneous localization and mapping of the human retina and vasculature during intraocular surgery, combining fast vessel detection with 2D scan-matching techniques to build and localize a probabilistic map of the vasculature. RESULTS: In the harsh imaging environment of retinal surgery with high magnification, quick shaky motions, textureless retina background, variable lighting and tool occlusion, EyeSLAM can map 75% of the vessels within two seconds of initialization and localize the retina in real time with a root mean squared (RMS) error of under 5.0 pixels (translation) and 1° (rotation). CONCLUSIONS: EyeSLAM robustly provides retinal maps and registration that enable intelligent surgical micromanipulators to aid surgeons in simulated retinal vessel tracing and photocoagulation tasks.

Techniques for robot-aided intraocular surgery using monocular vision.

This paper presents techniques for robot-aided intraocular surgery using monocular vision in order to overcome erroneous stereo reconstruction in an intact eye. We propose a new retinal surface estimation method based on a structured-light approach. A handheld robot known as the Micron enables automatic scanning of a laser probe, creating projected beam patterns on the retinal surface. Geometric analysis of the patterns then allows planar reconstruction of the surface. To realize automated surgery in an intact eye, monocular hybrid visual servoing is accomplished through a scheme that incorporates surface reconstruction and partitioned visual servoing. We investigate the sensitivity of the estimation method according to relevant parameters and also evaluate its performance in both dry and wet conditions. The approach is validated through experiments for automated laser photocoagulation in a realistic eye phantom in vitro. Finally, we present the first demonstration of automated intraocular laser surgery in porcine eyes ex vivo.

Tip Design for Safety of Steerable Needles for Robot-Controlled Brain Insertion.

BACKGROUND: Current practice in neurosurgical needle insertion is limited by the straight trajectories inherent with rigid probes. One technique allowing curvilinear trajectories involves flexible bevel-tipped needles, which bend during insertion due to their asymmetry. In the brain, safety will require avoidance of the sharp tips often used in laboratory studies, in favor of a more rounded profile. Steering performance, on the other hand, requires maximal asymmetry. Design of safe bevel-tipped brain needles thus involves management of this tradeoff by adjusting needle gauge, bevel angle, and fillet (or tip) radius to arrive at a design that is suitably asymmetrical while producing strain, strain rate, and stress below the levels that would damage brain tissue. METHODS: Designs with a variety of values of needle radius, bevel angle, and fillet radius were evaluated in finite-element simulations of simultaneous insertion and rotation. Brain tissue was modeled as a hyperelastic, linear viscoelastic material. Based on the literature available, safety thresholds of 0.19 strain, 10 s-1 strain rate, and 120 kPa stress were used. Safe values of needle radius, bevel angle, and fillet radius were selected, along with an appropriate velocity envelope for safe operation. The resulting needle was fabricated and compared with a Sedan side-cutting brain biopsy needle in a study in the porcine model in vivo (N=3). RESULTS: The prototype needle selected was 1.66 mm in diameter, with bevel angle of 10° and fillet radius of 0.25 mm. Upon examination of postoperative CT and histological images, no differences in tissue trauma or hemorrhage were noted between the prototype needle and the Sedan needle. CONCLUSIONS: The study indicates a general design technique for safe bevel-tipped brain needles based on comparison with relevant damage thresholds for strain, strain rate, and stress. The full potential of the technique awaits the determination of more exact safety thresholds.

Toward Monocular Camera-Guided Retinal Vein Cannulation with an Actively Stabilized Handheld Robot.

In this paper we describe work towards retinal vessel cannulation using an actively stabilized handheld robot, guided by monocular vision. We employ a previously developed monocular camera based surface reconstruction method using automated laser beam scanning over the retina. We use the reconstructed plane to find a coordinate transform between the 2D image plane coordinate system and the global 3D frame. Within a hemispherical region around the target, we use motion scaling for higher precision. The contribution of this work is the homography matrix estimation using monocular vision and application of the previously developed laser surface reconstruction to Micron guided vein cannulation. Experiments are conducted in a wet eye phantom to show the higher accuracy of the surface reconstruction as compared to standard stereo reconstruction. Further, experiments to show the increased surgical accuracy due to motion scaling are also carried out.

Techniques for epicardial mapping and ablation with a miniature robotic walker.

BACKGROUND: Present treatments for ventricular tachycardia have significant drawbacks. To ameliorate these drawbacks, it may be advantageous to employ an epicardial robotic walker that performs mapping and ablation with precise control of needle insertion depth. This paper examines the feasibility of such a system. METHODS: This paper describes techniques for epicardial mapping and depth-controlled ablation with the robotic walker. The mapping technique developed for the current form of the system uses a single equivalent moving dipole model combined with the navigation capability of the walker. The intervention technique provides saline-enhanced radio frequency ablation, with sensing of needle penetration depth. The mapping technique was demonstrated in an artificial heart model with a simulated arrhythmia focus, followed by preliminary testing in the porcine model in vivo. The ablation technique was demonstrated in an artificial tissue model, and then in chicken breast tissue ex vivo. RESULTS: The walker located targets to within 2 mm using the SEMDM technique. No epicardial damage was found subsequent to the porcine trial in vivo. Needle insertion for ablation was controlled to within 2 mm of the target depth. Lesion size was repeatable, with diameter varying consistently in proportion to volume of saline injected. CONCLUSIONS: The experiments demonstrated the general feasibility of the techniques for mapping and depth-controlled ablation with the robotic walker.