sEMG-based Motion Recognition for Robotic Surgery Training - A Preliminary Study.

Robotic surgery represents a major breakthrough in the evolution of medical technology. Accordingly, efficient skill training and assessment methods should be developed to meet the surgeon's need of acquiring such robotic skills over a relatively short learning curve in a safe manner. Different from conventional training and assessment methods, we aim to explore the surface electromyography (sEMG) signal during the training process in order to obtain semantic and interpretable information to help the trainee better understand and improve his/her training performance. As a preliminary study, motion primitive recognition based on sEMG signal is studied in this work. Using machine learning (ML) technique, it is shown that the sEMG-based motion recognition method is feasible and promising for hand motions along 3 Cartesian axes in the virtual reality (VR) environment of a commercial robotic surgery training platform, which will hence serve as the basis for new robotic surgical skill assessment criterion and training guidance based on muscle activity information. Considering certain motion patterns were less accurately recognized than others, more data collection and deep learning-based analysis will be carried out to further improve the recognition accuracy in future research.

Contactless surface registration of featureless anatomy using structured light camera: application to fibula navigation in mandible reconstruction.

PURPOSE: Mandibular reconstruction using fibula free flap is a challenging surgical procedure. To assist osteotomies, computer-assisted surgery (CAS) can be used. Nevertheless, precise registration is required and often necessitates anchored markers that disturb the patient and clinical flow. This work proposes a new contactless surface-based method adapted to featureless anatomies such as fibula to achieve a fast, precise, and reproducible registration. METHODS: Preoperatively, a CT-scan of the patient is realized and osteotomies are virtually planned. During surgery, a structured light camera digitizes the fibula. The obtained intraoperative point cloud is coarsely registered with the preoperative model using 3 points defined in the CT-scan and located on the patient's bone with a laser beam. Then, a fine registration is performed using an ICP algorithm. The registration accuracy was evaluated comparing the position of points engraved in a 3D-printed fibula with their position in the registered model and evaluating resulting osteotomies. Accuracy and execution time were compared to a conventional stylus-based registration method. The work was validated in vivo. RESULTS: The experiment performed on a 3D-printed model showed that execution time is equivalent to surface-based registration using a stylus, with a better accuracy (mean TRE of 0.9 mm vs 1.3 mm using stylus) and guarantee good osteotomies. The preliminary in vivo study proved the feasibility of the method. CONCLUSION: The proposed contactless surface-based registration method using structured light camera gave promising results in terms of accuracy and execution speed and should be useful to implement CAS for mandibular reconstruction.

Miniaturized Endoscopic 2D US Transducer for Volumetric Ultrasound Imaging of the Auditory System.

OBJECTIVE: In this paper, we focus on the carrying out and validation of minimally invasive three-dimensional (3D) ultrasound (US) imaging of the auditory system, which is based on a new miniaturized endoscopic 2D US transducer. METHODS: This unique probe consists of a 18 MHz 24 elements curved array transducer with a distal diameter of 4 mm so it can be inserted into the external auditory canal. Typical acquisition is achieved by rotating such a transducer around its own axis using a robotic platform. Reconstruction of a US volume from the set of acquired B-scans during the rotation is then performed using scan-conversion. The accuracy of the reconstruction procedure is evaluated using a dedicated phantom that includes a set of wires as reference geometry. RESULTS: Twelve acquisitions obtained from different probe poses are compared to a micro-computed tomographic model of the phantom, leading to a maximum error of 0.20 mm. Additionally, acquisitions with a cadaveric head highlight the clinical applicability of this set up. Structures of the auditory system such as the ossicles and the round window can be identified from the obtained 3D volumes. CONCLUSION: These results confirm that our technique enables the accurate imaging of the middle and inner ears without having to deteriorate the surrounding bone. SIGNIFICANCE: Since US is a real-time, wide available and non-ionizing imaging modality, our acquisition setup could facilitate the minimally invasive diagnosis and surgical navigation for otology in a fast, cost-effective and safe way.

Fibular registration using surface matching in navigation-guided osteotomies: a proof of concept study on 3D-printed models.

PURPOSE: Fibula free flap is currently used in mandibular reconstruction. The main difficulties involved in this surgery concern mandible shaping and therefore, osteotomy positioning on the fibula. The use of navigation could help in osteotomy positioning, but accurate registration is required. We assess a surface-matching method for fibula registration that relies on an iterative closest point (ICP) algorithm. Since the fibula shape is landmark free, a robust registration initialization approach is used to avoid non-optimal local minimums in the ICP. METHODS: Bone surface-matching registration was evaluated on a 3D printed fibula and compared to its virtual reference model. The registration initialization relied on 3 initialization points placed on the surgically exposed area, geometrically remote from the fibular distal extremity. The bone surface was digitized, and the obtained point clouds were registered to the virtual reference model. The position of 3 assessment points engraved on the 3D printed fibula was then compared to that of the equivalent points on the virtual model. RESULTS: The registration procedure was performed 24 times by an expert surgeon. Seventy-two target registration errors (TRE) were computed, corresponding to the distance between the paired assessment points. Most TRE (86.1%) were less than 1 mm, with a maximum of 1.552 mm. The overall mean value was 0.759 ± 0.302 mm. CONCLUSION: This study illustrates a surface-matching approach for fibula registration, with an initialization method based on points remote from the fibula distal extremity. This registration technique gave promising results and should be considered as a valid registration method for straight bones like the fibula. These findings indicate that navigation can be used for fibula flap shaping for mandibular reconstruction, with a noninvasive and accurate registration method.

Feasibility of Cochlea High-frequency Ultrasound and Microcomputed Tomography Registration for Cochlear Computer-assisted Surgery: A Testbed.

INTRODUCTION: There remains no standard imaging method that allows computer-assisted surgery of the cochlea in real time. However, recent evidence suggests that high-frequency ultrasound (HFUS) could permit real-time visualization of cochlear architecture. Registration with an imaging modality that suffers neither attenuation nor conical deformation could reveal useful anatomical landmarks to surgeons. Our study aimed to address the feasibility of an automated three-dimensional (3D) HFUS/microCT registration, and to evaluate the identification of cochlear structures using 2D/3D HFUS and microCT. METHODS: MicroCT, and 2D/3D 40 MHz US in B-mode were performed on ex vivo guinea pig cochlea. An automatic rigid registration algorithm was applied to segmented 3D images. This automatic registration was then compared to a reference method using manual annotated landmarks placed by two senior otologists. Inter- and intrarater reliabilities were evaluated using intraclass correlation coefficient (ICC) and the mean registration error was calculated. RESULTS: 3D HFUS/microCT automatic registration was successful. Excellent levels of concordance were achieved with regards intra-rater reliability for both raters with micro-CT and US images (ICC ranging from 0.98 to 1, p < 0.001) and with regards inter-rater reliability (ICC ranging from 0.99 to 1, p < 0.001). The mean HFUS/microCT automated RE for both observers was 0.17 ±â€Š0.03 mm [0.10-0.25]. Identification of the basilar membrane, modiolus, scala tympani, and scala vestibuli was possible with 2D/3D HFUS and micro-CT. CONCLUSIONS: HFUS/microCT image registration is feasible. 2D/3D HFUS and microCT allow the visualization of cochlear structures. Many potential clinical applications are conceivable.

Towards 3D Ultrasound Guided Needle Steering Robust to Uncertainties, Noise, and Tissue Heterogeneity.

This paper presents a new solution for 3D steering of flexible needles guided by 3D B-mode ultrasound imaging. It aims to realize a robust steering, by accounting for uncertainties, noise and tissue heterogeneities, while limiting tissue-related disturbances. The proposed solution features interconnected state observer, automatic needle tip segmentation and path planning algorithms. Measurement quality, state uncertainties and tissue heterogeneity are considered for robust needle steering with helical paths of variable curvature. Fast replanning allows for adaptability to unexpected disturbances. An experimental validation has been done through 62 insertions of 24 Gauge bevel-tip nitinol needles in various tissue. Results are promising, characterized by mean targeting errors of less than 1 mm in homogeneous phantoms, 1.5 ± 0.9 mm in heterogeneous phantoms and 1.7 ± 0.8 mm in ex-vivo tissue. This new approach is a step towards a precise and robust patient-specific gesture.

HFUS Imaging of the Cochlea: A Feasibility Study for Anatomical Identification by Registration with MicroCT.

Cochlear implantation consists in electrically stimulating the auditory nerve by inserting an electrode array inside the cochlea, a bony structure of the inner ear. In the absence of any visual feedback, the insertion results in many cases of damages of the internal structures. This paper presents a feasibility study on intraoperative imaging and identification of cochlear structures with high-frequency ultrasound (HFUS). 6 ex-vivo guinea pig cochleae were subjected to both US and microcomputed tomography (µCT) we respectively referred as intraoperative and preoperative modalities. For each sample, registration based on simulating US from the scanner was performed to allow a precise matching between the visible structures. According to two otologists, the procedure led to a target registration error of 0.32 mm ± 0.05. Thanks to referring to a better preoperative anatomical representation, we were able to intraoperatively identify the modiolus, both scalae vestibuli and tympani and deduce the location of the basilar membrane, all of which is of great interest for cochlear implantation. Our main objective is to extend this procedure to the human case and thus provide a new tool for inner ear surgery.

Sparse-view CBCT reconstruction via weighted Schatten p-norm minimization.

A novel iterative algorithm is proposed for sparse-view cone beam computed tomography (CBCT) reconstruction based on the weighted Schatten p-norm minimization (WSNM). By using the half quadratic splitting, the sparse-view CBCT reconstruction task is decomposed into two sub-problems that can be solved through alternating iteration: simple reconstruction and image denoising. The WSNM that fits well with the low-rank hypothesis of CBCT data is introduced to improve the denoising sub-problem as a regularization term. The experimental results based on the digital brain phantom and clinical CT data indicated the advantages of the proposed algorithm in both structural information preservation and artifacts suppression, which performs better than the classical algorithms in quantitative and qualitative evaluations.

Transrectal ultrasound image-based real-time augmented reality guidance in robot-assisted laparoscopic rectal surgery: a proof-of-concept study.

PURPOSE: Surgical treatments for low-rectal cancer require careful considerations due to the low location of cancer in rectums. Successful surgical outcomes highly depend on surgeons' ability to determine clear distal margins of rectal tumors. This is a challenge for surgeons in robot-assisted laparoscopic surgery, since tumors are often concealed in rectums and robotic surgical instruments do not provide tactile feedback for tissue diagnosis in real time. This paper presents the development and evaluation of an intraoperative ultrasound-based augmented reality framework for surgical guidance in robot-assisted rectal surgery. METHODS: Framework implementation consists in calibrating the transrectal ultrasound (TRUS) and the endoscopic camera (hand-eye calibration), generating a virtual model and registering it to the endoscopic image via optical tracking, and displaying the augmented view on a head-mounted display. An experimental validation setup is designed to evaluate the framework. RESULTS: The evaluation process yields a mean error of 0.9 mm for the TRUS calibration, a maximum error of 0.51 mm for the hand-eye calibration of endoscopic cameras, and a maximum RMS error of 0.8 mm for the whole framework. In the experiment with a rectum phantom, our framework guides the surgeon to accurately localize the simulated tumor and the distal resection margin. CONCLUSIONS: This framework is developed with our clinical partner, based on actual clinical conditions. The experimental protocol and the high level of accuracy show the feasibility of seamlessly integrating this framework within the surgical workflow.

Fast And Simple Automatic 3D Ultrasound Probe Calibration Based On 3D Printed Phantom And An Untracked Marker.

Tracking the pose of an ultrasound (US) probe is essential for an intraoperative US-based navigation system. The tracking requires mounting a marker on the US probe and calibrating the probe. The goal of the US probe calibration is to determine the rigid transformation between the coordinate system (CS) of the image and the CS of the marker mounted on the probe. We present a fast and automatic calibration method based on a 3D printed phantom and an untracked marker for three-dimensional (3D) US probe calibration. To simplify the conventional calibration procedures using and tracking at least two markers, we used only one marker and did not track it in the whole calibration process. Our automatic calibration method is fast, simple and does not require any experience from the user. The performance of our calibration method was evaluated by point reconstruction tests. The root mean square (RMS) of the point reconstruction errors was 1.39 mm.

Automatic Robotic Steering of Flexible Needles from 3D Ultrasound Images in Phantoms and Ex Vivo Biological Tissue.

Robotic control of needle bending aims at increasing the precision of percutaneous procedures. Ultrasound feedback is preferable for its clinical ease of use, cost and compactness but raises needle detection issues. In this paper, we propose a complete system dedicated to robotized guidance of a flexible needle under 3D ultrasound imaging. This system includes a medical robot dedicated to transperineal needle positioning and insertion, a rapid path planning for needle steering using bevel-tip needle natural curvature in tissue, and an ultrasound-based automatic needle detection algorithm. Since ultrasound-based automatic needle steering is often made difficult by the needle localization in biological tissue, we quantify the benefit of using flexible echogenic needles for robotized guidance under 3D ultrasound. The "echogenic" term refers to the etching of microstructures on the needle shaft. We prove that these structures improve needle visibility and detection robustness in ultrasound images. We finally present promising results when reaching targets using needle steering. The experiments were conducted with various needles in different media (synthetic phantoms and ex vivo biological tissue). For instance, with nitinol needles the mean accuracy is 1.2 mm (respectively 3.8 mm) in phantoms (resp. biological tissue).

Evaluation of contactless human-machine interface for robotic surgical training.

PURPOSE: Teleoperated robotic systems are nowadays routinely used for specific interventions. Benefits of robotic training courses have already been acknowledged by the community since manipulation of such systems requires dedicated training. However, robotic surgical simulators remain expensive and require a dedicated human-machine interface. METHODS: We present a low-cost contactless optical sensor, the Leap Motion, as a novel control device to manipulate the RAVEN-II robot. We compare peg manipulations during a training task with a contact-based device, the electro-mechanical Sigma.7. We perform two complementary analyses to quantitatively assess the performance of each control method: a metric-based comparison and a novel unsupervised spatiotemporal trajectory clustering. RESULTS: We show that contactless control does not offer as good manipulability as the contact-based. Where part of the metric-based evaluation presents the mechanical control better than the contactless one, the unsupervised spatiotemporal trajectory clustering from the surgical tool motions highlights specific signature inferred by the human-machine interfaces. CONCLUSIONS: Even if the current implementation of contactless control does not overtake manipulation with high-standard mechanical interface, we demonstrate that using the optical sensor complete control of the surgical instruments is feasible. The proposed method allows fine tracking of the trainee's hands in order to execute dexterous laparoscopic training gestures. This work is promising for development of future human-machine interfaces dedicated to robotic surgical training systems.

Unsupervised Trajectory Segmentation for Surgical Gesture Recognition in Robotic Training.

Dexterity and procedural knowledge are two critical skills that surgeons need to master to perform accurate and safe surgical interventions. However, current training systems do not allow us to provide an in-depth analysis of surgical gestures to precisely assess these skills. Our objective is to develop a method for the automatic and quantitative assessment of surgical gestures. To reach this goal, we propose a new unsupervised algorithm that can automatically segment kinematic data from robotic training sessions. Without relying on any prior information or model, this algorithm detects critical points in the kinematic data that define relevant spatio-temporal segments. Based on the association of these segments, we obtain an accurate recognition of the gestures involved in the surgical training task. We, then, perform an advanced analysis and assess our algorithm using datasets recorded during real expert training sessions. After comparing our approach with the manual annotations of the surgical gestures, we observe 97.4% accuracy for the learning purpose and an average matching score of 81.9% for the fully automated gesture recognition process. Our results show that trainees workflow can be followed and surgical gestures may be automatically evaluated according to an expert database. This approach tends toward improving training efficiency by minimizing the learning curve.

Motion prediction using dual Kalman filter for robust beating heart tracking.

A novel prediction method for robust beating heart tracking is proposed. The dual time-varying Fourier series is used to model the heart motion. The frequency parameters and Fourier coefficients in the model are estimated respectively by using a dual Kalman filter scheme. The instantaneous frequencies of breathing and heartbeat motion are measured online from the 3D trajectory of the point of interest using an orthogonal decomposition algorithm. The proposed method is evaluated based on both the simulated signals and the real motion signals, which are measured from the videos recorded using the da Vinci surgical system.

Enhanced position-force tracking of time-delayed teleoperation for robotic-assisted surgery.

The advance of minimally invasive surgery has been empowered by new medical/surgical robotic systems towards achieving less invasiveness, smaller or even no scars. Wireless communication possesses great potential to be utilized in miniaturized surgical robotic system. However time delay is inevitably introduced in the control loop which causes stability issues for robotic-assisted surgeries. Wave variable based teleoperation structure provides stable force reflecting teleoperation performance but with both position and force tracking performance compromised due to conservative passivity condition. Recently, we proposed a new wave variable compensated structure to improve position and force tracking performance together with energy reservoir based regulators for stability purpose. In this paper, different energy reservoir based regulators are proposed with consideration of passivity of master and slave system to avoid uncertain compensated wave variables. Experiments are designed to evaluate the performance of proposed structure in comparison with traditional wave variable structure. Quantitative analyses of the obtained results justify the efficiency of proposed method.

Synthesis of optimal electrical stimulation patterns for functional motion restoration: applied to spinal cord-injured patients.

We investigated the synthesis of electrical stimulation patterns for functional movement restoration in human paralyzed limbs. We considered the knee joint system, co-activated by the stimulated quadriceps and hamstring muscles. This synthesis is based on optimized functional electrical stimulation (FES) patterns to minimize muscular energy consumption and movement efficiency criteria. This two-part work includes a multi-scale physiological muscle model, based on Huxley's formulation. In the simulation, three synthesis strategies were investigated and compared in terms of muscular energy consumption and co-contraction levels. In the experimental validation, the synthesized FES patterns were carried out on the quadriceps-knee joint system of four complete spinal cord injured subjects. Surface stimulation was applied to all subjects, except for one FES-implanted subject who received neural stimulation. In each experimental validation, the model was adapted to the subject through a parameter identification procedure. Simulation results were successful and showed high co-contraction levels when reference trajectories were tracked. Experimental validation results were encouraging, as the desired and measured trajectories showed good agreement, with an 8.4 % rms error in a subject without substantial time-varying behavior. We updated the maximal isometric force in the model to account for time-varying behavior, which improved the average rms errors from 31.4 to 13.9 % for all subjects.

Using rotation for steerable needle detection in 3D color-Doppler ultrasound images.

This paper demonstrates a new way to detect needles in 3D color-Doppler volumes of biological tissues. It uses rotation to generate vibrations of a needle using an existing robotic brachytherapy system. The results of our detection for color-Doppler and B-Mode ultrasound are compared to a needle location reference given by robot odometry and robot ultrasound calibration. Average errors between detection and reference are 5.8 mm on needle tip for B-Mode images and 2.17 mm for color-Doppler images. These results show that color-Doppler imaging leads to more robust needle detection in noisy environment with poor needle visibility or when needle interacts with other objects.

Viscoelastic model based force control for soft tissue interaction and its application in physiological motion compensation.

Controlling the interaction between robots and living soft tissues has become an important issue as the number of robotic systems inside the operating room increases. Many researches have been done on force control to help surgeons during medical procedures, such as physiological motion compensation and tele-operation systems with haptic feedback. In order to increase the performance of such controllers, this work presents a novel force control scheme using Active Observer (AOB) based on a viscoelastic interaction model. The control scheme has shown to be stable through theoretical analysis and its performance was evaluated by in vitro experiments. In order to evaluate how the force control scheme behaves under the presence of physiological motion, experiments considering breathing and beating heart disturbances are presented. The proposed control scheme presented a stable behavior in both static and moving environment. The viscoelastic AOB presented a compensation ratio of 87% for the breathing motion and 79% for the beating heart motion.

On the use of fixed-intensity functional electrical stimulation for attenuating essential tremor.

A great proportion of essential tremor (ET) patients have not so far been able to receive functional benefits from traditional therapies. In this regard, the use of functional electrical stimulation (FES) has been proposed for reducing tremor amplitude by stimulating muscles in antiphase with respect to the trembling motion. Although some studies have reported success in terms of tremor attenuation, drawbacks still exist that prevent the method from being used in real-life applications. In this article, we explore an alternative approach: a strategy based on the hypothesis that FES-induced constant muscle contraction may provide functional benefit for tremor patients. To evaluate the proposed strategy, experiments were conducted in which stimulation was intermittently turned on and off while the subjects performed a static motor task. The results of the proposed experimental protocol indicate that tremor attenuation using this strategy is feasible, as consistent tremor attenuation levels were obtained in eight out of 10 ET patients. Nonetheless, tremor reduction was not instantaneous for all successful trials, indicating that prior training with FES may improve the overall response. Furthermore, although simpler assistive devices may potentially be designed based on this technique, some experimental difficulties still exist, which suggests that further studies are necessary.

Scaled position-force tracking for wireless teleoperation of miniaturized surgical robotic system.

Miniaturized surgical robotic system presents promising trend for reducing invasiveness during operation. However, cables used for power and communication may affect its performance. In this paper we chose Zigbee wireless communication as a means to replace communication cables for miniaturized surgical robot. Nevertheless, time delay caused by wireless communication presents a new challenge to performance and stability of the teleoperation system. We proposed a bilateral wireless teleoperation architecture taking into consideration of the effect of position-force scaling between operator and slave. Optimal position-force tracking performance is obtained and the overall system is shown to be passive with a simple condition on the scaling factors satisfied. Simulation studies verify the efficiency of the proposed scaled wireless teleoperation scheme.