sEMG-based Motion Recognition for Robotic Surgery Training - A Preliminary Study.

Robotic surgery represents a major breakthrough in the evolution of medical technology. Accordingly, efficient skill training and assessment methods should be developed to meet the surgeon's need of acquiring such robotic skills over a relatively short learning curve in a safe manner. Different from conventional training and assessment methods, we aim to explore the surface electromyography (sEMG) signal during the training process in order to obtain semantic and interpretable information to help the trainee better understand and improve his/her training performance. As a preliminary study, motion primitive recognition based on sEMG signal is studied in this work. Using machine learning (ML) technique, it is shown that the sEMG-based motion recognition method is feasible and promising for hand motions along 3 Cartesian axes in the virtual reality (VR) environment of a commercial robotic surgery training platform, which will hence serve as the basis for new robotic surgical skill assessment criterion and training guidance based on muscle activity information. Considering certain motion patterns were less accurately recognized than others, more data collection and deep learning-based analysis will be carried out to further improve the recognition accuracy in future research.

Development of a lumbar puncture virtual simulator for medical students training: A preliminary evaluation.

BACKGROUND: Lumbar puncture is an essential medical procedure whose objective is to obtain cerebrospinal fluid. Lumbar puncture is considered a complex procedure, mainly for novice residents who suffer from stress and low confidence, which may result in harm to the patient. METHODS: The LPVirSim, has been developed in four stages: i) requirements analysis through user-centred design; ii) prototyping of the virtual environment and the haptic component; iii) preliminary tests with Ph.D. students and physicians using two haptic devices (Omega.7 and Sigma.7); iv) a user study where physicians evaluated the usability and user experience. RESULTS: The LPVirSim integrates non-technical skills and the possibility of representing different patients for training. Usability increased from 61.76 to 68.75 in the preliminary tests to 71.43 in the user study. CONCLUSIONS: All the results showed good usability and demonstrated that the simulator arouses interest and realistically represents a Lumbar puncture, through the force and visual feedback.

Contactless surface registration of featureless anatomy using structured light camera: application to fibula navigation in mandible reconstruction.

PURPOSE: Mandibular reconstruction using fibula free flap is a challenging surgical procedure. To assist osteotomies, computer-assisted surgery (CAS) can be used. Nevertheless, precise registration is required and often necessitates anchored markers that disturb the patient and clinical flow. This work proposes a new contactless surface-based method adapted to featureless anatomies such as fibula to achieve a fast, precise, and reproducible registration. METHODS: Preoperatively, a CT-scan of the patient is realized and osteotomies are virtually planned. During surgery, a structured light camera digitizes the fibula. The obtained intraoperative point cloud is coarsely registered with the preoperative model using 3 points defined in the CT-scan and located on the patient's bone with a laser beam. Then, a fine registration is performed using an ICP algorithm. The registration accuracy was evaluated comparing the position of points engraved in a 3D-printed fibula with their position in the registered model and evaluating resulting osteotomies. Accuracy and execution time were compared to a conventional stylus-based registration method. The work was validated in vivo. RESULTS: The experiment performed on a 3D-printed model showed that execution time is equivalent to surface-based registration using a stylus, with a better accuracy (mean TRE of 0.9 mm vs 1.3 mm using stylus) and guarantee good osteotomies. The preliminary in vivo study proved the feasibility of the method. CONCLUSION: The proposed contactless surface-based registration method using structured light camera gave promising results in terms of accuracy and execution speed and should be useful to implement CAS for mandibular reconstruction.

Miniaturized Endoscopic 2D US Transducer for Volumetric Ultrasound Imaging of the Auditory System.

OBJECTIVE: In this paper, we focus on the carrying out and validation of minimally invasive three-dimensional (3D) ultrasound (US) imaging of the auditory system, which is based on a new miniaturized endoscopic 2D US transducer. METHODS: This unique probe consists of a 18 MHz 24 elements curved array transducer with a distal diameter of 4 mm so it can be inserted into the external auditory canal. Typical acquisition is achieved by rotating such a transducer around its own axis using a robotic platform. Reconstruction of a US volume from the set of acquired B-scans during the rotation is then performed using scan-conversion. The accuracy of the reconstruction procedure is evaluated using a dedicated phantom that includes a set of wires as reference geometry. RESULTS: Twelve acquisitions obtained from different probe poses are compared to a micro-computed tomographic model of the phantom, leading to a maximum error of 0.20 mm. Additionally, acquisitions with a cadaveric head highlight the clinical applicability of this set up. Structures of the auditory system such as the ossicles and the round window can be identified from the obtained 3D volumes. CONCLUSION: These results confirm that our technique enables the accurate imaging of the middle and inner ears without having to deteriorate the surrounding bone. SIGNIFICANCE: Since US is a real-time, wide available and non-ionizing imaging modality, our acquisition setup could facilitate the minimally invasive diagnosis and surgical navigation for otology in a fast, cost-effective and safe way.

Fibular registration using surface matching in navigation-guided osteotomies: a proof of concept study on 3D-printed models.

PURPOSE: Fibula free flap is currently used in mandibular reconstruction. The main difficulties involved in this surgery concern mandible shaping and therefore, osteotomy positioning on the fibula. The use of navigation could help in osteotomy positioning, but accurate registration is required. We assess a surface-matching method for fibula registration that relies on an iterative closest point (ICP) algorithm. Since the fibula shape is landmark free, a robust registration initialization approach is used to avoid non-optimal local minimums in the ICP. METHODS: Bone surface-matching registration was evaluated on a 3D printed fibula and compared to its virtual reference model. The registration initialization relied on 3 initialization points placed on the surgically exposed area, geometrically remote from the fibular distal extremity. The bone surface was digitized, and the obtained point clouds were registered to the virtual reference model. The position of 3 assessment points engraved on the 3D printed fibula was then compared to that of the equivalent points on the virtual model. RESULTS: The registration procedure was performed 24 times by an expert surgeon. Seventy-two target registration errors (TRE) were computed, corresponding to the distance between the paired assessment points. Most TRE (86.1%) were less than 1 mm, with a maximum of 1.552 mm. The overall mean value was 0.759 ± 0.302 mm. CONCLUSION: This study illustrates a surface-matching approach for fibula registration, with an initialization method based on points remote from the fibula distal extremity. This registration technique gave promising results and should be considered as a valid registration method for straight bones like the fibula. These findings indicate that navigation can be used for fibula flap shaping for mandibular reconstruction, with a noninvasive and accurate registration method.

Robust imaging using electrical impedance tomography: review of current tools.

Electrical impedance tomography (EIT) is a medical imaging technique with many advantages and great potential for development in the coming years. Currently, some limitations of EIT are related to the ill-posed nature of the problem. These limitations are translated on a practical level by a lack of genericity of the developed tools. In this paper, the main robust data acquisition and processing tools for EIT proposed in the scientific literature are presented. Their relevance and potential to improve the robustness of EIT are analysed, in order to conclude on the feasibility of a robust EIT tool capable of providing resistivity or difference of resistivity mapping in a wide range of applications. In particular, it is shown that certain measurement acquisition tools and algorithms, such as faulty electrode detection algorithm or particular electrode designs, can ensure the quality of the acquisition in many circumstances. Many algorithms, aiming at processing acquired data, are also described and allow to overcome certain difficulties such as an error in the knowledge of the position of the boundaries or the poor conditioning of the inverse problem. They have a strong potential to faithfully reconstruct a quality image in the presence of disturbances such as noise or boundary modelling error.

Feasibility of Cochlea High-frequency Ultrasound and Microcomputed Tomography Registration for Cochlear Computer-assisted Surgery: A Testbed.

INTRODUCTION: There remains no standard imaging method that allows computer-assisted surgery of the cochlea in real time. However, recent evidence suggests that high-frequency ultrasound (HFUS) could permit real-time visualization of cochlear architecture. Registration with an imaging modality that suffers neither attenuation nor conical deformation could reveal useful anatomical landmarks to surgeons. Our study aimed to address the feasibility of an automated three-dimensional (3D) HFUS/microCT registration, and to evaluate the identification of cochlear structures using 2D/3D HFUS and microCT. METHODS: MicroCT, and 2D/3D 40 MHz US in B-mode were performed on ex vivo guinea pig cochlea. An automatic rigid registration algorithm was applied to segmented 3D images. This automatic registration was then compared to a reference method using manual annotated landmarks placed by two senior otologists. Inter- and intrarater reliabilities were evaluated using intraclass correlation coefficient (ICC) and the mean registration error was calculated. RESULTS: 3D HFUS/microCT automatic registration was successful. Excellent levels of concordance were achieved with regards intra-rater reliability for both raters with micro-CT and US images (ICC ranging from 0.98 to 1, p < 0.001) and with regards inter-rater reliability (ICC ranging from 0.99 to 1, p < 0.001). The mean HFUS/microCT automated RE for both observers was 0.17 ±â€Š0.03 mm [0.10-0.25]. Identification of the basilar membrane, modiolus, scala tympani, and scala vestibuli was possible with 2D/3D HFUS and micro-CT. CONCLUSIONS: HFUS/microCT image registration is feasible. 2D/3D HFUS and microCT allow the visualization of cochlear structures. Many potential clinical applications are conceivable.

HFUS Imaging of the Cochlea: A Feasibility Study for Anatomical Identification by Registration with MicroCT.

Cochlear implantation consists in electrically stimulating the auditory nerve by inserting an electrode array inside the cochlea, a bony structure of the inner ear. In the absence of any visual feedback, the insertion results in many cases of damages of the internal structures. This paper presents a feasibility study on intraoperative imaging and identification of cochlear structures with high-frequency ultrasound (HFUS). 6 ex-vivo guinea pig cochleae were subjected to both US and microcomputed tomography (µCT) we respectively referred as intraoperative and preoperative modalities. For each sample, registration based on simulating US from the scanner was performed to allow a precise matching between the visible structures. According to two otologists, the procedure led to a target registration error of 0.32 mm ± 0.05. Thanks to referring to a better preoperative anatomical representation, we were able to intraoperatively identify the modiolus, both scalae vestibuli and tympani and deduce the location of the basilar membrane, all of which is of great interest for cochlear implantation. Our main objective is to extend this procedure to the human case and thus provide a new tool for inner ear surgery.

Transrectal ultrasound image-based real-time augmented reality guidance in robot-assisted laparoscopic rectal surgery: a proof-of-concept study.

PURPOSE: Surgical treatments for low-rectal cancer require careful considerations due to the low location of cancer in rectums. Successful surgical outcomes highly depend on surgeons' ability to determine clear distal margins of rectal tumors. This is a challenge for surgeons in robot-assisted laparoscopic surgery, since tumors are often concealed in rectums and robotic surgical instruments do not provide tactile feedback for tissue diagnosis in real time. This paper presents the development and evaluation of an intraoperative ultrasound-based augmented reality framework for surgical guidance in robot-assisted rectal surgery. METHODS: Framework implementation consists in calibrating the transrectal ultrasound (TRUS) and the endoscopic camera (hand-eye calibration), generating a virtual model and registering it to the endoscopic image via optical tracking, and displaying the augmented view on a head-mounted display. An experimental validation setup is designed to evaluate the framework. RESULTS: The evaluation process yields a mean error of 0.9 mm for the TRUS calibration, a maximum error of 0.51 mm for the hand-eye calibration of endoscopic cameras, and a maximum RMS error of 0.8 mm for the whole framework. In the experiment with a rectum phantom, our framework guides the surgeon to accurately localize the simulated tumor and the distal resection margin. CONCLUSIONS: This framework is developed with our clinical partner, based on actual clinical conditions. The experimental protocol and the high level of accuracy show the feasibility of seamlessly integrating this framework within the surgical workflow.

Fast And Simple Automatic 3D Ultrasound Probe Calibration Based On 3D Printed Phantom And An Untracked Marker.

Tracking the pose of an ultrasound (US) probe is essential for an intraoperative US-based navigation system. The tracking requires mounting a marker on the US probe and calibrating the probe. The goal of the US probe calibration is to determine the rigid transformation between the coordinate system (CS) of the image and the CS of the marker mounted on the probe. We present a fast and automatic calibration method based on a 3D printed phantom and an untracked marker for three-dimensional (3D) US probe calibration. To simplify the conventional calibration procedures using and tracking at least two markers, we used only one marker and did not track it in the whole calibration process. Our automatic calibration method is fast, simple and does not require any experience from the user. The performance of our calibration method was evaluated by point reconstruction tests. The root mean square (RMS) of the point reconstruction errors was 1.39 mm.

Evaluation of contactless human-machine interface for robotic surgical training.

PURPOSE: Teleoperated robotic systems are nowadays routinely used for specific interventions. Benefits of robotic training courses have already been acknowledged by the community since manipulation of such systems requires dedicated training. However, robotic surgical simulators remain expensive and require a dedicated human-machine interface. METHODS: We present a low-cost contactless optical sensor, the Leap Motion, as a novel control device to manipulate the RAVEN-II robot. We compare peg manipulations during a training task with a contact-based device, the electro-mechanical Sigma.7. We perform two complementary analyses to quantitatively assess the performance of each control method: a metric-based comparison and a novel unsupervised spatiotemporal trajectory clustering. RESULTS: We show that contactless control does not offer as good manipulability as the contact-based. Where part of the metric-based evaluation presents the mechanical control better than the contactless one, the unsupervised spatiotemporal trajectory clustering from the surgical tool motions highlights specific signature inferred by the human-machine interfaces. CONCLUSIONS: Even if the current implementation of contactless control does not overtake manipulation with high-standard mechanical interface, we demonstrate that using the optical sensor complete control of the surgical instruments is feasible. The proposed method allows fine tracking of the trainee's hands in order to execute dexterous laparoscopic training gestures. This work is promising for development of future human-machine interfaces dedicated to robotic surgical training systems.

Geometric and mechanical evaluation of 3D-printing materials for skull base anatomical education and endoscopic surgery simulation - A first step to create reliable customized simulators.

INTRODUCTION: Endoscopic skull base surgery allows minimal invasive therapy through the nostrils to treat infectious or tumorous diseases. Surgical and anatomical education in this field is limited by the lack of validated training models in terms of geometric and mechanical accuracy. We choose to evaluate several consumer-grade materials to create a patient-specific 3D-printed skull base model for anatomical learning and surgical training. METHODS: Four 3D-printed consumer-grade materials were compared to human cadaver bone: calcium sulfate hemihydrate (named Multicolor), polyamide, resin and polycarbonate. We compared the geometric accuracy, forces required to break thin walls of materials and forces required during drilling. RESULTS: All materials had an acceptable global geometric accuracy (from 0.083mm to 0.203mm of global error). Local accuracy was better in polycarbonate (0.09mm) and polyamide (0.15mm) than in Multicolor (0.90mm) and resin (0.86mm). Resin and polyamide thin walls were not broken at 200N. Forces needed to break Multicolor thin walls were 1.6-3.5 times higher than in bone. For polycarbonate, forces applied were 1.6-2.5 times higher. Polycarbonate had a mode of fracture similar to the cadaver bone. Forces applied on materials during drilling followed a normal distribution except for the polyamide which was melted. Energy spent during drilling was respectively 1.6 and 2.6 times higher on bone than on PC and Multicolor. CONCLUSION: Polycarbonate is a good substitute of human cadaver bone for skull base surgery simulation. Thanks to short lead times and reasonable production costs, patient-specific 3D printed models can be used in clinical practice for pre-operative training, improving patient safety.

Unsupervised Trajectory Segmentation for Surgical Gesture Recognition in Robotic Training.

Dexterity and procedural knowledge are two critical skills that surgeons need to master to perform accurate and safe surgical interventions. However, current training systems do not allow us to provide an in-depth analysis of surgical gestures to precisely assess these skills. Our objective is to develop a method for the automatic and quantitative assessment of surgical gestures. To reach this goal, we propose a new unsupervised algorithm that can automatically segment kinematic data from robotic training sessions. Without relying on any prior information or model, this algorithm detects critical points in the kinematic data that define relevant spatio-temporal segments. Based on the association of these segments, we obtain an accurate recognition of the gestures involved in the surgical training task. We, then, perform an advanced analysis and assess our algorithm using datasets recorded during real expert training sessions. After comparing our approach with the manual annotations of the surgical gestures, we observe 97.4% accuracy for the learning purpose and an average matching score of 81.9% for the fully automated gesture recognition process. Our results show that trainees workflow can be followed and surgical gestures may be automatically evaluated according to an expert database. This approach tends toward improving training efficiency by minimizing the learning curve.

Viscoelastic model based force control for soft tissue interaction and its application in physiological motion compensation.

Controlling the interaction between robots and living soft tissues has become an important issue as the number of robotic systems inside the operating room increases. Many researches have been done on force control to help surgeons during medical procedures, such as physiological motion compensation and tele-operation systems with haptic feedback. In order to increase the performance of such controllers, this work presents a novel force control scheme using Active Observer (AOB) based on a viscoelastic interaction model. The control scheme has shown to be stable through theoretical analysis and its performance was evaluated by in vitro experiments. In order to evaluate how the force control scheme behaves under the presence of physiological motion, experiments considering breathing and beating heart disturbances are presented. The proposed control scheme presented a stable behavior in both static and moving environment. The viscoelastic AOB presented a compensation ratio of 87% for the breathing motion and 79% for the beating heart motion.

Haptic feedback control in medical robots through fractional viscoelastic tissue model.

In this paper, we discuss the design of an adaptive control system for robot-assisted surgery with haptic feedback. Through a haptic device, the surgeon teleoperates the medical instrument in free space, fixed on a remote robot or in contact. In free space, the surgeon feels the motion of the robot. In the present paper, we evaluated the performance of the controller on viscoelastic tissue, modeled by a fractional derivative equation. In addition, we propose a novel controller using an integer formalization process that is suitable for these tissue properties. The simulation results suggested that performance, in terms of force control and telepresence, became poorer when the conventional controller, which was designed for elastic target object, was applied to the viscoelastic tissues. In contrast, the results suggested that our proposed controller maintained its performance on the viscoelastic tissues.

Real-time control and evaluation of a teleoperated miniature arm for Single Port Laparoscopy.

This paper presents the control architecture and the first performance evaluation results of a novel and highly-dexterous 18 degrees of freedom (DOF) miniature master/slave teleoperated robotic system called SPRINT (Single-Port la-paRoscopy bimaNual roboT). The system was evaluated in terms of positioning accuracy, repeatability, tracking error during local teleoperation and end-effector payload. Moreover, it was experimentally verified that the control architecture is real-time compliant at an operating frequency of 1 kHz and it is also reliable in terms of safety. The architecture accounts for cases when the robot is lead through singularities, and includes other safety mechanisms, such as supervision tasks and watchdog timers. Peliminary tests that were performed by surgeons in-vitro suggest that the SPRINT robot, along with its real-time control architecture, could become in the near future a reliable system in the field of Single Port Laparoscopy.

3D force control for robotic-assisted beating heart surgery based on viscoelastic tissue model.

Current cardiac surgery faces the challenging problem of heart beating motion even with the help of mechanical stabilizer which makes delicate operation on the heart surface difficult. Motion compensation methods for robotic-assisted beating heart surgery have been proposed recently in literature, but research on force control for such kind of surgery has hardly been reported. Moreover, the viscoelasticity property of the interaction between organ tissue and robotic instrument further complicates the force control design which is much easier in other applications by assuming the interaction model to be elastic (industry, stiff object manipulation, etc.). In this work, we present a three-dimensional force control method for robotic-assisted beating heart surgery taking into consideration of the viscoelastic interaction property. Performance studies based on our D2M2 robot and 3D heart beating motion information obtained through Da Vinci™ system are provided.

Some control-related issues in mini-robotics for endoluminal surgery.

This paper introduces some issues related to the development of robotics for endoluminal surgery from control point of view. Endoluminal surgery are incisionless procedures performed through natural orifices within the natural pathways. New devices are then required to achieve these new surgical procedures. Besides the development of new devices, control issues arise in both technological and theoretical aspects. The paper presents some of them and we propose a teleoperation architecture that has already been tested for needle insertion that could be used for teleoperated endoluminal surgery especially for instance for biopsies or anastomoses.

Design of an ultrasound-guided robotic brachytherapy needle-insertion system.

In this paper we describe a new robotic brachytherapy needle-insertion system that is designed to replace the template used in the manual technique. After a brief review of existing robotic systems, we describe the requirements that we based our design upon. A detailed description of the proposed system follows. Our design is capable of positioning and inclining a needle within the same workspace as the manual template. To help improve accuracy, the needle can be rotated about its axis during insertion into the prostate. The system can be mounted on existing steppers and also easily accommodates existing seed dispensers, such as the Mick Applicator.