Matching brain-machine interface performance to space applications.

A brain-machine interface (BMI) is a particular class of human-machine interface (HMI). BMIs have so far been studied mostly as a communication means for people who have little or no voluntary control of muscle activity. For able-bodied users, such as astronauts, a BMI would only be practical if conceived as an augmenting interface. A method is presented for pointing out effective combinations of HMIs and applications of robotics and automation to space. Latency and throughput are selected as performance measures for a hybrid bionic system (HBS), that is, the combination of a user, a device, and a HMI. We classify and briefly describe HMIs and space applications and then compare the performance of classes of interfaces with the requirements of classes of applications, both in terms of latency and throughput. Regions of overlap correspond to effective combinations. Devices requiring simpler control, such as a rover, a robotic camera, or environmental controls are suitable to be driven by means of BMI technology. Free flyers and other devices with six degrees of freedom can be controlled, but only at low-interactivity levels. More demanding applications require conventional interfaces, although they could be controlled by BMIs once the same levels of performance as currently recorded in animal experiments are attained. Robotic arms and manipulators could be the next frontier for noninvasive BMIs. Integrating smart controllers in HBSs could improve interactivity and boost the use of BMI technology in space applications.

On the use of longitudinal intrafascicular peripheral interfaces for the control of cybernetic hand prostheses in amputees.

Significant strides have been recently made to develop highly sensorized cybernetic prostheses aimed at restoring sensorimotor limb functions to those who have lost them because of a traumatic event (amputation). In these cases, one of the main goals is to create a bidirectional link between the artificial devices (e.g., robotic hands, arms, or legs) and the nervous system. Several human-machine interfaces (HMIs) are currently used to this aim. Among them, interfaces with the peripheral nervous system and in particular longitudinal intrafascicular electrodes can be a promising solution able to improve the current situation. In this paper, the potentials and limits of the use of this interface to control robotic devices are presented. Specific information is provided on: 1) the neurophysiological bases for the use peripheral nerve interfaces; 2) a comparison of the potentials of the different peripheral neural interfaces; 3) the possibility of extracting and appropriately interpreting the neural code for motor commands and of delivering sensory feedback by stimulating afferent fibers by using longitudinal intrafascicular electrodes; 4) a preliminary comparative analysis of the performance of this approach with the ones of others HMIs; 5) the open issues which have to be addressed for a chronic usability of this approach.

Hand-held robotic instrument for dextrous laparoscopic interventions.

BACKGROUND: Surgical instruments used in many types of minimally invasive procedures are rigid or only limitedly flexible. Some common tasks like suturing, require precise and dextrous movements that are difficult to perform by means of instruments with limited degrees of freedom (DOF). METHODS: A hand-held lightweight and ergonomic robotic instrument with a 3-DOF roll-pitch-roll end-effector has been developed, which can be controlled by the surgeon with one hand like a conventional instrument. RESULTS: The instrument is composed by a handle part and an end-effector. The handle part has been developed taking into account a control mode study for laparoscopic instruments and it allows direct mapping between the orientation of handle and that of the end-effector. CONCLUSION: The instrument presented is the result of a global study involving mechanical, electronic and ergonomic aspects, with the aim of developing an instrument that enhances the dexterity of the surgeon while having an intuitive interface.

EndoCAS navigator platform: a common platform for computer and robotic assistance in minimally invasive surgery.

BACKGROUND: Computer-assisted surgery (CAS) systems are currently used in only a few surgical specialties: ear, nose and throat (ENT), neurosurgery and orthopaedics. Almost all of these systems have been developed as dedicated platforms and work on rigid anatomical structures. The development of augmented reality systems for intra-abdominal organs remains problematic because of the anatomical complexity of the human peritoneal cavity and especially because of the deformability of its organs. The aim of the present work was to develop and implement a highly modular platform (targeted for minimally invasive laparoscopic surgery) generally suitable for CAS, and to produce a prototype for demonstration of its potential clinical application and use in laparoscopic surgery. METHODS: In this paper we outline details of a platform integrating several aspects of CAS and medical robotics into a modular open architecture: the EndoCAS navigator platform, which integrates all the functionalities necessary for provision of computer-based assistance to surgeons during all the management phases (diagnostic work-up, planning and intervention). A specific application for computer-assisted laparoscopic procedures has been developed on the basic modules of the platform. The system provides capabilities for three-dimensional (3D) surface model generation, 3D visualization, intra-operative registration, surgical guidance and robotic assistance during laparoscopic surgery. The description of specific modules and an account of the initial clinical experience with the system are reported. RESULTS: We developed a common platform for computer assisted surgery and implemented a system for intraoperative laparoscopic navigation. The preliminary clinical trials and feedback from the surgeons on its use in laparoscopic surgery have been positive, although experience has been limited to date. CONCLUSIONS: We have successfully developed a system for computer-assisted technologies for use in laparoscopic surgery and demonstrated, by early clinical trials, that the introduction of these technologies in operative laparoscopy, even though they are not yet sufficiently accurate (from an engineering viewpoint) for surgical treatment of intra-abdominal disease, brings added benefits to the execution of interventions by surgeons and hence represents concrete on-going progress in interventional technology.

Defining brain-machine interface applications by matching interface performance with device requirements.

Interaction with machines is mediated by human-machine interfaces (HMIs). Brain-machine interfaces (BMIs) are a particular class of HMIs and have so far been studied as a communication means for people who have little or no voluntary control of muscle activity. In this context, low-performing interfaces can be considered as prosthetic applications. On the other hand, for able-bodied users, a BMI would only be practical if conceived as an augmenting interface. In this paper, a method is introduced for pointing out effective combinations of interfaces and devices for creating real-world applications. First, devices for domotics, rehabilitation and assistive robotics, and their requirements, in terms of throughput and latency, are described. Second, HMIs are classified and their performance described, still in terms of throughput and latency. Then device requirements are matched with performance of available interfaces. Simple rehabilitation and domotics devices can be easily controlled by means of BMI technology. Prosthetic hands and wheelchairs are suitable applications but do not attain optimal interactivity. Regarding humanoid robotics, the head and the trunk can be controlled by means of BMIs, while other parts require too much throughput. Robotic arms, which have been controlled by means of cortical invasive interfaces in animal studies, could be the next frontier for non-invasive BMIs. Combining smart controllers with BMIs could improve interactivity and boost BMI applications.

Novel haptic tool and input device for real time bilateral biomanipulation addressing endoscopic surgery.

In this paper a sensorised polymer microgripper is presented which can be used as a suitable end effector on an endoscopic microinstrument for robot-assisted and possibly teleoperated surgery to enable the operator to receive haptic feedback information on the forces generated during the procedure. A novel tweezers- like haptic input device is also described, which gives the operator the ability to remotely feel these forces generated by grasping operations with the microgripper. This feedback is used to control the amount of force applied in manipulation of tissues during the procedure. The mechanical and electronic design of the microgripper, microinstrument and haptic tweezers is also presented and preliminary results detailed.

Tracking endoscopic instruments without a localizer: a shape-analysis-based approach.

We present an approach to localizing endoscopic instruments with respect to the camera position, based purely on processing of the endoscope image. No localizers are needed; the only requirement is a colored strip at the distal part of the instrument shaft to facilitate image segmentation. The method exploits perspective image analysis applied to the cylindrical shape of the instrument shaft, allowing measurement of the instrument position and orientation with five degrees of freedom. We describe the method theoretically, and experimentally derive calibration curves for tuning the parameters of the algorithm. Results show that the method can be used for applications where accuracy is not critical, such as workspace measurement, gesture analysis, augmented-reality guidance, telementoring, etc. If this method is used in combination with an endoscope tracker or a robotic camera holder, full localization with respect to the patient reference frame can be achieved.

Modelling and evaluation of surgical performance using hidden Markov models.

Minimally invasive surgery has become very widespread in the last ten years. Since surgeons experience difficulties in learning and mastering minimally invasive techniques, the development of training methods is of great importance. While the introduction of virtual reality-based simulators has introduced a new paradigm in surgical training, skill evaluation methods are far from being objective. This paper proposes a method for defining a model of surgical expertise and an objective metric to evaluate performance in laparoscopic surgery. Our approach is based on the processing of kinematic data describing movements of surgical instruments. We use hidden Markov model theory to define an expert model that describes expert surgical gesture. The model is trained on kinematic data related to exercises performed on a surgical simulator by experienced surgeons. Subsequently, we use this expert model as a reference model in the definition of an objective metric to evaluate performance of surgeons with different abilities. Preliminary results show that, using different topologies for the expert model, the method can be efficiently used both for the discrimination between experienced and novice surgeons, and for the quantitative assessment of surgical ability.

A biomechanical analysis of surgeon's gesture in a laparoscopic virtual scenario.

Minimally invasive surgery (MIS) has become very common in recent years thanks to many advantages that patients can get. However, due to the difficulties surgeons encounter to learn and manage this technique, several training methods and metrics have been proposed in order to, respectively, improve surgeon's abilities and assess his/her surgical skills. In this context, this paper presents a biomechanical analysis method of the surgeon's movements, during exercise involving instrument tip positioning and depth perception in a laparoscopic virtual environment. Estimation of some biomechanical parameters enables us to assess the abilities of surgeons and to distinguish an expert surgeon from a novice. A segmentation algorithm has been defined to deeply investigate the surgeon's movements and to divide them into many sub-movements.

Tracking endoscopic instruments without localizer: image analysis-based approach.

In this paper we present an approach to localize endoscopic instruments with respect to the camera position, purely based on video image processing. No localizers are required. The only requirement is a coloured strip at the distal part of the instrument shaft, to facilitate image segmentation. The method exploits perspective image analysis applied to the cylindrical shape of the instrument shaft, allowing to measure five degrees of freedom of the instrument position and orientation. We describe the method theoretically and experimentally derive calibration curves for tuning the parameters of the algorithm. Results show that the method can be used for applications where accuracy is not critical, e.g. workspace analysis, gesture analysis, augmented-reality guidance, telementoring, etc. If this method is used in combination with a robotic camera holder, full localization with respect to the operating room can be achieved.

Comparison of control modes of a hand-held robot for laparoscopic surgery.

Teleoperated robots for minimally invasive surgery make surgeons loose direct contact with the patient. We are developing a handheld, dexterous surgical robot that can be controlled with one hand only, while standing at the operating table. The instrument is composed of a master part (the handle) and a slave part (the tip). This work compares the performance of different control modes, i.e. different ways to map the degrees of freedom of the handle to those of the tip. We ask users to drive the tip along complex trajectories in a virtual environment, using the real master to drive a simulated slave, and assess their performance. Results show that, concerning time, users with no training in laparoscopy prefer a direct mapping of position and orientation, like in free hand motion. However, users trained in laparoscopy perform equally fast with our hand-held robot and, concerning precision, make a smaller number of errors.