Development and evaluation of an autonomous camera control algorithm on the da Vinci Surgical System.

BACKGROUND: Manual control of the camera arm in telerobotic surgical systems requires the surgeon to repeatedly interrupt the flow of the surgery. During surgery, there are instances when one or even both tools can drift out of the field of view. These issues may lead to increased workload and potential errors. METHODS: We performed a 20-participant subject study (including four surgeons) to compare different methods of camera control on a customized da Vinci Surgical System. We tested (a) an autonomous camera algorithm, (b) standard clutched control, and (c) an experienced camera operator using a joystick. RESULTS: The automated algorithm surpassed the traditional method of clutched camera control in measures of userperceived workload, efficiency, and progress. Additionally, it was consistently able to generate more centered and appropriately zoomed viewpoints than the other methods while keeping both tools safely inside the camera's field of view. CONCLUSIONS: Clinical systems of the future should consider automating the camera control aspects of robotic surgery.

Automated robot-assisted surgical skill evaluation: Predictive analytics approach.

BACKGROUND: Surgical skill assessment has predominantly been a subjective task. Recently, technological advances such as robot-assisted surgery have created great opportunities for objective surgical evaluation. In this paper, we introduce a predictive framework for objective skill assessment based on movement trajectory data. Our aim is to build a classification framework to automatically evaluate the performance of surgeons with different levels of expertise. METHODS: Eight global movement features are extracted from movement trajectory data captured by a da Vinci robot for surgeons with two levels of expertise - novice and expert. Three classification methods - k-nearest neighbours, logistic regression and support vector machines - are applied. RESULTS: The result shows that the proposed framework can classify surgeons' expertise as novice or expert with an accuracy of 82.3% for knot tying and 89.9% for a suturing task. CONCLUSION: This study demonstrates and evaluates the ability of machine learning methods to automatically classify expert and novice surgeons using global movement features.

Distance-based time series classification approach for task recognition with application in surgical robot autonomy.

BACKGROUND: Robotic-assisted surgery allows surgeons to perform many types of complex operations with greater precision than is possible with conventional surgery. Despite these advantages, in current systems, a surgeon should communicate with the device directly and manually. To allow the robot to adjust parameters such as camera position, the system needs to know automatically what task the surgeon is performing. METHODS: A distance-based time series classification framework has been developed which measures dynamic time warping distance between temporal trajectory data of robot arms and classifies surgical tasks and gestures using a k-nearest neighbor algorithm. RESULTS: Results on real robotic surgery data show that the proposed framework outperformed state-of-the-art methods by up to 9% across three tasks and by 8% across gestures. CONCLUSION: The proposed framework is robust and accurate. Therefore, it can be used to develop adaptive control systems that will be more responsive to surgeons' needs by identifying next movements of the surgeon. Copyright © 2016 John Wiley & Sons, Ltd.

Towards the Implementation of an Autonomous Camera Algorithm on the da Vinci Platform.

Camera positioning is critical for all telerobotic surgical systems. Inadequate visualization of the remote site can lead to serious errors that can jeopardize the patient. An autonomous camera algorithm has been developed on a medical robot (da Vinci) simulator. It is found to be robust in key scenarios of operation. This system behaves with predictable and expected actions for the camera arm with respect to the tool positions. The implementation of this system is described herein. The simulation closely models the methodology needed to implement autonomous camera control in a real hardware system. The camera control algorithm follows three rules: (1) keep the view centered on the tools, (2) keep the zoom level optimized such that the tools never leave the field of view, and (3) avoid unnecessary movement of the camera that may distract/disorient the surgeon. Our future work will apply this algorithm to the real da Vinci hardware.

Task analysis of laparoscopic camera control schemes.

BACKGROUND: Minimally invasive surgeries rely on laparoscopic camera views to guide the procedure. Traditionally, an expert surgical assistant operates the camera. In some cases, a robotic system is used to help position the camera, but the surgeon is required to direct all movements of the system. Some prior research has focused on developing automated robotic camera control systems, but that work has been limited to rudimentary control schemes due to a lack of understanding of how the camera should be moved for different surgical tasks. METHODS: This research used task analysis with a sample of eight expert surgeons to discover and document several salient methods of camera control and their related task contexts. RESULTS: Desired camera placements and behaviours were established for two common surgical subtasks (suturing and knot tying). CONCLUSION: The results can be used to develop better robotic control algorithms that will be more responsive to surgeons' needs. Copyright © 2015 John Wiley & Sons, Ltd.

Voice-Activated Lightweight Reacher to Assist with Upper Extremity Movement Limitations: A Case Study.

The focus of this research was to design a functional and user-friendly reacher for people with spinal cord injuries (SCIs). Engineering advancements have taken assistive robotics to new dimensions. Technologies such as wheelchair robotics and myo-electronically controlled systems have opened up a wide range of new applications to assist people with physical disabilities. Similarly, exo-skeletal limbs and body suits have provided new foundations from which technologies can aid function. Unfortunately, these devices have issues of usability, weight, and discomfort with donning. The Smart Assistive Reacher Arm (SARA) system, developed in this research, is a voice-activated, lightweight, mobile device that can be used when needed. SARA was built to help overcome daily reach challenges faced by individuals with limited arm and hand movement capability, such as people with cervical level 5-6 (C5-6) SCI. This article shows that a functional reacher arm with voice control can be beneficial for this population. Comparison study with healthy participants and an SCI participant shows that, when using SARA, a person with SCI can perform simple reach and grasp tasks independently, without someone else's help. This suggests that the interface is intuitive and can be easily used to a high level of proficiency by a SCI individual.

Towards an autonomous robot for camera control during laparoscopic surgery.

INTRODUCTION: During laparoscopic surgery, the surgeon currently must instruct a human camera operator or a robotic arm to move the camera. This process is distracting, and the camera is not always placed in an ideal location. To mitigate these problems, we have developed a test platform that tracks laparoscopic instruments and automatically moves a camera with no explicit human direction. MATERIALS AND METHODS: The test platform is designed to mimic a typical laparoscopic working environment, where two hand-operated tools are manipulated through small ports. A pan-tilt-zoom camera is positioned over the tools, which emulates the positioning capabilities of a straight (0°) scope placed through a trocar. A camera control algorithm automatically keeps the tools in the view. In addition, two test tasks that require camera movement have been developed to aid in future evaluation of the system. RESULTS: The system was found to successfully track the laparoscopic instruments in the camera view as intended. The camera is moved and zoomed to follow the instruments in a smooth and consistent fashion. CONCLUSIONS: This technology shows that it is possible to create an autonomous camera system that cooperates with a surgeon without requiring any explicit user input. However, the currently implemented camera control behaviors are not ideal or sufficient for many surgical tasks. Future work will be performed to develop, test, and refine more complex behaviors that are optimized for different kinds of surgical tasks. In addition, portions of the test platform will be redesigned to enable its use in actual laparoscopic procedures.

Human performance in the task of port placement for biosensor use.

BACKGROUND: We conducted a study of participants' abilities to place a laparoscopic port for in vivo biosensor use. Biosensors have physical limitations that make port placement crucial to proper data collection. A new port placement algorithm enabled evaluation of port locations, using segmented patient data in a virtual environment. METHODS: Port placement scoring algorithms were integrated into an image-guided surgery system. Virtual test scenes were created to evaluate various scenarios encountered during biosensor use. Participants were scored based on their ability to choose a port location from which points of interest could be scanned with a biosensor. Participants' scores were also compared to those of a port placement algorithm. RESULTS: The port placement algorithm consistently outscored participants by 10-25%. Participants were inconsistent from trial to trial and from participant to participant. CONCLUSION: Port placement for biosensor procedures could be improved through training or augmentation.

An integrated movement capture and control platform applied towards autonomous movements of surgical robots.

Robotic surgery has gradually gained acceptance due to its numerous advantages such as tremor filtration, increased dexterity and motion scaling. There remains, however, a significant scope for improvement, especially in the areas of surgeon-robot interface and autonomous procedures. Previous studies have attempted to identify factors affecting a surgeon's performance in a master-slave robotic system by tracking hand movements. These studies relied on conventional optical or magnetic tracking systems, making their use impracticable in the operating room. This study concentrated on building an intrinsic movement capture platform using microcontroller based hardware wired to a surgical robot. Software was developed to enable tracking and analysis of hand movements while surgical tasks were performed. Movement capture was applied towards automated movements of the robotic instruments. By emulating control signals, recorded surgical movements were replayed by the robot's end-effectors. Though this work uses a surgical robot as the platform, the ideas and concepts put forward are applicable to telerobotic systems in general.

Optimized port placement for in vivo biosensors.

BACKGROUND: We discuss the implementation of an automated port placement system for use with laparoscopic in vivo biosensors. Biosensors have physical limitations that make port placement crucial to proper data collection. The port placement process is prohibitively complex to execute optimally by human estimation. METHODS: Port placement algorithms were integrated into an image-guided surgery system. Variables for optimization of ports include biosensor length, data collection restrictions, obstacles, and regions of interest. The port placement is applied to 3D virtual 'patients' created from medical imaging data. An example implementation for a Raman biosensor probe was created. RESULTS: The prototype system can correctly find ideal port locations on a virtual patient based on biosensor limitations and regions of interest. Conversely, the system can evaluate and score an individual port selected by a user. CONCLUSION: As biosensors become incorporated into laparoscopic surgical environments, an automated port placement system would enable their optimized integration.

NASA TLX: software for assessing subjective mental workload.

The NASA Task Load Index (TLX) is a popular technique for measuring subjective mental workload. It relies on a multidimensional construct to derive an overall workload score based on a weighted average of ratings on six subscales: mental demand, physical demand, temporal demand, performance, effort, and frustration level. A program for implementing a computerized version of the NASA TLX is described. The software version assists in simplifying collection, postprocessing, and storage of raw data. The program collects raw data from the subject and calculates the weighted (or unweighted) workload score, which is output to a text file. The program can also be tailored to a specific experiment using a simple input text file, if desired. The program was designed in Visual Studio 2005 and is capable of running on a Pocket PC with Windows CE or on a PC with Windows 2000 or higher. The NASA TLX program is available for free download.

Performance of basic manipulation and intracorporeal suturing tasks in a robotic surgical system: single- versus dual-monitor views.

BACKGROUND: Technical advances in the application of laparoscopic and robotic surgical systems have improved platform usability. The authors hypothesized that using two monitors instead of one would lead to faster performance with fewer errors. METHODS: All tasks were performed using a surgical robot in a training box. One of the monitors was a standard camera with two preset zoom levels (zoomed in and zoomed out, single-monitor condition). The second monitor provided a static panoramic view of the whole surgical field. The standard camera was static at the zoomed-in level for the dual-monitor condition of the study. The study had two groups of participants: 4 surgeons proficient in both robotic and advanced laparoscopic skills and 10 lay persons (nonsurgeons) who were given adequate time to train and familiarize themselves with the equipment. Running a 50-cm rope was the basic task. Advanced tasks included running a suture through predetermined points and intracorporeal knot tying with 3-0 silk. Trial completion times and errors, categorized into three groups (orientation, precision, and task), were recorded. RESULTS: The trial completion times for all the tasks, basic and advanced, in the two groups were not significantly different. Fewer orientation errors occurred in the nonsurgeon group during knot tying (p=0.03) and in both groups during suturing (p=0.0002) in the dual-monitor arm of the study. Differences in precision and task error were not significant. CONCLUSIONS: Using two camera views helps both surgeons and lay persons perform complex tasks with fewer errors. These results may be due to better awareness of the surgical field with regard to the location of the instruments, leading to better field orientation. This display setup has potential for use in complex minimally invasive surgeries such as esophagectomy and gastric bypass. This technique also would be applicable to open microsurgery.

Evaluation of pancreatic cancer with Raman spectroscopy in a mouse model.

OBJECTIVES: Detection of neoplastic changes using optical spectroscopy has been an active area of research in recent times. Raman spectroscopy is a vibrational spectroscopic technique that can be used to diagnose various tumors with high sensitivity and specificity. We evaluated the ability of Raman spectroscopy to differentiate normal pancreatic tissue from malignant tumors in a mouse model. METHODS: We collected 920 spectra, 475 from 31 normal pancreatic tissue and 445 from 29 tumor nodules using a 785-nm near-infrared laser excitation. Discriminant function analysis was used for classification of normal and tumor samples. RESULTS: Using principal component analysis, we were able to highlight subtle chemical differences in normal and malignant tissue. Using histopathology as the gold standard, Raman analysis gave sensitivities between 91% and 96% and specificities between 88% and 96%. CONCLUSIONS: Raman spectroscopy along with discriminant function analysis is a useful method to detect cancerous changes in the pancreas. Pancreatic tumors were characterized by increased collagen content and decreased DNA, RNA, and lipids components compared with normal pancreatic tissue.

Comparison of a supplemental wide field of view versus a single field of view with zoom on performance in minimally invasive surgery.

BACKGROUND: The limited space and high magnification involved in minimally invasive surgery (MIS) can cause surgeons to lose sight of an instrument while performing tasks such as suturing and knot-tying. A current strategy employed to locate the instrument is zooming out and in with the endoscope, which can be a time-intensive and iterative task. This study investigates the use of a supplemental wide field of view (FOV) via a second endoscope for locating an instrument outside the FOV in a MIS setting. METHODS: Ten surgically naïve subjects performed a simple aimed movement task with either hand (dominant or nondominant) under two display conditions: (1) conventional single monitor with zoom, and (2) supplemental wide FOV monitor with no zoom. The task emulated the need to locate an instrument outside the surgeon's FOV and return it to a home position. RESULTS: The supplemental wide FOV produced significantly faster times [F(3,716) = 173.2, p < 0.001)] compared to a single monitor. The task was accomplished most quickly with the dual monitor with the dominant hand, followed by dual monitor with nondominant hand followed by a single monitor with either hand. There were also significantly fewer errors (t = 3.734, df = 9, p = 0.005) with the supplemental wide FOV. None of the subjects were slower with the dual monitor, and all but one had fewer errors. The variance for both task times and errors were also significantly smaller (p < 0.001 and p = 0.008, respectively) with the supplemental wide FOV indicating that subjects performed with increased reliability. CONCLUSION: The supplemental wide FOV gave the subjects the ability to see their instrument at all times providing a more efficient display than zooming out and in. This enabled faster times and fewer errors while allowing the user to perform the task with more consistency.

Raman spectroscopy can differentiate malignant tumors from normal breast tissue and detect early neoplastic changes in a mouse model.

Raman spectroscopy shows potential in differentiating tumors from normal tissue. We used Raman spectroscopy with near-infrared light excitation to study normal breast tissue and tumors from 11 mice injected with a cancer cell line. Spectra were collected from 17 tumors, 18 samples of adjacent breast tissue and lymph nodes, and 17 tissue samples from the contralateral breast and its adjacent lymph nodes. Discriminant function analysis was used for classification with principal component analysis scores as input data. Tissues were examined by light microscopy following formalin fixation and hematoxylin and eosin staining. Discriminant function analysis and histology agreed on the diagnosis of all contralateral normal, tumor, and mastitis samples, except one tumor which was found to be more similar to normal tissue. Normal tissue adjacent to each tumor was examined as a separate data group called tumor bed. Scattered morphologically suspicious atypical cells not definite for tumor were present in the tumor bed samples. Classification of tumor bed tissue showed that some tumor bed tissues are diagnostically different from normal, tumor, and mastitis tissue. This may reflect malignant molecular alterations prior to morphologic changes, as expected in preneoplastic processes. Raman spectroscopy not only distinguishes tumor from normal breast tissue, but also detects early neoplastic changes prior to definite morphologic alteration.

Supplemental wide field-of-view monitor improves performance in surgical telerobotic movement time.

BACKGROUND: This study investigates the effect of a supplemental wide field-of-view (FOV) monitor on performance at high magnifications where the benefits of robotic surgery have greater importance. METHODS: Ten surgically naïve participants performed a simple aimed movement task under two different monitor conditions. The task is intended to emulate the need to locate an instrument outside the surgeon's FOV and return it to a 'home' position without the need to zoom. One monitor condition used a narrow FOV (25x) coupled with a supplemental wide FOV (3x). The second monitor condition used only a narrow FOV. RESULTS: Using a supplemental wide FOV in addition to a narrow FOV improved task performance by at least 33%, with greater consistency and reliability. CONCLUSION: The supplemental wide FOV monitor provided additional information to the participant, allowing more efficient performance at high magnifications without the need for zooming.