Mirage events & driver haptic steering alerts in a motion-base driving simulator: A method for selecting an optimal HMI.

This paper describes a new method, a 'mirage scenario,' to support formative evaluation of driver alerting or warning displays for manual and automated driving. This method provides driving contexts (e.g., various Times-To-Collision (TTCs) to a lead vehicle) briefly presented and then removed. In the present study, during each mirage event, a haptic steering display was evaluated. This haptic display indicated a steering response may be initiated to drive around an obstacle ahead. A motion-base simulator was used in a 32-participant study to present vehicle motion cues similar to the actual application. Surprise was neither present nor of concern, as it would be for a summative evaluation of a forward collision warning system. Furthermore, no collision avoidance maneuvers were performed, thereby reducing the risk of simulator sickness. This paper illustrates the mirage scenario procedures, the rating methods and definitions used with the mirage scenario, and analysis of the ratings obtained, together with a multi-attribute utility theory (MAUT) approach to evaluate and select among alternative designs for future summative evaluation.

Distance-based time series classification approach for task recognition with application in surgical robot autonomy.

BACKGROUND: Robotic-assisted surgery allows surgeons to perform many types of complex operations with greater precision than is possible with conventional surgery. Despite these advantages, in current systems, a surgeon should communicate with the device directly and manually. To allow the robot to adjust parameters such as camera position, the system needs to know automatically what task the surgeon is performing. METHODS: A distance-based time series classification framework has been developed which measures dynamic time warping distance between temporal trajectory data of robot arms and classifies surgical tasks and gestures using a k-nearest neighbor algorithm. RESULTS: Results on real robotic surgery data show that the proposed framework outperformed state-of-the-art methods by up to 9% across three tasks and by 8% across gestures. CONCLUSION: The proposed framework is robust and accurate. Therefore, it can be used to develop adaptive control systems that will be more responsive to surgeons' needs by identifying next movements of the surgeon. Copyright © 2016 John Wiley & Sons, Ltd.

Task analysis of laparoscopic camera control schemes.

BACKGROUND: Minimally invasive surgeries rely on laparoscopic camera views to guide the procedure. Traditionally, an expert surgical assistant operates the camera. In some cases, a robotic system is used to help position the camera, but the surgeon is required to direct all movements of the system. Some prior research has focused on developing automated robotic camera control systems, but that work has been limited to rudimentary control schemes due to a lack of understanding of how the camera should be moved for different surgical tasks. METHODS: This research used task analysis with a sample of eight expert surgeons to discover and document several salient methods of camera control and their related task contexts. RESULTS: Desired camera placements and behaviours were established for two common surgical subtasks (suturing and knot tying). CONCLUSION: The results can be used to develop better robotic control algorithms that will be more responsive to surgeons' needs. Copyright © 2015 John Wiley & Sons, Ltd.

Towards an autonomous robot for camera control during laparoscopic surgery.

INTRODUCTION: During laparoscopic surgery, the surgeon currently must instruct a human camera operator or a robotic arm to move the camera. This process is distracting, and the camera is not always placed in an ideal location. To mitigate these problems, we have developed a test platform that tracks laparoscopic instruments and automatically moves a camera with no explicit human direction. MATERIALS AND METHODS: The test platform is designed to mimic a typical laparoscopic working environment, where two hand-operated tools are manipulated through small ports. A pan-tilt-zoom camera is positioned over the tools, which emulates the positioning capabilities of a straight (0°) scope placed through a trocar. A camera control algorithm automatically keeps the tools in the view. In addition, two test tasks that require camera movement have been developed to aid in future evaluation of the system. RESULTS: The system was found to successfully track the laparoscopic instruments in the camera view as intended. The camera is moved and zoomed to follow the instruments in a smooth and consistent fashion. CONCLUSIONS: This technology shows that it is possible to create an autonomous camera system that cooperates with a surgeon without requiring any explicit user input. However, the currently implemented camera control behaviors are not ideal or sufficient for many surgical tasks. Future work will be performed to develop, test, and refine more complex behaviors that are optimized for different kinds of surgical tasks. In addition, portions of the test platform will be redesigned to enable its use in actual laparoscopic procedures.

Communication strategies and timeliness of response to life critical telemetry alarms.

BACKGROUND: A centralized electrocardiogram telemetry monitoring system (TMS) facilitates early identification of critical arrhythmias and acute medical decompensation. Timely intervention can only be performed if abnormalities are communicated rapidly to the direct caregiver. The study objectives were to measure effectiveness of bi-directional voice communication badges versus one-way alphanumeric pagers for telemetry alarm response and communication loop closure. METHODS: A sequential observational pilot study of nursing response to TMS alarms compared communication technologies on four nursing units in a 1,061 bed tertiary care hospital with 264 TMS channels of telemetry over a 2-year period. Subsequently, the communication technologies were compared in a randomized fashion on a 68-bed progressive cardiac care unit. Caregivers were blinded to the protocol. All alarm responses were recorded during two periods using either pagers or voice communication devices. Alarm response time and closure of the communication loop were analyzed in a blinded fashion. RESULTS: The direct communication functionality of the badge significantly shortened the time to first contact, time to completion, and rate of closure of the communication loop in both the pilot and study phases. Median time to first contact with the communication badge was 0.5 min, compared to 1.6 min with pager communication (p < 0.0003). Communication loop closure was achieved in 100% of clinical alarms using the badge versus 19% with the pager (p < 0.0001). CONCLUSIONS: Communication badge technology reduced alarm time to first contact and completion as well as facilitated communication loop closures. Immediate two-way communication significantly impacted practice, alarm management, and resulted in faster bedside care.

Human performance in the task of port placement for biosensor use.

BACKGROUND: We conducted a study of participants' abilities to place a laparoscopic port for in vivo biosensor use. Biosensors have physical limitations that make port placement crucial to proper data collection. A new port placement algorithm enabled evaluation of port locations, using segmented patient data in a virtual environment. METHODS: Port placement scoring algorithms were integrated into an image-guided surgery system. Virtual test scenes were created to evaluate various scenarios encountered during biosensor use. Participants were scored based on their ability to choose a port location from which points of interest could be scanned with a biosensor. Participants' scores were also compared to those of a port placement algorithm. RESULTS: The port placement algorithm consistently outscored participants by 10-25%. Participants were inconsistent from trial to trial and from participant to participant. CONCLUSION: Port placement for biosensor procedures could be improved through training or augmentation.

Optimized port placement for in vivo biosensors.

BACKGROUND: We discuss the implementation of an automated port placement system for use with laparoscopic in vivo biosensors. Biosensors have physical limitations that make port placement crucial to proper data collection. The port placement process is prohibitively complex to execute optimally by human estimation. METHODS: Port placement algorithms were integrated into an image-guided surgery system. Variables for optimization of ports include biosensor length, data collection restrictions, obstacles, and regions of interest. The port placement is applied to 3D virtual 'patients' created from medical imaging data. An example implementation for a Raman biosensor probe was created. RESULTS: The prototype system can correctly find ideal port locations on a virtual patient based on biosensor limitations and regions of interest. Conversely, the system can evaluate and score an individual port selected by a user. CONCLUSION: As biosensors become incorporated into laparoscopic surgical environments, an automated port placement system would enable their optimized integration.

NASA TLX: software for assessing subjective mental workload.

The NASA Task Load Index (TLX) is a popular technique for measuring subjective mental workload. It relies on a multidimensional construct to derive an overall workload score based on a weighted average of ratings on six subscales: mental demand, physical demand, temporal demand, performance, effort, and frustration level. A program for implementing a computerized version of the NASA TLX is described. The software version assists in simplifying collection, postprocessing, and storage of raw data. The program collects raw data from the subject and calculates the weighted (or unweighted) workload score, which is output to a text file. The program can also be tailored to a specific experiment using a simple input text file, if desired. The program was designed in Visual Studio 2005 and is capable of running on a Pocket PC with Windows CE or on a PC with Windows 2000 or higher. The NASA TLX program is available for free download.

Performance of basic manipulation and intracorporeal suturing tasks in a robotic surgical system: single- versus dual-monitor views.

BACKGROUND: Technical advances in the application of laparoscopic and robotic surgical systems have improved platform usability. The authors hypothesized that using two monitors instead of one would lead to faster performance with fewer errors. METHODS: All tasks were performed using a surgical robot in a training box. One of the monitors was a standard camera with two preset zoom levels (zoomed in and zoomed out, single-monitor condition). The second monitor provided a static panoramic view of the whole surgical field. The standard camera was static at the zoomed-in level for the dual-monitor condition of the study. The study had two groups of participants: 4 surgeons proficient in both robotic and advanced laparoscopic skills and 10 lay persons (nonsurgeons) who were given adequate time to train and familiarize themselves with the equipment. Running a 50-cm rope was the basic task. Advanced tasks included running a suture through predetermined points and intracorporeal knot tying with 3-0 silk. Trial completion times and errors, categorized into three groups (orientation, precision, and task), were recorded. RESULTS: The trial completion times for all the tasks, basic and advanced, in the two groups were not significantly different. Fewer orientation errors occurred in the nonsurgeon group during knot tying (p=0.03) and in both groups during suturing (p=0.0002) in the dual-monitor arm of the study. Differences in precision and task error were not significant. CONCLUSIONS: Using two camera views helps both surgeons and lay persons perform complex tasks with fewer errors. These results may be due to better awareness of the surgical field with regard to the location of the instruments, leading to better field orientation. This display setup has potential for use in complex minimally invasive surgeries such as esophagectomy and gastric bypass. This technique also would be applicable to open microsurgery.

Development and human factors analysis of neuronavigation vs. augmented reality.

This paper is focused on the human factors analysis comparing a standard neuronavigation system with an augmented reality system. We use a passive articulated arm (Microscribe, Immersion technology) to track a calibrated end-effector mounted video camera. In real time, we superimpose the live video view with the synchronized graphical view of CT-derived segmented object(s) of interest within a phantom skull. Using the same robotic arm, we have developed a neuronavigation system able to show the end-effector of the arm on orthogonal CT scans. Both the AR and the neuronavigation systems have been shown to be within 3mm of accuracy. A human factors study was conducted in which subjects were asked to draw craniotomies and answer questions to gage their understanding of the phantom objects. The human factors study included 21 subjects and indicated that the subjects performed faster, with more accuracy and less errors using the Augmented Reality interface.

Computational modeling of foveal target detection.

This paper presents the VDM2000, a computational model of target detection designed for use in military developmental test and evaluation settings. The model integrates research results from the fields of early vision, object recognition, and psychophysics. The VDM2000 is image based and provides a criterion-independent measure of target conspicuity, referred to as the vehicle metric (VM). A large data set of human responses to photographs of military vehicles in a field setting was used to validate the model. The VM adjusted by a single calibration parameter accounts for approximately 80% of the variance in the validation data. The primary application of this model is to predict detection of military targets in daylight with the unaided eye. The model also has application to target detection prediction using infrared night vision systems. The model has potential as a tool to evaluate the visual properties of more general task settings.