Affordable, web-based surgical skill training and evaluation tool.

Advances in the medical field have increased the need to incorporate modern techniques into surgical resident training and surgical skills learning. To facilitate this integration, one approach that has gained credibility is the incorporation of simulator based training to supplement traditional training programs. However, existing implementations of these training methods still require the constant presence of a competent surgeon to assess the surgical dexterity of the trainee, which limits the evaluation methods and relies on subjective evaluation. This research proposes an efficient, effective, and economic video-based skill assessment technique for minimally invasive surgery (MIS). It analyzes a surgeon's hand and surgical tool movements and detects features like smoothness, efficiency, and preciseness. The system is capable of providing both real time on-screen feedback and a performance score at the end of the surgery. Finally, we present a web-based tool where surgeons can securely upload MIS training videos and receive evaluation scores and an analysis of trainees' performance trends over time.

Sociotechnical evaluation of the safety and effectiveness of point-of-care mobile computing devices: a case study conducted in India.

The potential for health information technology (IT) to enhance quality of care is limited by unanticipated problems following adoption of new systems and technologies. Proactive assessment of system vulnerabilities can help improve existing systems and ease implementation of new innovations. We applied a comprehensive socio-technical model of safe and effective health IT use to the formative evaluation of a novel tablet-based device designed to support primary care practice in rural India. Based on our conceptual model, we developed an assessment guide for the tablet system that was informed by literature review, interviews, and observations of health workers and supervisors. Our assessment revealed and addressed both technical (functionality, content, usability, user interface) and non-technical (workflow, processes and policies etc.)areas of improvement.

Avatar-based simulation in the evaluation of diagnosis and management of mental health disorders in primary care.

Major Depressive Disorder (MDD) and Posttraumatic Stress Disorder (PTSD) are highly prevalent illnesses, but the literature suggests they are under-detected and suboptimally managed by primary care practitioners (PCPs). In this paper, we propose and use an evaluation method, using digitally simulated patients (avatars) to evaluate the diagnostic and therapeutic reasoning of PCPs and compared it to the traditional use of paper-based cases. Verbal (think-aloud) protocols were captured in the context of a diagnostic and therapeutic reasoning task. Propositional and semantic representational analysis of simulation data during evaluation, showed specific deficiencies in PCP reasoning, suggesting a promise of this technology in training and evaluation in mental health. Avatars are flexible and easily modifiable and are also a cost-effective and easy-to-disseminate educational tool.

High-fidelity, low-cost, automated method to assess laparoscopic skills objectively.

BACKGROUND: We sought to define the extent to which a motion analysis-based assessment system constructed with simple equipment could measure technical skill objectively and quantitatively. METHODS: An "off-the-shelf" digital video system was used to capture the hand and instrument movement of surgical trainees (beginner level = PGY-1, intermediate level = PGY-3, and advanced level = PGY-5/fellows) while they performed a peg transfer exercise. The video data were passed through a custom computer vision algorithm that analyzed incoming pixels to measure movement smoothness objectively. RESULTS: The beginner-level group had the poorest performance, whereas those in the advanced group generated the highest scores. Intermediate-level trainees scored significantly (p < 0.04) better than beginner trainees. Advanced-level trainees scored significantly better than intermediate-level trainees and beginner-level trainees (p < 0.04 and p < 0.03, respectively). CONCLUSIONS: A computer vision-based analysis of surgical movements provides an objective basis for technical expertise-level analysis with construct validity. The technology to capture the data is simple, low cost, and readily available, and it obviates the need for expert human assessment in this setting.

Laparoscopic warm-up exercises improve performance of senior-level trainees during laparoscopic renal surgery.

BACKGROUND AND PURPOSE: Surgery is a high-stakes "performance." Yet, unlike athletes or musicians, surgeons do not engage in routine "warm-up" exercises before "performing" in the operating room. We study the impact of a preoperative warm-up exercise routine (POWER) on surgeon performance during laparoscopic surgery. MATERIALS AND METHODS: Serving as their own controls, each subject performed two pairs of laparoscopic cases, each pair consisting of one case with POWER (+POWER) and one without (-POWER). Subjects were randomly assigned to +POWER or -POWER for the initial case of each pairing, and all cases were performed ≥ 1 week apart. POWER consisted of completing an electrocautery skill task on a virtual reality simulator and 15 minutes of laparoscopic suturing and knot tying in a pelvic box trainer. For each case, cognitive, psychomotor, and technical performance data were collected during two different tasks: mobilization of the colon (MC) and intracorporeal suturing and knot tying (iSKT). Statistical analysis was performed using SYSTAT v11.0. RESULTS: A total of 28 study cases (14+POWER, 14-POWER) were performed by seven different subjects. Cognitive and psychomotor performance (attention, distraction, workload, spatial reasoning, movement smoothness, posture stability) were found to be significantly better in the +POWER group (P ≤ 0.05) and technical performance, as scored by two blinded laparoscopic experts, was found to be better in the +POWER group for MC (P=0.04) but not iSKT (P=0.92). Technical scores demonstrated excellent reliability using our assessment tool (Cronbach ∝=0.88). Subject performance during POWER was also found to correlate with intraoperative performance scores. CONCLUSIONS: Urologic trainees who perform a POWER approximately 1 hour before laparoscopic renal surgery demonstrate improved cognitive, psychomotor, and technical performance.

The effect of call on neurosurgery residents' skills: implications for policy regarding resident call periods.

OBJECT: Although fatigue and its effects on surgical proficiency have been an actively researched area, previous studies have not examined the effect of fatigue on neurosurgery residents specifically. This study aims to quantify the effect of fatigue on the psychomotor and cognitive skills of neurosurgery residents. METHODS: Seven neurosurgery residents performed a minimum of 3 and a maximum of 4 sessions of 6 surgical exercises precall and postcall. The simulation exercises were designed to measure a surgeon's cognitive abilities, such as memory and attention, while performing simulated surgical tasks and exercises that have been previously validated in several studies, including studies measuring the impact of fatigue on general surgery residents. Each exercise measured tool-movement smoothness, time elapsed, and cognitive errors. The change in surgical skills in precall and postcall conditions was assessed by means of an ANOVA, with p < 0.05 considered statistically significant. RESULTS: The neurosurgery residents did not show a statistically significant difference in their surgical skills between the pre- and postcall states (p < 0.3, p < 0.4, and p < 0.2 for movement smoothness, time elapsed, and cognitive errors, respectively). The mean decrement for all residents in the postcall condition was 13.1%. CONCLUSIONS: Postcall fatigue is associated with a marginal decrease in proficiency during simulated surgery in neurosurgery residents. In a similar study, general surgery residents showed a statistically significant decrement of 27.3% in the postcall condition. The impact of fatigue on different specialties should be further investigated prior to implementation of a national physician work-hour policy.

Honing a surgeon's decision making skills in the presence of mechanical tasks.

BACKGROUND: Although surgeons perform procedures and tasks under a significant amount of cognitive load, current simulators focus on training psychomotor skills in isolation. This may limit the transfer of learned skills to actual surgical environments. METHODS: Visuohaptic simulations were created that required participants to hone psychomotor skills in the presence of cognitive load. A control group (n = 7) trained on conventional skills simulator and their performance was compared with the experimental group (n = 7) in terms of learning curves and performance on a transfer task. RESULTS: The experimental group performed significantly better than the control group in terms of both learning curves and performance on the transfer task. CONCLUSIONS: Simulations that combine psychomotor and cognitive skills training are more effective than simulators that focus on psychomotor skills in isolation.

Are the yips a task-specific dystonia or "golfer's cramp"?

This study compared golfers with and without the yips using joint movement and surface electromyographic detectors. Fifty golfers (25 with and 25 without complaints of the yips) were studied while putting. All putts were videotaped. Surface electromyography assessed arm cocontraction. A CyberGlove II (Immersion Technologies, Palo Alto, CA) assessed right-arm angular movements. Primary analysis was done by subjective complaint of the yips, whereas secondary analysis was done by video evidence of an involuntary movement. When grouped by subjective complaints, there were no differences in any movement parameter. When grouped by video evidence of an involuntary movement, yips cases had more (P < 0.001) angular movement in wrist pronation/supination and a trend (P = 0.08) for wrist flexor/extensor cocontraction (yips: 7 of 17, 41.2%; no yips: 6 of 33, 18.2%). Golfers with video evidence of an involuntary movement while putting have excessive rotation of the right wrist in a pronation/supination motion and, as previously reported, a trend for wrist flexor/extensor cocontraction.

Integrative gaming: a framework for sustainable game-based diabetes management.

Obesity and diabetes have reached epidemic proportions in both developing and developed nations. While doctors and caregivers stress the importance of physical exercise in maintaining a healthy lifestyle, many people have difficulty subscribing to a healthy lifestyle. Virtual reality games offer a potentially exciting aid in accelerating and sustaining behavior change. However, care needs to be taken to develop sustainable models of employing games for the management of diabetes and obesity. In this article, we propose an integrative gaming paradigm designed to combine multiple activities involving physical exercises and cognitive skills through a game-based storyline. The persuasive story acts as a motivational binder that enables a user to perform multiple activities such as running, cycling, and problem solving. These activities guide a virtual character through different stages of the game. While performing the activities in the games, users wear sensors that can measure movement (accelerometers, gyrometers, magnetometers) and sense physiological measures (heart rate, pulse oximeter oxygen saturation). These measures drive the game and are stored and analyzed on a cloud computing platform. A prototype integrative gaming system is described and design considerations are discussed. The system is highly configurable and allows researchers to build games for the system with ease and drive the games with different types of activities. The capabilities of the system allow for engaging and motivating the user in the long term. Clinicians can employ the system to collect clinically relevant data in a seamless manner.

Deviations from protocol in a complex trauma environment: errors or innovations?

Protocol standardizations are important for consistent and safe practices. However, complex clinical environments are highly dynamic in nature and often require clinicians, confronted with non-standard situations, to adjust and deviate from standard protocol. Some of these deviations are errors which can result in harmful outcomes. On the other hand, some of the deviations can be innovations, which are dynamic adjustments to the protocols made by people to adapt the current operational conditions and achieve high accuracy and efficiency. However, there is very little known about the underlying cognitive processes that are related to errors and innovations. In this study we investigate the extent to which deviations are classified as errors or innovations, as a function of expertise in a trauma setting. Field observations were conducted in a Level 1 trauma unit. A total of 10 trauma cases were observed and collected data was analyzed using measures that included customized activity-error-innovation ontology, timestamps and expertise of the team members. The results show that expertise of the caregivers and criticality of a patient's condition in critical care environment influence the number and type of deviations from standard protocol. Experts' deviations were a combination of errors and innovations; whereas the novices' deviations were mostly errors. This research suggests that a novel approach must be taken into consideration for the design of protocols (including standards) and compliance measurements in complex clinical environments.

Impact of fatigue on neurophysiologic measures of surgical residents.

BACKGROUND: To gain additional insight into the impact of fatigue on surgery resident proficiency, we set out to quantify its impact on behavioral and neurophysiologic measures. STUDY DESIGN: Simulations were first created using a visio-haptic joystick attached to a surgical instrument (tool) that allows realistic interactions. Before baseline (pre-call) and after call, 7 PGY1 surgery residents performed simulation tasks that required varying levels of psychomotor and cognitive skill. Residents completed 3 sessions per week for 4 weeks. Surgical proficiency was established using 5 metrics: hand movement smoothness; instrument movement smoothness; time to task completion; gesture level proficiency and cognitive errors. Data (percent change from baseline, pre-call) were analyzed using repeated measures ANOVA. While performing these tasks, the residents also wore an EEG cap (B-Alert; Advanced Brain Monitoring), the data from which provided second to second insight into the effects of workload, distraction, and attention on task performance. Mean (±SD) pre-call and post-call values for each were analyzed using repeated measures ANOVA. RESULTS: Residents experienced significant (p < 0.014) post-call erosions in surgical proficiency, punctuated by dramatic increases in cognitive errors. EEG-based attention scores showed a significant (p < 0.014) concomitant decrement of 40%; distraction/drowsiness scores increased by 91%; and workload score increased by 51%. CONCLUSIONS: Fatigue adversely affects PGY1 resident surgical proficiency and neurophysiologic performance.

The validity of take-home surgical simulators to enhance resident technical skill proficiency.

BACKGROUND: It is unknown whether surgical residents who learn minimal-access surgery skills in an unstructured environment (ie, at home), will develop a technical skill set that rivals that of those trained in the more traditional, structured learning environment. METHODS: Seven surgery residents were provided structured learning through didactic and hands-on skills training sessions and consistent supervision throughout training. A second group of 7 residents participated in an unstructured learning curriculum of training without supervision. End points were determined at the end of training using a standardized simulator based on predetermined performance measures. RESULTS: Both groups achieved high task scores, with comparable scores on gesture proficiency, hand movement smoothness, instrument movement smoothness, errors, and time elapsed. There was no significant difference between group differences in final skills scores. CONCLUSIONS: Unstructured learning is equally effective in delivering quality skills training when compared with structured training.

Toward effective pediatric minimally invasive surgical simulation.

BACKGROUND/PURPOSE: Simulation is increasingly being recognized as an important tool in the training and evaluation of surgeons. Currently, there is no simulator that is specific to pediatric minimally invasive surgery (MIS). A fundamental technical difference between adult and pediatric MIS is the degree of motion scaling. Smaller instruments and areas of dissection under greater optical magnification require finer, more precise hand movements. We hypothesized that this can be used to detect differences in skills proficiency between pediatric and general surgeons. METHODS: We programmed a virtual reality simulation of intracorporeal suturing with modes that used motion scaling to mimic conditions of either adult or pediatric MIS. The participants consisted of pediatric and general surgeons who wore motion-sensing gloves. Metrics included time elapsed, penetration errors, tool movement smoothness, hand movement smoothness, and gesture level proficiency. RESULTS: For all measures, pediatric surgeons demonstrated superior proficiency on exercises conducted in pediatric conditions (P < .05). Performance in adult conditions was similar between the 2 groups. CONCLUSION: Pediatric surgeons possess unique skills compared with general surgeons that relate to the technical challenges they routinely face, reinforcing the need for a surgical simulator specific to pediatric MIS. This validates our simulator and the manipulation of motion scaling as a useful training tool.

Initial experiences in embedding core competency education in entry-level surgery residents through a nonclinical rotation.

BACKGROUND: Health care continues to expand in scope and in complexity. In this changing environment, residents are challenged with understanding its intricacies and the impact it will have on their professional activities and careers. AIM: Embedding each of the competency elements in residents in a meaningful way remains a challenge for many surgery residency program directors. METHODS: We established a nonclinical rotation to provide surgery postgraduate year-1 (PGY-1) residents with a structured, multifaceted, largely self-directed curriculum into which each of the 6 core competencies are woven. Posttesting strategies were established for most curricular experiences to ensure to the greatest possible extent that each resident will have achieved an acceptable level of understanding of each of the competency areas before being given credit for the rotation. RESULTS: By uniformly exceeding satisfactory scores on respective objective analyses, residents demonstrated an increased (at least short-term) understanding of each of the assessed competency areas. CONCLUSION: Our project sought to address a prior lack of opportunity for our residents to develop a sound foundation for our residents in systems-based practice. Our new rotation addresses systems-based practice in several different learning environments, including emergency medical service ride-along, sentinel event participation, and hospice visits. Several research projects have enhanced the overall learning program. Our experience shows that a rotation dedicated to competency training can provide an innovative and engaging means of teaching residents the value of each element.

Toward automated workflow analysis and visualization in clinical environments.

Lapses in patient safety have been linked to unexpected perturbations in clinical workflow. The effectiveness of workflow analysis becomes critical to understanding the impact of these perturbations on patient outcome. The typical methods used for workflow analysis, such as ethnographic observations and interviewing, are limited in their ability to capture activities from different perspectives simultaneously. This limitation, coupled with the complexity and dynamic nature of clinical environments makes understanding the nuances of clinical workflow difficult. The methods proposed in this research aim to provide a quantitative means of capturing and analyzing workflow. The approach taken utilizes recordings of motion and location of clinical teams that are gathered using radio identification tags and observations. This data is used to model activities in critical care environments. The detected activities can then be replayed in 3D virtual reality environments for further analysis and training. Using this approach, the proposed system augments existing methods of workflow analysis, allowing for capture of workflow in complex and dynamic environments. The system was tested with a set of 15 simulated clinical activities that when combined represent workflow in trauma units. A mean recognition rate of 87.5% was obtained in automatically recognizing the activities.

Quantitative evaluation of retention of surgical skills learned in simulation.

BACKGROUND: While initial results suggest that simulation does promote learning, there is a dearth of studies that define the extent to which skills learned through simulation are retained. METHODS: Residents skills were measured upon completion of an initial simulation training (baseline scores) and then every month for 6 months. Analysis was also performed to identify the number of iterations of practice required to regain baseline scores. RESULTS: While skill scores did not deteriorate from baseline after the first 3 months (p = 0.61, p = 0.44, p = 0.2, respectively), all scores (except time elapsed) reflected significant deterioration from the fourth month onward (p < 0.05, p < 0.032, p < 0.02). However the number of practice sessions required to regain baseline scores was significantly less than that required to achieve the baseline skill set (p < 0.0003). CONCLUSIONS: Skills learned through simulation show significant deterioration over long periods of time, suggesting that periodic retraining of skills may be necessary to maintain surgical proficiency.

Effects of duty hours and time of day on surgery resident proficiency.

BACKGROUND: Night floats have evolved in the era of limited resident work hours. This study was designed to define the effect of restricted nighttime duty hours on the psychomotor and cognitive skills of surgery residents. METHODS: To quantify the effect of fatigue on the skills of residents on day-shift and night-float rotations, residents were asked to complete visuohaptic simulations before and after 12-hour duty periods and to rate their fatigue level with questionnaires. RESULTS: Both groups showed significant decrements in proficiency measures after their shifts compared with baseline. The night-float group showed more significant declines (P < .05) in all areas assessed than the day-shift group. The night-float group was significantly less proficient in cognitive tasks after their shifts compared with the day-shift group. CONCLUSIONS: The deterioration of surgical proficiency is to a degree dependent on the time of day during which call occurs, not solely on the length of call.

A virtual reality simulator for orthopedic basic skills: a design and validation study.

Orthopedic drilling as a skill demands high levels of dexterity and expertise from the surgeon. It is a basic skill that is required in many orthopedic procedures. Inefficient drilling can be a source of avoidable medical errors that may lead to adverse events. It is hence important to train and evaluate residents in safe environments for this skill. This paper presents a virtual orthopedic drilling simulator that was designed to provide visiohaptic interaction with virtual bones. The simulation provides a realistic basic training environment for orthopedic surgeons. It contains modules to track and analyze movements of surgeons, in order to determine their surgical proficiency. The simulator was tested with senior surgeons, residents and medical students for validation purposes. Through the multi-tiered testing strategy it was shown that the simulator was able to produce a learning effect that transfers to real-world drilling. Further, objective measures of surgical performance were found to be able to differentiate between experts and novices.