Deep Learning for Video-Based Assessment in Surgery.

This surgical innovation explains how applying deep neural networks could ensure the continued use of video-based assessment.

Mechanical behavior of full-thickness burn human skin is rate-independent.

Skin tissue is recognized to exhibit rate-dependent mechanical behavior under various loading conditions. Here, we report that the full-thickness burn human skin exhibits rate-independent behavior under uniaxial tensile loading conditions. Mechanical properties, namely, ultimate tensile stress, ultimate tensile strain, and toughness, and parameters of Veronda-Westmann hyperelastic material law were assessed via uniaxial tensile tests. Univariate hypothesis testing yielded no significant difference (p > 0.01) in the distributions of these properties for skin samples loaded at three different rates of 0.3 mm/s, 2 mm/s, and 8 mm/s. Multivariate multiclass classification, employing a logistic regression model, failed to effectively discriminate samples loaded at the aforementioned rates, with a classification accuracy of only 40%. The median values for ultimate tensile stress, ultimate tensile strain, and toughness are computed as 1.73 MPa, 1.69, and 1.38 MPa, respectively. The findings of this study hold considerable significance for the refinement of burn care training protocols and treatment planning, shedding new light on the unique, rate-independent behavior of burn skin.

One-shot skill assessment in high-stakes domains with limited data via meta learning.

Deep Learning (DL) has achieved robust competency assessment in various high-stakes fields. However, the applicability of DL models is often hampered by their substantial data requirements and confinement to specific training domains. This prevents them from transitioning to new tasks where data is scarce. Therefore, domain adaptation emerges as a critical element for the practical implementation of DL in real-world scenarios. Herein, we introduce A-VBANet, a novel meta-learning model capable of delivering domain-agnostic skill assessment via one-shot learning. Our methodology has been tested by assessing surgical skills on five laparoscopic and robotic simulators and real-life laparoscopic cholecystectomy. Our model successfully adapted with accuracies up to 99.5 % in one-shot and 99.9 % in few-shot settings for simulated tasks and 89.7 % for laparoscopic cholecystectomy. This study marks the first instance of a domain-agnostic methodology for skill assessment in critical fields setting a precedent for the broad application of DL across diverse real-life domains with limited data.

Endoscopic sleeve gastroplasty: stomach location and task classification for evaluation using artificial intelligence.

PURPOSE: We have previously developed grading metrics to objectively measure endoscopist performance in endoscopic sleeve gastroplasty (ESG). One of our primary goals is to automate the process of measuring performance. To achieve this goal, the repeated task being performed (grasping or suturing) and the location of the endoscopic suturing device in the stomach (Incisura, Anterior Wall, Greater Curvature, or Posterior Wall) need to be accurately recorded. METHODS: For this study, we populated our dataset using screenshots and video clips from experts carrying out the ESG procedure on ex vivo porcine specimens. Data augmentation was used to enlarge our dataset, and synthetic minority oversampling (SMOTE) to balance it. We performed stomach localization for parts of the stomach and task classification using deep learning for images and computer vision for videos. RESULTS: Classifying the stomach's location from the endoscope without SMOTE for images resulted in 89% and 84% testing and validation accuracy, respectively. For classifying the location of the stomach from the endoscope with SMOTE, the accuracies were 97% and 90% for images, while for videos, the accuracies were 99% and 98% for testing and validation, respectively. For task classification, the accuracies were 97% and 89% for images, while for videos, the accuracies were 100% for both testing and validation, respectively. CONCLUSION: We classified the four different stomach parts manipulated during the ESG procedure with 97% training accuracy and classified two repeated tasks with 99% training accuracy with images. We also classified the four parts of the stomach with a 99% training accuracy and two repeated tasks with a 100% training accuracy with video frames. This work will be essential in automating feedback mechanisms for learners in ESG.

Validation of a Virtual Simulator With Haptic Feedback for Endotracheal Intubation Training.

INTRODUCTION: Endotracheal intubation (ETI) is a procedure that varies in difficulty because of patient characteristics and clinical conditions. Existing physical simulators do not encompass these variations. The Virtual Airway Skills Trainer for Endotracheal Intubation (VAST-ETI) was developed to provide different patient characteristics and high-fidelity haptic feedback to improve training. METHODS: We demonstrate the effectiveness of VAST-ETI as a training and evaluation tool for ETI. Construct validation was evaluated by scoring the performance of experts ( N = 15) and novices ( N = 15) on the simulator to ensure its ability to distinguish technical proficiency. Convergent and predictive validity were evaluated by performing a learning curve study, in which a group of novices ( N = 7) were trained for 2 weeks using VAST-ETI and then compared with a control group ( N = 9). RESULTS: The VAST-ETI was able to distinguish between expert and novice based on mean simulator scores ( t [88] = -6.61, P < 0.0005). When used during repeated practice, individuals demonstrated a significant increase in their score on VAST-ETI over the learning period ( F [11,220] = 7206, P < 0.001); however when compared with a control group, there was not a significant interaction effect on the simulator score. There was a significant difference between the simulator-trained and control groups ( t [12.85] = -2.258, P = 0.042) when tested in the operating room. CONCLUSIONS: Our results demonstrate the effectiveness of virtual simulation with haptic feedback for assessing performance and training of ETI. The simulator was not able to differentiate performance between more experienced trainees and experts because of limits in simulator difficulty.

Distinguishing Laparoscopic Surgery Experts from Novices Using EEG Topographic Features.

The study aimed to differentiate experts from novices in laparoscopic surgery tasks using electroencephalogram (EEG) topographic features. A microstate-based common spatial pattern (CSP) analysis with linear discriminant analysis (LDA) was compared to a topography-preserving convolutional neural network (CNN) approach. Expert surgeons (N = 10) and novice medical residents (N = 13) performed laparoscopic suturing tasks, and EEG data from 8 experts and 13 novices were analysed. Microstate-based CSP with LDA revealed distinct spatial patterns in the frontal and parietal cortices for experts, while novices showed frontal cortex involvement. The 3D CNN model (ESNet) demonstrated a superior classification performance (accuracy > 98%, sensitivity 99.30%, specificity 99.70%, F1 score 98.51%, MCC 97.56%) compared to the microstate based CSP analysis with LDA (accuracy ~90%). Combining spatial and temporal information in the 3D CNN model enhanced classifier accuracy and highlighted the importance of the parietal-temporal-occipital association region in differentiating experts and novices.

Cognitive Task Analysis of Escharotomy.

INTRODUCTION: With the Army's emerging doctrine of prolonged field care, and with burns being a common injury among soldiers, non-expert providers must be trained to perform escharotomy when indicated. However, the existing physical simulators and training protocols are not sufficient for training non-experts for performing effective escharotomy. Hence, to provide guidance in developing realistic escharotomy simulators and effective training protocols suitable for prolonged field care, a cognitive task analysis (CTA) is needed. This work aims to obtain educative information from expert burn surgeons regarding escharotomy procedures via the CTA. MATERIALS AND METHODS: The CTA was done by interviewing five subject matter experts with experience in performing escharotomy ranging from 20 to over 100 procedures and analyzing their responses. Interview questions were developed to obtain educative information from expert burn surgeons regarding the escharotomy procedure. A "gold standard protocol" was developed based on the CTA of each of the subject matter experts. RESULTS: The CTA helped identify general themes, including objectives, conditions that mandate escharotomy, signs of successful escharotomy, precautions, challenges, decisions, and performance standards, and specific learning goals such as the use of equipment, vital signs, performing the procedure, and preoperative and postoperative care. A unique aspect of this CTA is that it identifies the background information and preparations that could be useful to the practitioners at various levels of expertise. CONCLUSIONS: The CTA enabled us to compile a "gold standard protocol" for escharotomy that may serve as a guide for practitioners at various levels of expertise. This information will provide a framework for escharotomy training systems and simulators.

Specular Reflection Removal for 3D Reconstruction of Tissues using Endoscopy Videos.

Endoscopy is widely employed for diagnostic examination of the interior of organs and body cavities and numerous surgical interventions. Still, the inability to correlate individual 2D images with 3D organ morphology limits its applications, especially in intra-operative planning and navigation, disease physiology, cancer surveillance, etc. As a result, most endoscopy videos, which carry enormous data potential, are used only for real-time guidance and are discarded after collection. We present a complete method for the 3D reconstruction of inner organs that suggests image extraction techniques from endoscopic videos and a novel image pre-processing technique to reconstruct and visualize a 3D model of organs from an endoscopic video. We use advanced computer vision methods and do not require any modifications to the clinical-grade endoscopy hardware. We have also formalized an image acquisition protocol through experimentation with a calibrated test bed. We validate the accuracy and robustness of our reconstruction using a test bed with known ground truth. Our method can significantly contribute to endoscopy-based diagnostic and surgical procedures using comprehensive tissue and tumor 3D visualization.

Assessment of Surgical Tasks Using Neuroimaging Dataset (ASTaUND).

Functional near-infrared spectroscopy (fNIRS) is a neuroimaging tool for studying brain activity in mobile subjects. Open-access fNIRS datasets are limited to simple and/or motion-restricted tasks. Here, we report a fNIRS dataset acquired on mobile subjects performing Fundamentals of Laparoscopic Surgery (FLS) tasks in a laboratory environment. Demonstrating competency in the FLS tasks is a prerequisite for board certification in general surgery in the United States. The ASTaUND data set was acquired over four different studies. We provide the relevant information about the hardware, FLS task execution protocols, and subject demographics to facilitate the use of this open-access data set. We also provide the concurrent FLS scores, a quantitative metric for surgical skill assessment developed by the FLS committee. This data set is expected to support the growing field of assessing surgical skills via neuroimaging data and provide an example of data processing pipeline for use in realistic, non-restrictive environments.

What are essential laparoscopic skills these days? Results of the SAGES Fundamentals of Laparoscopic Surgery (FLS) Committee technical skills survey.

INTRODUCTION: The Fundamentals of Laparoscopic Surgery (FLS) program tests basic knowledge and skills required to perform laparoscopic surgery. Educational experiences in laparoscopic training and development of associated competencies have evolved since FLS inception, making it important to review the definition of fundamental laparoscopic skills. The Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) assigned an FLS Technical Skills Working Group to characterize technical skills used in basic laparoscopic surgery in current practice contexts and their possible application to future FLS tests. METHODS: A group of subject matter experts defined an inventory of 65 laparoscopic skills using a Nominal Group Technique. From these, a survey was developed rating these items for importance, frequency of use, and priority for testing for FLS certification. This survey was distributed to SAGES members, recent recipients of FLS certification, and members of the Association of Program Directors in Surgery (APDS). Results were collected using a secure web-based survey platform. RESULTS: Complete data were available for 1742 surveys. Of these, 1143 comprised results for post-residency participants who performed advanced procedures. Seventeen competencies were identified for FLS testing prioritization by determining the proportion of respondents who identified them of highest priority, at median (50th percentile) of the maximum survey scale rating. These included basic peritoneal access, laparoscope and instrument use, tissue manipulation, and specific problem management skills. Sixteen could be used to show appropriateness of the domain construct by confirmatory factor analysis. Of these 8 could be characterized as manipulative tasks. Of these 5 mapped to current FLS tasks. CONCLUSIONS: This survey-identified competencies, some of which are currently assessed in FLS, with a high level of priority for testing. Further work is needed to determine if this should prompt consideration of changes or additions to the FLS technical skills test component.

Changes in functional neuroimaging measures as novices gain proficiency on the fundamentals of laparoscopic surgery suturing task.

Significance: As trainees practice fundamental surgical skills, they typically rely on performance measures such as time and errors, which are limited in their sensitivity. Aim: The goal of our study was to evaluate the use of portable neuroimaging measures to map the neural processes associated with learning basic surgical skills. Approach: Twenty-one subjects completed 15 sessions of training on the fundamentals of laparoscopic surgery (FLS) suture with intracorporeal knot-tying task in a box trainer. Functional near infrared spectroscopy data were recorded using an optode montage that covered the prefrontal and sensorimotor brain areas throughout the task. Average oxy-hemoglobin (HbO) changes were determined for repetitions performed during the first week of training compared with the third week of training. Statistical differences between the time periods were evaluated using a general linear model of the HbO changes. Results: Average performance scores across task repetitions increased significantly from the first day to the last day of training ( p < 0.01 ). During the first day of training, there was significant lateral prefrontal cortex (PFC) activation. On the final day, significant activation was observed in the PFC, as well as the sensorimotor areas. When comparing the two periods, significant differences in activation ( p < 0.05 ) were found for the right medial PFC and the right inferior parietal gyrus. While gaining proficiency, trainees activated the perception-action cycle to build a perceptual model and then apply the model to improve task execution. Conclusions: Learners engaged the sensorimotor areas more substantially as they developed skill on the FLS suturing task. These findings are consistent with findings for the FLS pattern cutting task and contribute to the development of objective metrics for skill evaluation.

A statistical machine learning approach linking molecular conformational changes to altered mechanical characteristics of skin due to thermal injury.

This article develops statistical machine learning models to predict the mechanical properties of skin tissue subjected to thermal injury based on the Raman spectra associated with conformational changes of the molecules in the burned tissue. Ex vivo porcine skin tissue samples were exposed to controlled burn conditions at 200 °F for five different durations: (i) 10s, (ii) 20s, (iii) 30s, (iv) 40s, and (v) 50s. For each burn condition, Raman spectra of wavenumbers 500-2000 cm-1 were measured from the tissue samples, and tensile testing on the same samples yielded their material properties, including, ultimate tensile strain, ultimate tensile stress, and toughness. Partial least squares regression models were established such that the Raman spectra, describing conformational changes in the tissue, could accurately predict ultimate tensile stress, toughness, and ultimate tensile strain of the burned skin tissues with R2 values of 0.8, 0.8, and 0.7, respectively, using leave-two-out cross validation scheme. An independent assessment of the resultant models showed that amino acids, proteins & lipids, and amide III components of skin tissue significantly influence the prediction of the properties of the burned skin tissue. In contrast, amide I has a lesser but still noticeable effect. These results are consistent with similar observations found in the literature on the mechanical characterization of burned skin tissue.

Intelligent Virtual Operating Room for Enhancing Nontechnical Skills.

This article discusses an intelligent immersive virtual operating room to enable teams to train in a distributed fashion wearing head-mounted displays.

Skill-level classification and performance evaluation for endoscopic sleeve gastroplasty.

BACKGROUND: We previously developed grading metrics for quantitative performance measurement for simulated endoscopic sleeve gastroplasty (ESG) to create a scalar reference to classify subjects into experts and novices. In this work, we used synthetic data generation and expanded our skill level analysis using machine learning techniques. METHODS: We used the synthetic data generation algorithm SMOTE to expand and balance our dataset of seven actual simulated ESG procedures using synthetic data. We performed optimization to seek optimum metrics to classify experts and novices by identifying the most critical and distinctive sub-tasks. We used support vector machine (SVM), AdaBoost, K-nearest neighbors (KNN) Kernel Fisher discriminant analysis (KFDA), random forest, and decision tree classifiers to classify surgeons as experts or novices after grading. Furthermore, we used an optimization model to create weights for each task and separate the clusters by maximizing the distance between the expert and novice scores. RESULTS: We split our dataset into a training set of 15 samples and a testing dataset of five samples. We put this dataset through six classifiers, SVM, KFDA, AdaBoost, KNN, random forest, and decision tree, resulting in 0.94, 0.94, 1.00, 1.00, 1.00, and 1.00 accuracy, respectively, for training and 1.00 accuracy for the testing results for SVM and AdaBoost. Our optimization model maximized the distance between the expert and novice groups from 2 to 53.72. CONCLUSION: This paper shows that feature reduction, in combination with classification algorithms such as SVM and KNN, can be used in tandem to classify endoscopists as experts or novices based on their results recorded using our grading metrics. Furthermore, this work introduces a non-linear constraint optimization to separate the two clusters and find the most important tasks using weights.

Video-based formative and summative assessment of surgical tasks using deep learning.

To ensure satisfactory clinical outcomes, surgical skill assessment must be objective, time-efficient, and preferentially automated-none of which is currently achievable. Video-based assessment (VBA) is being deployed in intraoperative and simulation settings to evaluate technical skill execution. However, VBA is manual, time-intensive, and prone to subjective interpretation and poor inter-rater reliability. Herein, we propose a deep learning (DL) model that can automatically and objectively provide a high-stakes summative assessment of surgical skill execution based on video feeds and low-stakes formative assessment to guide surgical skill acquisition. Formative assessment is generated using heatmaps of visual features that correlate with surgical performance. Hence, the DL model paves the way for the quantitative and reproducible evaluation of surgical tasks from videos with the potential for broad dissemination in surgical training, certification, and credentialing.

Portable neuroimaging differentiates novices from those with experience for the Fundamentals of Laparoscopic Surgery (FLS) suturing with intracorporeal knot tying task.

BACKGROUND: The goal of this study was to compare the brain activation patterns of experienced and novice individuals when performing the Fundamentals of Laparoscopic Surgery (FLS) suture with intracorporeal knot tying task, which requires bimanual motor control. METHODS: Twelve experienced and fourteen novice participants completed this cross-sectional observational study. Participants performed three repetitions of the FLS suture with intracorporeal knot tying task in a standard box trainer. Functional near infrared spectroscopy (fNIRS) data was recorded using an optode montage that covered the prefrontal and sensorimotor brain areas throughout the task. Data processing was conducted using the HOMER3 and AtlasViewer toolboxes to determine the oxy-hemoglobin (HbO) and deoxyhemoglobin (HbR) concentrations. The hemodynamic response function based on HbO changes during the task relative to the resting state was averaged for each repetition and by participant. Group-level differences were evaluated using a general linear model of the HbO changes with significance set at p < 0.05. RESULTS: The average performance score for the experienced group was significantly higher than the novice group (p < 0.01). There were significant cortical activations (p < 0.05) in the prefrontal and sensorimotor areas for the experienced and novice groups. Areas of statistically significant differences in activation included the right dorsolateral prefrontal cortex (PFC), the right precentral gyrus, and the right postcentral gyrus. CONCLUSIONS: Portable neuroimaging allowed for the differentiation of brain regions activated by experienced and novice participants for a complex surgical motor task. This information can be used to support the objective evaluation of expertise during surgical skills training and assessment.

Objective metrics for hand-sewn bowel anastomoses can differentiate novice from expert surgeons.

BACKGROUND: Assessing performance automatically in a virtual reality trainer or from recorded videos is advantageous but needs validated objective metrics. The purpose of this study is to obtain expert consensus and validate task-specific metrics developed for assessing performance in double-layered end-to-end anastomosis. MATERIALS AND METHODS: Subjects were recruited into expert (PGY 4-5, colorectal surgery residents, and attendings) and novice (PGY 1-3) groups. Weighted average scores of experts for each metric item, completion time, and the total scores computed using global and task-specific metrics were computed for assessment. RESULTS: A total of 43 expert surgeons rated our task-specific metric items with weighted averages ranging from 3.33 to 4.5 on a 5-point Likert scale. A total of 20 subjects (10 novices and 10 experts) participated in validation study. The novice group completed the task significantly more slowly than the experienced group (37.67 ± 7.09 vs 25.47 ± 7.82 min, p = 0.001). In addition, both the global rating scale (23.47 ± 4.28 vs 28.3 ± 3.85, p = 0.016) and the task-specific metrics showed a significant difference in performance between the two groups (38.77 ± 2.83 vs 42.58 ± 4.56 p = 0.027) following partial least-squares (PLS) regression. Furthermore, PLS regression showed that only two metric items (Stay suture tension and Tool handling) could reliably differentiate the performance between the groups (20.41 ± 2.42 vs 24.28 ± 4.09 vs, p = 0.037). CONCLUSIONS: Our study shows that our task-specific metrics have significant discriminant validity and can be used to evaluate the technical skills for this procedure.

Brain-behavior analysis of transcranial direct current stimulation effects on a complex surgical motor task.

Transcranial Direct Current Stimulation (tDCS) has demonstrated its potential in enhancing surgical training and performance compared to sham tDCS. However, optimizing its efficacy requires the selection of appropriate brain targets informed by neuroimaging and mechanistic understanding. Previous studies have established the feasibility of using portable brain imaging, combining functional near-infrared spectroscopy (fNIRS) with tDCS during Fundamentals of Laparoscopic Surgery (FLS) tasks. This allows concurrent monitoring of cortical activations. Building on these foundations, our study aimed to explore the multi-modal imaging of the brain response using fNIRS and electroencephalogram (EEG) to tDCS targeting the right cerebellar (CER) and left ventrolateral prefrontal cortex (PFC) during a challenging FLS suturing with intracorporeal knot tying task. Involving twelve novices with a medical/premedical background (age: 22-28 years, two males, 10 females with one female with left-hand dominance), our investigation sought mechanistic insights into tDCS effects on brain areas related to error-based learning, a fundamental skill acquisition mechanism. The results revealed that right CER tDCS applied to the posterior lobe elicited a statistically significant (q < 0.05) brain response in bilateral prefrontal areas at the onset of the FLS task, surpassing the response seen with sham tDCS. Additionally, right CER tDCS led to a significant (p < 0.05) improvement in FLS scores compared to sham tDCS. Conversely, the left PFC tDCS did not yield a statistically significant brain response or improvement in FLS performance. In conclusion, right CER tDCS demonstrated the activation of bilateral prefrontal brain areas, providing valuable mechanistic insights into the effects of CER tDCS on FLS peformance. These insights motivate future investigations into the effects of CER tDCS on error-related perception-action coupling through directed functional connectivity studies.

Split & Join: An Efficient Approach for Simulating Stapled Intestinal Anastomosis in Virtual Reality.

Colorectal cancer is a life-threatening disease. It is the second leading cause of cancer-related deaths in the United States. Stapled anastomosis is a rapid treatment for colorectal cancer and other intestinal diseases and has become an integral part of routine surgical practice. However, to the best of our knowledge, there is no existing work simulating intestinal anastomosis that often involves sophisticated soft tissue manipulations such as cutting and stitching. In this paper, for the first time, we propose a novel split and join approach to simulate a side-to-side stapled intestinal anastomosis in virtual reality. We mimic the intestine model using a new hybrid representation - a grid-linked particles model for physics simulation and a surface mesh for rendering. The proposed split and join operations handle the updates of both the grid-linked particles model and the surface mesh during the anastomosis procedure. The simulation results demonstrate the feasibility of the proposed approach in simulating intestine models and the side-to-side anastomosis operation.

Error-related brain state analysis using electroencephalography in conjunction with functional near-infrared spectroscopy during a complex surgical motor task.

Error-based learning is one of the basic skill acquisition mechanisms that can be modeled as a perception-action system and investigated based on brain-behavior analysis during skill training. Here, the error-related chain of mental processes is postulated to depend on the skill level leading to a difference in the contextual switching of the brain states on error commission. Therefore, the objective of this paper was to compare error-related brain states, measured with multi-modal portable brain imaging, between experts and novices during the Fundamentals of Laparoscopic Surgery (FLS) "suturing and intracorporeal knot-tying" task (FLS complex task)-the most difficult among the five psychomotor FLS tasks. The multi-modal portable brain imaging combined functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) for brain-behavior analysis in thirteen right-handed novice medical students and nine expert surgeons. The brain state changes were defined by quasi-stable EEG scalp topography (called microstates) changes using 32-channel EEG data acquired at 250 Hz. Six microstate prototypes were identified from the combined EEG data from experts and novices during the FLS complex task that explained 77.14% of the global variance. Analysis of variance (ANOVA) found that the proportion of the total time spent in different microstates during the 10-s error epoch was significantly affected by the skill level (p < 0.01), the microstate type (p < 0.01), and the interaction between the skill level and the microstate type (p < 0.01). Brain activation based on the slower oxyhemoglobin (HbO) changes corresponding to the EEG band power (1-40 Hz) changes were found using the regularized temporally embedded Canonical Correlation Analysis of the simultaneously acquired fNIRS-EEG signals. The HbO signal from the overlying the left inferior frontal gyrus-opercular part, left superior frontal gyrus-medial orbital, left postcentral gyrus, left superior temporal gyrus, right superior frontal gyrus-medial orbital cortical areas showed significant (p < 0.05) difference between experts and novices in the 10-s error epoch. We conclude that the difference in the error-related chain of mental processes was the activation of cognitive top-down attention-related brain areas, including left dorsolateral prefrontal/frontal eye field and left frontopolar brain regions, along with a 'focusing' effect of global suppression of hemodynamic activation in the experts, while the novices had a widespread stimulus(error)-driven hemodynamic activation without the 'focusing' effect.