Challenges in multi-centric generalization: phase and step recognition in Roux-en-Y gastric bypass surgery.

PURPOSE: Most studies on surgical activity recognition utilizing artificial intelligence (AI) have focused mainly on recognizing one type of activity from small and mono-centric surgical video datasets. It remains speculative whether those models would generalize to other centers. METHODS: In this work, we introduce a large multi-centric multi-activity dataset consisting of 140 surgical videos (MultiBypass140) of laparoscopic Roux-en-Y gastric bypass (LRYGB) surgeries performed at two medical centers, i.e., the University Hospital of Strasbourg, France (StrasBypass70) and Inselspital, Bern University Hospital, Switzerland (BernBypass70). The dataset has been fully annotated with phases and steps by two board-certified surgeons. Furthermore, we assess the generalizability and benchmark different deep learning models for the task of phase and step recognition in 7 experimental studies: (1) Training and evaluation on BernBypass70; (2) Training and evaluation on StrasBypass70; (3) Training and evaluation on the joint MultiBypass140 dataset; (4) Training on BernBypass70, evaluation on StrasBypass70; (5) Training on StrasBypass70, evaluation on BernBypass70; Training on MultiBypass140, (6) evaluation on BernBypass70 and (7) evaluation on StrasBypass70. RESULTS: The model's performance is markedly influenced by the training data. The worst results were obtained in experiments (4) and (5) confirming the limited generalization capabilities of models trained on mono-centric data. The use of multi-centric training data, experiments (6) and (7), improves the generalization capabilities of the models, bringing them beyond the level of independent mono-centric training and validation (experiments (1) and (2)). CONCLUSION: MultiBypass140 shows considerable variation in surgical technique and workflow of LRYGB procedures between centers. Therefore, generalization experiments demonstrate a remarkable difference in model performance. These results highlight the importance of multi-centric datasets for AI model generalization to account for variance in surgical technique and workflows. The dataset and code are publicly available at https://github.com/CAMMA-public/MultiBypass140.

CholecTriplet2022: Show me a tool and tell me the triplet - An endoscopic vision challenge for surgical action triplet detection.

Formalizing surgical activities as triplets of the used instruments, actions performed, and target anatomies is becoming a gold standard approach for surgical activity modeling. The benefit is that this formalization helps to obtain a more detailed understanding of tool-tissue interaction which can be used to develop better Artificial Intelligence assistance for image-guided surgery. Earlier efforts and the CholecTriplet challenge introduced in 2021 have put together techniques aimed at recognizing these triplets from surgical footage. Estimating also the spatial locations of the triplets would offer a more precise intraoperative context-aware decision support for computer-assisted intervention. This paper presents the CholecTriplet2022 challenge, which extends surgical action triplet modeling from recognition to detection. It includes weakly-supervised bounding box localization of every visible surgical instrument (or tool), as the key actors, and the modeling of each tool-activity in the form of triplet. The paper describes a baseline method and 10 new deep learning algorithms presented at the challenge to solve the task. It also provides thorough methodological comparisons of the methods, an in-depth analysis of the obtained results across multiple metrics, visual and procedural challenges; their significance, and useful insights for future research directions and applications in surgery.

Weakly Supervised Temporal Convolutional Networks for Fine-Grained Surgical Activity Recognition.

Automatic recognition of fine-grained surgical activities, called steps, is a challenging but crucial task for intelligent intra-operative computer assistance. The development of current vision-based activity recognition methods relies heavily on a high volume of manually annotated data. This data is difficult and time-consuming to generate and requires domain-specific knowledge. In this work, we propose to use coarser and easier-to-annotate activity labels, namely phases, as weak supervision to learn step recognition with fewer step annotated videos. We introduce a step-phase dependency loss to exploit the weak supervision signal. We then employ a Single-Stage Temporal Convolutional Network (SS-TCN) with a ResNet-50 backbone, trained in an end-to-end fashion from weakly annotated videos, for temporal activity segmentation and recognition. We extensively evaluate and show the effectiveness of the proposed method on a large video dataset consisting of 40 laparoscopic gastric bypass procedures and the public benchmark CATARACTS containing 50 cataract surgeries.

TRandAugment: temporal random augmentation strategy for surgical activity recognition from videos.

PURPOSE: Automatic recognition of surgical activities from intraoperative surgical videos is crucial for developing intelligent support systems for computer-assisted interventions. Current state-of-the-art recognition methods are based on deep learning where data augmentation has shown the potential to improve the generalization of these methods. This has spurred work on automated and simplified augmentation strategies for image classification and object detection on datasets of still images. Extending such augmentation methods to videos is not straightforward, as the temporal dimension needs to be considered. Furthermore, surgical videos pose additional challenges as they are composed of multiple, interconnected, and long-duration activities. METHODS: This work proposes a new simplified augmentation method, called TRandAugment, specifically designed for long surgical videos, that treats each video as an assemble of temporal segments and applies consistent but random transformations to each segment. The proposed augmentation method is used to train an end-to-end spatiotemporal model consisting of a CNN (ResNet50) followed by a TCN. RESULTS: The effectiveness of the proposed method is demonstrated on two surgical video datasets, namely Bypass40 and CATARACTS, and two tasks, surgical phase and step recognition. TRandAugment adds a performance boost of 1-6% over previous state-of-the-art methods, that uses manually designed augmentations. CONCLUSION: This work presents a simplified and automated augmentation method for long surgical videos. The proposed method has been validated on different datasets and tasks indicating the importance of devising temporal augmentation methods for long surgical videos.

Towards automatic verification of the critical view of the myopectineal orifice with artificial intelligence.

BACKGROUND: Visualization of key anatomical landmarks is required during surgical Trans Abdominal Pre Peritoneal repair (TAPP) of inguinal hernia. The Critical View of the MyoPectineal Orifice (CVMPO) was proposed to ensure correct dissection. An artificial intelligence (AI) system that automatically validates the presence of key and marks during the procedure is a critical step towards automatic dissection quality assessment and video-based competency evaluation. The aim of this study was to develop an AI system that automatically recognizes the TAPP key CVMPO landmarks in hernia repair videos. METHODS: Surgical videos of 160 TAPP procedures were used in this single-center study. A deep neural network-based object detector was developed to automatically recognize the pubic symphysis, direct hernia orifice, Cooper's ligament, the iliac vein, triangle of Doom, deep inguinal ring, and iliopsoas muscle. The system was trained using 130 videos, annotated and verified by two board-certified surgeons. Performance was evaluated in 30 videos of new patients excluded from the training data. RESULTS: Performance was validated in 2 ways: first, single-image validation where the AI model detected landmarks in a single laparoscopic image (mean average precision (MAP) of 51.2%). The second validation is video evaluation where the model detected landmarks throughout the myopectineal orifice visual inspection phase (mean accuracy and F-score of 77.1 and 75.4% respectively). Annotation objectivity was assessed between 2 surgeons in video evaluation, showing a high agreement of 88.3%. CONCLUSION: This study establishes the first AI-based automated recognition of critical structures in TAPP surgical videos, and a major step towards automatic CVMPO validation with AI. Strong performance was achieved in the video evaluation. The high inter-rater agreement confirms annotation quality and task objectivity.

Proposal and multicentric validation of a laparoscopic Roux-en-Y gastric bypass surgery ontology.

BACKGROUND: Phase and step annotation in surgical videos is a prerequisite for surgical scene understanding and for downstream tasks like intraoperative feedback or assistance. However, most ontologies are applied on small monocentric datasets and lack external validation. To overcome these limitations an ontology for phases and steps of laparoscopic Roux-en-Y gastric bypass (LRYGB) is proposed and validated on a multicentric dataset in terms of inter- and intra-rater reliability (inter-/intra-RR). METHODS: The proposed LRYGB ontology consists of 12 phase and 46 step definitions that are hierarchically structured. Two board certified surgeons (raters) with > 10 years of clinical experience applied the proposed ontology on two datasets: (1) StraBypass40 consists of 40 LRYGB videos from Nouvel Hôpital Civil, Strasbourg, France and (2) BernBypass70 consists of 70 LRYGB videos from Inselspital, Bern University Hospital, Bern, Switzerland. To assess inter-RR the two raters' annotations of ten randomly chosen videos from StraBypass40 and BernBypass70 each, were compared. To assess intra-RR ten randomly chosen videos were annotated twice by the same rater and annotations were compared. Inter-RR was calculated using Cohen's kappa. Additionally, for inter- and intra-RR accuracy, precision, recall, F1-score, and application dependent metrics were applied. RESULTS: The mean ± SD video duration was 108 ± 33 min and 75 ± 21 min in StraBypass40 and BernBypass70, respectively. The proposed ontology shows an inter-RR of 96.8 ± 2.7% for phases and 85.4 ± 6.0% for steps on StraBypass40 and 94.9 ± 5.8% for phases and 76.1 ± 13.9% for steps on BernBypass70. The overall Cohen's kappa of inter-RR was 95.9 ± 4.3% for phases and 80.8 ± 10.0% for steps. Intra-RR showed an accuracy of 98.4 ± 1.1% for phases and 88.1 ± 8.1% for steps. CONCLUSION: The proposed ontology shows an excellent inter- and intra-RR and should therefore be implemented routinely in phase and step annotation of LRYGB.

Anatomical Engineering and 3D Printing for Surgery and Medical Devices: International Review and Future Exponential Innovations.

Three-dimensional printing (3DP) has recently gained importance in the medical industry, especially in surgical specialties. It uses different techniques and materials based on patients' needs, which allows bioprofessionals to design and develop unique pieces using medical imaging provided by computed tomography (CT) and magnetic resonance imaging (MRI). Therefore, the Department of Biology and Medicine and the Department of Physics and Engineering, at the Bioastronautics and Space Mechatronics Research Group, have managed and supervised an international cooperation study, in order to present a general review of the innovative surgical applications, focused on anatomical systems, such as the nervous and craniofacial system, cardiovascular system, digestive system, genitourinary system, and musculoskeletal system. Finally, the integration with augmented, mixed, virtual reality is analyzed to show the advantages of personalized treatments, taking into account the improvements for preoperative, intraoperative planning, and medical training. Also, this article explores the creation of devices and tools for space surgery to get better outcomes under changing gravity conditions.

Rendezvous: Attention mechanisms for the recognition of surgical action triplets in endoscopic videos.

Out of all existing frameworks for surgical workflow analysis in endoscopic videos, action triplet recognition stands out as the only one aiming to provide truly fine-grained and comprehensive information on surgical activities. This information, presented as 〈instrument, verb, target〉 combinations, is highly challenging to be accurately identified. Triplet components can be difficult to recognize individually; in this task, it requires not only performing recognition simultaneously for all three triplet components, but also correctly establishing the data association between them. To achieve this task, we introduce our new model, the Rendezvous (RDV), which recognizes triplets directly from surgical videos by leveraging attention at two different levels. We first introduce a new form of spatial attention to capture individual action triplet components in a scene; called Class Activation Guided Attention Mechanism (CAGAM). This technique focuses on the recognition of verbs and targets using activations resulting from instruments. To solve the association problem, our RDV model adds a new form of semantic attention inspired by Transformer networks; called Multi-Head of Mixed Attention (MHMA). This technique uses several cross and self attentions to effectively capture relationships between instruments, verbs, and targets. We also introduce CholecT50 - a dataset of 50 endoscopic videos in which every frame has been annotated with labels from 100 triplet classes. Our proposed RDV model significantly improves the triplet prediction mAP by over 9% compared to the state-of-the-art methods on this dataset.

Multi-task temporal convolutional networks for joint recognition of surgical phases and steps in gastric bypass procedures.

PURPOSE: Automatic segmentation and classification of surgical activity is crucial for providing advanced support in computer-assisted interventions and autonomous functionalities in robot-assisted surgeries. Prior works have focused on recognizing either coarse activities, such as phases, or fine-grained activities, such as gestures. This work aims at jointly recognizing two complementary levels of granularity directly from videos, namely phases and steps. METHODS: We introduce two correlated surgical activities, phases and steps, for the laparoscopic gastric bypass procedure. We propose a multi-task multi-stage temporal convolutional network (MTMS-TCN) along with a multi-task convolutional neural network (CNN) training setup to jointly predict the phases and steps and benefit from their complementarity to better evaluate the execution of the procedure. We evaluate the proposed method on a large video dataset consisting of 40 surgical procedures (Bypass40). RESULTS: We present experimental results from several baseline models for both phase and step recognition on the Bypass40. The proposed MTMS-TCN method outperforms single-task methods in both phase and step recognition by 1-2% in accuracy, precision and recall. Furthermore, for step recognition, MTMS-TCN achieves a superior performance of 3-6% compared to LSTM-based models on all metrics. CONCLUSION: In this work, we present a multi-task multi-stage temporal convolutional network for surgical activity recognition, which shows improved results compared to single-task models on a gastric bypass dataset with multi-level annotations. The proposed method shows that the joint modeling of phases and steps is beneficial to improve the overall recognition of each type of activity.

A Standard Set of Value-Based Patient-Centered Outcomes for Pancreatic Carcinoma: An International Delphi Survey.

BACKGROUND: Global health systems are shifting toward value-based health care to improve patient outcomes in the face of rising health care costs. The challenge is to identify standardized outcome measurements that allow optimal quality-of-care monitoring and comparison to optimize medical practices and patient pathways. A common outcomes definition is required, including medical results (Clinical Reported Outcomes Measurements [CROMs]) and quality-of-life components that matter most to patients (Patient-Reported Outcomes Measurements [PROMs]), which are particularly important for severe pathologies with short life expectancy such as pancreatic cancer. This study aimed to create standardized metrics that could be used for outcomes analysis of pancreatic cancer care. METHODS: A multidisciplinary working group (WG) was assembled. A systematic review was performed to collect the most used outcomes in clinical studies of pancreatic cancers. The study reviewed 570 studies published in the last 10 years. From these studies, 3370 outcomes, including CROMs, and PROMs, were listed and prioritized. The WG reached a consensus on key outcomes, proposed groupings for CROMs and PROMs, identified existing questionnaires that could be used for PROMs collection, and set the timeline for data collection. To refine and validate the final outcomes set, an international external committee completed a Delphi process (two rounds for both CROMS and PROMs). RESULTS: After the systematic literature review, the WG selected 102 outcomes (92 CROMs and 10 PROMs) for submission to the international Delphi vote committee. The committee retrained 89 outcomes (78 CROMs and 11 PROMs). For the PROMs, the WG and the international external committee chose a validated questionnaire, the Functional Assessment of Cancer Therapy-Hepatobiliary, which covers all of the 11 selected PROMs. CONCLUSIONS: A standardized set of outcome measures that need to be validated through international health outcome comparisons and quality-of-care assessments was built. Pilot projects are underway to test and optimize the approach in real-life conditions.

Mixed reality navigation system for ultrasound-guided percutaneous punctures: a pre-clinical evaluation.

Image-guided surgery is growing in importance with each year. Various imaging technologies are used. The objective of this study was to test whether a new mixed reality navigation system (MRNS) improved percutaneous punctures. This system allowed to clearly visualize the needle tip, needle orientation, US probe and puncture target simultaneously with an interactive 3D computer user inferface. Prospective pre-clinical comparative study. An opaque ballistic gel phantom containing grapes of different sizes was used to simulate puncture targets. The evaluation consisted of ultrasound-guided (US-guided) needle punctures divided into two groups, standard group consisted of punctures using the standard approach (US-guided), and assisted navigation group consisted of punctures using MRNS. Once a puncture was completed, a computed tomography scan was made of the phantom and needle. The distance between the needle tip and the center of the target was measured. The time required to complete the puncture and puncture attempts was also calculated. Total participants was n = 23, between surgeons, medical technicians and radiologist. The participants were divided into novices (without experience, 69.6%) and experienced (with experience > 25 procedures, 30.4%). Each participant performed the puncture of six targets. For puncture completion time, the assisted navigation group was faster (42.1%) compared to the standard group (57.9%) (28.3 s ± 24.7 vs. 39.3 s ± 46.3-p 0.775). The total punctures attempts was lower in the assisted navigation group (35.4%) compared to the standard group (64.6%) (1.0 mm ± 0.2 vs. 1.8 mm ± 1.1-p 0.000). The assisted navigation group was more accurate than the standard group (4.2 ± 2.9 vs. 6.5 ± 4.7-p 0.003), observed in both novices and experienced groups. The use of MRNS improved ultrasound-guided percutaneous punctures parameters compared to the standard approach.

Hybrid endoluminal stapled pyloroplasty: an alternative treatment option for gastric outlet obstruction syndrome.

BACKGROUND: Gastroparesis is a rapidly increasing problem with sometimes devastating consequences. While surgical treatments, particularly laparoscopic pyloroplasty, have recently gained popularity, they require general anesthesia, advanced skills, and can lead to leaks. Peroral pyloromyotomy is a less invasive alternative; however, this technique is technically demanding and not widely available. We describe a hybrid laparo-endoscopic collaborative approach using a novel gastric access device to allow endoluminal stapled pyloroplasty as an alternative treatment option for gastric outlet obstruction. METHODS: Under general anesthesia, six pigs (mean weight 33 kg) underwent endoscopic placement of intragastric ports using a technique similar to percutaneous endoscopic gastrostomy. A 5 mm laparoscope was used for visualization. A functional lumen imagine probe was used to measure the cross-sectional area (CSA) and diameter of the pylorus before, after, and at 1 week after intervention. Pyloroplasty was performed using a 5 mm articulating laparoscopic stapler. Gastrotomies were closed by endoscopic clips, endoscopic suture, or combination. After 6-8 days, a second evaluation was performed. At the end of the protocol, all animals were euthanized. RESULTS: Six pyloroplasties were performed. In all cases, this technique was effective in achieving significant pyloric dilatation. The median pre-pyloroplasty pyloric diameter (D) and cross-sectional area (CSA) were 8 mm (4.9-11.6 mm) and 58.6 mm2 (19-107 mm2), respectively. After the procedure, these values increased to 13.41 mm (9.8-17.6 mm) and 147.7 mm2 (76-244 mm2), respectively (p = 0.0152). No important intraoperative events were observed. Postoperatively, all animals did well, with adequate oral intake and no relevant complications. At follow-up endoscopy, all incisions were healed and the pylorus widely patent. CONCLUSIONS: Hybrid endoluminal stapled pyloroplasty is a feasible, safe, and effective alternative method for the treatment of gastric outlet obstruction syndrome.

Stereotactic Pelvic Navigation With Augmented Reality for Transanal Total Mesorectal Excision.

INTRODUCTION: Technical difficulty and unfamiliar surgical anatomy are the main challenges in transanal total mesorectal excision. Precise 3-dimensional real-time image guidance may facilitate the safety, accuracy, and efficiency of transanal total mesorectal excision. TECHNIQUE: A preoperative CT was obtained with 10 skin fiducials and further processed to emphasize the border of the anatomical structure by 3-dimensional modeling and pelvic organ segmentation. A forced sacral tilt by placing a 10-degree wedge under the patient's sacrum was induced to minimize pelvic organ movement caused by lithotomy position. An optical navigation system with cranial software was used. Preoperative CT images were loaded into the navigation system, and patient tracker was mounted onto the iliac bone. Once the patient-to-image paired point registration using skin fiducials was completed, the laparoscopic instrument mounted with instrument tracker was calibrated for instrument tracking. After validating the experimental setup and process of registration by navigating laparoscopic anterior resection, stereotactic navigation for transanal total mesorectal excision was performed in the low rectal neuroendocrine tumor. RESULTS: The fiducial registration error was 1.7 mm. The accuracy of target positioning was sufficient at less than 3 mm (1.8 ± 0.9 mm). Qualitative assessment using a Likert scale was well matched between the 2 observers. Of the 20 scores, 19 were judged as 4 (very good) or 5 (excellent). There was no statistical difference between mean Likert scales of the abdominal or transanal landmarks (4.4 ± 0.5 vs 4.3 ± 1.0, p = 0.965). CONCLUSIONS: Application of an existing navigation system to transanal total mesorectal excision for a low rectal tumor is feasible. The acceptable accuracy of target positioning justifies its clinical use. Further research is needed to prove the clinical need for the procedure and its impact on clinical outcomes.

Percutaneous Image-guided Surgery Training: Model IHU-DAICIM.

The evolution of guided imaging surgery is well known in recent years. As the field of action becomes more specific, learning and teaching are also more specific. State-of-the-art medical training should be mandatory in the field of general medicine and surgery in particular. In this work, we report on how to create a model for the formation of guided surgery by images in a simple and fast way, and its implementation by young surgeons. Pig models have been used in which collections made by bovine small intestine and simulated tumor lesions have been placed. Several types of image-guided procedures have been performed. No major complications were found during the development of the model or during its use. It is possible to develop a quick, simple, and safe living training model that can be used immediately after preparation.

Carcinoma sarcomatoide de hipofaringe: reporte de un caso / Sarcomatoid carcinoma of hypopharynx: report of a case

El carcinoma sarcomatoide es una rara neoplasia maligna de alto grado, la cual, ha sido motivo de controversias en su diagnóstico y su tratamiento. Se observa con frecuencia en el tracto aerodigestivo superior, aunque su localización en la hipofaringe es rara, habiéndose reportado pocos casos a nivel mundial. Se presenta una paciente femenina de 42 años sin historia de consumo de tabaco ni alcohol, quien presentaba disfagia, pérdida de peso, disfonía y masa cervical bilateral, mediante laringoscopia de fibra óptica se evidenció un extenso tumor de hipofaringe a nivel del seno piriforme izquierdo que invalida la laringe supraglótica. La histología del tumor reportó un carcinoma sarcomatoide. La paciente fue intervenida quirúrgicamente realizándose una laringofaringoesofagectomía con reconstrucción con ascenso gástrico más disección cervical radical bilateral, Presentó complicaciones pulmonares y abdominales, falleciendo a los 2 meses del posoperatorio. El carcinoma sarcomatoide es una rara y agresiva neoplasia cuyo pronóstico depende de su localización, tamaño y presencia de metástasis cervicales; aunque por lo general el pronóstico es malo independientemente del tratamiento recibido