Surgeons vs ChatGPT: Assessment and Feedback Performance Based on Real Surgical Scenarios.

INTRODUCTION: Artificial intelligence tools are being progressively integrated into medicine and surgical education. Large language models, such as ChatGPT, could provide relevant feedback aimed at improving surgical skills. The purpose of this study is to assess ChatGPT´s ability to provide feedback based on surgical scenarios. METHODS: Surgical situations were transformed into texts using a neutral narrative. Texts were evaluated by ChatGPT 4.0 and 3 surgeons (A, B, C) after a brief instruction was delivered: identify errors and provide feedback accordingly. Surgical residents were provided with each of the situations and feedback obtained during the first stage, as written by each surgeon and ChatGPT, and were asked to assess the utility of feedback (FCUR) and its quality (FQ). As control measurement, an Education-Expert (EE) and a Clinical-Expert (CE) were asked to assess FCUR and FQ. RESULTS: Regarding residents' evaluations, 96.43% of times, outputs provided by ChatGPT were considered useful, comparable to what surgeons' B and C obtained. Assessing FQ, ChatGPT and all surgeons received similar scores. Regarding EE's assessment, ChatGPT obtained a significantly higher FQ score when compared to surgeons A and B (p = 0.019; p = 0.033) with a median score of 8 vs. 7 and 7.5, respectively; and no difference respect surgeon C (score of 8; p = 0.2). Regarding CE´s assessment, surgeon B obtained the highest FQ score while ChatGPT received scores comparable to that of surgeons A and C. When participants were asked to identify the source of the feedback, residents, CE, and EE perceived ChatGPT's outputs as human-provided in 33.9%, 28.5%, and 14.3% of cases, respectively. CONCLUSION: When given brief written surgical situations, ChatGPT was able to identify errors with a detection rate comparable to that of experienced surgeons and to generate feedback that was considered useful for skill improvement in a surgical context performing as well as surgical instructors across assessments made by general surgery residents, an experienced surgeon, and a nonsurgeon feedback expert.

Comparative study of video recordings of non-medical devices in laparoscopic surgery: a cross-sectional study.

INTRODUCTION: Laparoscopic surgery is the approach of choice for multiple procedures, being laparoscopic cholecystectomy one of the most frequently performed surgeries. Likewise, video recording of these surgeries has become widespread. Currently, the market offers medical recording devices (MRD) with an approximate cost of 2000 USD, and alternative non-medical recording devices (NMRD) with a cost ranging from 120 to 200 USD. To our knowledge, no comparative studies between the available recording devices have been done. We aim to compare the perception of the quality of videos recorded by MRD and NMRD in a group of surgeons and surgical residents. METHODS: A cross-sectional study was conducted using an online survey to compare recordings from three NMRDs (Elgato 30 fps, AverMedia 60 fps, Hauppauge 30 fps) and one MRD (MediCap 20 fps) during a laparoscopic cholecystectomy. The survey assessed: definition of anatomical structures (DA), fluidity of movements (FM), similarity with the operating room screen (ORsim), and overall quality (OQ). Descriptive and nonparametric analytical statistics tests were applied. Results were analyzed using JMP-15 software. RESULTS: Forty surveys were collected (80% surgeons, 20% residents). NMRDs scored significantly higher than MRD in DA (p = 0.003), FM (p < 0.001), ORsim (p < 0.001), and OQ (p < 0.001). One NMRD was chosen as the highest quality device (70%), and MRD as the poorest (78%). No significant differences were found when analyzing by surgical experience. CONCLUSIONS: In terms of recording laparoscopic procedures, non-medical video recording devices (NMRDs) outperformed medical-grade recording device (MRD) with a higher overall score. This suggests that NMRDs could serve as a cost-effective alternative with superior video quality for recording laparoscopic surgeries.

Recommendations to Develop a Laparoscopic Surgical Simulation Training Program. Insights Gained After 12 Years of Training Surgeons / Recomendaciones para desarrollar un programa de formación en simulación quirúrgica laparoscópica. Perspectivas obtenidas tras 12 años de formación de cirujanos / Recomendações para o desenvolvimento de um programa de formação em simulação cirúrgica laparoscópica. Perspectivas obtidas após 12 anos de formação de cirurgiões

Introduction. The use of simulation in surgery has made it possible to shorten learning curves through deliberate practice. Although it has been incorporated long ago, there are still no clear recommendations to standardize its development and implementation. This manuscript aims to share recommendations based on our experience of more than twelve years of employing and improving a methodology in laparoscopic surgical simulation. Topics for Reflection. To transfer surgical skills to a trainee, we base our methodology on a three-pillar framework: The hardware and infrastructure (tools to train with), the training program itself (what to do), and the feedback (how to improve). Implementing a cost-effective program is feasible: the hardware does not need to be high fidelity to transfer skills, but the program needs to be validated. These pillars have evolved over time by incorporating technology: the on-site guidance from experts has changed to a remote and asynchronous modality by video recording the trainee's execution, and by enabling remote and asynchronous feedback. The feedback provider does not necessarily have to be an expert clinician in the subject, but a person previously trained to be a trainer. This allows for deliberate practice until mastery has been reached and learning curves are consolidated. Conclusions. Recommendations based on the experience of our center have been presented, explaining the framework of our strategy. Considering these suggestions, it is hoped that our simulation methodology can aid the development and implementation of effective simulationbased programs for other groups and institutions.

Introducción. El uso de la simulación en cirugía ha permitido acortar las curvas de aprendizaje mediante la práctica deliberada. A pesar de que se ha incorporado previamente, aún no existen recomendaciones claras para estandarizar su desarrollo e implementación. Este manuscrito pretende compartir recomendaciones basadas en nuestra experiencia, con más de doce años empleando y mejorando una metodología en la simulación quirúrgica laparoscópica. Temas de reflexión. Para transferir las habilidades quirúrgicas a un aprendiz, basamos nuestra metodología en un marco de tres pilares: El hardware y la infraestructura (herramientas con las que entrenar), el programa de entrenamiento (qué hacer), y la retroalimentación (cómo mejorar). La implementación de un programa rentable es factible: el hardware no necesita ser de alta fidelidad para transferir las habilidades, pero el programa necesita ser validado. Estos pilares han evolucionado a lo largo del tiempo incorporando tecnología: la presencia de expertos ha evolucionado a una modalidad remota y asincrónica mediante la grabación en vídeo de la ejecución del alumno, y permitiendo su retroalimentación. Aquel que entrega retroalimentación no tiene que ser necesariamente un clínico experto en la materia, sino una persona previamente formada como instructor. Esto permite una práctica deliberada hasta dominar la habilidad y establecer curvas de aprendizaje. Conclusiones. Se han presentado recomendaciones basadas en la experiencia de nuestro centro, explicando el marco de nuestra estrategia. Teniendo en cuenta estas sugerencias, se espera que nuestra metodología de simulación pueda ayudar al desarrollo e implementación de programas efectivos basados en la simulación a otros grupos e instituciones.

Introdução. O uso de simulação em cirurgia tornou possível encurtar as curvas de aprendizagem por meio da prática deliberada. Embora tenha sido incorporado anteriormente, ainda não há recomendações claras para padronizar seu desenvolvimento e implementação. Este manuscrito pretende compartilhar recomendações com base em nossa experiência, com mais de doze anos usando e aprimorando uma metodologia em simulação cirúrgica laparoscópica. Temas de reflexão. Para transferir habilidades cirúrgicas para um aprendiz, baseamos nossa metodologia em uma estrutura de três pilares: o hardware e a infraestrutura (ferramentas para treinar), o programa de treinamento (o que fazer) e feedback (como melhorar). A implementação de um programa rentável é viável: o hardware não precisa ser de alta fidelidade para transferir as habilidades, mas o programa precisa ser validado. Esses pilares evoluíram ao longo do tempo incorporando a tecnologia: a presença de especialistas evoluiu para uma modalidade remota e assíncrona por meio da gravação em vídeo do desempenho do aluno e permitindo seu feedback. Quem dá feedback não precisa ser necessariamente um clínico especialista na área, mas sim uma pessoa previamente treinada como instrutor. Isso permite a prática deliberada até que a habilidade seja dominada e estabeleça curvas de aprendizado. Conclusões. Foram apresentadas recomendações baseadas na experiência do nosso centro, explicando o enquadramento da nossa estratégia. Levando em consideração essas sugestões, espera-se que nossa metodologia de simulação possa ajudar outros grupos e instituições a desenvolver e implementar programas eficazes baseados em simulação.

Evaluación objetiva de las habilidades técnicas en cirugía / Objective evaluation of technical skills in surgery

La enseñanza tradicional de la cirugía se ha basado en la observación y retroalimentación directa por parte de un experto. Esto corresponde a una evaluación subjetiva y requiere de un tiempo mínimo de exposición no siempre fácil de obtener. Los resultados adversos y complicaciones se traducen en gastos innecesarios para los servicios de salud. Surge la necesidad de nuevos métodos de evaluación objetiva de las habilidades quirúrgicas. Se presenta una revisión de la literatura respecto a los métodos objetivos de evaluación de las habilidades técnicas en cirugía que actualmente existen y dar a conocer una nueva metodología de evaluación actualmente en desarrollo. Métodos: Se realizó una revisión de la literatura, utilizando PubMed, Google Scholar y Scielo. Se incluyeron estudios en inglés y español que utilizaron métodos objetivos estandarizados de evaluación de técnicas quirúrgicas, ya sea con técnica abierta o laparoscópica, además de revisiones de la literatura. Resultados: Se incluyeron las escalas OSATS, GOALS, GAGES, métodos basados en simulación y otros aún en desarrollo. Las escalas de evaluación han sido validadas en un gran número de procedimientos. La simulación en el área quirúrgica ha permitido generar una evaluación y retroalimentación previa al escenario operatorio real, reduciendo la tasa de errores e incrementando la eficiencia en éste. La Caja Negra es un método novedoso que incluye una gran cantidad de variables, permitiendo detectar pequeñas variaciones o errores en la ejecución de la técnica, lo que permite una evaluación máscertera. Conclusiones: Múltiples métodos de evaluación objetiva de las habilidades técnicas han sido desarrollados y validados. Se espera que exista una incorporación sistemática de estas herramientas en los programas de formación de cirujanos de nuestro país.(AU)

The traditional teaching of surgery has been based on observation and direct feedback from an expert. This is a subjective evaluation and requires a minimum exposure time not always easy to obtain. Adverse outcomes and complications translate into unnecessary expenses for health services. The need for new methods of objective evaluation of surgical skills arises. We present a review of the literature regarding the objective methods of evaluation of the technical skills in surgery that currently exist and present a new evaluation methodology currently under development. Methods: A review of the literature was performed, using PubMed, Google Scholar and Scielo. Studies in English and Spanish were included that used objective standardized methods of evaluation of surgical techniques, either with open or laparoscopic technique, in addition to reviews of the literature. Results: The scales OSATS, GOALS, GAGES, simulation-based methods and others still in development were included. The evaluation scales have been validated in a large number of procedures. The simulation in the surgical area has allowed to generate an evaluation and previous feedback to the real operating scenario, reducing the error rate and increasing the efficiency in it. The Black Box is a novel method that includes a large number of variables, allowing to detect small variations or errors in the execution of the technique, which allows a more accurate evaluation. Conclusions: Multiple methods of objective evaluation of technical skills have been developed and validated. It is expected that there is a systematic incorporation of these tools in the training programs of surgeons in our country. (AU)

Simulación en cirugía laparoscópica / Simulation in laparoscopic surgery

Hoy en día los cirujanos en formación se ven enfrentados a un menor entrenamiento quirúrgico debido a limitaciones legales y restricciones horarias, sumadas a la exigencia actual de dominar técnicas más complejas como la laparoscopia. La simulación surge como una herramienta complementaria de aprendizaje en cirugía laparoscópica, mediante el entrenamiento en un ambiente seguro, controlado y estandarizado, sin comprometer la seguridad del paciente. El objetivo de la simulación es que las habilidades adquiridas sean transferidas al quirófano, permitiendo disminuir las curvas de aprendizaje. Programas de simulación se han incorporado progresivamente en todo el mundo en residencias quirúrgicas y cursos de entrenamiento en cirugía laparoscópica, exigiéndose como requisito en algunos países para certificar la especialidad. En este artículo se revisan los aspectos más importantes de la simulación en cirugía laparoscópica, incluyendo los tipos de simuladores y programas de entrenamiento conocidos, así como las metodologías de aprendizaje y escalas de evaluación utilizadas

Nowadays surgical trainees are faced with a more reduced surgical practice, due to legal limitations and work hourly constraints. Also, currently surgeons are expected to dominate more complex techniques such as laparoscopy. Simulation emerges as a complementary learning tool in laparoscopic surgery, by training in a safe, controlled and standardized environment, without jeopardizing patient’ safety. Simulation’ objective is that the skills acquired should be transferred to the operating room, allowing reduction of learning curves. The use of simulation has increased worldwide, becoming an important tool in different surgical residency programs and laparoscopic training courses. For several countries, the approval of these training courses are a prerequisite for the acquisition of surgeon title certifications. This article reviews the most important aspects of simulation in laparoscopic surgery, including the most used simulators and training programs, as well as the learning methodologies and the different key ways to assess learning in simulation

Simulation in laparoscopic surgery.

Nowadays surgical trainees are faced with a more reduced surgical practice, due to legal limitations and work hourly constraints. Also, currently surgeons are expected to dominate more complex techniques such as laparoscopy. Simulation emerges as a complementary learning tool in laparoscopic surgery, by training in a safe, controlled and standardized environment, without jeopardizing patient' safety. Simulation' objective is that the skills acquired should be transferred to the operating room, allowing reduction of learning curves. The use of simulation has increased worldwide, becoming an important tool in different surgical residency programs and laparoscopic training courses. For several countries, the approval of these training courses are a prerequisite for the acquisition of surgeon title certifications. This article reviews the most important aspects of simulation in laparoscopic surgery, including the most used simulators and training programs, as well as the learning methodologies and the different key ways to assess learning in simulation.