Surgeons vs ChatGPT: Assessment and Feedback Performance Based on Real Surgical Scenarios.

INTRODUCTION: Artificial intelligence tools are being progressively integrated into medicine and surgical education. Large language models, such as ChatGPT, could provide relevant feedback aimed at improving surgical skills. The purpose of this study is to assess ChatGPT´s ability to provide feedback based on surgical scenarios. METHODS: Surgical situations were transformed into texts using a neutral narrative. Texts were evaluated by ChatGPT 4.0 and 3 surgeons (A, B, C) after a brief instruction was delivered: identify errors and provide feedback accordingly. Surgical residents were provided with each of the situations and feedback obtained during the first stage, as written by each surgeon and ChatGPT, and were asked to assess the utility of feedback (FCUR) and its quality (FQ). As control measurement, an Education-Expert (EE) and a Clinical-Expert (CE) were asked to assess FCUR and FQ. RESULTS: Regarding residents' evaluations, 96.43% of times, outputs provided by ChatGPT were considered useful, comparable to what surgeons' B and C obtained. Assessing FQ, ChatGPT and all surgeons received similar scores. Regarding EE's assessment, ChatGPT obtained a significantly higher FQ score when compared to surgeons A and B (p = 0.019; p = 0.033) with a median score of 8 vs. 7 and 7.5, respectively; and no difference respect surgeon C (score of 8; p = 0.2). Regarding CE´s assessment, surgeon B obtained the highest FQ score while ChatGPT received scores comparable to that of surgeons A and C. When participants were asked to identify the source of the feedback, residents, CE, and EE perceived ChatGPT's outputs as human-provided in 33.9%, 28.5%, and 14.3% of cases, respectively. CONCLUSION: When given brief written surgical situations, ChatGPT was able to identify errors with a detection rate comparable to that of experienced surgeons and to generate feedback that was considered useful for skill improvement in a surgical context performing as well as surgical instructors across assessments made by general surgery residents, an experienced surgeon, and a nonsurgeon feedback expert.

Comparative study of video recordings of non-medical devices in laparoscopic surgery: a cross-sectional study.

INTRODUCTION: Laparoscopic surgery is the approach of choice for multiple procedures, being laparoscopic cholecystectomy one of the most frequently performed surgeries. Likewise, video recording of these surgeries has become widespread. Currently, the market offers medical recording devices (MRD) with an approximate cost of 2000 USD, and alternative non-medical recording devices (NMRD) with a cost ranging from 120 to 200 USD. To our knowledge, no comparative studies between the available recording devices have been done. We aim to compare the perception of the quality of videos recorded by MRD and NMRD in a group of surgeons and surgical residents. METHODS: A cross-sectional study was conducted using an online survey to compare recordings from three NMRDs (Elgato 30 fps, AverMedia 60 fps, Hauppauge 30 fps) and one MRD (MediCap 20 fps) during a laparoscopic cholecystectomy. The survey assessed: definition of anatomical structures (DA), fluidity of movements (FM), similarity with the operating room screen (ORsim), and overall quality (OQ). Descriptive and nonparametric analytical statistics tests were applied. Results were analyzed using JMP-15 software. RESULTS: Forty surveys were collected (80% surgeons, 20% residents). NMRDs scored significantly higher than MRD in DA (p = 0.003), FM (p < 0.001), ORsim (p < 0.001), and OQ (p < 0.001). One NMRD was chosen as the highest quality device (70%), and MRD as the poorest (78%). No significant differences were found when analyzing by surgical experience. CONCLUSIONS: In terms of recording laparoscopic procedures, non-medical video recording devices (NMRDs) outperformed medical-grade recording device (MRD) with a higher overall score. This suggests that NMRDs could serve as a cost-effective alternative with superior video quality for recording laparoscopic surgeries.