Development of an artificial intelligence system for real-time intraoperative assessment of the Critical View of Safety in laparoscopic cholecystectomy.

BACKGROUND: The Critical View of Safety (CVS) was proposed in 1995 to prevent bile duct injury during laparoscopic cholecystectomy (LC). The achievement of CVS was evaluated subjectively. This study aimed to develop an artificial intelligence (AI) system to evaluate CVS scores in LC. MATERIALS AND METHODS: AI software was developed to evaluate the achievement of CVS using an algorithm for image classification based on a deep convolutional neural network. Short clips of hepatocystic triangle dissection were converted from 72 LC videos, and 23,793 images were labeled for training data. The learning models were examined using metrics commonly used in machine learning. RESULTS: The mean values of precision, recall, F-measure, specificity, and overall accuracy for all the criteria of the best model were 0.971, 0.737, 0.832, 0.966, and 0.834, respectively. It took approximately 6 fps to obtain scores for a single image. CONCLUSIONS: Using the AI system, we successfully evaluated the achievement of the CVS criteria using still images and videos of hepatocystic triangle dissection in LC. This encourages surgeons to be aware of CVS and is expected to improve surgical safety.

Impact of AI system on recognition for anatomical landmarks related to reducing bile duct injury during laparoscopic cholecystectomy.

BACKGROUND: According to the National Clinical Database of Japan, the incidence of bile duct injury (BDI) during laparoscopic cholecystectomy has hovered around 0.4% for the last 10 years and has not declined. On the other hand, it has been found that about 60% of BDI occurrences are due to misidentifying anatomical landmarks. However, the authors developed an artificial intelligence (AI) system that gave intraoperative data to recognize the extrahepatic bile duct (EHBD), cystic duct (CD), inferior border of liver S4 (S4), and Rouviere sulcus (RS). The purpose of this research was to evaluate how the AI system affects landmark identification. METHODS: We prepared a 20-s intraoperative video before the serosal incision of Calot's triangle dissection and created a short video with landmarks overwritten by AI. The landmarks were defined as landmark (LM)-EHBD, LM-CD, LM-RS, and LM-S4. Four beginners and four experts were recruited as subjects. After viewing a 20-s intraoperative video, subjects annotated the LM-EHBD and LM-CD. Then, a short video is shown with the AI overwriting landmark instructions; if there is a change in each perspective, the annotation is changed. The subjects answered a three-point scale questionnaire to clarify whether the AI teaching data advanced their confidence in verifying the LM-RS and LM-S4. Four external evaluation committee members investigated the clinical importance. RESULTS: In 43 of 160 (26.9%) images, the subjects transformed their annotations. Annotation changes were primarily observed in the gallbladder line of the LM-EHBD and LM-CD, and 70% of these shifts were considered safer changes. The AI-based teaching data encouraged both beginners and experts to affirm the LM-RS and LM-S4. CONCLUSION: The AI system provided significant awareness to beginners and experts and prompted them to identify anatomical landmarks linked to reducing BDI.

Development of a cross-artificial intelligence system for identifying intraoperative anatomical landmarks and surgical phases during laparoscopic cholecystectomy.

BACKGROUND: Attention to anatomical landmarks in the appropriate surgical phase is important to prevent bile duct injury (BDI) during laparoscopic cholecystectomy (LC). Therefore, we created a cross-AI system that works with two different AI algorithms simultaneously, landmark detection and phase recognition. We assessed whether landmark detection was activated in the appropriate phase by phase recognition during LC and the potential contribution of the cross-AI system in preventing BDI through a clinical feasibility study (J-SUMMIT-C-02). METHODS: A prototype was designed to display landmarks during the preparation phase and Calot's triangle dissection. A prospective clinical feasibility study using the cross-AI system was performed in 20 LC cases. The primary endpoint of this study was the appropriateness of the detection timing of landmarks, which was assessed by an external evaluation committee (EEC). The secondary endpoint was the correctness of landmark detection and the contribution of cross-AI in preventing BDI, which were assessed based on the annotation and 4-point rubric questionnaire. RESULTS: Cross-AI-detected landmarks in 92% of the phases where the EEC considered landmarks necessary. In the questionnaire, each landmark detected by AI had high accuracy, especially the landmarks of the common bile duct and cystic duct, which were assessed at 3.78 and 3.67, respectively. In addition, the contribution to preventing BDI was relatively high at 3.65. CONCLUSIONS: The cross-AI system provided landmark detection at appropriate situations. The surgeons who previewed the model suggested that the landmark information provided by the cross-AI system may be effective in preventing BDI. Therefore, it is suggested that our system could help prevent BDI in practice. Trial registration University Hospital Medical Information Network Research Center Clinical Trial Registration System (UMIN000045731).

An intraoperative artificial intelligence system identifying anatomical landmarks for laparoscopic cholecystectomy: a prospective clinical feasibility trial (J-SUMMIT-C-01).

BACKGROUND: We have implemented Smart Endoscopic Surgery (SES), a surgical system that uses artificial intelligence (AI) to detect the anatomical landmarks that expert surgeons base on to perform certain surgical maneuvers. No report has verified the use of AI-based support systems for surgery in clinical practice, and no evaluation method has been established. To evaluate the detection performance of SES, we have developed and established a new evaluation method by conducting a clinical feasibility trial. METHODS: A single-center prospective clinical feasibility trial was conducted on 10 cases of LC performed at Oita University hospital. Subsequently, an external evaluation committee (EEC) evaluated the AI detection accuracy for each landmark using five-grade rubric evaluation and DICE coefficient. We defined LM-CBD as the expert surgeon's "judge" of the cystic bile duct in endoscopic images. RESULTS: The average detection accuracy on the rubric by the EEC was 4.2 ± 0.8 for the LM-CBD. The DICE coefficient between the AI detection area of the LM-CBD and the EEC members' evaluation was similar to the mean value of the DICE coefficient between the EEC members. The DICE coefficient was high score for the case that was highly evaluated by the EEC on a five-grade scale. CONCLUSION: This is the first feasible clinical trial of an AI system designed for intraoperative use and to evaluate the AI system using an EEC. In the future, this concept of evaluation for the AI system would contribute to the development of new AI navigation systems for surgery.

Artificial intelligence software available for medical devices: surgical phase recognition in laparoscopic cholecystectomy.

BACKGROUND: Surgical process modeling automatically identifies surgical phases, and further improvement in recognition accuracy is expected with deep learning. Surgical tool or time series information has been used to improve the recognition accuracy of a model. However, it is difficult to collect this information continuously intraoperatively. The present study aimed to develop a deep convolution neural network (CNN) model that correctly identifies the surgical phase during laparoscopic cholecystectomy (LC). METHODS: We divided LC into six surgical phases (P1-P6) and one redundant phase (P0). We prepared 115 LC videos and converted them to image frames at 3 fps. Three experienced doctors labeled the surgical phases in all image frames. Our deep CNN model was trained with 106 of the 115 annotation datasets and was evaluated with the remaining datasets. By depending on both the prediction probability and frequency for a certain period, we aimed for highly accurate surgical phase recognition in the operation room. RESULTS: Nine full LC videos were converted into image frames and were fed to our deep CNN model. The average accuracy, precision, and recall were 0.970, 0.855, and 0.863, respectively. CONCLUSION: The deep learning CNN model in this study successfully identified both the six surgical phases and the redundant phase, P0, which may increase the versatility of the surgical process recognition model for clinical use. We believe that this model can be used in artificial intelligence for medical devices. The degree of recognition accuracy is expected to improve with developments in advanced deep learning algorithms.

Development of Endoscopic Surgery Navigated by Artificial Intelligence.

Endoscopic surgery, which was first introduced in the late 1980s, has rapidly become widespread. However, despite its popularity, the occurrence of intraoperative organ damage has not necessarily decreased. To avoid intraoperative bile duct injury in laparoscopic cholecystectomy, which is one of the most popular procedures in endoscopic surgery, we are developing a laparoscopic surgical system that uses Artificial Intelligence (AI) to identify four anatomical landmarks (cystic duct of the gallbladder, common bile duct, lower surface of hepatic S4, and Rouviere's sulcus, related to "Calot's triangle") in real time during surgery. The development process consists of 5 steps: 1) identification of anatomical landmarks, 2) collection and creation of teaching data, 3) annotation and deep learning, 4) validation of development model, and 5) actual clinical performance evaluation. At present, anatomical landmarks can be identified with high accuracy in an actual clinical performance test in laparoscopic cholecystectomy, whereas issues for practical clinical use, such as a need to recognize the scene of surgical steps and surgical difficulties related to inflammation of the gallbladder, have also been clarified. The development of an AI-navigation system for endoscopic surgery, which could identify anatomical landmarks in real time during surgery, could be expected to support surgeons' decisions, reduce surgical complications, and contribute to improving the quality of surgical treatments.

Development of an artificial intelligence system using deep learning to indicate anatomical landmarks during laparoscopic cholecystectomy.

BACKGROUND: The occurrence of bile duct injury (BDI) during laparoscopic cholecystectomy (LC) is an important medical issue. Expert surgeons prevent intraoperative BDI by identifying four landmarks. The present study aimed to develop a system that outlines these landmarks on endoscopic images in real time. METHODS: An intraoperative landmark indication system was constructed using YOLOv3, which is an algorithm for object detection based on deep learning. The training datasets comprised approximately 2000 endoscopic images of the region of Calot's triangle in the gallbladder neck obtained from 76 videos of LC. The YOLOv3 learning model with the training datasets was applied to 23 videos of LC that were not used in training, to evaluate the estimation accuracy of the system to identify four landmarks: the cystic duct, common bile duct, lower edge of the left medial liver segment, and Rouviere's sulcus. Additionally, we constructed a prototype and used it in a verification experiment in an operation for a patient with cholelithiasis. RESULTS: The YOLOv3 learning model was quantitatively and subjectively evaluated in this study. The average precision values for each landmark were as follows: common bile duct: 0.320, cystic duct: 0.074, lower edge of the left medial liver segment: 0.314, and Rouviere's sulcus: 0.101. The two expert surgeons involved in the annotation confirmed consensus regarding valid indications for each landmark in 22 of the 23 LC videos. In the verification experiment, the use of the intraoperative landmark indication system made the surgical team more aware of the landmarks. CONCLUSIONS: Intraoperative landmark indication successfully identified four landmarks during LC, which may help to reduce the incidence of BDI, and thus, increase the safety of LC. The novel system proposed in the present study may prevent BDI during LC in clinical practice.