ABSTRACT
PURPOSE: Surgical workflow and skill analysis are key technologies for the next generation of cognitive surgical assistance systems. These systems could increase the safety of the operation through context-sensitive warnings and semi-autonomous robotic assistance or improve training of surgeons via data-driven feedback. In surgical workflow analysis up to 91% average precision has been reported for phase recognition on an open data single-center video dataset. In this work we investigated the generalizability of phase recognition algorithms in a multicenter setting including more difficult recognition tasks such as surgical action and surgical skill. METHODS: To achieve this goal, a dataset with 33 laparoscopic cholecystectomy videos from three surgical centers with a total operation time of 22 h was created. Labels included framewise annotation of seven surgical phases with 250 phase transitions, 5514 occurences of four surgical actions, 6980 occurences of 21 surgical instruments from seven instrument categories and 495 skill classifications in five skill dimensions. The dataset was used in the 2019 international Endoscopic Vision challenge, sub-challenge for surgical workflow and skill analysis. Here, 12 research teams trained and submitted their machine learning algorithms for recognition of phase, action, instrument and/or skill assessment. RESULTS: F1-scores were achieved for phase recognition between 23.9% and 67.7% (n = 9 teams), for instrument presence detection between 38.5% and 63.8% (n = 8 teams), but for action recognition only between 21.8% and 23.3% (n = 5 teams). The average absolute error for skill assessment was 0.78 (n = 1 team). CONCLUSION: Surgical workflow and skill analysis are promising technologies to support the surgical team, but there is still room for improvement, as shown by our comparison of machine learning algorithms. This novel HeiChole benchmark can be used for comparable evaluation and validation of future work. In future studies, it is of utmost importance to create more open, high-quality datasets in order to allow the development of artificial intelligence and cognitive robotics in surgery.
Subject(s)
Artificial Intelligence , Benchmarking , Humans , Workflow , Algorithms , Machine LearningABSTRACT
Severity and progression of degenerative neuromuscular diseases can be sensitively captured by evaluating the fat infiltration of muscle tissue in T1-weighted MRI scans of human limbs. For computing the fat fraction, the original muscle needs to be first separated from other tissue. Five conceptionally different approaches were investigated and evaluated with respect to the segmentation of muscles of human thighs. Besides a rather basic thresholding approach, local (level set) as well as global (graph cut) energy-minimizing segmentation approaches with and without a shape prior energy term were examined. For experimental evaluations, a dataset containing 37 subjects was divided into four classes according to the degree of fat infiltration. Results show that the choice of the best method depends on the severity of fat infiltration. In severe cases, the best results were obtained with shape prior based graph cuts, whereas in marginal cases thresholding was sufficient. With the best approach, the worst-case error in fat fraction computation was always below 11% and on average between 2% for tissue showing no fat infiltrations and 6% for heavily infiltrated tissue. The obtained Dice similarity coefficients, measuring the segmentation quality, were on average between 0.85 and 0.92. Although segmentation of heavily infiltrated muscle tissue is extremely difficult, an approach for reasonably segmenting these image data was identified. Especially the negative impact on the calculated fat fraction can be reduced significantly.
