A surgical activity model of laparoscopic cholecystectomy for co-operation with collaborative robots.

BACKGROUND: Laparoscopic cholecystectomy is a very frequent surgical procedure. However, in an ageing society, less surgical staff will need to perform surgery on patients. Collaborative surgical robots (cobots) could address surgical staff shortages and workload. To achieve context-awareness for surgeon-robot collaboration, the intraoperative action workflow recognition is a key challenge. METHODS: A surgical process model was developed for intraoperative surgical activities including actor, instrument, action and target in laparoscopic cholecystectomy (excluding camera guidance). These activities, as well as instrument presence and surgical phases were annotated in videos of laparoscopic cholecystectomy performed on human patients (n = 10) and on explanted porcine livers (n = 10). The machine learning algorithm Distilled-Swin was trained on our own annotated dataset and the CholecT45 dataset. The validation of the model was conducted using a fivefold cross-validation approach. RESULTS: In total, 22,351 activities were annotated with a cumulative duration of 24.9 h of video segments. The machine learning algorithm trained and validated on our own dataset scored a mean average precision (mAP) of 25.7% and a top K = 5 accuracy of 85.3%. With training and validation on our dataset and CholecT45, the algorithm scored a mAP of 37.9%. CONCLUSIONS: An activity model was developed and applied for the fine-granular annotation of laparoscopic cholecystectomies in two surgical settings. A machine recognition algorithm trained on our own annotated dataset and CholecT45 achieved a higher performance than training only on CholecT45 and can recognize frequently occurring activities well, but not infrequent activities. The analysis of an annotated dataset allowed for the quantification of the potential of collaborative surgical robots to address the workload of surgical staff. If collaborative surgical robots could grasp and hold tissue, up to 83.5% of the assistant's tissue interacting tasks (i.e. excluding camera guidance) could be performed by robots.

Ensuring privacy protection in the era of big laparoscopic video data: development and validation of an inside outside discrimination algorithm (IODA).

BACKGROUND: Laparoscopic videos are increasingly being used for surgical artificial intelligence (AI) and big data analysis. The purpose of this study was to ensure data privacy in video recordings of laparoscopic surgery by censoring extraabdominal parts. An inside-outside-discrimination algorithm (IODA) was developed to ensure privacy protection while maximizing the remaining video data. METHODS: IODAs neural network architecture was based on a pretrained AlexNet augmented with a long-short-term-memory. The data set for algorithm training and testing contained a total of 100 laparoscopic surgery videos of 23 different operations with a total video length of 207 h (124 min ± 100 min per video) resulting in 18,507,217 frames (185,965 ± 149,718 frames per video). Each video frame was tagged either as abdominal cavity, trocar, operation site, outside for cleaning, or translucent trocar. For algorithm testing, a stratified fivefold cross-validation was used. RESULTS: The distribution of annotated classes were abdominal cavity 81.39%, trocar 1.39%, outside operation site 16.07%, outside for cleaning 1.08%, and translucent trocar 0.07%. Algorithm training on binary or all five classes showed similar excellent results for classifying outside frames with a mean F1-score of 0.96 ± 0.01 and 0.97 ± 0.01, sensitivity of 0.97 ± 0.02 and 0.0.97 ± 0.01, and a false positive rate of 0.99 ± 0.01 and 0.99 ± 0.01, respectively. CONCLUSION: IODA is able to discriminate between inside and outside with a high certainty. In particular, only a few outside frames are misclassified as inside and therefore at risk for privacy breach. The anonymized videos can be used for multi-centric development of surgical AI, quality management or educational purposes. In contrast to expensive commercial solutions, IODA is made open source and can be improved by the scientific community.

Does speed equal quality? Time pressure impairs minimally invasive surgical skills in a prospective crossover trial.

BACKGROUND: Time pressure can cause stress, subsequently influencing surgeons during minimally invasive procedures. This trial aimed to investigate the effect of time pressure on surgical quality, as assessed by force application and errors during minimally invasive surgical tasks. METHODS: Sixty-three participants (43 surgical novices trained to proficiency and 20 surgeons) performed four laparoscopic tasks (PEG transfer, precise Cutting, balloon resection, surgical knot) both with and without time pressure. The primary endpoint was the mean and maximal force exertion during each task. Secondary endpoints were the occurrence of predefined errors and the self-assessed stress level. RESULTS: Time pressure led to a significant shortening of the task time in all four tasks. However, significantly more errors were noticed under time pressure in one task (suture precision P < 0.001). Moreover, time pressure led to a significant increase in mean force in all tasks (PEG: P < 0.001; precision cutting: P = 0.001; surgical knot: P < 0.001; balloon: P = 0.004). In three tasks the maximal force application (PEG: P < 0.001; precision cutting: P < 0.001; surgical knot: P = 0.006) increased significantly. Performing the tasks under time pressure significantly increased the stress level. Cohort analysis revealed that time pressure impaired the performance of both, surgical novices and surgeons but novices were more strongly affected compared to surgeons. CONCLUSION: Time pressure during minimally invasive surgery may improve procedural time but impair the quality of surgical performance in terms of the incidence of errors and force exertion. Experience may only partially compensate for the negative influence of time pressure.

[Technical innovations and future perspectives]. / Technische Innovationen und Blick in die Zukunft.

BACKGROUND: Digital systems have increasingly become integrated into the modern operating room in the last few decades. This has brought about a massive change, especially in minimally invasive surgery. OBJECTIVE: The article provides an overview of the current technical innovations and the perspectives of digitalization and artificial intelligence (AI) in surgery. MATERIAL AND METHODS: The article is based on a literature search via PubMed and research work by the participating coauthors. RESULTS: Current research is increasingly looking at machine learning techniques that take advantage of the complex data in surgery; however, the integration of artificial intelligence systems into the operating room and clinical practice has only just begun. DISCUSSION: Translational research of artificial intelligence in surgery is still in its infancy but has great potential to improve patient care; however, to accelerate the incorporation of intelligent systems into the clinical practice, the creation of interdisciplinary research groups led by surgeons is necessary.

Nanoscale Spin Manipulation with Pulsed Magnetic Gradient Fields from a Hard Disc Drive Writer.

The individual and coherent control of solid-state based electron spins is important covering fields from quantum information processing and quantum metrology to material research and medical imaging. Especially for the control of individual spins in nanoscale networks, the generation of strong, fast, and localized magnetic fields is crucial. Highly engineered devices that demonstrate most of the desired features are found in nanometer size magnetic writers of hard disk drives (HDD). Currently, however, their nanoscale operation in particular comes at the cost of excessive magnetic noise. Here, we present HDD writers as a tool for the efficient manipulation of single as well as multiple spins. We show that their tunable gradients of up to 100 µT/nm can be used to spectrally address individual spins on the nanoscale. Their gigahertz bandwidth allows one to switch control fields within nanoseconds, faster than characteristic time scales such as Rabi and Larmor periods, spin-spin couplings, or optical transitions, thus extending the set of feasible spin manipulations. We used the fields to drive spin transitions through nonadiabatic fast passages or to enable the optical readout of spin states in strong misaligned fields. Building on these techniques, we further apply the large magnetic field gradients for microwave selective addressing of single spins and show its use for the nanoscale optical colocalization of two emitters.

Comparative validation of single-shot optical techniques for laparoscopic 3-D surface reconstruction.

Intra-operative imaging techniques for obtaining the shape and morphology of soft-tissue surfaces in vivo are a key enabling technology for advanced surgical systems. Different optical techniques for 3-D surface reconstruction in laparoscopy have been proposed, however, so far no quantitative and comparative validation has been performed. Furthermore, robustness of the methods to clinically important factors like smoke or bleeding has not yet been assessed. To address these issues, we have formed a joint international initiative with the aim of validating different state-of-the-art passive and active reconstruction methods in a comparative manner. In this comprehensive in vitro study, we investigated reconstruction accuracy using different organs with various shape and texture and also tested reconstruction robustness with respect to a number of factors like the pose of the endoscope as well as the amount of blood or smoke present in the scene. The study suggests complementary advantages of the different techniques with respect to accuracy, robustness, point density, hardware complexity and computation time. While reconstruction accuracy under ideal conditions was generally high, robustness is a remaining issue to be addressed. Future work should include sensor fusion and in vivo validation studies in a specific clinical context. To trigger further research in surface reconstruction, stereoscopic data of the study will be made publically available at www.open-CAS.com upon publication of the paper.