Non-relevant segment recognition via hard example mining under sparsely distributed events.

We propose on/offline hard example mining (HEM) techniques to alleviate the degradation of the generalization performance in the sparse distribution of events in non-relevant segment (NRS) recognition and to examine their utility for long-duration surgery. Through on/offline HEM, higher recognition performance can be achieved by extracting hard examples that help train NRS events, for a given training dataset. Furthermore, we provide two performance measurement metrics to quantitatively evaluate NRS recognition in the clinical field. The existing precision and recall-based performance measurement method provides accurate quantitative statistics. However, it is not an efficient evaluation metric in tasks where false positive recognition errors are fatal, such as NRS recognition. We measured the false discovery rate (FDR) and threat score (TS) to provide quantitative values that meet the needs of the clinical setting. Finally, unlike previous studies, the utility of NRS recognition was improved by applying our model to long-duration surgeries, instead of short-length surgical operations such as cholecystectomy. In addition, the proposed training methodology was applied to robotic and laparoscopic surgery datasets to verify that it can be robustly applied to various clinical environments.

Molecular and cellular basis of sodium sensing in Drosophila labellum.

Appropriate ingestion of salt is essential for physiological processes such as ionic homeostasis and neuronal activity. Generally, low concentrations of salt elicit attraction, while high concentrations elicit aversive responses. Here, we observed that sugar neurons in the L sensilla of the Drosophila labellum cf. responses to NaCl, while sugar neurons in the S-c sensilla do not respond to NaCl, suggesting that gustatory receptor neurons involved in NaCl sensing may employ diverse molecular mechanisms. Through an RNAi screen of the entire Ir and ppk gene families and molecular genetic approaches, we identified IR76b, IR25a, and IR56b as necessary components for NaCl sensing in the Drosophila labellum. Co-expression of these three IRs in heterologous systems such as S2 cells or Xenopus oocytes resulted in a current in response to sodium stimulation, suggesting formation of a sodium-sensing complex. Our results should provide insights for research on the diverse combinations constituting salt receptor complexes.

Deep learning-based surgical phase recognition in laparoscopic cholecystectomy.

Backgrounds/Aims: Artificial intelligence (AI) technology has been used to assess surgery quality, educate, and evaluate surgical performance using video recordings in the minimally invasive surgery era. Much attention has been paid to automating surgical workflow analysis from surgical videos for an effective evaluation to achieve the assessment and evaluation. This study aimed to design a deep learning model to automatically identify surgical phases using laparoscopic cholecystectomy videos and automatically assess the accuracy of recognizing surgical phases. Methods: One hundred and twenty cholecystectomy videos from a public dataset (Cholec80) and 40 laparoscopic cholecystectomy videos recorded between July 2022 and December 2022 at a single institution were collected. These datasets were split into training and testing datasets for the AI model at a 2:1 ratio. Test scenarios were constructed according to structural characteristics of the trained model. No pre- or post-processing of input data or inference output was performed to accurately analyze the effect of the label on model training. Results: A total of 98,234 frames were extracted from 40 cases as test data. The overall accuracy of the model was 91.2%. The most accurate phase was Calot's triangle dissection (F1 score: 0.9421), whereas the least accurate phase was clipping and cutting (F1 score: 0.7761). Conclusions: Our AI model identified phases of laparoscopic cholecystectomy with a high accuracy.

Molecular and cellular organization of odorant binding protein genes in Drosophila.

Chemosensation is important for the survival and reproduction of animals. The odorant binding proteins (OBPs) are thought to be involved in chemosensation together with chemosensory receptors. While OBPs were initially considered to deliver hydrophobic odorants to olfactory receptors in the aqueous lymph solution, recent studies suggest more complex roles in various organs. Here, we use GAL4 transgenes to systematically analyze the expression patterns of all 52 members of the Obp gene family and 3 related chemosensory protein genes in adult Drosophila, focusing on chemosensory organs such as the antenna, maxillary palp, pharynx, and labellum, and other organs such as the brain, ventral nerve cord, leg, wing, and intestine. The OBPs were observed to express in diverse organs and in multiple cell types, suggesting that these proteins can indeed carry out diverse functional roles. Also, we constructed 10 labellar-expressing Obp mutants, and obtained behavioral evidence that these OBPs may be involved in bitter sensing. The resources we constructed should be useful for future Drosophila OBP gene family research.

Visual modalities-based multimodal fusion for surgical phase recognition.

Surgical workflow analysis is essential to help optimize surgery by encouraging efficient communication and the use of resources. However, the performance of phase recognition is limited by the use of information related to the presence of surgical instruments. To address the problem, we propose visual modality-based multimodal fusion for surgical phase recognition to overcome the limited diversity of information such as the presence of instruments. Using the proposed methods, we extracted a visual kinematics-based index related to using instruments, such as movement and their interrelations during surgery. In addition, we improved recognition performance using an effective convolutional neural network (CNN)-based fusion method for visual features and a visual kinematics-based index (VKI). The visual kinematics-based index improves the understanding of a surgical procedure since information is related to instrument interaction. Furthermore, these indices can be extracted in any environment, such as laparoscopic surgery, and help obtain complementary information for system kinematics log errors. The proposed methodology was applied to two multimodal datasets, a virtual reality (VR) simulator-based dataset (PETRAW) and a private distal gastrectomy surgery dataset, to verify that it can help improve recognition performance in clinical environments. We also explored the influence of a visual kinematics-based index to recognize each surgical workflow by the instrument's existence and the instrument's trajectory. Through the experimental results of a distal gastrectomy video dataset, we validated the effectiveness of our proposed fusion approach in surgical phase recognition. The relatively simple yet index-incorporated fusion we propose can yield significant performance improvements over only CNN-based training and exhibits effective training results compared to fusion based on Transformers, which require a large amount of pre-trained data.

PEg TRAnsfer Workflow recognition challenge report: Do multimodal data improve recognition?

BACKGROUND AND OBJECTIVE: In order to be context-aware, computer-assisted surgical systems require accurate, real-time automatic surgical workflow recognition. In the past several years, surgical video has been the most commonly-used modality for surgical workflow recognition. But with the democratization of robot-assisted surgery, new modalities, such as kinematics, are now accessible. Some previous methods use these new modalities as input for their models, but their added value has rarely been studied. This paper presents the design and results of the "PEg TRAnsfer Workflow recognition" (PETRAW) challenge with the objective of developing surgical workflow recognition methods based on one or more modalities and studying their added value. METHODS: The PETRAW challenge included a data set of 150 peg transfer sequences performed on a virtual simulator. This data set included videos, kinematic data, semantic segmentation data, and annotations, which described the workflow at three levels of granularity: phase, step, and activity. Five tasks were proposed to the participants: three were related to the recognition at all granularities simultaneously using a single modality, and two addressed the recognition using multiple modalities. The mean application-dependent balanced accuracy (AD-Accuracy) was used as an evaluation metric to take into account class balance and is more clinically relevant than a frame-by-frame score. RESULTS: Seven teams participated in at least one task with four participating in every task. The best results were obtained by combining video and kinematic data (AD-Accuracy of between 93% and 90% for the four teams that participated in all tasks). CONCLUSION: The improvement of surgical workflow recognition methods using multiple modalities compared with unimodal methods was significant for all teams. However, the longer execution time required for video/kinematic-based methods(compared to only kinematic-based methods) must be considered. Indeed, one must ask if it is wise to increase computing time by 2000 to 20,000% only to increase accuracy by 3%. The PETRAW data set is publicly available at www.synapse.org/PETRAW to encourage further research in surgical workflow recognition.

Reducing redundancy in wireless capsule endoscopy videos.

We eliminate similar frames from a wireless capsule endoscopy video of the human intestines to maximize spatial coverage and minimize the redundancy in images. We combine an intensity correction method with a method based an optical flow and features to detect and reduce near-duplicate images acquired during the repetitive backward and forward egomotions due to peristalsis. In experiments, this technique reduced duplicate image of 52.3% from images of the small intestine.