ABSTRACT
Stereo matching and instrument segmentation of laparoscopic surgical scenarios are key tasks in robotic surgical automation. Many researchers have been studying the two tasks separately for stereo matching and instrument segmentation. However, the relationship between these two tasks is often neglected. In this paper, we propose a model framework for multi-tasking with complementary functions for stereo matching and surgical instrument segmentation (MCF-SMSIS). We aim to complement the features of instrument prediction segmentation to the parallax matching block of stereo matching. We also propose two new evaluation metrics (MINPD and MAXPD) for assessing how well the parallax range matches the migrated domain when the model used for the stereo matching task undergoes domain migration. We performed stereo matching experiments on the SCARED , SERV-CT dataset as well as instrumentation segmentation experiments on the AutoLaparo dataset. The results demonstrate the effectiveness of the proposed method. In particular, stereo matching supplemented with instrument features reduced EPE, >3px and RMSE Depth in the surgical instrument section by 9.5%, 12.7% and 6.51%, respectively. The instrumentation segmentation performance also achieves a DSC value of 0.9233. Moreover, MCF-SMSIS takes only 0.14 s to infer a set of images. The model code and model weights for each stage are available from https://github.com/wurenkai/MCF-SMSIS.
ABSTRACT
The use of computer-assisted clinical dermatologists to diagnose skin diseases is an important aid. And computer-assisted techniques mainly use deep neural networks. Recently, the proposal of higher-order spatial interaction operations in deep neural networks has attracted a lot of attention. It has the advantages of both convolution and transformers, and additionally has the advantages of efficient, extensible and translation-equivariant. However, the selection of the interaction order in higher-order interaction operations requires tedious manual selection of a suitable interaction order. In this paper, a hybrid selective higher-order interaction U-shaped model HSH-UNet is proposed to solve the problem that requires manual selection of the order. Specifically, we design a hybrid selective high-order interaction module HSHB embedded in the U-shaped model. The HSHB adaptively selects the appropriate order for the interaction operation channel-by-channel under the computationally obtained guiding features. The hybrid order interaction also solves the problem of fixed order of interaction at each level. We performed extensive experiments on three public skin lesion datasets and our own dataset to validate the effectiveness of our proposed method. The ablation experiments demonstrate the effectiveness of our hybrid selective higher order interaction module. The comparison with state-of-the-art methods also demonstrates the superiority of our proposed HSH-UNet performance. The code is available at https://github.com/wurenkai/HSH-UNet.
