Towards multimodal graph neural networks for surgical instrument anticipation.

PURPOSE: Decision support systems and context-aware assistance in the operating room have emerged as the key clinical applications supporting surgeons in their daily work and are generally based on single modalities. The model- and knowledge-based integration of multimodal data as a basis for decision support systems that can dynamically adapt to the surgical workflow has not yet been established. Therefore, we propose a knowledge-enhanced method for fusing multimodal data for anticipation tasks. METHODS: We developed a holistic, multimodal graph-based approach combining imaging and non-imaging information in a knowledge graph representing the intraoperative scene of a surgery. Node and edge features of the knowledge graph are extracted from suitable data sources in the operating room using machine learning. A spatiotemporal graph neural network architecture subsequently allows for interpretation of relational and temporal patterns within the knowledge graph. We apply our approach to the downstream task of instrument anticipation while presenting a suitable modeling and evaluation strategy for this task. RESULTS: Our approach achieves an F1 score of 66.86% in terms of instrument anticipation, allowing for a seamless surgical workflow and adding a valuable impact for surgical decision support systems. A resting recall of 63.33% indicates the non-prematurity of the anticipations. CONCLUSION: This work shows how multimodal data can be combined with the topological properties of an operating room in a graph-based approach. Our multimodal graph architecture serves as a basis for context-sensitive decision support systems in laparoscopic surgery considering a comprehensive intraoperative operating scene.

Improving Theatre Productivity by Digitising Surgical Equipment Repairs.

Introduction A few cancelled surgeries are due to surgical equipment issues representing a significant burden to both patients and National Health Service (NHS) hospitals on waiting lists. Despite this, there remain very few strategies designed to tackle these avoidable cancellations, especially in combination with digitisation. Our aim was to demonstrate improved efficiency through a pilot study in collaboration with Broomfield Hospital (Broomfield, United Kingdom), MediShout Ltd (London, United Kingdom), and B. Braun Medical Ltd (Sheffield, United Kingdom) with the digitalisation of the equipment repair pathway. Methods MediShout digitised two distinct repair pathways: ad-hoc repairs and maintenance equipment services (MES). Pre- and post-digitisation outcome measures were collected including the number of process steps, staff contribution time, non-staff continuation time, turnaround time, cancelled surgeries, planned preventative maintenance compliance, and staff satisfaction. The number of steps, staff contribution time, and non-staff contribution time were calculated using cognitive task analyses and time-motion studies, respectively. Turnaround time and cancellation data were taken from existing hospital data sets and staff satisfaction was measured through two staff surveys. Results Digitising the ad-hoc repair pathway reduced the number of steps by 18 (118 to 100) and saved 74 minutes of total staff time (Broomfield Hospital and B. Braun) per repair, resulting in annual efficiency savings of £21,721.48. Digitising the MES repair pathway reduced the number of steps by 13 (74 to 61) and saved 56 minutes of total staff time per repair, resulting in annual efficiency savings of £3469.44. Turnaround time for the repaired kit decreased by 14 days and 29 days for the digital ad-hoc and digital MES pathways, respectively. Elective operations cancelled due to equipment issues decreased by 44%, from 1.5 operations/month pre-pilot to 0.83 operations/month post-pilot. Planned preventative maintenance compliance across the MES pathway increased by 67% (33% to 100%). Staff satisfaction with the repair pathway improved from 12% to 96%. Conclusion This pilot study showcases the numerous benefits that can be achieved through digitisation and offers an innovative case study to approach avoidable cancellations due to equipment failure.

Application of microwave surgical instrument to lung segmentectomy for small-sized lung cancer.

Objectives: For lung segmentectomy of small lung cancers, we used a microwave surgical instrument for lung parenchymal dissection mainly at the pulmonary hilum, which is difficult to handle with surgical staplers. This technique facilitated the insertion of staples. Methods: In total, 116 patients with cStage 0-1A3 non-small cell lung cancer who underwent lung segmentectomy were included in this study. We compared the perioperative factors of the group in which a microwave surgical instrument was used for dissection procedures, including lung parenchymal dissection at the pulmonary hilum, and peripheral intersegmental dissection was performed with surgical staplers (group M+S: N = 69), with those of the group in which parenchymal dissection was performed mainly with surgical staplers without using the microwave surgical instrument (group S: N = 47). Results: Although more complex segmentectomies were performed in the M+S group (P = .001), the number of staple cartridges (7 staple cartridges vs 8 staple cartridges, P = .005), the surgical times (179 vs 221 minutes, P < .0001), and the blood loss (5 mL vs 30 mL, P = .012) were significantly lower in the M+S group. The duration of chest tube placement was significantly shorter in the M+S group (P = .019), and postoperative complications of grade 2 or greater were significantly lower in the M+S group (P = .049). Conclusions: The effective use of the microwave surgical instrument combined with surgical staplers can simplify pulmonary hilar and intersegmental plane dissections not only for simple segmentectomy but also for complex segmentectomy, leading to favorable intraoperative and postoperative outcomes.

Enhancing surgical instrument segmentation: integrating vision transformer insights with adapter.

PURPOSE: In surgical image segmentation, a major challenge is the extensive time and resources required to gather large-scale annotated datasets. Given the scarcity of annotated data in this field, our work aims to develop a model that achieves competitive performance with training on limited datasets, while also enhancing model robustness in various surgical scenarios. METHODS: We propose a method that harnesses the strengths of pre-trained Vision Transformers (ViTs) and data efficiency of convolutional neural networks (CNNs). Specifically, we demonstrate how a CNN segmentation model can be used as a lightweight adapter for a frozen ViT feature encoder. Our novel feature adapter uses cross-attention modules that merge the multiscale features derived from the CNN encoder with feature embeddings from ViT, ensuring integration of the global insights from ViT along with local information from CNN. RESULTS: Extensive experiments demonstrate our method outperforms current models in surgical instrument segmentation. Specifically, it achieves superior performance in binary segmentation on the Robust-MIS 2019 dataset, as well as in multiclass segmentation tasks on the EndoVis 2017 and EndoVis 2018 datasets. It also showcases remarkable robustness through cross-dataset validation across these 3 datasets, along with the CholecSeg8k and AutoLaparo datasets. Ablation studies based on the datasets prove the efficacy of our novel adapter module. CONCLUSION: In this study, we presented a novel approach integrating ViT and CNN. Our unique feature adapter successfully combines the global insights of ViT with the local, multi-scale spatial capabilities of CNN. This integration effectively overcomes data limitations in surgical instrument segmentation. The source code is available at: https://github.com/weimengmeng1999/AdapterSIS.git .

Enabling the design of surgical instruments for under-resourced patients through metal additive manufacturing: ulnar shortening osteotomy as an example.

BACKGROUND: Ulnar shortening osteotomy (USO) has demonstrated good outcomes for patients with ulnar impaction syndrome. To minimize complications such as non-union, precise osteotomy and firm fixation are warranted. Despite various ulnar shortening systems have been developed, current technology does not meet all needs. A considerable portion of patients could not afford those designated USO systems. To tackle this challenge, our team reported successful results in standardized free-hand predrilled USO technique. However, it is still technical demanding and requires sufficient experience and confidence to excel. Therefore, our team designed an ulnar shortening system based on our free-hand technique principle, using metal additive manufacturing technology. The goal of this study is to describe the development process and report the performance of the system. METHODS: Utilizing metal additive manufacturing technology, our team developed an ulnar shortening system that requires minimal exposure, facilitates precise cutting, and allows for the easy placement of a 3.5 mm dynamic compression plate, available to patients at zero out-of-pocket cost. For performance testing, two surgeons with different levels of experience in ulnar shortening procedures were included: one fellow-trained hand and wrist surgeon and one senior resident. They performed ulnar shortening osteotomy (USO) using both the free-hand technique and the USO system-assisted technique on ulna sawbones, repeating each method three times. The recorded parameters included time-to-complete-osteotomy, total procedure time, chip diameter, shortening length, maximum residual gap, and deviation angle. RESULTS: For the hand and wrist fellow, with the USO system, the time-to-complete osteotomy was significantly reduced. (468.7 ± 63.6 to 260.0 ± 5 s, p < 0.05). Despite the preop goal was shortening 3 mm, the average shortening length was significantly larger in the free-hand group (5 ± 0.1; 3.2 ± 0.2 mm, p < 0.05). Both maximum residual gap and deviation angle reported no statistical difference between the two techniques for the hand surgeon. As for the senior resident, the maximum residual gap was significantly reduced, using the USO system (2.9 ± 0.8; 0.4 ± 0.4 mm, p = 0.02). Between two surgeons, significant larger maximum residual gap and deviation angle were noted on the senior resident doctor, in the free-hand technique group, but not in the USO system group. CONCLUSION: The developed USO system may serve as a valuable tool, aiding in reliable and precise cutting as well as fixation for patients undergoing ulnar shortening osteotomy with a 3.5 mm dynamic compression plate, even for less experienced surgeons. The entire process, from concept generation and sketching to creating the CAD file and final production, serves as a translatable reference for other surgical scenarios.

DBH-YOLO: a surgical instrument detection method based on feature separation in laparoscopic surgery.

PURPOSE: Accurately locating and analysing surgical instruments in laparoscopic surgical videos can assist doctors in postoperative quality assessment. This can provide patients with more scientific and rational solutions for healing surgical complications. Therefore, we propose an end-to-end algorithm for the detection of surgical instruments. METHODS: Dual-Branched Head (DBH) and Overall Intersection over Union Loss (OIoU Loss) are introduced to solve the problem of inaccurate surgical instrument detection, both in terms of localization and classification. An effective method (DBHYOLO) for the detection for laparoscopic surgery in complex scenarios is proposed. This study manually annotates a new laparoscopic gastric cancer resection surgical instrument location dataset LGIL, which provides a better validation platform for surgical instrument detection methods. RESULTS: The proposed method's performance was tested using the m2cai16-tool-locations, LGIL, and Onyeogulu datasets. The mean Average Precision (mAP) values obtained were 96.8%, 95.6%, and 98.4%, respectively, which were higher than the other classical models compared. The improved model is more effective than the benchmark network in distinguishing between surgical instrument classes with high similarity and avoiding too many missed detection cases. CONCLUSIONS: In this paper, the problem of inaccurate detection of surgical instruments is addressed from two different perspectives: classification and localization. And the experimental results on three representative datasets verify the performance of DBH-YOLO. It is shown that this method has a good generalization capability.

Camera sheath with transformable head for minimally invasive surgical instruments.

INTRODUCTION: This paper presents a camera sheath that can be assembled to various minimally invasive surgical instruments and provide the localized view of the instrument tip. MATERIAL AND METHODS: The advanced transformable head structure (ATHS) that overcomes the trade-off between the camera resolution and the instrument size is designed for the sheath. Design solutions to maintain the alignment between the camera's line of sight and the instrument tip direction during the transformation of the ATHS are derived and applied to the prototype of the sheath. RESULTS: The design solution ensured proper alignment between the line of sight and the tip direction. The prototype was used with the curved micro-debrider blades in simulated functional endoscopic sinus surgery (FESS). Deep regions of the sinus that were not observable with the conventional endoscopes was accessed and observed using the prototype. CONCLUSIONS: The presented camera sheath allows the delivery of the instrument and camera to the surgical site with minimal increase in port size. It may be applied to various surgeries to reduce invasiveness and provide additional visual information to the surgeons.

Observed rates of surgical instrument errors point to visualization tasks as being a critically vulnerable point in sterile processing and a significant cause of lost chargeable OR minutes.

BACKGROUND: The reporting of surgical instrument errors historically relies on cumbersome, non-automated, human-dependent, data entry into a computer database that is not integrated into the electronic medical record. The limitations of these reporting systems make it difficult to accurately estimate the negative impact of surgical instrument errors on operating room efficiencies. We set out to determine the impact of surgical instrument errors on a two-hospital healthcare campus using independent observers trained in the identification of Surgical Instrument Errors. METHODS: This study was conducted in the 7 pediatric ORs at an academic healthcare campus. Direct observations were conducted over the summer of 2021 in the 7 pediatric ORs by 24 trained student observers during elective OR days. Surgical service line, error type, case type (inpatient or outpatient), and associated length of delay were recorded. RESULTS: There were 236 observed errors affecting 147 individual surgical cases. The three most common errors were Missing+ (n = 160), Broken/poorly functioning instruments (n = 44), and Tray+ (n = 13). Errors arising from failures in visualization (i.e. inspection, identification, function) accounted for 88.6% of all errors (Missing+/Broken/Bioburden). Significantly more inpatient cases (42.73%) had errors than outpatient cases (22.32%) (p = 0.0129). For cases in which data was collected on whether an error caused a delay (103), over 50% of both IP and OP cases experienced a delay. The average length of delays per case was 10.16 min. The annual lost charges in dollars for surgical instrument associated delays in chargeable minutes was estimated to be between $6,751,058.06 and $9,421,590.11. CONCLUSIONS: These data indicate that elimination of surgical instrument errors should be a major target of waste reduction. Most observed errors (88.6%) have to do with failures in the visualization required to identify, determine functionality, detect the presence of bioburden, and assemble instruments into the correct trays. To reduce these errors and associated waste, technological advances in instrument identification, inspection, and assembly will need to be made and applied to the process of sterile processing.

Intraoperative robotic surgical system-related problems in robot-assisted thoracoscopic surgery.

BACKGROUND: Malfunctions of robotic instruments during robotic surgery are well known to occur; however, detailed reports on the inherent problems associated with robotic instruments and robotic surgical systems are scarce. The objective of this study was to retrospectively investigate the intraoperative problems associated with robotic surgical systems and robotic instruments. MATERIALS AND METHODS: This was a single-center retrospective study. Between April 2012 and December 2022, 544 patients with consecutive lung malignancies and/or mediastinal tumors underwent robot-assisted thoracoscopic surgery. Among these, 15 cases had intraoperative problems associated with the robotic surgical system. Human error was defined as a problem caused by the incorrect operation of the robotic surgical system and human factors as problems in which the robotic surgical system stopped owing to damage to the instruments of the robotic surgical system or the self-diagnosis of the robotic surgical system. We retrospectively investigated the causes of intraoperative problems in these cases. RESULTS: There were 4 cases (0.7%) with problems related to the robotic surgical system, 2 of which were human errors, and 11 (2.0%) with problems related to robotic surgical instruments, 6 of these were related to instruments and 5 were related to robotic staplers. Five of these were related to human factors. CONCLUSION: Teams performing robot-assisted thoracoscopic surgery should be familiar with the features of robotic surgical systems and various robotic devices, be aware of reported problems during robot-assisted thoracoscopic surgery, and be prepared for emergencies.

Distributed visual positioning for surgical instrument tracking.

In clinical operations, it is crucial for surgeons to know the location of the surgical instrument. Traditional positioning systems have difficulty dealing with camera occlusion, marker occlusion, and environmental interference. To address these issues, we propose a distributed visual positioning system for surgical instrument tracking in surgery. First, we design the marker pattern with a black and white triangular grid and dot that can be adapted to various instrument surfaces and improve the marker location accuracy of the feature. The cross-points in the marker are the features that each feature has a unique ID. Furthermore, we proposed detection and identification for the position-sensing marker to realize the accurate location and identification of features. Second, we introduce multi Perspective-n-Point (mPnP) method, which fuses feature coordinates from all cameras to deduce the final result directly by the intrinsic and extrinsic parameters. This method provides a reliable initial value for the Bundle Adjustment algorithms. During instrument tracking, we assess the motion state of the instrument and select either dynamic or static Kalman filtering to mitigate any jitter in the instrument's movement. The core algorithms comparison experiment indicates our positioning algorithm has a lower reprojection error comparison to the mainstream algorithms. A series of quantitative experiments showed that the proposed system positioning error is below 0.207 mm, and the run time is below 118.842 ms. The results demonstrate the tremendous clinical application potential of our system providing accurate positioning of instruments promoting the efficiency and safety of clinical surgery.

Methods and datasets for segmentation of minimally invasive surgical instruments in endoscopic images and videos: A review of the state of the art.

In the field of computer- and robot-assisted minimally invasive surgery, enormous progress has been made in recent years based on the recognition of surgical instruments in endoscopic images and videos. In particular, the determination of the position and type of instruments is of great interest. Current work involves both spatial and temporal information, with the idea that predicting the movement of surgical tools over time may improve the quality of the final segmentations. The provision of publicly available datasets has recently encouraged the development of new methods, mainly based on deep learning. In this review, we identify and characterize datasets used for method development and evaluation and quantify their frequency of use in the literature. We further present an overview of the current state of research regarding the segmentation and tracking of minimally invasive surgical instruments in endoscopic images and videos. The paper focuses on methods that work purely visually, without markers of any kind attached to the instruments, considering both single-frame semantic and instance segmentation approaches, as well as those that incorporate temporal information. The publications analyzed were identified through the platforms Google Scholar, Web of Science, and PubMed. The search terms used were "instrument segmentation", "instrument tracking", "surgical tool segmentation", and "surgical tool tracking", resulting in a total of 741 articles published between 01/2015 and 07/2023, of which 123 were included using systematic selection criteria. A discussion of the reviewed literature is provided, highlighting existing shortcomings and emphasizing the available potential for future developments.

Observing vessel stumps axially: A novel exoscope for end-to-end microvascular anastomosis.

Although many free tissue transfers have been performed, free flap loss can still occur because of vascular compromise. To facilitate microsurgery, we invented the axial-view microscope (aMS), a new type of microscope that can axially visualize vessel stumps. The aMS was combined with an optical microscope, the so-called bird's-eye-view microscope (bMS). Using our aMS, we observed the cross-sections of the following 12 arteries during vascular anastomosis: three deep inferior epigastric arteries, three suprathyroid arteries, two thoracodorsal arteries, two jejunal arteries, one lateral circumflex femoral artery, and one facial artery. For each artery, we measured the vessel height-to-width (H-W) ratio to determine the roundness of the vessel stump. Based on the aMS and bMS, the average H-W ratios were 0.877 ± 0.187 and 0.445 ± 0.172, respectively. The H-W ratio obtained using the aMS was significantly higher than that of the bMS (P < 0.001). Providing the surgeon with a bidirectional view of the vessel stump reduced blind spots at the anastomotic site. In this report, we describe our new microscope and associated clinical cases.

Odontogenic maxillary sinusitis associated with foreign body displacement after tooth extraction: report of an unusual case / Sinusite maxilar odontogênica por corpo estranho após extração dentária: relato de um caso incomum

Background: Odontogenic maxillary sinusitis caused by a foreign body presents diagnostic and therapeutic challenges due to its infrequent occurrence and unique characteristics compared to sinusitis originating from other sources. CaseReport:Illustrating such fact, this report presents the clinical case of a 37-year-old woman referred complaining of pain in the same region where she had extracted her upper right first molar five days before. The intraoral examination revealed the presence of an orifice in the region, suggesting oroantral communication. Imaging exams revealed opacification of the right maxillary sinus and the unexpected presence of a highly radiodense object. With the diagnosis of maxillary sinusitis due to a foreign body established, the surgical approach initially consisted of administering preoperative medication, preceded by access to the maxillary antrum using the Caldwell-Luc technique. The object was found and removed, consisting of a surgical drill. At follow-up there was complete absence of symptoms and complete closure of communication. Conclusion: Cases of odontogenic maxillary sinusitis caused by drill detachment after tooth extraction are fairly uncommon. A thorough clinical evaluation proved to be essential and the Caldwell-Luc access was effective, safe and with good postoperative results, even with the absence of standardized diagnostic and management methods(AU)

Contexto: A sinusite maxilar odontogênica causada por corpo estranho apresenta desafios diagnósticos e terapêuticos devido à sua ocorrência infrequente e características únicas em comparação com sinusites originadas de outras fontes. Relato do Caso: Ilustrando tal fato, este relato apresenta o caso clínico de uma mulher de 37 anos de idade encaminhada com queixa de dor em mesma região que havia extraído o primeiro molar superior direito cinco dias antes. Ao exame intraoral verificou-se a presença de um orifício na região, sugerindo comunicação oroantral. Os exames de imagem revelaram opacificação do SM direito e a inesperada presença de um objeto altamente radiodenso. Com o diagnóstico de sinusite maxilar por corpo estranho estabelecido, a abordagem cirúrgica consistiu inicialmente na administração de medicação pré-operatória, precedida pelo acesso ao antro maxilar através da técnica de Caldwell-Luc. O objeto foi encontrado e removido, consistindo em uma broca cirúrgica. Ao acompanhamento houve ausência completa dos sintomas e total fechamento da comunicação. Conclusão: Casos de sinusite maxilar odontogênica causada por descolamento da broca após extração dentária são bastante incomuns. Uma avaliação clínica minuciosa mostrou-se primordial e o acesso de Caldwell-Luc eficaz, seguro e com bons resultados pós-operatórios, mesmo com as ausências de métodos de diagnóstico e manejo padronizados.(AU)

CLAD-Net: cross-layer aggregation attention network for real-time endoscopic instrument detection.

As medical treatments continue to advance rapidly, minimally invasive surgery (MIS) has found extensive applications across various clinical procedures. Accurate identification of medical instruments plays a vital role in comprehending surgical situations and facilitating endoscopic image-guided surgical procedures. However, the endoscopic instrument detection poses a great challenge owing to the narrow operating space, with various interfering factors (e.g. smoke, blood, body fluids) and inevitable issues (e.g. mirror reflection, visual obstruction, illumination variation) in the surgery. To promote surgical efficiency and safety in MIS, this paper proposes a cross-layer aggregated attention detection network (CLAD-Net) for accurate and real-time detection of endoscopic instruments in complex surgical scenarios. We propose a cross-layer aggregation attention module to enhance the fusion of features and raise the effectiveness of lateral propagation of feature information. We propose a composite attention mechanism (CAM) to extract contextual information at different scales and model the importance of each channel in the feature map, mitigate the information loss due to feature fusion, and effectively solve the problem of inconsistent target size and low contrast in complex contexts. Moreover, the proposed feature refinement module (RM) enhances the network's ability to extract target edge and detail information by adaptively adjusting the feature weights to fuse different layers of features. The performance of CLAD-Net was evaluated using a public laparoscopic dataset Cholec80 and another set of neuroendoscopic dataset from Sun Yat-sen University Cancer Center. From both datasets and comparisons, CLAD-Net achieves the AP0.5 of 98.9% and 98.6%, respectively, that is better than advanced detection networks. A video for the real-time detection is presented in the following link: https://github.com/A0268/video-demo.

MFF-Net: Multiscale feature fusion semantic segmentation network for intracranial surgical instruments.

BACKGROUND: In robot-assisted surgery, automatic segmentation of surgical instrument images is crucial for surgical safety. The proposed method addresses challenges in the craniotomy environment, such as occlusion and illumination, through an efficient surgical instrument segmentation network. METHODS: The network uses YOLOv8 as the target detection framework and integrates a semantic segmentation head to achieve detection and segmentation capabilities. A concatenation of multi-channel feature maps is designed to enhance model generalisation by fusing deep and shallow features. The innovative GBC2f module ensures the lightweight of the network and the ability to capture global information. RESULTS: Experimental validation of the intracranial glioma surgical instrument dataset shows excellent performance: 94.9% MPA score, 89.9% MIoU value, and 126.6 FPS. CONCLUSIONS: According to the experimental results, the segmentation model proposed in this study has significant advantages over other state-of-the-art models. This provides a valuable reference for the further development of intelligent surgical robots.

Can gamified surgical sets improve surgical instrument recognition and student performance retention in the operating room? A multi-institutional experimental crossover study.

INTRODUCTION: Surgery requires a high degree of precision, speed, and concentration. Owing to the complexity of the modern world, traditional methods cannot meet these requirements. Therefore, in this study, we investigated students' diagnostic skills in the Operating Room in the context of surgical instruments by using gamification of surgical instruments and a crossover design. METHOD: The study design was a multi-institutional quasi-experimental crossover and involved a three-arm intervention (with gender-specific block randomisation: Group A, B, and C) with a pre-test and three post-tests. A total of 90 students fell into three groups of 30 participants each. The surgical sets were learned for one semester through game-based instruction and traditional teaching, and then three OSCE tests were administered with time and location differences. Using one-way ANOVA, OSCE results were compared in the game, traditional, and control groups. The effectiveness of the intervention was tested in each group by repeated measures. RESULT: The pretest scores of all three groups did not differ significantly. In the OSCE tests, both groups, A and B, performed similarly. However, these tests showed a significant difference in grouping between training through games and training in the traditional way. There was no significant difference between OSCE tests 2 and 3 in the game-based training group, indicating that what was learned was retained, while in the traditional method training group, OSCE 3 test scores declined significantly. Furthermore, repeated measures showed the effectiveness of game-based training. CONCLUSION: In this study, gamification has turned out to be very effective in helping learners learn practical skills and leading to more sustainable learning.

InstrumentNet: An integrated model for real-time segmentation of intracranial surgical instruments.

In robot-assisted surgery, precise surgical instrument segmentation technology can provide accurate location and pose data for surgeons, helping them perform a series of surgical operations efficiently and safely. However, there are still some interfering factors, such as surgical instruments being covered by tissue, multiple surgical instruments interlacing with each other, and instrument shaking during surgery. To better address these issues, an effective surgical instrument segmentation network called InstrumentNet is proposed, which adopts YOLOv7 as the object detection framework to achieve a real-time detection solution. Specifically, a multiscale feature fusion network is constructed, which aims to avoid problems such as feature redundancy and feature loss and enhance the generalization ability. Furthermore, an adaptive feature-weighted fusion mechanism is introduced to regulate network learning and convergence. Finally, a semantic segmentation head is introduced to integrate the detection and segmentation functions, and a multitask learning loss function is specifically designed to optimize the surgical instrument segmentation performance. The proposed segmentation model is validated on a dataset of intracranial surgical instruments provided by seven experts from Beijing Tiantan Hospital and achieved an mAP score of 93.5 %, Dice score of 82.49 %, and MIoU score of 85.48 %, demonstrating its universality and superiority. The experimental results demonstrate that the proposed model achieves good segmentation performance on surgical instruments compared to other advanced models and can provide a reference for developing intelligent medical robots.

Evaluating the Feasibility and Reproducibility of a Novel Insertion Method for Modular Acetabular Ceramic Liners.

Modern hip implants have a modular design. In case of wear or other damage it allows surgeons to change the tribological partners, i.e., the acetabular liner and femoral ball. In both revision and primary surgery, the secure joining of the implant components is important for the success of the operation. The two components, namely the ceramic liner and hip cup, are connected via a conical press connection and should be concentrically aligned to avoid chipping. Malseated liners can reduce the life span in situ. The amount of the joining force, which is usually applied via a hammer, depends on the surgeon. In this study, an alternative joining method for acetabular ceramic liners in hip cups was investigated, which intends to make the process more reproducible and thus safer. For this purpose, a handpiece was used to apply a defined force impulse of 4 kN. For the concentric alignment of a ceramic liner in the hip cup, an adapter was developed based on findings via a qualitative finite element (FE) analysis. Insertion and pushout tests of the acetabular cup-liner connection were performed in the laboratory with the new instrument (handpiece with the connected adapter) to evaluate the functionality of the instrument and the reproducibility of the new insertion method. For comparison, liners and acetabular cups were joined using a testing machine according to the standard. The presented results demonstrate the technical proof-of-concept of the new joining method under laboratory conditions. They meet the acceptance criteria of established manufacturers, which proves the equivalency to proven methods for joining modular implant components. To verify the improvement of the new joining method compared to the conventionally used joining method, an application-oriented study with different surgeons and the new joining instrument under clinical conditions is necessary.

Improving instrument detection for a robotic scrub nurse using multi-view voting.

PURPOSE: A basic task of a robotic scrub nurse is surgical instrument detection. Deep learning techniques could potentially address this task; nevertheless, their performance is subject to some degree of error, which could render them unsuitable for real-world applications. In this work, we aim to demonstrate how the combination of a trained instrument detector with an instance-based voting scheme that considers several frames and viewpoints is enough to guarantee a strong improvement in the instrument detection task. METHODS: We exploit the typical setup of a robotic scrub nurse to collect RGB data and point clouds from different viewpoints. Using trained Mask R-CNN models, we obtain predictions from each view. We propose a multi-view voting scheme based on predicted instances that combines the gathered data and predictions to produce a reliable map of the location of the instruments in the scene. RESULTS: Our approach reduces the number of errors by more than 82% compared with the single-view case. On average, the data from five viewpoints are sufficient to infer the correct instrument arrangement with our best model. CONCLUSION: Our approach can drastically improve an instrument detector's performance. Our method is practical and can be applied during an actual medical procedure without negatively affecting the surgical workflow. Our implementation and data are made available for the scientific community ( https://github.com/Jorebs/Multi-view-Voting-Scheme ).