Intraoperative Needle Tip Tracking with an Integrated Fibre-Optic Ultrasound Sensor.

Ultrasound is an essential tool for guidance of many minimally-invasive surgical and interventional procedures, where accurate placement of the interventional device is critical to avoid adverse events. Needle insertion procedures for anaesthesia, fetal medicine and tumour biopsy are commonly ultrasound-guided, and misplacement of the needle may lead to complications such as nerve damage, organ injury or pregnancy loss. Clear visibility of the needle tip is therefore critical, but visibility is often precluded by tissue heterogeneities or specular reflections from the needle shaft. This paper presents the in vitro and ex vivo accuracy of a new, real-time, ultrasound needle tip tracking system for guidance of fetal interventions. A fibre-optic, Fabry-Pérot interferometer hydrophone is integrated into an intraoperative needle and used to localise the needle tip within a handheld ultrasound field. While previous, related work has been based on research ultrasound systems with bespoke transmission sequences, the new system-developed under the ISO 13485 Medical Devices quality standard-operates as an adjunct to a commercial ultrasound imaging system and therefore provides the image quality expected in the clinic, superimposing a cross-hair onto the ultrasound image at the needle tip position. Tracking accuracy was determined by translating the needle tip to 356 known positions in the ultrasound field of view in a tank of water, and by comparison to manual labelling of the the position of the needle in B-mode US images during an insertion into an ex vivo phantom. In water, the mean distance between tracked and true positions was 0.7 ± 0.4 mm with a mean repeatability of 0.3 ± 0.2 mm. In the tissue phantom, the mean distance between tracked and labelled positions was 1.1 ± 0.7 mm. Tracking performance was found to be independent of needle angle. The study demonstrates the performance and clinical compatibility of ultrasound needle tracking, an essential step towards a first-in-human study.

Ultrasonic Needle Tracking with Dynamic Electronic Focusing.

Accurate identification of the needle tip is a key challenge with ultrasound-guided percutaneous interventions in regional anaesthesia, foetal surgery and cardiovascular medicine. In this study, we developed an ultrasonic needle tracking system in which the measured needle tip location was used to set the electronic focus of the external ultrasound imaging probe. In this system, needle tip tracking was enabled with a fibre-optic ultrasound sensor that was integrated into a needle stylet, and the A-lines recorded by the sensor were processed to generate tracking images of the needle tip. The needle tip position was estimated from the tracking images. The dependency of the tracking image on the electronic focal depth of the external ultrasound imaging probe was studied in a water bath and with needle insertions into a clinical training phantom. The variability in the estimated tracked position of the needle tip, with the needle tip at fixed depths in the imaging plane across a depth range from 0.5 to 7.5 cm, was studied. When the electronic focus was fixed, the variability of tracked position was found to increase with distance from that focus. The variability with the fixed focus was found to depend on the the relative distance between the needle tip and focal depth. It was found that with dynamic focusing, the maximum variability of tracked position was below 0.31 mm, as compared with 3.97 mm for a fixed focus.

Pruning strategies for efficient online globally consistent mosaicking in fetoscopy.

Twin-to-twin transfusion syndrome is a condition in which identical twins share a certain pattern of vascular connections in the placenta. This leads to an imbalance in the blood flow that, if not treated, may result in a fatal outcome for both twins. To treat this condition, a surgeon explores the placenta with a fetoscope to find and photocoagulate all intertwin vascular connections. However, the reduced field of view of the fetoscope complicates their localization and general overview. A much more effective exploration could be achieved with an online mosaic created at exploration time. Currently, accurate, globally consistent algorithms such as bundle adjustment cannot be used due to their offline nature, while online algorithms lack sufficient accuracy. We introduce two pruning strategies facilitating the use of bundle adjustment in a sequential fashion: (1) a technique that efficiently exploits the potential of using an electromagnetic tracking system to avoid unnecessary matching attempts between spatially inconsistent image pairs, and (2) an aggregated representation of images, which we refer to as superframes, that allows decreasing the computational complexity of a globally consistent approach. Quantitative and qualitative results on synthetic and phantom-based datasets demonstrate a better trade-off between efficiency and accuracy.

NiftyNet: a deep-learning platform for medical imaging.

BACKGROUND AND OBJECTIVES: Medical image analysis and computer-assisted intervention problems are increasingly being addressed with deep-learning-based solutions. Established deep-learning platforms are flexible but do not provide specific functionality for medical image analysis and adapting them for this domain of application requires substantial implementation effort. Consequently, there has been substantial duplication of effort and incompatible infrastructure developed across many research groups. This work presents the open-source NiftyNet platform for deep learning in medical imaging. The ambition of NiftyNet is to accelerate and simplify the development of these solutions, and to provide a common mechanism for disseminating research outputs for the community to use, adapt and build upon. METHODS: The NiftyNet infrastructure provides a modular deep-learning pipeline for a range of medical imaging applications including segmentation, regression, image generation and representation learning applications. Components of the NiftyNet pipeline including data loading, data augmentation, network architectures, loss functions and evaluation metrics are tailored to, and take advantage of, the idiosyncracies of medical image analysis and computer-assisted intervention. NiftyNet is built on the TensorFlow framework and supports features such as TensorBoard visualization of 2D and 3D images and computational graphs by default. RESULTS: We present three illustrative medical image analysis applications built using NiftyNet infrastructure: (1) segmentation of multiple abdominal organs from computed tomography; (2) image regression to predict computed tomography attenuation maps from brain magnetic resonance images; and (3) generation of simulated ultrasound images for specified anatomical poses. CONCLUSIONS: The NiftyNet infrastructure enables researchers to rapidly develop and distribute deep learning solutions for segmentation, regression, image generation and representation learning applications, or extend the platform to new applications.

Probabilistic visual and electromagnetic data fusion for robust drift-free sequential mosaicking: application to fetoscopy.

The most effective treatment for twin-to-twin transfusion syndrome is laser photocoagulation of the shared vascular anastomoses in the placenta. Vascular connections are extremely challenging to locate due to their caliber and the reduced field-of-view of the fetoscope. Therefore, mosaicking techniques are beneficial to expand the scene, facilitate navigation, and allow vessel photocoagulation decision-making. Local vision-based mosaicking algorithms inherently drift over time due to the use of pairwise transformations. We propose the use of an electromagnetic tracker (EMT) sensor mounted at the tip of the fetoscope to obtain camera pose measurements, which we incorporate into a probabilistic framework with frame-to-frame visual information to achieve globally consistent sequential mosaics. We parametrize the problem in terms of plane and camera poses constrained by EMT measurements to enforce global consistency while leveraging pairwise image relationships in a sequential fashion through the use of local bundle adjustment. We show that our approach is drift-free and performs similarly to state-of-the-art global alignment techniques like bundle adjustment albeit with much less computational burden. Additionally, we propose a version of bundle adjustment that uses EMT information. We demonstrate the robustness to EMT noise and loss of visual information and evaluate mosaics for synthetic, phantom-based and ex vivo datasets.

Medical-grade Sterilizable Target for Fluid-immersed Fetoscope Optical Distortion Calibration.

We have developed a calibration target for use with fluid-immersed endoscopes within the context of the GIFT-Surg (Guided Instrumentation for Fetal Therapy and Surgery) project. One of the aims of this project is to engineer novel, real-time image processing methods for intra-operative use in the treatment of congenital birth defects, such as spina bifida and the twin-to-twin transfusion syndrome. The developed target allows for the sterility-preserving optical distortion calibration of endoscopes within a few minutes. Good optical distortion calibration and compensation are important for mitigating undesirable effects like radial distortions, which not only hamper accurate imaging using existing endoscopic technology during fetal surgery, but also make acquired images less suitable for potentially very useful image computing applications, like real-time mosaicing. In this paper proposes a novel fabrication method to create an affordable, sterilizable calibration target suitable for use in a clinical setup. This method involves etching a calibration pattern by laser cutting a sandblasted stainless steel sheet. This target was validated using the camera calibration module provided by OpenCV, a state-of-the-art software library popular in the computer vision community.

GIFT-Cloud: A data sharing and collaboration platform for medical imaging research.

OBJECTIVES: Clinical imaging data are essential for developing research software for computer-aided diagnosis, treatment planning and image-guided surgery, yet existing systems are poorly suited for data sharing between healthcare and academia: research systems rarely provide an integrated approach for data exchange with clinicians; hospital systems are focused towards clinical patient care with limited access for external researchers; and safe haven environments are not well suited to algorithm development. We have established GIFT-Cloud, a data and medical image sharing platform, to meet the needs of GIFT-Surg, an international research collaboration that is developing novel imaging methods for fetal surgery. GIFT-Cloud also has general applicability to other areas of imaging research. METHODS: GIFT-Cloud builds upon well-established cross-platform technologies. The Server provides secure anonymised data storage, direct web-based data access and a REST API for integrating external software. The Uploader provides automated on-site anonymisation, encryption and data upload. Gateways provide a seamless process for uploading medical data from clinical systems to the research server. RESULTS: GIFT-Cloud has been implemented in a multi-centre study for fetal medicine research. We present a case study of placental segmentation for pre-operative surgical planning, showing how GIFT-Cloud underpins the research and integrates with the clinical workflow. CONCLUSIONS: GIFT-Cloud simplifies the transfer of imaging data from clinical to research institutions, facilitating the development and validation of medical research software and the sharing of results back to the clinical partners. GIFT-Cloud supports collaboration between multiple healthcare and research institutions while satisfying the demands of patient confidentiality, data security and data ownership.

Acquisition models in intraoperative positron surface imaging.

PURPOSE: Intraoperative imaging aims at identifying residual tumor during surgery. Positron Surface Imaging (PSI) is one of the solutions to help surgeons in a better detection of resection margins of brain tumor, leading to an improved patient outcome. This system relies on a tracked freehand beta probe, using [Formula: see text]F-based radiotracer. Some acquisition models have been proposed in the literature in order to enhance image quality, but no comparative validation study has been performed for PSI. METHODS: In this study, we investigated the performance of different acquisition models by considering validation criteria and normalized metrics. We proposed a reference-based validation framework to perform the comparative study between acquisition models and a basic method. We estimated the performance of several acquisition models in light of four validation criteria: efficiency, computational speed, spatial accuracy and tumor contrast. RESULTS: Selected acquisition models outperformed the basic method, albeit with the real-time aspect compromised. One acquisition model yielded the best performance among all according to the validation criteria: efficiency (1-Spe: 0.1, Se: 0.94), spatial accuracy (max Dice: 0.77) and tumor contrast (max T/B: 5.2). We also found out that above a minimum threshold value of the sampling rate, the reconstruction quality does not vary significantly. CONCLUSION: Our method allowed the comparison of different acquisition models and highlighted one of them according to our validation criteria. This novel approach can be extended to 3D datasets, for validation of future acquisition models dedicated to intraoperative guidance of brain surgery.

Detection models for freehand SPECT reconstruction.

Nuclear imaging modalities are commonly used tools in today's diagnostics and therapy planning. However for interventional use they suffer from drawbacks which limit their application. Freehand SPECT was developed to provide 3D functional imaging during interventions. It combines a nuclear detector with an optical tracking system to obtain its position and orientation in space and synchronizes this with the detector readings. This information can be used to compute a 3D tomographic reconstruction of an activity distribution of a nuclear tracer. As there is no fixed geometry, the system matrix has to be computed on the fly. This is done with models of the detection process for completely arbitrary freehand acquisitions. The accuracy of the reconstructions is highly dependent on the used models of the detection process. Different models of the detection process were developed and evaluated in this work, in particular two analytical models as well as lookup tables generated from either real measurements or Monte Carlo simulations. We showed that it is possible to perform acceptable reconstructions with a simple but efficient analytical model. The use of lookup tables to generate the system matrix in Freehand SPECT is a fast solution with good accuracy.

Towards intra-operative PET for head and neck cancer: lymph node localization using high-energy probes.

We present a novel approach for intra-operative localization of lymph nodes and metastases in the head and neck region using the radio-tracer [18F]FDG. By combining an optical tracking system with a high-energy gamma probe to detect 511keV annihilation gammas, we enable intra-operative PET to visualize activity distributions. Detection of these gammas is modeled ad-hoc analytically, taking into account several factors affecting the detection process. This allows us to iteratively reconstruct the radio-tracer distribution within a localized volume of interest. As a feasibility study we analyze clinical data of 7 patients with tumors in the head and neck region, and derive a realistic neck phantom configuration with [18F]FDG-filled lesions mimicking tumors and lymph nodes. We demonstrate the capabilities and limitations of our approach using that neck phantom. We also outline possible improvements to make our method clinically viable towards less invasive surgeries.