Total Variation Regularization of Pose Signals with an Application to 3D Freehand Ultrasound.

Three-dimensional freehand imaging techniques are gaining wider adoption due to their ?exibility and cost ef?ciency. Typical examples for such a combination of a tracking system with an imaging device are freehand SPECT or freehand 3D ultrasound. However, the quality of the resulting image data is heavily dependent on the skill of the human operator and on the level of noise of the tracking data. The latter aspect can introduce blur or strong artifacts, which can signi?cantly hamper the interpretation of image data. Unfortunately, the most commonly used tracking systems to date, i.e. optical and electromagnetic, present a trade-off between invading the surgeon's workspace (due to line-of-sight requirements) and higher levels of noise and sensitivity due to the interference of surrounding metallic objects. In this work, we propose a novel approach for total variation regularization of data from tracking systems (which we term pose signals) based on a variational formulation in the manifold of Euclidean transformations. The performance of the proposed approach was evaluated using synthetic data as well as real ultrasound sweeps executed on both a Lego phantom and human anatomy, showing signi?cant improvement in terms of tracking data quality and compounded ultrasound images. Source code can be found at https://github.com/IFL-CAMP/pose_regularization.

Robotic ultrasound-guided facet joint insertion.

PURPOSE: Facet joint insertion is a common treatment of chronic pain in the back and spine. This procedure is often performed under fluoroscopic guidance, where the staff's repetitive radiation exposure remains an unsolved problem. Robotic ultrasound (rUS) has the potential to reduce or even eliminate the use of radiation by using ultrasound with a robotic-guided needle insertion. This work presents first clinical data of rUS-based needle insertions extending previous work of our group. METHODS: Our system implements an automatic US acquisition protocol combined with a calibrated needle targeting system. This approach assists the physician by positioning the needle holder on a trajectory selected in a 3D US volume of the spine. RESULTS: By the time of submission, nine facets were treated with our approach as first data from an ongoing clinical study. The insertion success rate was shown to be comparable to current clinical practice. Furthermore, US imaging offers additional anatomical context for needle trajectory planning. CONCLUSION: This work shows first clinical data for robotic ultrasound-assisted facet joint insertion as a promising solution that can easily be incorporated into the clinical workflow. Presented results show the clinical value of such a system.

Trackerless panoramic optoacoustic imaging: a first feasibility evaluation.

PURPOSE: Optoacoustic imaging provides high spatial resolution and the possibility to image specific functional parameters in real-time, therefore positioning itself as a promising modality for various applications. However, despite these advantages, the applicability of real-time optoacoustic imaging is generally limited due to a relatively small field of view. METHODS: With this work, we aim at presenting a path towards panoramic optoacoustic tomographic imaging without requiring additional sensors or position trackers. We propose a two-step seamless stitching method for the compounding of multiple datasets acquired with a real-time 3D optoacoustic imaging system within a panoramic scan. The employed workflow is specifically tailored to the image properties and respective challenges. RESULTS: A comparison of the presented alignment on in-vivo data shows a mean error of [Formula: see text] compared to ground truth tracking data. The presented compounding scheme integrates the physical resolution of optoacoustic data and hence can provide improved contrast in comparison with other compounding approaches based on addition or averaging. CONCLUSION: The proposed method can produce optoacoustic volumes with an enlarged field of view and improved quality compared to current methods in optoacoustic imaging. However, our study also shows challenges for panoramic scans. In this view, we discuss relevant properties, challenges, and opportunities and present an evaluation of the performance of the presented approach with different input data.

Use the force: deformation correction in robotic 3D ultrasound.

PURPOSE: Ultrasound acquisitions are typically affected by deformations due to the pressure applied onto the contact surface. While a certain amount of pressure is necessary to ensure good acoustic coupling and visibility of the anatomy under examination, the caused deformations hinder accurate localization and geometric analysis of anatomical structures. These complications have even greater impact in case of 3D ultrasound scans as they limit the correct reconstruction of acquired volumes. METHODS: In this work, we propose a method to estimate and correct the induced deformation based solely on the tracked ultrasound images and information about the applied force. This is achieved by modeling estimated displacement fields of individual image sequences using the measured force information. By representing the computed displacement fields using a graph-based approach, we are able to recover a deformation-less 3D volume. RESULTS: Validation is performed on 30 in vivo human datasets acquired using a robotic ultrasound framework. Compared to ground truth, the presented deformation correction shows errors of [Formula: see text] for an applied force of 5 N at a penetration depth of 55 mm. CONCLUSION: The proposed technique allows for the correction of deformations induced by the transducer pressure in entire 3D ultrasound volumes. Our technique does not require biomechanical models, patient-specific assumptions or information about the tissue properties; it can be employed based on the information from readily available robotic ultrasound platforms.

SUPRA: open-source software-defined ultrasound processing for real-time applications : A 2D and 3D pipeline from beamforming to B-mode.

PURPOSE: Research in ultrasound imaging is limited in reproducibility by two factors: First, many existing ultrasound pipelines are protected by intellectual property, rendering exchange of code difficult. Second, most pipelines are implemented in special hardware, resulting in limited flexibility of implemented processing steps on such platforms. METHODS: With SUPRA, we propose an open-source pipeline for fully software-defined ultrasound processing for real-time applications to alleviate these problems. Covering all steps from beamforming to output of B-mode images, SUPRA can help improve the reproducibility of results and make modifications to the image acquisition mode accessible to the research community. We evaluate the pipeline qualitatively, quantitatively, and regarding its run time. RESULTS: The pipeline shows image quality comparable to a clinical system and backed by point spread function measurements a comparable resolution. Including all processing stages of a usual ultrasound pipeline, the run-time analysis shows that it can be executed in 2D and 3D on consumer GPUs in real time. CONCLUSIONS: Our software ultrasound pipeline opens up the research in image acquisition. Given access to ultrasound data from early stages (raw channel data, radiofrequency data), it simplifies the development in imaging. Furthermore, it tackles the reproducibility of research results, as code can be shared easily and even be executed without dedicated ultrasound hardware.

3D intra-operative ultrasound and MR image guidance: pursuing an ultrasound-based management of brainshift to enhance neuronavigation.

BACKGROUND: Brainshift is still a major issue in neuronavigation. Incorporating intra-operative ultrasound (iUS) with advanced registration algorithms within the surgical workflow is regarded as a promising approach for a better understanding and management of brainshift. This work is intended to (1) provide three-dimensional (3D) ultrasound reconstructions specifically for brain imaging in order to detect brainshift observed intra-operatively, (2) evaluate a novel iterative intra-operative ultrasound-based deformation correction framework, and (3) validate the performance of the proposed image-registration-based deformation estimation in a clinical environment. METHODS: Eight patients with brain tumors undergoing surgical resection are enrolled in this study. For each patient, a 3D freehand iUS system is employed in combination with an intra-operative navigation (iNav) system, and intra-operative ultrasound data are acquired at three timepoints during surgery. On this foundation, we present a novel resolution-preserving 3D ultrasound reconstruction, as well as a framework to detect brainshift through iterative registration of iUS images. To validate the system, the target registration error (TRE) is evaluated for each patient, and both rigid and elastic registration algorithms are analyzed. RESULTS: The mean TRE based on 3D-iUS improves significantly using the proposed brainshift compensation compared to neuronavigation (iNav) before (2.7 vs. 5.9 mm; [Formula: see text]) and after dural opening (4.2 vs. 6.2 mm, [Formula: see text]), but not after resection (6.7 vs. 7.5 mm; [Formula: see text]). iUS depicts a significant ([Formula: see text]) dynamic spatial brainshift throughout the three timepoints. Accuracy of registration can be improved through rigid and elastic registrations by 29.2 and 33.3%, respectively, after dural opening, and by 5.2 and 0.4%, after resection. CONCLUSION: 3D-iUS systems can improve the detection of brainshift and significantly increase the accuracy of the navigation in a real scenario. 3D-iUS can thus be regarded as a robust, reliable, and feasible technology to enhance neuronavigation.

Acoustic window planning for ultrasound acquisition.

Autonomous robotic ultrasound has recently gained considerable interest, especially for collaborative applications. Existing methods for acquisition trajectory planning are solely based on geometrical considerations, such as the pose of the transducer with respect to the patient surface. PURPOSE: This work aims at establishing acoustic window planning to enable autonomous ultrasound acquisitions of anatomies with restricted acoustic windows, such as the liver or the heart. METHODS: We propose a fully automatic approach for the planning of acquisition trajectories, which only requires information about the target region as well as existing tomographic imaging data, such as X-ray computed tomography. The framework integrates both geometrical and physics-based constraints to estimate the best ultrasound acquisition trajectories with respect to the available acoustic windows. We evaluate the developed method using virtual planning scenarios based on real patient data as well as for real robotic ultrasound acquisitions on a tissue-mimicking phantom. RESULTS: The proposed method yields superior image quality in comparison with a naive planning approach, while maintaining the necessary coverage of the target. CONCLUSION: We demonstrate that by taking image formation properties into account acquisition planning methods can outperform naive plannings. Furthermore, we show the need for such planning techniques, since naive approaches are not sufficient as they do not take the expected image quality into account.

3D ultrasound registration-based visual servoing for neurosurgical navigation.

PURPOSE: We present a fully image-based visual servoing framework for neurosurgical navigation and needle guidance. The proposed servo-control scheme allows for compensation of target anatomy movements, maintaining high navigational accuracy over time, and automatic needle guide alignment for accurate manual insertions. METHOD: Our system comprises a motorized 3D ultrasound (US) transducer mounted on a robotic arm and equipped with a needle guide. It continuously registers US sweeps in real time with a pre-interventional plan based on CT or MR images and annotations. While a visual control law maintains anatomy visibility and alignment of the needle guide, a force controller is employed for acoustic coupling and tissue pressure. We validate the servoing capabilities of our method on a geometric gel phantom and real human anatomy, and the needle targeting accuracy using CT images on a lumbar spine gel phantom under neurosurgery conditions. RESULTS: Despite the varying resolution of the acquired 3D sweeps, we achieved direction-independent positioning errors of [Formula: see text] mm and [Formula: see text], respectively. Our method is capable of compensating movements of around 25 mm/s and works reliably on human anatomy with errors of [Formula: see text] mm. In all four manual insertions by an expert surgeon, a needle could be successfully inserted into the facet joint, with an estimated targeting accuracy of [Formula: see text] mm, superior to the gold standard. CONCLUSION: The experiments demonstrated the feasibility of robotic ultrasound-based navigation and needle guidance for neurosurgical applications such as lumbar spine injections.

Towards MRI-Based Autonomous Robotic US Acquisitions: A First Feasibility Study.

Robotic ultrasound has the potential to assist and guide physicians during interventions. In this work, we present a set of methods and a workflow to enable autonomous MRI-guided ultrasound acquisitions. Our approach uses a structured-light 3D scanner for patient-to-robot and image-to-patient calibration, which in turn is used to plan 3D ultrasound trajectories. These MRI-based trajectories are followed autonomously by the robot and are further refined online using automatic MRI/US registration. Despite the low spatial resolution of structured light scanners, the initial planned acquisition path can be followed with an accuracy of 2.46 ± 0.96 mm. This leads to a good initialization of the MRI/US registration: the 3D-scan-based alignment for planning and acquisition shows an accuracy (distance between planned ultrasound and MRI) of 4.47 mm, and 0.97 mm after an online-update of the calibration based on a closed loop registration.

Personalized, relevance-based Multimodal Robotic Imaging and augmented reality for Computer Assisted Interventions.

In the last decade, many researchers in medical image computing and computer assisted interventions across the world focused on the development of the Virtual Physiological Human (VPH), aiming at changing the practice of medicine from classification and treatment of diseases to that of modeling and treating patients. These projects resulted in major advancements in segmentation, registration, morphological, physiological and biomechanical modeling based on state of art medical imaging as well as other sensory data. However, a major issue which has not yet come into the focus is personalizing intra-operative imaging, allowing for optimal treatment. In this paper, we discuss the personalization of imaging and visualization process with particular focus on satisfying the challenging requirements of computer assisted interventions. We discuss such requirements and review a series of scientific contributions made by our research team to tackle some of these major challenges.

Multimodal US-gamma imaging using collaborative robotics for cancer staging biopsies.

PURPOSE: The staging of female breast cancer requires detailed information about the level of cancer spread through the lymphatic system. Common practice to obtain this information for patients with early-stage cancer is sentinel lymph node (SLN) biopsy, where LNs are radioactively identified for surgical removal and subsequent histological analysis. Punch needle biopsy is a less invasive approach but suffers from the lack of combined anatomical and nuclear information. We present and evaluate a system that introduces live collaborative robotic 2D gamma imaging in addition to live 2D ultrasound to identify SLNs in the surrounding anatomy. METHODS: The system consists of a robotic arm equipped with both a gamma camera and a stereoscopic tracking system that monitors the position of an ultrasound probe operated by the physician. The arm cooperatively places the gamma camera parallel to the ultrasound imaging plane to provide live multimodal visualization and guidance. We validate the system by evaluating the target registration errors between fused nuclear and US image data in a phantom consisting of two spheres, one of which is filled with radioactivity. Medical experts perform punch biopsies on agar-gelatine phantoms with complex configurations of hot and cold lesions to provide a qualitative and quantitative evaluation of the system. RESULTS: The average point registration error for the overlay is [Formula: see text] mm. The time of the entire procedure was reduced by 36 %, with 80v of the biopsies being successful. The users' feedback was very positive, and the system was deemed to be very intuitive, with handling similar to classic US-guided needle biopsy. CONCLUSION: We present and evaluate the first medical collaborative robotic imaging system. Feedback from potential users for SLN punch needle biopsy is encouraging. Ongoing work investigates the clinical feasibility with more complex and realistic phantoms.

Standardized 2D ultrasound versus 3D/4D ultrasound and image fusion for measurement of aortic aneurysm diameter in follow-up after EVAR.

PURPOSE: To compare standardised 2D ultrasound (US) to the novel ultrasonographic imaging techniques 3D/4D US and image fusion (combined real-time display of B mode and CT scan) for routine measurement of aortic diameter in follow-up after endovascular aortic aneurysm repair (EVAR). METHOD AND MATERIALS: 300 measurements were performed on 20 patients after EVAR by one experienced sonographer (3rd degree of the German society of ultrasound (DEGUM)) with a high-end ultrasound machine and a convex probe (1-5 MHz). An internally standardized scanning protocol of the aortic aneurysm diameter in B mode used a so called leading-edge method. In summary, five different US methods (2D, 3D free-hand, magnetic field tracked 3D - Curefab™, 4D volume sweep, image fusion), each including contrast-enhanced ultrasound (CEUS), were used for measurement of the maximum aortic aneurysm diameter. Standardized 2D sonography was the defined reference standard for statistical analysis. CEUS was used for endoleak detection. RESULTS: Technical success was 100%. In augmented transverse imaging the mean aortic anteroposterior (AP) diameter was 4.0±1.3 cm for 2D US, 4.0±1.2 cm for 3D Curefab™, and 3.9±1.3 cm for 4D US and 4.0±1.2 for image fusion. The mean differences were below 1 mm (0.2-0.9 mm). Concerning estimation of aneurysm growth, agreement was found between 2D, 3D and 4D US in 19 of the 20 patients (95%). Definitive decision could always be made by image fusion. CEUS was combined with all methods and detected two out of the 20 patients (10%) with an endoleak type II. In one case, endoleak feeding arteries remained unclear with 2D CEUS but could be clearly localized by 3D CEUS and image fusion. CONCLUSION: Standardized 2D US allows adequate routine follow-up of maximum aortic aneurysm diameter after EVAR. Image Fusion enables a definitive statement about aneurysm growth without the need for new CT imaging by combining the postoperative CT scan with real-time B mode in a dual image display. 3D/4D CEUS and image fusion can improve endoleak characterization in selected cases but are not mandatory for routine practice.

Multimodal image-guided prostate fusion biopsy based on automatic deformable registration.

PURPOSE: Transrectal ultrasound (TRUS)-guided random prostate biopsy is, in spite of its low sensitivity, the gold standard for the diagnosis of prostate cancer. The recent advent of PET imaging using a novel dedicated radiotracer, [Formula: see text]-labeled prostate-specific membrane antigen (PSMA), combined with MRI provides improved pre-interventional identification of suspicious areas. This work proposes a multimodal fusion image-guided biopsy framework that combines PET-MRI images with TRUS, using automatic segmentation and registration, and offering real-time guidance. METHODS: The prostate TRUS images are automatically segmented with a Hough transform-based random forest approach. The registration is based on the Coherent Point Drift algorithm to align surfaces elastically and to propagate the deformation field calculated from thin-plate splines to the whole gland. RESULTS: The method, which has minimal requirements and temporal overhead in the existing clinical workflow, is evaluated in terms of surface distance and landmark registration error with respect to the clinical ground truth. Evaluations on agar-gelatin phantoms and clinical data of 13 patients confirm the validity of this approach. CONCLUSION: The system is able to successfully map suspicious regions from PET/MRI to the interventional TRUS image.

Multi-scale tubular structure detection in ultrasound imaging.

We propose a novel, physics-based method for detecting multi-scale tubular features in ultrasound images. The detector is based on a Hessian-matrix eigenvalue method, but unlike previous work, our detector is guided by an optimal model of vessel-like structures with respect to the ultrasound-image formation process. Our method provides a voxel-wise probability map, along with estimates of the radii and orientations of the detected tubes. These results can then be used for further processing, including segmentation and enhanced volume visualization. Most Hessian-based algorithms, including the well-known Frangi filter, were developed for CTA or MRA; they implicitly assume symmetry about the vessel centerline. This is not consistent with ultrasound data. We overcome this limitation by introducing a novel filter that allows multi-scale estimation both with respect to the vessel's centerline and with respect to the vessel's border. We use manually-segmented ultrasound imagery from 35 patients to show that our method is superior to standard Hessian-based methods. We evaluate the performance of the proposed methods based on the sensitivity and specificity like measures, and finally demonstrate further applicability of our method to vascular ultrasound images of the carotid artery, as well as ultrasound data for abdominal aortic aneurysms.

A quadratic energy minimization framework for signal loss estimation from arbitrarily sampled ultrasound data.

We present a flexible and general framework to iteratively solve quadratic energy problems on a non uniform grid, targeted at ultrasound imaging. Therefore, we model input samples as the nodes of an irregular directed graph, and define energies according to the application by setting weights to the edges. To solve the energy, we derive an effective optimization scheme, which avoids both the explicit computation of a linear system, as well as the compounding of the input data on a regular grid. The framework is validated in the context of 3D ultrasound signal loss estimation with the goal of providing an uncertainty estimate for each 3D data sample. Qualitative and quantitative results for 5 subjects and two target regions, namely US of the bone and the carotid artery, show the benefits of our approach, yielding continuous loss estimates.

3D velocity field and flow profile reconstruction from arbitrarily sampled Doppler ultrasound data.

With the need for adequate analysis of blood flow dynamics, different maging modalities have been developed to measure varying blood velocities over time. Due to its numerous advantages, Doppler ultrasound sonography remains one of the most widely used techniques in clinical routine, but requires additional preprocessing to recover 3D velocity information. Despite great progress in the last years, recent approaches do not jointly consider spatial and temporal variation in blood flow. In this work, we present a novel gating- and compounding-free method to simultaneously reconstruct a 3D velocity field and a temporal flow profile from arbitrarily sampled Doppler ultrasound measurements obtained from multiple directions. Based on a laminar flow assumption, a patch-wise B-spline formulation of blood velocity is coupled for the first time with a global waveform model acting as temporal regularization. We evaluated our method on three virtual phantom datasets, demonstrating robustness in terms of noise, angle between measurements and data sparsity, and applied it successfully to five real case datasets of carotid artery examination.

FhSPECT-US guided needle biopsy of sentinel lymph nodes in the axilla: is it feasible?

Until now, core needle biopsy of the axillary sentinel lymph nodes in early stage breast cancer patients is not possible, due to the lack of a proper combination of functional and anatomical information. In this work we present the first fully 3D freehand SPECT--ultrasound fusion, combining the advantages of both modalities. By using spatial positioning either with optical or with electromagnetic tracking for the ultrasound probe, and a mini gamma camera as radiation detector for freehand SPECT reconstructions, we investigate the capability of the introduced multi-model imaging system, where we compare both 3D freehand SPECT and 3D ultrasound to ground truth for a realistic breast mimicking phantom and further analyze the effect of tissue deformation by ultrasound. Finally, we also show its application in a real clinical setting.