3D magnetic seed localization for augmented reality in surgery.

PURPOSE: For tumor resection, surgeons need to localize the tumor. For this purpose, a magnetic seed can be inserted into the tumor by a radiologist and, during surgery, a magnetic detection probe informs the distance to the seed for localization. In this case, the surgeon still needs to mentally reconstruct the position of the tumor from the probe's information. The purpose of this study is to develop and assess a method for 3D localization and visualization of the seed, facilitating the localization of the tumor. METHODS: We propose a method for 3D localization of the magnetic seed by extending the magnetic detection probe with a tracking-based localization. We attach a position sensor (QR-code or optical marker) to the probe in order to track its 3D pose (respectively, using a head-mounted display with a camera or optical tracker). Following an acquisition protocol, the 3D probe tip and seed position are subsequently obtained by solving a system of equations based on the distances and the 3D probe poses. RESULTS: The method was evaluated with an optical tracking system. An experimental setup using QR-code tracking (resp. using an optical marker) achieves an average of 1.6 mm (resp. 0.8 mm) 3D distance between the localized seed and the ground truth. Using a breast phantom setup, the average 3D distance is 4.7 mm with a QR-code and 2.1 mm with an optical marker. CONCLUSION: Tracking the magnetic detection probe allows 3D localization of a magnetic seed, which opens doors for augmented reality target visualization during surgery. Such an approach should enhance the perception of the localized region of interest during the intervention, especially for breast tumor resection where magnetic seeds can already be used in the protocol.

Stabilization and visual analysis of video-recorded sailing sessions.

One common way to aid coaching and seek to improve athletes' performance is by recording training sessions for posterior analysis. In the case of sailing, coaches record videos from another boat, but usually rely on handheld devices, which may lead to issues with the footage and missing important moments. On the other hand, by autonomously recording the entire session with a fixed camera, the analysis becomes challenging owing to the length of the video and possible stabilization issues. In this work, we aim to facilitate the analysis of such full-session videos by automatically extracting maneuvers and providing a visualization framework to readily locate interesting moments. Moreover, we address issues related to image stability. Finally, an evaluation of the framework points to the benefits of video stabilization in this scenario and an appropriate accuracy of the maneuver detection method.

On GPU Connected Components and Properties: A Systematic Evaluation of Connected Component Labeling Algorithms and Their Extension for Property Extraction.

Connected component labeling (CCL) is a fundamental image processing problem that has been studied in many platforms, including GPUs. A common approach to CCL performance analysis is studying the total processing time as a function of abstract image features, like the number of connected components or the fraction of foreground pixels, and input data usually includes synthetic images and segmented video datasets. In this paper, we develop on these ideas and propose an evaluation methodology for GPU CCL algorithms based on synthetic image patterns, addressing the nonexistence of a standard and reliable benchmark in the literature. Our methodology, applied on two important algorithms from existing literature, uncovers their data dependency with great detail, and allows us to model their processing time in three real-world video data sets as functions of abstract, high-level image concepts. We also apply our methodology for studying the memory and performance requirements of two strategies for computing connected component properties: an existing memory-hungry approach, and a new memory-preserving strategy.

Generation of 3D ultrasound biomicroscopic images: technique validation and in vivo volumetric imaging of rat lateral gastrocnemius

Introduction Ultrasound biomicroscopy (UBM) is a technique for generating high-resolution images, with frequencies from 20 MHz to 100 MHz. For example, it has been used in animal research related to models of injury and diseases that mimic human conditions. With a three-dimensional ultrasound (3D) image system, an organ can be viewed at various angles and the volume estimated, contributing to an accurate diagnosis. This work refers to the generation of 3D-UBM images, employing a 35 MHz ultrasound system, from multiple two-dimensional (2D) images. Phantoms were used to validate the technique and to determine its reliability of volume measurements. Additionally, the technique was used to obtain 3D images of the rat gastrocnemius muscle. Methods Four different phantoms were used and ten acquisition sequences of 2D-images acquired for each one. Thereafter, 5 volume segmentations were performed for each acquisition sequence, resulting in 50 measured volumes for each phantom. The physical volumes of all phantoms were used to validate the technique based on the coefficient of variation (CV) and the intraclass correlation coefficient (ICC). Images of the gastrocnemius muscle were acquired and the partial volume quantified. Results The CV and ICC confirmed the reliability of volume measurements obtained by segmentation. Moreover, cross-sectional 2D images of rat hindlimb were obtained, allowing to identify the gastrocnemius muscle and to partially quantify the muscle volume from 3D images. Conclusion The results indicated that the technique is valid to generate 3D images and quantify the volume of a muscle compatible with the dimensions of a small animal.

Flow-based local optimization for image-to-geometry projection.

The projection of a photographic data set on a 3D model is a robust and widely applicable way to acquire appearance information of an object. The first step of this procedure is the alignment of the images on the 3D model. While any reconstruction pipeline aims at avoiding misregistration by improving camera calibrations and geometry, in practice a perfect alignment cannot always be reached. Depending on the way multiple camera images are fused on the object surface, remaining misregistrations show up either as ghosting or as discontinuities at transitions from one camera view to another. In this paper we propose a method, based on the computation of Optical Flow between overlapping images, to correct the local misalignment by determining the necessary displacement. The goal is to correct the symptoms of misregistration, instead of searching for a globally consistent mapping, which might not exist. The method scales up well with the size of the data set (both photographic and geometric) and is quite independent of the characteristics of the 3D model (topology cleanliness, parametrization, density). The method is robust and can handle real world cases that have different characteristics: low level geometric details and images that lack enough features for global optimization or manual methods. It can be applied to different mapping strategies, such as texture or per-vertex attribute encoding.