Impact of Background, Foreground, and Manipulated Object Rendering on Egocentric Depth Perception in Virtual and Augmented Indoor Environments.

This research investigated how the similarity of the rendering parameters of background and foreground objects affected egocentric depth perception in indoor virtual and augmented environments. We refer to the similarity of the rendering parameters as visual 'congruence'. Study participants manipulated the depth of a sphere to match the depth of a designated target peg. In the first experiment, the sphere and peg were both virtual, while in the second experiment, the sphere is virtual and the peg is real. In both experiments, depth perception accuracy was found to depend on the levels of realism and congruence between the sphere, pegs, and background. In Experiment 1, realistic backgrounds lead to overestimation of depth, but resulted in underestimation when the background was virtual, and when depth cues were applied to the sphere and target peg. In Experiment 2, background and target pegs were real but matched with the virtual sphere; in comparison to Experiment 1, realistically rendered targets prompted an underestimation and more accuracy with the manipulated object. These findings suggest that congruence can affect distance estimation and the underestimation effect in the AR environment resulted from increased graphical fidelity of the foreground target and background.

Open vs. Closed Shapes: New Perceptual Categories?

Effective communication using visualization relies in part on the use of viable encoding strategies. For example, a viewer's ability to rapidly and accurately discern between two or more categorical variables in a chart or figure is contingent upon the distinctiveness of the encodings applied to each variable. Research in perception suggests that color is a more salient visual feature when compared to shape and although that finding is supported by visualization studies, characteristics of shape also yield meaningful differences in distinctiveness. We propose that open or closed shapes (that is, whether shapes are composed of line segments that are bounded across a region of space or not) represent a salient characteristic that influences perceptual processing. Three experiments were performed to test the reliability of the open/closed category; the first two from the perspective of attentional allocation, and the third experiment in the context of multi-class scatterplot displays. In the first, a flanker paradigm was used to test whether perceptual load and open/closed feature category would modulate the effect of the flanker on target processing. Results showed an influence of both variables. The second experiment used a Same/Different reaction time task to replicate and extend those findings. Results from both show that responses are faster and more accurate when closed rather than open shapes are processed as targets, and there is more processing interference when two competing shapes come from the same rather than different open or closed feature categories. The third experiment employed three commonly used visual analytic tasks - perception of average value, numerosity, and linear relationships with both single and dual displays of open and closed symbols. Our findings show that for numerosity and trend judgments, in particular, that different symbols from the same open or closed feature category cause more perceptual interference when they are presented together in a plot than symbols from different categories. Moreover, the extent of the interference appears to depend upon whether the participant is focused on processing open or closed symbols.

An optical flow approach to tracking colonoscopy video.

We can supplement the clinical value of an optical colonoscopy procedure if we can continuously co-align corresponding virtual colonoscopy (from preoperative X-ray CT exam) and optical colonoscopy images. In this work, we demonstrate a computer vision algorithm based on optical flow to compute egomotion from live colonoscopy video, which is then used to navigate and visualize the corresponding patient anatomy from X-ray CT data. The key feature of the algorithm lies in the effective combination of sparse and dense optical flow fields to compute the focus of expansion (FOE); FOE permits independent computation of camera translational and rotational parameters, directly contributing to the algorithm's accuracy and robustness. We performed extensive evaluation via a colon phantom and clinical colonoscopy data. We constructed two colon like phantoms, a straight phantom and a curved phantom to measure actual colonoscopy motion; tracking accuracy was quantitatively evaluated by comparing estimated motion parameters (velocity and displacement) to ground truth. Thirty straight and curved phantom sequences were collected at 10, 15 and 20 mm/s (5 trials at each speed), to simulate typical velocities during colonoscopy procedures. The average error in velocity estimation was within 3 mm/s in both straight and curved phantoms. Displacement error was under 7 mm over a total distance of 287-288 mm in the straight and curved phantoms. Algorithm robustness was successfully demonstrated on 27 optical colonoscopy image sequences from 20 different patients, and spanning 5 different colon segments. Specific sequences among these were chosen to illustrate the algorithm's decreased sensitivity to (1) recording interruptions, (2) errors in colon segmentation, (3) illumination artifacts, (4) presence of fluid, and (5) changes in colon structure, such as deformation, polyp removal, and surgical tool movement during a procedure.

A robust method to track colonoscopy videos with non-informative images.

PURPOSE: Continuously, optical and virtual image alignment can significantly supplement the clinical value of colonoscopy. However, the co-alignment process is frequently interrupted by non-informative images. A video tracking framework to continuously track optical colonoscopy images was developed and tested. METHODS: A video tracking framework with immunity to non-informative images was developed with three essential components: temporal volume flow, region flow, and incremental egomotion estimation. Temporal volume flow selects two similar images interrupted by non-informative images; region flow measures large visual motion between selected images; and incremental egomotion processing estimates significant camera motion by decomposing each large visual motion vector into a sequence of small optical flow vectors. The framework was extensively evaluated via phantom and colonoscopy image sequences. We constructed two colon-like phantoms, a straight phantom and a curved phantom, to measure actual colonoscopy motion. RESULTS: In the straight phantom, after 48 frames were excluded, the tracking error was [Formula: see text]3 mm of 16 mm traveled. In the curved phantom, the error was [Formula: see text]4 mm of 23.88 mm traveled after 72 frames were excluded. Through evaluations with clinical sequences, the robustness of the tracking framework was demonstrated on 30 colonoscopy image sequences from 22 different patients. Four specific sequences among these were chosen to illustrate the algorithm's decreased sensitivity to (1) fluid immersion, (2) wall contact, (3) surgery-induced colon deformation, and (4) multiple non-informative image sequences. CONCLUSION: A robust tracking framework for real-time colonoscopy was developed that facilitates continuous alignment of optical and virtual images, immune to non-informative images that enter the video stream. The system was validated in phantom testing and achieved success with clinical image sequences.

Region flow: a multi-stage method for colonoscopy tracking.

Co-located optical and virtual colonoscopy images provide important clinical information during routine colonoscopy procedures. Tracking algorithms that rely on image features to align virtual and optical images can fail when they encounter blurry image sequences. This is a common occurrence in colonoscopy images, when the endoscope touches a wall or is immersed in fluid. We propose a region-flow based matching algorithm to determine the large changes between images that bridge such interruptions in the visual field. The region flow field is used as the means to limit the search space for computing corresponding feature points; a sequence of refining steps is performed to identify the most reliable and accurate feature point pairs. The feature point pairs are then used in a deformation based scheme to compute the final camera parameters. We have successfully tested this algorithm on four clinical colonoscopy image sequences containing anywhere from 9-57 consecutive blurry images. Two additional tabletop experiments were performed to quantitatively validate the algorithm: the endoscope was moved along a slightly curved path by 24 mm and along a straight path by 40 mm. Our method reported errors within 1-5% in these experiments.

Interactive detection and visualization of breast lesions from dynamic contrast enhanced MRI volumes.

Mammography is currently regarded as the most effective and widely used method for early detection of breast cancer, but recently its sensitivity in certain high risk cases has been less than desired. The use of Dynamic Contrast Enhanced Magnetic Resonance Imaging (DCE-MRI) has gained considerable attention in the past 10 years, especially for high risk cases, for smaller multi-focal lesions, or very sparsely distributed lesions. In this work, we present an interactive visualization system to identify, process, visualize and quantify lesions from DCE-MRI volumes. Our approach has the following key features: (1) we determine a confidence measure for each voxel, representing the probability that the voxel is part of the tumor, using a rough goodness-of-fit for the shape of the intensity-time curves, (2) our system takes advantage of low-cost, readily available 3D texture mapping hardware to produce both 2D and 3D visualizations of the segmented MRI volume in near real-time, enabling improved spatial perception of the tumor location, shape, size, distribution, and other characteristics useful in staging and treatment courses, and (3) our system permits interactive manipulation of the signal-time curves, adapts to different tumor types and morphology, thus making it a powerful tool for radiologists/physicians to rapidly assess probable malignant volumes. We illustrate the application of our system with four case studies: invasive ductal cancer, benign fibroadenoma, ductal carcinoma in situ and lobular carcinoma.