Computer-based System for the Virtual-Endoscopic Guidance of Bronchoscopy.

The standard procedure for diagnosing lung cancer involves two stages: three-dimensional (3D) computed-tomography (CT) image assessment, followed by interventional bronchoscopy. In general, the physician has no link between the 3D CT image assessment results and the follow-on bronchoscopy. Thus, the physician essentially performs bronchoscopic biopsy of suspect cancer sites blindly. We have devised a computer-based system that greatly augments the physician's vision during bronchoscopy. The system uses techniques from computer graphics and computer vision to enable detailed 3D CT procedure planning and follow-on image-guided bronchoscopy. The procedure plan is directly linked to the bronchoscope procedure, through a live registration and fusion of the 3D CT data and bronchoscopic video. During a procedure, the system provides many visual tools, fused CT-video data, and quantitative distance measures; this gives the physician considerable visual feedback on how to maneuver the bronchoscope and where to insert the biopsy needle. Central to the system is a CT-video registration technique, based on normalized mutual information. Several sets of results verify the efficacy of the registration technique. In addition, we present a series of test results for the complete system for phantoms, animals, and human lung-cancer patients. The results indicate that not only is the variation in skill level between different physicians greatly reduced by the system over the standard procedure, but that biopsy effectiveness increases.

Three-dimensional path planning for virtual bronchoscopy.

Multidetector computed-tomography (MDCT) scanners provide large high-resolution three-dimensional (3-D) images of the chest. MDCT scanning, when used in tandem with bronchoscopy, provides a state-of-the-art approach for lung-cancer assessment. We have been building and validating a lung-cancer assessment system, which enables virtual-bronchoscopic 3-D MDCT image analysis and follow-on image-guided bronchoscopy. A suitable path planning method is needed, however, for using this system. We describe a rapid, robust method for computing a set of 3-D airway-tree paths from MDCT images. The method first defines the skeleton of a given segmented 3-D chest image and then performs a multistage refinement of the skeleton to arrive at a final tree structure. The tree consists of a series of paths and branch structural data, suitable for quantitative airway analysis and smooth virtual navigation. A comparison of the method to a previously devised path-planning approach, using a set of human MDCT images, illustrates the efficacy of the method. Results are also presented for human lung-cancer assessment and the guidance of bronchoscopy.

Automatic axis generation for virtual bronchoscopic assessment of major airway obstructions.

Virtual bronchoscopy (VB) has emerged as a paradigm for more effective 3D CT image evaluation. Systematic evaluation of a 3D CT chest image using VB techniques, however, requires precomputed guidance data. This guidance data takes the form of central axes, or centerlines, through the major airways. We propose an axes-generation algorithm for VB assessment of 3D CT chest images. For a typical high-resolution 3D CT chest image, the algorithm produces a series of airway-tree axes, corresponding airway cross-sectional area measurements, and a segmented airway tree in a few minutes on a standard PC. Results for phantom and human airway-obstruction cases demonstrate the efficacy of the algorithm. Also, the algorithm is demonstrated in the context of VB-based 3D CT assessment.

New techniques for efficient sliding thin-slab volume visualization.

High-resolution three-dimensional (3-D) volumetric images obtained by today's radiologic imaging scanners are rich in detailed diagnostic information. Despite the many visualization techniques available to assess such images, there remains information that is challenging to uncover, such as the location of small structures (e.g., mediastinal lymph nodes, narrowed-airway regions). Recently, sliding thin-slab (STS) visualization was proposed to improve the visualization of interior structures. These STS techniques sometimes depend on user opacity specifications or extra preprocessing, and other rendering approaches that use the general STS mechanism are conceivable. We introduce two techniques for STS volume visualization. The first, a depth (perspective) rendering process, produces an unobstructed, high-contrast 3-D view of the information within a thin volume of image data. Results are a function of relative planar locations. Thus, rendered views accurately depict the internal properties that were initially captured as position and intensity. The second method produces a gradient-like view of the intensity changes in a thin volume. Results can effectively detect the occurrence and location of dramatic tissue variations, often not visually recognized otherwise. Both STS techniques exploit the concept of temporal coherence to form sequences of consecutive slabs, using information from previously computed slabs. This permits efficient real-time computation on a general-purpose computer. Further, these techniques require no preprocessing, and results are not dependent on user knowledge. Results using 3-D computed tomography chest images show the computational efficiency and visual efficacy of the new STS techniques.

Videoendoscopic distortion correction and its application to virtual guidance of endoscopy.

Modern video-based endoscopes offer physicians a wide-angle field of view (FOV) for minimally invasive procedures. Unfortunately, inherent barrel distortion prevents accurate perception of range. This makes measurement and distance judgment difficult and causes difficulties in emerging applications, such as virtual guidance of endoscopic procedures. Such distortion also arises in other wide FOV camera circumstances. This paper presents a distortion-correction technique that can automatically calculate correction parameters, without precise knowledge of horizontal and vertical orientation. The method is applicable to any camera-distortion correction situation. Based on a least-squares estimation, our proposed algorithm considers line fits in both FOV directions and gives a globally consistent set of expansion coefficients and an optimal image center. The method is insensitive to the initial orientation of the endoscope and provides more exhaustive FOV correction than previously proposed algorithms. The distortion-correction procedure is demonstrated for endoscopic video images of a calibration test pattern, a rubber bronchial training device, and real human circumstances. The distortion correction is also shown as a necessary component of an image-guided virtual-endoscopy system that matches endoscope images to corresponding rendered three-dimensional computed tomography views.

Extraction of the hepatic vasculature in rats using 3-D micro-CT images.

High-resolution micro-computed tomography (CT) scanners now exist for imaging small animals. In particular, such a scanner can generate very large three-dimensional (3-D) digital images of the rat's hepatic vasculature. These images provide data on the overall structure and function of such complex vascular trees. Unfortunately, human operators have extreme difficulty in extracting the extensive vasculature contained in the images. Also, no suitable tree representation exists that permits straight-forward structural analysis and information retrieval. This work proposes an automatic procedure for extracting and representing such a vascular tree. The procedure is both computation and memory efficient and runs on current PCs. As the results demonstrate, the procedure faithfully follows human-defined measurements and provides far more information than can be defined interactively.

Interactive dynamic navigation for virtual endoscopy.

Complex anatomical information can be obtained from a 3D radiologic image by navigating through it in a manner similar to an endoscopic examination. Real-time computation of 'virtual' endoscopic views, however, is needed to permit interactive navigation. We present a fast volume-rendering method for computing such views. Our method, motivated by the temporal-coherence concept, performs dynamic volume rendering at interactive frame rates. Results demonstrate the method's efficiency and accuracy. Also, our method constitutes part of a complete virtual-endoscopic system we have devised. This system is illustrated for 3D pulmonary analysis.

Watershed-based maximum-homogeneity filtering.

We introduce an image enhancement method referred to as the watershed-based maximum-homogeneity filter. This method first uses watershed analysis to subdivide the image into homogeneous pixel clusters called catchment basins. Next, using an adaptive, local, catchment-basin selection scheme, similar neighboring catchment basins are combined together to produce an enhanced image. Because the method starts with watershed analysis, it can preserve edge information and run with high computational efficiency. Illustrative results show that the method performs well relative to other popular nonlinear filters.

Virtual bronchoscopy for three--dimensional pulmonary image assessment: state of the art and future needs.

Virtual bronchoscopy is emerging as a useful approach for assessment of three-dimensional (3D) computed tomographic (CT) pulmonary images. A protocol for virtual bronchoscopic assessment of a 3D CT pulmonary image would have two main stages: (a) preprocessing of image data, which involves extracting objects of interest, defining paths through major airways, and preparing the extracted objects for 3D rendering; and (b) interactive image assessment, which involves use of graphics-based software tools such as surface-rendered views, projection images, virtual endoscopic views, tube views, oblique section images, measurement data, global two-dimensional section images, and cross-sectional views. Although a virtual bronchoscope offers a unique opportunity for exploration and quantitation, it cannot replace a real bronchoscope. Limitations of current virtual endoscopy systems include high cost, lack of visual aids beyond simulated endoscopic views, difficulty in performing interactive anatomic exploration, lack of quantitative information, use of surface rendering instead of volume rendering, and need for substantial off-line display computation. Future needs include development of fully integrated user-friendly virtual bronchoscopes, development of optimal CT protocols for generating artifact-free data sets, and improvements in automated preprocessing of 3D CT images.

Distributed system for processing 3D medical images.

Three-dimensional (3D) image data generated by radiological imaging modalities such as CT and MRI can provide detailed structural insight. Automating the analysis of these images can improve the consistency of the results and reduce user interaction time, but introduces a tremendous computational burden. To address this problem, we have designed a distributed processing environment for the rapid processing of 3D medical images. Our system allows a user to perform automatic 3D filtering, segmentation, and measurement on a 3D image using a heterogeneous network of processors and the PVM protocol.

System for analyzing high-resolution three-dimensional coronary angiograms.

Three-dimensional (3-D) high-resolution coronary angiograms offer a means for visualizing the entire coronary arterial tree from any orientation and for detecting and quantitating coronary arterial stenoses. Previously, a skilled operator had to perform several hours of tedious manual analysis using an interactive graphical user-interface (GUI) system (Tree Trace) to analyze a 3-D angiogram. The authors have devised an improved GUI system, consisting of three tools for analyzing 3-D angiograms. The Artery Extractor first performs automatic image-analysis operations to extract the central axes of the arterial tree. Next, using the Artery Display tool and results from the Artery Extractor, the operator can visualize structures in the angiogram and compute various measurements. Finally, the aforementioned Tree Trace tool can be used to manually correct irregularities in the automatically generated results of the Artery Extractor. The system greatly reduces operator analysis time, gives exactly reproducible results, uses true 3-D image-processing operations, and provides a comprehensive interface for visualizing and quantifying features of the 3-D coronary arteries.

Nonlinear filtering approach to 3-D gray-scale image interpolation.

Three-dimensional (3-D) images are now common in radiology. A 3-D image is formed by stacking a contiguous sequence of two-dimensional cross-sectional images, or slices. Typically, the spacing between known slices is greater than the spacing between known points on a slice. Many visualization and image-analysis tasks, however, require the 3-D image to have equal sample spacing in all directions. To meet this requirement, one applies an interpolation technique to the known 3-D image to generate a new uniformly sampled 3-D image. The authors propose a nonlinear-filter-based approach to gray-scale interpolation of 3-D images. The method, referred to as column-fitting interpolation, is reminiscent of the maximum-homogeneity filter used for image enhancement. The authors also draw upon the paradigm of relaxation labeling to devise an improved column-fitting interpolator. Both methods are typically more effective than traditional gray-scale interpolation techniques.

Efficient morphological shape representation.

Mathematical morphology is well suited to capturing geometric information. Hence, morphology-based approaches have been popular for object shape representation. The two primary morphology-based approaches-the morphological skeleton and the morphological shape decomposition (MSD)-each represent an object as a collection of disjoint sets. A practical shape representation scheme, though, should give a representation that is computationally efficient to use. Unfortunately, little work has been done for the morphological skeleton and the MSD to address efficiency. We propose a flexible search-based shape representation scheme that typically gives more efficient representations than the morphological skeleton and MSD. Our method decomposes an object into a number of simple components based on homothetics of a set of structuring elements. To form the representation, the components are combined using set union and set difference operations. We use three constituent component types and a thorough cost-based search strategy to find efficient representations. We also consider allowing object representation error, which may yield even more efficient representations.

Optimal Gabor filters for texture segmentation.

Texture segmentation involves subdividing an image into differently textured regions. Many texture segmentation schemes are based on a filter-bank model, where the filters, called Gabor filters, are derived from Gabor elementary functions. The goal is to transform texture differences into detectable filter-output discontinuities at texture boundaries. By locating these discontinuities, one can segment the image into differently textured regions. Distinct discontinuities occur, however, only if the Gabor filter parameters are suitably chosen. Some previous analysis has shown how to design filters for discriminating simple textures. Designing filters for more general natural textures, though, has largely been done ad hoc. We have devised a more rigorously based method for designing Gabor filters. It assumes that an image contains two different textures and that prototype samples of the textures are given a priori. We argue that Gabor filter outputs can be modeled as Rician random variables (often approximated well as Gaussian rv's) and develop a decision-theoretic algorithm for selecting optimal filter parameters. To improve segmentations for difficult texture pairs, we also propose a multiple-filter segmentation scheme, motivated by the Rician model. Experimental results indicate that our method is superior to previous methods in providing useful Gabor filters for a wide range of texture pairs.

IMPROMPTU: a system for automatic 3D medical image-analysis.

The utility of three-dimensional (3D) medical imaging is hampered by difficulties in extracting anatomical regions and making measurements in 3D images. Presently, a user is generally forced to use time-consuming, subjective, manual methods, such as slice tracing and region painting, to define regions of interest. Automatic image-analysis methods can ameliorate the difficulties of manual methods. This paper describes a graphical user interface (GUI) system for constructing automatic image-analysis processes for 3D medical-imaging applications. The system, referred to as IMPROMPTU, provides a user-friendly environment for prototyping, testing and executing complex image-analysis processes. IMPROMPTU can stand alone or it can interact with an existing graphics-based 3D medical image-analysis package (VIDA), giving a strong environment for 3D image-analysis, consisting of tools for visualization, manual interaction, and automatic processing. IMPROMPTU links to a large library of 1D, 2D, and 3D image-processing functions, referred to as VIPLIB, but a user can easily link in custom-made functions. 3D applications of the system are given for left-ventricular chamber, myocardial, and upper-airway extractions.

Interactive morphological watershed analysis for 3D medical images.

Combining automatic processing with interactive techniques is proving to be an effective strategy for segmenting complex three-dimensional (3D) medical images. We describe a general 3D image segmentation strategy that draws upon morphological watershed analysis and operator-defined topological cues. Watershed analysis segments a gray scale image into different regions by interpreting the image as a topographic surface. Using readily available interactive techniques, a human operator can easily define cues that specify spatial relationships between regions of interest. Cues defined in such a manner greatly assist subsequent watershed analysis. Results using 3D cardiac images show that this method leads to rapid robust image segmentation.

Shape-based interpolation of tree-like structures in three-dimensional images.

Many three-dimensional (3-D) medical images have lower resolution in the z direction than in the x or y directions. Before extracting and displaying objects in such images, an interpolated 3-D gray-scale image is usually generated via a technique such as linear interpolation to fill in the missing slices. Unfortunately, when objects are extracted and displayed from the interpolated image, they often exhibit a blocky and generally unsatisfactory appearance, a problem that is particularly acute for thin treelike structures such as the coronary arteries. Two methods for shape-based interpolation that offer an improvement to linear interpolation are presented. In shape-based interpolation, the object of interest is first segmented (extracted) from the initial 3-D image to produce a low-z-resolution binary-valued image, and the segmented image is interpolated to produce a high-resolution binary-valued 3-D image. The first method incorporates geometrical constraints and takes as input a segmented version of the original 3-D image. The second method builds on the first in that it also uses the original gray-scale image as a second input. Tests with 3-D images of the coronary arterial tree demonstrate the efficacy of the methods.

LV chamber extraction from 3-D CT images--accuracy and precision.

Measurement of left ventricular (LV) chamber volume and shape from three-dimensional (3-D) CT images, generated by the fast X-ray CT scanner known as the dynamic spatial reconstructor, has previously been done using manual slice-image editing. To reduce the exorbitant operator analysis time and operator-dependent measurement variations of manual slice-image editing, we have devised a semiautomatic method for LV-chamber extraction. The method draws upon a minimum requirement for selective manual slice-image editing and mostly makes use of automatic image-analysis operations. Detailed validation results over a wide range of hemodynamic and image-analysis conditions show that the measurements of the semiautomatic method strongly correlate with those made via manual slice-image editing and exhibit a lower intertrial variability. Further, the method reduces operator interaction time by nearly an order of magnitude over that of manual slice-image editing, but provides more detailed 3-D structural definition.

Extraction of left-ventricular chamber from 3-D CT images of the heart.

A semiautomatic method is described for extracting the volume and shape of the left ventricular (LV) chamber from a dynamic spatial reconstructor cardiac volume. For a given volume, the operator first performs some simple manual edits. Then, an automated stage, which incorporates concepts from 3-D mathematical morphology and technology, the maximum-homogeneity filter, and an adaptive 3-D thresholder, extracts the LV chamber. The method gives more consistent measurements and demands considerably less operator time than manual slice-editing.

A Hankel transform approach to tomographic image reconstruction.

A relatively unexplored algorithm is developed for reconstructing a two-dimensional image from a finite set of its sampled projections. The algorithm, referred to as the Hankel-transform-reconstruction (HTR) algorithm, is polar-coordinate based. The algorithm expands the polar-form Fourier transform F(r,theta) of an image into a Fourier series in theta calculates the appropriately ordered Hankel transform of the coefficients of this series, giving the coefficients for the Fourier series of the polar-form image f(p,phi); resolves this series, giving a polar-form reconstruction; and interpolates this reconstruction to a rectilinear grid. The HTR algorithm is outlined, and it is shown that its performance compares favorably to the popular convolution-backprojection algorithm.