Spectral Super-Resolution for High Dynamic Range Images.

The images we commonly use are RGB images that contain three pieces of information: red, green, and blue. On the other hand, hyperspectral (HS) images retain wavelength information. HS images are utilized in various fields due to their rich information content, but acquiring them requires specialized and expensive equipment that is not easily accessible to everyone. Recently, Spectral Super-Resolution (SSR), which generates spectral images from RGB images, has been studied. Conventional SSR methods target Low Dynamic Range (LDR) images. However, some practical applications require High Dynamic Range (HDR) images. In this paper, an SSR method for HDR is proposed. As a practical example, we use the HDR-HS images generated by the proposed method as environment maps and perform spectral image-based lighting. The rendering results by our method are more realistic than conventional renderers and LDR SSR methods, and this is the first attempt to utilize SSR for spectral rendering.

High-Resolution Representation for Mobile Mapping Data in Curved Regular Grid Model.

As mobile mapping systems become a mature technology, there are many applications for the process of the measured data. One interesting application is the use of driving simulators that can be used to analyze the data of tire vibration or vehicle simulations. In previous research, we presented our proposed method that can create a precise three-dimensional point cloud model of road surface regions and trajectory points. Our data sets were obtained by a vehicle-mounted mobile mapping system (MMS). The collected data were converted into point cloud data and color images. In this paper, we utilize the previous results as input data and present a solution that can generate an elevation grid for building an OpenCRG model. The OpenCRG project was originally developed to describe road surface elevation data, and also defined an open file format. As it can be difficult to generate a regular grid from point cloud directly, the road surface is first divided into straight lines, circular arcs, and and clothoids. Secondly, a non-regular grid which contains the elevation of road surface points is created for each road surface segment. Then, a regular grid is generated by accurately interpolating the elevation values from the non-regular grid. Finally, the curved regular grid (CRG) model files are created based on the above procedures, and can be visualized by OpenCRG tools. The experimental results on real-world data show that the proposed approach provided a very-high-resolution road surface elevation model.

Primal-dual approach to optical tomography with discretized path integral with efficient formulations.

We propose an efficient optical tomography with discretized path integral. We first introduce the primal-dual approach to solve the inverse problem formulated as a constraint optimization problem. Next, we develop efficient formulations for computing Jacobian and Hessian of the cost function of the constraint nonlinear optimization problem. Numerical experiments show that the proposed formulation is faster ([Formula: see text]) than the previous work with the log-barrier interior point method ([Formula: see text]) for the Shepp-Logan phantom with a grid size of [Formula: see text], while keeping the quality of the estimation results (root-mean-square error increasing by up to 12%).

Defocus-aware Dirichlet particle filter for stable endoscopic video frame recognition.

BACKGROUND AND OBJECTIVE: A computer-aided system for colorectal endoscopy could provide endoscopists with important helpful diagnostic support during examinations. A straightforward means of providing an objective diagnosis in real time might be for using classifiers to identify individual parts of every endoscopic video frame, but the results could be highly unstable due to out-of-focus frames. To address this problem, we propose a defocus-aware Dirichlet particle filter (D-DPF) that combines a particle filter with a Dirichlet distribution and defocus information. METHODS: We develop a particle filter with a Dirichlet distribution that represents the state transition and likelihood of each video frame. We also incorporate additional defocus information by using isolated pixel ratios to sample from a Rayleigh distribution. RESULTS: We tested the performance of the proposed method using synthetic and real endoscopic videos with a frame-wise classifier trained on 1671 images of colorectal endoscopy. Two synthetic videos comprising 600 frames were used for comparisons with a Kalman filter and D-DPF without defocus information, and D-DPF was shown to be more robust against the instability of frame-wise classification results. Computation time was approximately 88ms/frame, which is sufficient for real-time applications. We applied our method to 33 endoscopic videos and showed that the proposed method can effectively smoothen highly unstable probability curves under actual defocus of the endoscopic videos. CONCLUSION: The proposed D-DPF is a useful tool for smoothing unstable results of frame-wise classification of endoscopic videos to support real-time diagnosis during endoscopic examinations.

Computer-aided diagnosis of colorectal polyp histology by using a real-time image recognition system and narrow-band imaging magnifying colonoscopy.

BACKGROUND AND AIMS: It is necessary to establish cost-effective examinations and treatments for diminutive colorectal tumors that consider the treatment risk and surveillance interval after treatment. The Preservation and Incorporation of Valuable Endoscopic Innovations (PIVI) committee of the American Society for Gastrointestinal Endoscopy published a statement recommending the establishment of endoscopic techniques that practice the resect and discard strategy. The aims of this study were to evaluate whether our newly developed real-time image recognition system can predict histologic diagnoses of colorectal lesions depicted on narrow-band imaging and to satisfy some problems with the PIVI recommendations. METHODS: We enrolled 41 patients who had undergone endoscopic resection of 118 colorectal lesions (45 nonneoplastic lesions and 73 neoplastic lesions). We compared the results of real-time image recognition system analysis with that of narrow-band imaging diagnosis and evaluated the correlation between image analysis and the pathological results. RESULTS: Concordance between the endoscopic diagnosis and diagnosis by a real-time image recognition system with a support vector machine output value was 97.5% (115/118). Accuracy between the histologic findings of diminutive colorectal lesions (polyps) and diagnosis by a real-time image recognition system with a support vector machine output value was 93.2% (sensitivity, 93.0%; specificity, 93.3%; positive predictive value (PPV), 93.0%; and negative predictive value, 93.3%). CONCLUSIONS: Although further investigation is necessary to establish our computer-aided diagnosis system, this real-time image recognition system may satisfy the PIVI recommendations and be useful for predicting the histology of colorectal tumors.

Image segmentation of pyramid style identifier based on Support Vector Machine for colorectal endoscopic images.

With the increase of colorectal cancer patients in recent years, the needs of quantitative evaluation of colorectal cancer are increased, and the computer-aided diagnosis (CAD) system which supports doctor's diagnosis is essential. In this paper, a hardware design of type identification module in CAD system for colorectal endoscopic images with narrow band imaging (NBI) magnification is proposed for real-time processing of full high definition image (1920 × 1080 pixel). A pyramid style image segmentation with SVMs for multi-size scan windows, which can be implemented on an FPGA with small circuit area and achieve high accuracy, is proposed for actual complex colorectal endoscopic images.

Optical tomography with discretized path integral.

We present a framework for optical tomography based on a path integral. Instead of directly solving the radiative transport equations, which have been widely used in optical tomography, we use a path integral that has been developed for rendering participating media based on the volume rendering equation in computer graphics. For a discretized two-dimensional layered grid, we develop an algorithm to estimate the extinction coefficients of each voxel with an interior point method. Numerical simulation results are shown to demonstrate that the proposed method works well.

Transfer learning for Bag-of-Visual words approach to NBI endoscopic image classification.

We address a problem of endoscopic image classification taken by different (e.g., old and new) endoscopies. Our proposed method formulates the problem as a constraint optimization that estimates a linear transformation between feature vectors (or Bag-of-Visual words histograms) in a framework of transfer learning. Experimental results show that the proposed method works much better than the case without feature transformation.

A computer system to be used with laser-based endoscopy for quantitative diagnosis of early gastric cancer.

GOALS: To evaluate the usefulness of a newly devised computer system for use with laser-based endoscopy in differentiating between early gastric cancer, reddened lesions, and surrounding tissue. BACKGROUND: Narrow-band imaging based on laser light illumination has come into recent use. We devised a support vector machine (SVM)-based analysis system to be used with the newly devised endoscopy system to quantitatively identify gastric cancer on images obtained by magnifying endoscopy with blue-laser imaging (BLI). We evaluated the usefulness of the computer system in combination with the new endoscopy system. STUDY: We evaluated the system as applied to 100 consecutive early gastric cancers in 95 patients examined by BLI magnification at Hiroshima University Hospital. We produced a set of images from the 100 early gastric cancers; 40 flat or slightly depressed, small, reddened lesions; and surrounding tissues, and we attempted to identify gastric cancer, reddened lesions, and surrounding tissue quantitatively. RESULTS: The average SVM output value was 0.846 ± 0.220 for cancerous lesions, 0.381 ± 0.349 for reddened lesions, and 0.219 ± 0.277 for surrounding tissue, with the SVM output value for cancerous lesions being significantly greater than that for reddened lesions or surrounding tissue. The average SVM output value for differentiated-type cancer was 0.840 ± 0.207 and for undifferentiated-type cancer was 0.865 ± 0.259. CONCLUSIONS: Although further development is needed, we conclude that our computer-based analysis system used with BLI will identify gastric cancers quantitatively.

SVM-MRF segmentation of colorectal NBI endoscopic images.

In this paper we investigate a method for segmentation of colorectal Narrow Band Imaging (NBI) endoscopic images with Support Vector Machine (SVM) and Markov Random Field (MRF). SVM classifiers recognize each square patch of an NBI image and output posterior probabilities that represent how likely the given patch falls into a certain label. To prevent the spatial inconsistency between adjacent patches and encourage segmented regions to have smoother shapes, MRF is introduced by using the posterior outputs of SVMs as a unary term of MRF energy function. Segmentation results of 1191 NBI images are evaluated in experiments in which SVMs were trained with 480 trimmed NBI images and the MRF energy was minimized by an α - ß swap Graph Cut.

Labeling colorectal NBI zoom-videoendoscope image sequences with MRF and SVM.

In this paper, we propose a sequence labeling method by using SVM posterior probabilities with a Markov Random Field (MRF) model for colorectal Narrow Band Imaging (NBI) zoom-videoendoscope. Classifying each frame of a video sequence by SVM classifiers independently leads to an output sequence which is unstable and hard to understand by endoscopists. To make it more stable and readable, we use an MRF model to label the sequence of posterior probabilities. In addition, we introduce class asymmetry for the NBI images in order to keep and enhance frames where there is a possibility that cancers might have been detected. Experimental results with NBI video sequences demonstrate that the proposed MRF model with class asymmetry performs much better than a model without asymmetry.

Quantitative identification of mucosal gastric cancer under magnifying endoscopy with flexible spectral imaging color enhancement.

BACKGROUND AND AIM: Magnifying endoscopy with flexible spectral imaging color enhancement (FICE) is clinically useful in diagnosing gastric cancer and determining treatment options; however, there is a learning curve. Accurate FICE-based diagnosis requires training and experience. In addition, objectivity is necessary. Thus, a software program that can identify gastric cancer quantitatively was developed. METHODS: A bag-of-features framework with densely sampled scale-invariant feature transform descriptors to magnifying endoscopy images of 46 mucosal gastric cancers was applied. Computer-based findings were compared with histologic findings. The probability of gastric cancer was calculated by means of logistic regression, and sensitivity and specificity of the system were determined. RESULTS: The average probability was 0.78 ± 0.25 for the images of cancer and 0.31 ± 0.25 for the images of noncancer tissue, with a significant difference between the two groups. An optimal cut-off point of 0.59 was determined on the basis of the receiver operating characteristic curves. The computer-aided diagnosis system yielded a detection accuracy of 85.9% (79/92), sensitivity for a diagnosis of cancer of 84.8% (39/46), and specificity of 87.0% (40/46). CONCLUSION: Further development of this system will allow for quantitative evaluation of mucosal gastric cancers on magnifying gastrointestinal endoscopy images obtained with FICE.

Computer-aided colorectal tumor classification in NBI endoscopy using local features.

An early detection of colorectal cancer through colorectal endoscopy is important and widely used in hospitals as a standard medical procedure. During colonoscopy, the lesions of colorectal tumors on the colon surface are visually inspected by a Narrow Band Imaging (NBI) zoom-videoendoscope. By using the visual appearance of colorectal tumors in endoscopic images, histological diagnosis is presumed based on classification schemes for NBI magnification findings. In this paper, we report on the performance of a recognition system for classifying NBI images of colorectal tumors into three types (A, B, and C3) based on the NBI magnification findings. To deal with the problem of computer-aided classification of NBI images, we explore a local feature-based recognition method, bag-of-visual-words (BoW), and provide extensive experiments on a variety of technical aspects. The proposed prototype system, used in the experiments, consists of a bag-of-visual-words representation of local features followed by Support Vector Machine (SVM) classifiers. A number of local features are extracted by using sampling schemes such as Difference-of-Gaussians and grid sampling. In addition, in this paper we propose a new combination of local features and sampling schemes. Extensive experiments with varying the parameters for each component are carried out, for the performance of the system is usually affected by those parameters, e.g. the sampling strategy for the local features, the representation of the local feature histograms, the kernel types of the SVM classifiers, the number of classes to be considered, etc. The recognition results are compared in terms of recognition rates, precision/recall, and F-measure for different numbers of visual words. The proposed system achieves a recognition rate of 96% for 10-fold cross validation on a real dataset of 908 NBI images collected during actual colonoscopy, and 93% for a separate test dataset.

Computer-aided system for predicting the histology of colorectal tumors by using narrow-band imaging magnifying colonoscopy (with video).

BACKGROUND: Narrow-band imaging (NBI) classification of colorectal lesions is clinically useful in determining treatment options for colorectal tumors. There is a learning curve, however. Accurate NBI-based diagnosis requires training and experience. In addition, objective diagnosis is necessary. Thus, we developed a computerized system to automatically classify NBI magnifying colonoscopic images. OBJECTIVE: To evaluate the utility and limitations of our automated NBI classification system. DESIGN: Retrospective study. SETTING: Department of endoscopy, university hospital. MAIN OUTCOME MEASUREMENTS: Performance of our computer-based system for classification of NBI magnifying colonoscopy images in comparison to classification by two experienced endoscopists and to histologic findings. RESULTS: For the 371 colorectal lesions depicted on validation images, the computer-aided classification system yielded a detection accuracy of 97.8% (363/371); sensitivity and specificity of types B-C3 lesions for a diagnosis of neoplastic lesion were 97.8% (317/324) and 97.9% (46/47), respectively. Diagnostic concordance between the computer-aided classification system and the two experienced endoscopists was 98.7% (366/371), with no significant difference between methods. LIMITATIONS: Retrospective, single-center in this initial report. CONCLUSION: Our new computer-aided system is reliable for predicting the histology of colorectal tumors by using NBI magnifying colonoscopy.

Quantitative analysis of colorectal lesions observed on magnified endoscopy images.

BACKGROUND: Various surface mucosal pit patterns, as recognized by endoscopists, correlate with the histologic features of colorectal cancers. We investigated whether magnified endoscopy images of these pit patterns could be analyzed quantitatively and thus facilitate computer-aided diagnosis of colorectal lesions. METHODS: We applied both texture analysis and scale-invariant feature transform (SIFT) descriptors and discriminant analysis to magnified endoscopy images of 165 neoplastic colorectal lesions (pit patterns: type III(L)/IV, n = 44; type V(I)-mildly irregular, n = 36; type V(I)-severely irregular, n = 45; type V(N), n = 40) [histologic findings: tubular adenoma (TA), n = 56; carcinoma with intramucosal or even scant submucosal invasion (M/SM-s), n = 52, carcinoma with massive submucosal invasion (SM-m), n = 57]. We analyzed differences in pit pattern values and corresponding histologic values to determine whether the values were diagnostically meaningful. RESULTS: Gray-level difference matrix (GLDM) inverse difference moment and spatial gray-level dependence matrix (SGLDM) local homogeneity values differed significantly between type III(L)/IV and type V(N) pit patterns. Values differed significantly for each analyzed feature between type III(L)/IV and type V(I)-severely irregular patterns and were high but descending for type III(L)/IV, type V(I)-mildly irregular, and type V(I)-severely irregular pit patterns (in that order). Similarly, texture analysis yielded high but descending values for TA, M/SM-s, and SM-m (in that order). Furthermore, SIFT descriptors and discriminant analysis yielded differences that were superior to those obtained by texture analyses. CONCLUSIONS: Computer analysis of magnified endoscopy images for the diagnosis of colorectal lesions appears feasible. We anticipate further developments in the computer-aided diagnosis of pit patterns on magnified endoscopy images.

Quantitative analysis and development of a computer-aided system for identification of regular pit patterns of colorectal lesions.

BACKGROUND: Because pit pattern classification of colorectal lesions is clinically useful in determining treatment options for colorectal tumors but requires extensive training, we developed a computerized system to automatically quantify and thus classify pit patterns depicted on magnifying endoscopy images. OBJECTIVE: To evaluate the utility and limitations of our automated pit pattern classification system. DESIGN: Retrospective study. SETTING: Department of endoscopy at a university hospital. MAIN OUTCOME MEASUREMENTS: Performance of our automated computer-based system for classification of pit patterns on magnifying endoscopic images in comparison to classification by diagnosis of the 134 regular pit pattern images by an endoscopist. RESULTS: For type I and II pit patterns, the results of discriminant analysis were in complete agreement with the endoscopic diagnoses. Type IIIl was diagnosed in 29 of 30 cases (96.7%) and type IV was diagnosed in 1 case. Twenty-nine of 30 cases (96.7%) were diagnosed as type IV pit pattern. The overall accuracy of our computerized recognition system was 132 of 134 (98.5%). CONCLUSIONS: Our system is best characterized as semiautomated but is a step toward the development of a fully automated system to assist in the diagnosis of colorectal lesions based on classification of pit patterns.