High Dynamic Range Image Reconstruction from Saturated Images of Metallic Objects.

This study considers a method for reconstructing a high dynamic range (HDR) original image from a single saturated low dynamic range (LDR) image of metallic objects. A deep neural network approach was adopted for the direct mapping of an 8-bit LDR image to HDR. An HDR image database was first constructed using a large number of various metallic objects with different shapes. Each captured HDR image was clipped to create a set of 8-bit LDR images. All pairs of HDR and LDR images were used to train and test the network. Subsequently, a convolutional neural network (CNN) was designed in the form of a deep U-Net-like architecture. The network consisted of an encoder, a decoder, and a skip connection to maintain high image resolution. The CNN algorithm was constructed using the learning functions in MATLAB. The entire network consisted of 32 layers and 85,900 learnable parameters. The performance of the proposed method was examined in experiments using a test image set. The proposed method was also compared with other methods and confirmed to be significantly superior in terms of reconstruction accuracy, histogram fitting, and psychological evaluation.

Spectral Reflectance Estimation from Camera Responses Using Local Optimal Dataset.

A novel method is proposed to estimate surface-spectral reflectance from camera responses using a local optimal reflectance dataset. We adopt a multispectral imaging system that involves an RGB camera capturing multiple images under multiple light sources. A spectral reflectance database is utilized to locally determine the candidates to optimally estimate the spectral reflectance. The proposed estimation method comprises two stages: (1) selecting the local optimal reflectance dataset and (2) determining the best estimate using only the local optimal dataset. In (1), the camera responses are predicted for the respective reflectances in the database, and then the prediction errors are calculated to select the local optimal dataset. In (2), multiple methods are used; in particular, the Wiener and linear minimum mean square error estimators are used to calculate all statistics, based only on the local optimal dataset, and linear and quadratic programming methods are used to solve optimization problems with constraints. Experimental results using different mobile phone cameras show that the estimation accuracy has improved drastically. A much smaller local optimal dataset among spectral reflectance databases is enough to obtain the optimal estimates. The method has potential applications including fields of color science, image science and technology, computer vision, and graphics.

Improved method for spectral reflectance estimation and application to mobile phone cameras.

We propose an improved method for estimating surface-spectral reflectance from the image data acquired by an RGB digital camera. We suppose a multispectral image acquisition system in the visible range, where a camera captures multiple images for the scene of an object under multiple light sources. First, the observed image data are described using the camera spectral sensitivities, the surface-spectral reflectance, the illuminant spectral power distributions, an additive noise term, and a gain parameter. Then, the optimal reflectance estimate is determined to minimize the mean-square error between the estimate and the original surface-spectral reflectance. We attempt to further improve the estimation accuracy and develop a novel linear estimator in a more general form than the Wiener estimator. Furthermore, we calibrate the imaging system using a reference standard sample. Finally, experiments are performed to validate the proposed method for estimating the surface-spectral reflectance using different mobile phone cameras.

Measurement and Estimation of Spectral Sensitivity Functions for Mobile Phone Cameras.

Mobile phone cameras are often significantly more useful than professional digital single-lens reflex (DSLR) cameras. Knowledge of the camera spectral sensitivity function is important in many fields that make use of images. In this study, methods for measuring and estimating spectral sensitivity functions for mobile phone cameras are developed. In the direct measurement method, the spectral sensitivity at each wavelength is measured using monochromatic light. Although accurate, this method is time-consuming and expensive. The indirect estimation method is based on color samples, in which the spectral sensitivities are estimated from the input data of color samples and the corresponding output RGB values from the camera. We first present an imaging system for direct measurements. A variety of mobile phone cameras are measured using the system to create a database of spectral sensitivity functions. The features of the measured spectral sensitivity functions are then studied using principal component analysis (PCA) and the statistical features of the spectral functions extracted. We next describe a normal method to estimate the spectral sensitivity functions using color samples and point out some drawbacks of the method. A method to solve the estimation problem using the spectral features of the sensitivity functions in addition to the color samples is then proposed. The estimation is stable even when only a small number of spectral features are selected. Finally, the results of the experiments to confirm the feasibility of the proposed method are presented. We establish that our method is excellent in terms of both the data volume of color samples required and the estimation accuracy of the spectral sensitivity functions.

Spectral reconstruction of fluorescent objects with mutual illumination effects.

An approach is proposed for appearance reconstruction of fluorescent objects with mutual illumination effects under different conditions of material and illumination. We focus on the problem of reconstructing a realistic scene appearance, including mutual illumination effects, under different conditions of materials and illumination. First, spectral images of two closely located fluorescent objects are acquired under different illumination directions. The observed images are then decomposed into five components. Each component is expanded into spectral composition functions and geometric factors. Second, aiming at effective appearance reconstruction, we define the reference geometric factors to be invariant representations of the geometric factors, independent of illumination. The reference factors are estimated by taking the models of reflectance, luminescence, and mutual illumination into account. A novel appearance is rendered with different fluorescence materials, different illumination conditions, and the corresponding geometric factor estimates. The spectral image is reconstructed as a linear sum of the five components, combining the spectral functions and the geometric factors at the level of each component. The accuracy of appearance reconstruction of fluorescent objects is examined in experiments in detail.

Evaluating an image-based bidirectional reflectance distribution function measurement setup: erratum.

Two typographical errors in [Appl. Opt.57, 1918 (2018)]APOPAI0003-693510.1364/AO.57.001918 are identified and have been corrected here.

Evaluating an image-based bidirectional reflectance distribution function measurement setup.

We evaluate an image-based multiangle bidirectional reflectance distribution function measurement setup by comparing it to measurements from two commercially available goniospectrophotometers. The image-based setup uses an RGB camera to perform bidirectional measurements of the sample material. We use a conversion matrix to calculate luminance from the captured data. The matrix is calculated using camera spectral sensitivities that are measured with a monochromator. Radiance factor of the sample material is measured using a commercially available tabletop goniospectrophotometer and compared to measurements made using the image-based setup in the colorimetric domain. Our measurement setup is validated by comparing the measurements performed using a goniospectrophotometer. Uncertainty and error propagation is calculated and taken into account for validation. The sample material measured is wax-based ink printed on packaging paper substrate commonly used in the print and packaging industry. Results obtained show that the image-based setup can perform bidirectional reflectance measurements with a known uncertainty. The goniospectrophotometer measurements lie within the uncertainty of the measurements performed by the image-based measurement setup. The setup can be used to perform bidirectional reflectance measurements on samples with properties similar to the samples used in this paper.

Estimation of fluorescent Donaldson matrices using a spectral imaging system.

The present paper proposes a method to estimate the bispectral Donaldson matrices of fluorescent objects in a scene with a spectral imaging system. Multiple ordinary light sources with continuous spectral-power distributions are projected sequentially onto object surfaces without controlling the spectral shape of the illumination source. The estimation problem of the Donaldson matrices is solved as an optimization problem, where the residual error of observations by the spectral imaging system is minimized. The reflection, emission, and excitation spectral functions are estimated at each wavelength without using a basis function approximation. To improve the estimation efficiency, the output visible range is segmented into two types of wavelength ranges: one for only reflection and another for both reflection and emission. An iterative algorithm is then developed based on the wavelength segmentation and the physical excitation model. The usefulness of the proposed method is examined in experiments using different fluorescent objects and illuminants. We show the estimation accuracy of the Donaldson matrices, discuss the effective selection of illuminants, and demonstrate an application to spectral analysis and reconstruction of a fluorescent image.

Image processing pipeline for segmentation and material classification based on multispectral high dynamic range polarimetric images.

We propose a method for the capture of high dynamic range (HDR), multispectral (MS), polarimetric (Pol) images of indoor scenes using a liquid crystal tunable filter (LCTF). We have included the adaptive exposure estimation (AEE) method to fully automatize the capturing process. We also propose a pre-processing method which can be applied for the registration of HDR images after they are already built as the result of combining different low dynamic range (LDR) images. This method is applied to ensure a correct alignment of the different polarization HDR images for each spectral band. We have focused our efforts in two main applications: object segmentation and classification into metal and dielectric classes. We have simplified the segmentation using mean shift combined with cluster averaging and region merging techniques. We compare the performance of our segmentation with that of Ncut and Watershed methods. For the classification task, we propose to use information not only in the highlight regions but also in their surrounding area, extracted from the degree of linear polarization (DoLP) maps. We present experimental results which proof that the proposed image processing pipeline outperforms previous techniques developed specifically for MSHDRPol image cubes.

Spectral image analysis of mutual illumination between florescent objects.

This paper proposes a method for modeling and component estimation of the spectral images of the mutual illumination phenomenon between two fluorescent objects. First, we briefly describe the bispectral characteristics of a single fluorescent object, which are summarized as a Donaldson matrix. We suppose that two fluorescent objects with different bispectral characteristics are located close together under a uniform illumination. Second, we model the mutual illumination between two objects. It is shown that the spectral composition of the mutual illumination is summarized with four components: (1) diffuse reflection, (2) diffuse-diffuse interreflection, (3) fluorescent self-luminescence, and (4) interreflection by mutual fluorescent illumination. Third, we develop algorithms for estimating the spectral image components from the observed images influenced by the mutual illumination. When the exact Donaldson matrices caused by the mutual illumination influence are unknown, we have to solve a non-linear estimation problem to estimate both the spectral functions and the location weights. An iterative algorithm is then proposed to solve the problem based on the alternate estimation of the spectral functions and the location weights. In our experiments, the feasibility of the proposed method is shown in three cases: the known Donaldson matrices, weak interreflection, and strong interreflection.

Estimation of bispectral Donaldson matrices of fluorescent objects by using two illuminant projections.

This paper proposes a method for estimating the bispectral Donaldson matrices of fluorescent objects by using only two illuminant projections with continuous spectral power distributions. The Donaldson matrix represents the spectral radiance factor consisting of the sum of two components: a reflected radiance factor and a luminescent radiance factor. First, we describe the spectral characteristics of the observed matrix and model the matrix so that the luminescent radiance factor is separable into the emission and excitation wavelength components. We make no assumption as to the spectral shapes of any components, but derive a physical model that is useful for predicting the excitation spectral component from the reflected radiance component. An algorithm is developed to estimate the entire elements of the Donaldson matrix based on only two sets of spectral sensor outputs under two different illuminants. We suggest that the difference between the observed reflected radiance factors under the two different illuminants is not caused by the reflected radiance component, but only the luminescent radiance component. The algorithm is a sequential estimation of three radiance components of luminescent excitation, luminescent emission, and reflection. The feasibility of the proposed method is confirmed in experiments using a variety of fluorescent samples. The estimation accuracy is evaluated numerically in root-mean squared error and the color difference under the assumption of a viewing illuminant. An optimal selection of the illuminant pair is shown based on a simulation experiment using blackbody radiators with different color temperatures.

Imaging systems and applications: introduction to the feature.

Imaging systems have numerous applications in industrial, military, consumer, and medical settings. Assembling a complete imaging system requires the integration of optics, sensing, image processing, and display rendering. This issue features original research ranging from design of stimuli for human perception, optics applications, and image enhancement to novel imaging modalities in both color and infrared spectral imaging, gigapixel imaging as well as a systems perspective to imaging.

Spectral imaging by synchronizing capture and illumination.

This paper proposes a spectral imaging technology by synchronizing a programmable light source and a high-speed monochrome camera. The light source is capable of emitting arbitrary spectrum in high speed, so that the system has the advantage of capturing spectral images without using filters. The camera and the light source are controlled by a computer in order to capture image sequence synchronously with camera and illuminant control signals. First, we describe a projector for spectrally rendering a real scene as a fundamental usage of the spectral imaging system. Second, we describe the effective applications to (1) spectral reflectance recovery and (2) tristimulus imager. The performances of the proposed algorithms to solve the applied problems are examined in experiments in detail. We demonstrate potential applicability of the proposed spectral imaging technology.