Design of a Dual-Mode Multispectral Filter Array.

Multispectral imaging is valuable in many vision-related fields as it provides an additional modality to observe the world. Cameras equipped with multispectral filter arrays (MSFAs) are typically impractical for everyday use due to their intractable demosaicking and chromatic reproduction processes, which restrict their applicability beyond academic research. In this work, a novel MSFA design is proposed to enable dual-mode imaging for multispectral cameras. In addition to a conventional multispectral image, the camera is also able to produce a Bayer-formed RGB image from a single shot by grouping and merging adjacent pixels in the proposed MSFA, making it suitable for scenarios where display-ready RGB images are required. Furthermore, a two-stage optimization scheme is implemented to jointly optimize objective functions for both imaging modes. The evaluation results on multiple datasets suggest that the proposed MSFA design is able to simultaneously achieve competitive spectral reconstruction accuracy compared to elaborate multispectral cameras and chromatic accuracy compared to commercial RGB cameras.

Color Constancy by Reweighting Image Feature Maps.

In this study, a novel illuminant color estimation framework is proposed for computational color constancy, which incorporates the high representational capacity of deep-learningbased models and the great interpretability of assumptionbased models. The well-designed building block, feature map reweight unit (ReWU), helps to achieve comparative accuracy on benchmark datasets with respect to prior state-of-the-art deep learning based models while requiring more compact model size and cheaper computational cost. In addition to local color estimation, a confidence estimation branch is also included such that the model is able to simultaneously produce point estimate and its uncertainty estimate, which provides useful clues for local estimates aggregation and multiple illumination estimation. The source code and the dataset have been made available1.

Image quality degradation of object-color metamer mismatching in digital camera color reproduction.

Metamer mismatching is a phenomenon where two objects that are colorimetrically indistinguishable under one lighting condition become distinguishable under another one. Due to the unavailability of spectral information, metamer mismatching introduces an inherent uncertainty into cameras' color reproduction. To investigate the degree of image quality degradation by the metamer mismatching, a large spectral reflectance database was compiled in this study to search the object-color metamers sets of the spectra in hyperspectral images. Then, metamer-degraded images were constructed and compared with the ground truth images by directional statistics-based color similarity index image quality assessment metrics to evaluate the perceptual image degradation. The results indicate that the object-color metamer mismatching has only little impact on the image quality degradation, whereas the inappropriate selection of color correction matrices involved with the illumination metamerism is the primary factor for the accuracy decrease in the digital camera color reproduction.

Self-training-based spectral image reconstruction for art paintings with multispectral imaging.

A self-training-based spectral reflectance recovery method was developed to accurately reconstruct the spectral images of art paintings with multispectral imaging. By partitioning the multispectral images with the k-means clustering algorithm, the training samples are directly extracted from the art painting itself to restrain the deterioration of spectral estimation caused by the material inconsistency between the training samples and the art painting. Coordinate paper is used to locate the extracted training samples. The spectral reflectances of the extracted training samples are acquired indirectly with a spectroradiometer, and the circle Hough transform is adopted to detect the circle measuring area of the spectroradiometer. Through simulation and a practical experiment, the implementation of the proposed method is explained in detail, and it is verified to have better reflectance recovery performance than that using the commercial target and is comparable to the approach using a painted color target.