Unsupervised design for broadband multispectral and polarization filter array patterns.

Imaging multiple wavelength and polarization components is problematic due to the complexity of equipment and the increase in the number of imaging shots, so imaging using filter arrays with various patterns has been widely reported from elemental research to practical applications. Most of them use bandpass filters with different center wavelengths for each pixel. Recently, however, filter arrays with multimodal transmission characteristics have been proposed using photonic crystals or Fabry-Perot filters. In any of these methods, the design of the filter array arrangement pattern is important to improve the quality of the captured image, as well as the improvement of the demosaicking algorithm. One way to design a filter array pattern is to minimize the mean squared error (MSE) between the ideal image and the demosaicked image. However, the more multidimensional the imaging components, the more difficult it becomes to collect training data. In such cases, it is necessary to empirically determine candidate transmission characteristics and patterns of filter arrays. In this study, we propose a method for evaluating filter array patterns without using any training data in the design of filter arrays for multispectral and polarization imaging. The proposed method estimates the MSE by approximating the autocorrelation matrix without using image data by expressing the imaging model as a linear forward problem and the demosaicking as a linear inverse problem. Since this method can be applied not only to ideal bandpass filter arrangements, but also to multispectral filter arrays with multimodal spectral transmission characteristics and even multispectral polarization filter arrays with different extinction ratios at different wavelengths, we will show that image quality can be improved over empirical arrangements by evaluating these patterns and by testing examples of optimal designs using genetic algorithms.

General demosaicking for multispectral polarization filter arrays using total generalized variation and weighted tensor nuclear norm minimization.

We focus on a demosaicking method for recovering multispectral polarization images (MSPIs) from a single image captured by a multispectral polarization filter array (MSPFA). Since the image captured by the MSPFA can be represented by a linear model, an algorithm to solve the inverse problem can be designed to enable general-purpose demosaicking regardless of the transmission characteristics and patterns of the MSPFA. Thus, we propose a method for demosaicking MSPIs by solving an inverse problem that introduces the decorrelated vectorial total generalized variation (D-VTGV) and weighted tensor nuclear norm (WTNN) regularization functions. D-VTGV evaluates the edge-preserving property in the spatial direction while preserving the correlation between bands and polarization angles, while WTNN exploits the correlation and low-rank property in nonlocal regions of the image to perform proper texture restoration and denoising. The experimental results show that the proposed method can restore images well for both the ideal MSPFA and an MSPFA manufactured from photonic crystals.

Alignment-free filter array: Snapshot multispectral polarization imaging based on a Voronoi-like random photonic crystal filter.

We develop a photonic crystal filter with a new structure and propose a method to realize a snapshot multispectral polarization camera by mounting the filter on a monochrome imager with no requirement for a specific alignment. The developed filter is based on the Voronoi structure, which forms multilayered photonic crystals with random wave-like structures in each of the Voronoi cells. Because the transmission characteristics of the multilayered photonic crystal can be controlled simply by changing the microstructure, there is no need to change the manufacturing process and materials for each Voronoi cell. Furthermore, the Voronoi cell is randomly distributed so that the filter can be junctioned with the imager at arbitrary positions and angles without the need to position the filter during mounting, although it requires measurement of the camera characteristics and an image restoration process after filter mounting. In this experiment, we evaluated to reconstruct spectra as well as linearly polarized components and RGB images in the visible wavelength range from a single exposure image.

Direct estimation of NIR reflection spectra utilizing a snapshot-type spectrometer with photonic crystal multi-spectral filter arrays.

We fabricated 16-, 25-, 36-, and 64-channel distributed passband-type multi-spectral filter arrays by utilizing a multilayer-type photonic crystal and integrated them onto a CCD to form a snapshot-type spectroscopic sensor. Reflection spectra from target objects (fruits) under broadband light illumination were estimated directly using the Wiener estimation method. A root mean square error of the reflectivity on the order of 2∼5% was obtained under optical shot noise with 6×6 pixel binning. A number of constituent filters of 36 was sufficient for this type of fruit spectral measurement. We also visualized reflection images at specified wavelengths by applying the estimation method to a multiple filter region on the sensor.

Semi-Automatic Calibration Method for a Bed-Monitoring System Using Infrared Image Depth Sensors.

With the aging of society, the number of fall accidents has increased in hospitals and care facilities, and some accidents have happened around beds. To help prevent accidents, mats and clip sensors have been used in these facilities but they can be invasive, and their purpose may be misinterpreted. In recent years, research has been conducted using an infrared-image depth sensor as a bed-monitoring system for detecting a patient getting up, exiting the bed, and/or falling; however, some manual calibration was required initially to set up the sensor in each instance. We propose a bed-monitoring system that retains the infrared-image depth sensors but uses semi-automatic rather than manual calibration in each situation where it is applied. Our automated methods robustly calculate the bed region, surrounding floor, sensor location, and attitude, and can recognize the spatial position of the patient even when the sensor is attached but unconstrained. Also, we propose a means to reconfigure the spatial position considering occlusion by parts of the bed and also accounting for the gravity center of the patient's body. Experimental results of multi-view calibration and motion simulation showed that our methods were effective for recognition of the spatial position of the patient.

Detection of pancreatic tumor cell nuclei via a hyperspectral analysis of pathological slides based on stain spectra.

Hyperspectral imaging (HSI) provides more detailed information than red-green-blue (RGB) imaging, and therefore has potential applications in computer-aided pathological diagnosis. This study aimed to develop a pattern recognition method based on HSI, called hyperspectral analysis of pathological slides based on stain spectrum (HAPSS), to detect cancers in hematoxylin and eosin-stained pathological slides of pancreatic tumors. The samples, comprising hyperspectral cubes of 420-750 nm, were harvested for HSI and tissue microarray (TMA) analysis. As a result of conducting HAPSS experiments with a support vector machine (SVM) classifier, we obtained maximal accuracy of 94%, a 14% improvement over the widely used RGB images. Thus, HAPSS is a suitable method to automatically detect tumors in pathological slides of the pancreas.

NIR spectrum estimation utilizing a photonic crystal distributed passband-type multiple filter array.

A near-infrared (NIR) spectral sensor consisting of a 25-channel dielectric multi-patterned filter array (MFA) and CCD is proposed and fabricated. The MFA consists of a wavy dielectric multilayer with a gradient layer profile on a silica substrate with surface grating. The incoming NIR spectrum is predicted by Wiener estimation utilizing the MFA's spectral responses and a set of training spectral data. Estimation performance is evaluated under various optical shot-noise conditions.

Demosaicking Using a Spatial Reference Image for an Anti-Aliasing Multispectral Filter Array.

Multispectral imaging with a multispectral filter array (MSFA) facilitates snapshot imaging; however, a demosaicking process is required to estimate a fully defined multispectral image based on undersampled sensor data. Undersampling induces aliasing and adverse artifacts in the reconstructed image. To solve this problem, Jia et al. proposed the Fourier spectral filter array (FSFA), which can reduce aliasing. In this paper, we analyze the FSFA and a more generalized anti-aliasing MSFA, and we identify the property that makes MSFAs anti-aliasing. Furthermore, we propose a novel demosaicking method that is a hybrid of frequency-decomposition-based and compressive-sensing-based demosaicking. Anti-aliasing MSFAs enable demosaicking to comprehend the precise spatial structures of an image. The image assists our proposed method in precisely reconstructing images using compressive sensing. Our experimental results demonstrated that the proposed method performs better than the existing demosaicking methods, especially in terms of spatial reconstruction.

Snapshot multispectral polarization imaging using a photonic crystal filter array.

A new filter array and a demosaicking method for snapshot multispectral polarization imaging are proposed in this paper. The proposed filter array is a thin-film wavy multilayer structure regarded as a photonic crystal that can be fabricated using the autocloning method. The multispectral polarization filter array is developed by altering the wave structure of the photonic crystal at each pixel. In addition, we propose a demosaicking method for multispectral polarization images by considering snapshot imaging as a linear model. In the experiments, we evaluated the recovered spectrum error in some color charts and showed various demosaicked images such as multispectral polarization images, specific-band degree of linear polarization images, polarized RGB images, and non-polarized RGB images.

Feature analysis of cell nuclear chromatin distribution in support of cervical cytology.

Cytology, a method of estimating cancer or cellular atypia from microscopic images of scraped specimens, is used according to the pathologist's experience to diagnose cases based on the degree of structural changes and atypia. Several methods of cell feature quantification, including nuclear size, nuclear shape, cytoplasm size, and chromatin texture, have been studied. We focus on chromatin distribution in the cell nucleus and propose new feature values that indicate the chromatin complexity, spreading, and bias, including convex hull ratio on multiple binary images, intensity distribution from the gravity center, and tangential component intensity and texture biases. The characteristics and cellular classification accuracies of the proposed features were verified through experiments using cervical smear samples, for which clear nuclear morphologic diagnostic criteria are available. In this experiment, we also used a stepwise support vector machine to create a machine learning model and a cross-validation algorithm with which to derive identification accuracy. Our results demonstrate the effectiveness of our proposed feature values.