Unsupervised spectral reconstruction from RGB images under two lighting conditions.

Unsupervised spectral reconstruction (SR) aims to recover the hyperspectral image (HSI) from corresponding RGB images without annotations. Existing SR methods achieve it from a single RGB image, hindered by the significant spectral distortion. Although several deep learning-based methods increase the SR accuracy by adding RGB images, their networks are always designed for other image recovery tasks, leaving huge room for improvement. To overcome this problem, we propose a novel, to our knowledge, approach that reconstructs the HSI from a pair of RGB images captured under two illuminations, significantly improving reconstruction accuracy. Specifically, an SR iterative model based on two illuminations is constructed at first. By unfolding the proximal gradient algorithm solving this SR model, an interpretable unsupervised deep network is proposed. All the modules in the proposed network have precise physical meanings, which enable our network to have superior performance and good generalization capability. Experimental results on two public datasets and our real-world images show the proposed method significantly improves both visually and quantitatively as compared with state-of-the-art methods.

DCT-net: Dual-domain cross-fusion transformer network for MRI reconstruction.

Current challenges in Magnetic Resonance Imaging (MRI) include long acquisition times and motion artifacts. To address these issues, under-sampled k-space acquisition has gained popularity as a fast imaging method. However, recovering fine details from under-sampled data remains challenging. In this study, we introduce a pioneering deep learning approach, namely DCT-Net, designed for dual-domain MRI reconstruction. DCT-Net seamlessly integrates information from the image domain (IRM) and frequency domain (FRM), utilizing a novel Cross Attention Block (CAB) and Fusion Attention Block (FAB). These innovative blocks enable precise feature extraction and adaptive fusion across both domains, resulting in a significant enhancement of the reconstructed image quality. The adaptive interaction and fusion mechanisms of CAB and FAB contribute to the method's effectiveness in capturing distinctive features and optimizing image reconstruction. Comprehensive ablation studies have been conducted to assess the contributions of these modules to reconstruction quality and accuracy. Experimental results on the FastMRI (2023) and Calgary-Campinas datasets (2021) demonstrate the superiority of our MRI reconstruction framework over other typical methods (most are illustrated in 2023 or 2022) in both qualitative and quantitative evaluations. This holds for knee and brain datasets under 4× and 8× accelerated imaging scenarios.

Hyperspectral image super-resolution via spectral matching and correction.

Fusing a low-spatial-resolution hyperspectral image (LR-HSI) and a high-spatial-resolution RGB image (HR-RGB) is an important technique for HR-HSI obtainment. In this paper, we propose a dual-illuminance fusion-based super-resolution method consisting of spectral matching and correction. In the spectral matching stage, an LR-HSI patch is first searched for each HR-RGB pixel; with the minimum color difference as a constraint, the matching spectrum is constructed by linear mixing the spectrum in the HSI patch. In the spectral correlation stage, we establish a polynomial model to correct the matched spectrum with the aid of the HR-RGBs illuminated by two illuminances, and the target spectrum is obtained. All pixels in the HR-RGB are traversed by the spectral matching and correction process, and the target HR-HSI is eventually reconstructed. The effectiveness of our method is evaluated on three public datasets and our real-world dataset. Experimental results demonstrate the effectiveness of our method compared with eight fusion methods.

Hyperspectral image super-resolution via a multi-stage scheme without employing spatial degradation.

Recently, it has become popular to obtain a high spatial resolution hyperspectral image (HR-HSI) by fusing a low spatial resolution hyperspectral image (LR-HSI) with a high spatial resolution RGB image (HR-RGB). Existing HSI super-resolution methods are designed based on a known spatial degeneration. In practice, it is difficult to obtain correct spatial degradation, which restricts the performance of existing methods. Therefore, we propose a multi-stage scheme without employing the spatial degradation model. The multi-stage scheme consists of three stages: initialization, modification, and refinement. According to the angle similarity between the HR-RGB pixel and LR-HSI spectra, we first initialize a spectrum for each HR-RGB pixel. Then, we propose a polynomial function to modify the initialized spectrum so that the RGB color values of the modified spectrum are the same as the HR-RGB. Finally, the modified HR-HSI is refined by a proposed optimization model, in which a novel, to the best of our knowledge, spectral-spatial total variation (SSTV) regularizer is investigated to keep the spectral and spatial structure of the reconstructed HR-HSI. The experimental results on two public datasets and our real-world images demonstrate our method outperforms eight state-of-the-art existing methods in terms of both reconstruction accuracy and computational efficiency.

Hyperspectral image super-resolution based on the transfer of both spectra and multi-level features.

Existing hyperspectral image (HSI) super-resolution methods fusing a high-resolution RGB image (HR-RGB) and a low-resolution HSI (LR-HSI) always rely on spatial degradation and handcrafted priors, which hinders their practicality. To address these problems, we propose a novel, to the best of our knowledge, method with two transfer models: a window-based linear mixing (W-LM) model and a feature transfer model. Specifically, W-LM initializes a high-resolution HSI (HR-HSI) by transferring the spectra from the LR-HSI to the HR-RGB. By using the proposed feature transfer model, the HR-RGB multi-level features extracted by a pre-trained convolutional neural network (CNN) are then transferred to the initialized HR-HSI. The proposed method fully exploits spectra of LR-HSI and multi-level features of HR-RGB and achieves super-resolution without requiring the spatial degradation model and any handcrafted priors. The experimental results for 32 × super-resolution on two public datasets and our real image set demonstrate the proposed method outperforms eight state-of-the-art existing methods.

Optimizing selection of the test color sample set for the CIE 2017 color fidelity index.

We have developed a new method for selecting the test color sample set (TCSS) used to calculate CIE 2017 color fidelity index (CIE-Rf). Taking a Large Set as a starting point, a new optimized color sample set (OCSS) is obtained by clustering analysis. Taking metamerism phenomenon into account, spectra clustering is performed within the class obtained from color appearance attributes clustering. The CIE-Rf of 1202 light sources are calculated and analyzed by taking the Large Set, OCSS and CIE color evaluation sample set (CIE CESS-99) as TCSS. Through analyzing CIE-Rf, the performance of the OCSS is further investigated. The results show that the clustering analysis method developed in this paper can be well used in selecting test color samples, and the obtained OCSS can represent Large Set well and be better used for color fidelity metrics of light sources.

[A novel spatial modulation Fourier transform spectrometer with adjustable spectral resolution].

In the premise of fulfilling the application requirement, the adjustment of spectral resolution can improve efficiency of data acquisition, data processing and data saving. So, by adjusting the spectral resolution, the performance of spectrometer can be improved, and its application range can be extended. To avoid the problems of the fixed spectral resolution of classical Fourier transform spectrometer, a novel type of spatial modulation Fourier transform spectrometer with adjustable spectral resolution is proposed in this paper. The principle of the novel spectrometer and its interferometer is described. The general expressions of the optical path difference and the lateral shear are induced by a ray tracing procedure. The equivalent model of the novel interferometer is analyzed. Meanwhile, the principle of the adjustment of spectral resolution is analyzed. The result shows that the novel spectrometer has the merits of adjustable spectral resolution, high stability, easy assemblage and adjustment etc. This theoretical study will provide the theoretical basis for the design of the spectrometer with adjustable spectral resolution and expand the application range of Fourier transform spectrometer.