NeRF-NQA: No-Reference Quality Assessment for Scenes Generated by NeRF and Neural View Synthesis Methods.

Neural View Synthesis (NVS) has demonstrated efficacy in generating high-fidelity dense viewpoint videos using a image set with sparse views. However, existing quality assessment methods like PSNR, SSIM, and LPIPS are not tailored for the scenes with dense viewpoints synthesized by NVS and NeRF variants, thus, they often fall short in capturing the perceptual quality, including spatial and angular aspects of NVS-synthesized scenes. Furthermore, the lack of dense ground truth views makes the full reference quality assessment on NVS-synthesized scenes challenging. For instance, datasets such as LLFF provide only sparse images, insufficient for complete full-reference assessments. To address the issues above, we propose NeRF-NQA, the first no-reference quality assessment method for densely-observed scenes synthesized from the NVS and NeRF variants. NeRF-NQA employs a joint quality assessment strategy, integrating both viewwise and pointwise approaches, to evaluate the quality of NVS-generated scenes. The viewwise approach assesses the spatial quality of each individual synthesized view and the overall inter-views consistency, while the pointwise approach focuses on the angular qualities of scene surface points and their compound inter-point quality. Extensive evaluations are conducted to compare NeRF-NQA with 23 mainstream visual quality assessment methods (from fields of image, video, and light-field assessment). The results demonstrate NeRF-NQA outperforms the existing assessment methods significantly and it shows substantial superiority on assessing NVS-synthesized scenes without references. An implementation of this paper are available at https://github.com/VincentQQu/NeRF-NQA.

LFACon: Introducing Anglewise Attention to No-Reference Quality Assessment in Light Field Space.

Light field imaging can capture both the intensity information and the direction information of light rays. It naturally enables a six-degrees-of-freedom viewing experience and deep user engagement in virtual reality. Compared to 2D image assessment, light field image quality assessment (LFIQA) needs to consider not only the image quality in the spatial domain but also the quality consistency in the angular domain. However, there is a lack of metrics to effectively reflect the angular consistency and thus the angular quality of a light field image (LFI). Furthermore, the existing LFIQA metrics suffer from high computational costs due to the excessive data volume of LFIs. In this paper, we propose a novel concept of "anglewise attention" by introducing a multihead self-attention mechanism to the angular domain of an LFI. This mechanism better reflects the LFI quality. In particular, we propose three new attention kernels, including anglewise self-attention, anglewise grid attention, and anglewise central attention. These attention kernels can realize angular self-attention, extract multiangled features globally or selectively, and reduce the computational cost of feature extraction. By effectively incorporating the proposed kernels, we further propose our light field attentional convolutional neural network (LFACon) as an LFIQA metric. Our experimental results show that the proposed LFACon metric significantly outperforms the state-of-the-art LFIQA metrics. For the majority of distortion types, LFACon attains the best performance with lower complexity and less computational time.

Light Field Spatial Super-Resolution Using Deep Efficient Spatial-Angular Separable Convolution.

Light field (LF) photography is an emerging paradigm for capturing more immersive representations of the real-world. However, arising from the inherent trade-off between the angular and spatial dimensions, the spatial resolution of LF images captured by commercial micro-lens based LF cameras are significantly constrained. In this paper, we propose effective and efficient end-to-end convolutional neural network models for spatially super-resolving LF images. Specifically, the proposed models have an hourglass shape, which allows feature extraction to be performed at the low resolution level to save both computational and memory costs. To fully make use of the four-dimensional (4-D) structure information of LF data in both spatial and angular domains, we propose to use 4-D convolution to characterize the relationship among pixels. Moreover, as an approximation of 4-D convolution, we also propose to use spatialangular separable (SAS) convolutions for more computationallyand memory-efficient extraction of spatial-angular joint features. Extensive experimental results on 57 test LF images with various challenging natural scenes show significant advantages from the proposed models over state-of-the-art methods. That is, an average PSNR gain of more than 3.0 dB and better visual quality are achieved, and our methods preserve the LF structure of the super-resolved LF images better, which is highly desirable for subsequent applications. In addition, the SAS convolutionbased model can achieve 3× speed up with only negligible reconstruction quality decrease when compared with the 4-D convolution-based one. The source code of our method is online available at https://github.com/spatialsr/DeepLightFieldSSR.

A novel tamper detection-recovery and watermarking system for medical image authentication and EPR hiding.

Recently, the literature has witnessed an increasing interest in the study of medical image watermarking and recovery techniques. In this article, a novel image tamper localization and recovery technique for medical image authentication is proposed. The sparse coding of the Electronic Patient Record (EPR) and the reshaped region of Interest (ROI) is embedded in the transform domain of the Region of Non-Interest (RONI). The first part of the sparse coded watermark is use for saving the patient information along with the image, whereas the second part is used for authentication purpose. When the watermarked image is tampered during transmission between hospitals and medical clinics, the embedded sparse coded ROI can be extracted to recover the tampered image. The experimental results demonstrate the efficiency of the proposed technique in term of tamper correction capability, robustness to attacks, and imperceptibility.