Compression for Bayer CFA Images: Review and Performance Comparison.

Bayer color filter array (CFA) images are captured by a single-chip image sensor covered with a Bayer CFA pattern which has been widely used in modern digital cameras. In the past two decades, many compression methods have been proposed to compress Bayer CFA images. These compression methods can be roughly divided into the compression-first-based (CF-based) scheme and the demosaicing-first-based (DF-based) scheme. However, in the literature, no review article for the two compression schemes and their compression performance is reported. In this article, the related CF-based and DF-based compression works are reviewed first. Then, the testing Bayer CFA images created from the Kodak, IMAX, screen content images, videos, and classical image datasets are compressed on the Joint Photographic Experts Group-2000 (JPEG-2000) and the newly released Versatile Video Coding (VVC) platform VTM-16.2. In terms of the commonly used objective quality, perceptual quality metrics, the perceptual effect, and the quality-bitrate tradeoff metric, the compression performance comparison of the CF-based compression methods, in particular the reversible color transform-based compression methods and the DF-based compression methods, is reported and discussed.

Effective Three-Stage Demosaicking Method for RGBW CFA Images Using The Iterative Error-Compensation Based Approach.

As the color filter array (CFA)2.0, the RGBW CFA pattern, in which each CFA pixel contains only one R, G, B, or W color value, provides more luminance information than the Bayer CFA pattern. Demosaicking RGBW CFA images I R G B W is necessary in order to provide high-quality RGB full-color images as the target images for human perception. In this letter, we propose a three-stage demosaicking method for I R G B W . In the first-stage, a cross shape-based color difference approach is proposed in order to interpolate the missing W color pixels in the W color plane of I R G B W . In the second stage, an iterative error compensation-based demosaicking process is proposed to improve the quality of the demosaiced RGB full-color image. In the third stage, taking the input image I R G B W as the ground truth RGBW CFA image, an I R G B W -based refinement process is proposed to refine the quality of the demosaiced image obtained by the second stage. Based on the testing RGBW images that were collected from the Kodak and IMAX datasets, the comprehensive experimental results illustrated that the proposed three-stage demosaicking method achieves substantial quality and perceptual effect improvement relative to the previous method by Hamilton and Compton and the two state-of-the-art methods, Kwan et al.'s pansharpening-based method, and Kwan and Chou's deep learning-based method.

Effective Content-Aware Chroma Reconstruction Method for Screen Content Images.

In this paper, we propose an effective novel content-aware chroma reconstruction (CACR) method for screen content images (SCIs). After receiving the decoded downsampled YUV image on the client side, our fast chroma-copy approach reconstructs the missing chroma pixels in the flat regions of SCI. Then, for non-flat regions, a non-flat region-based winner-first voting (NRWV) strategy is proposed to identify the chroma subsampling scheme used on the server side prior to compression. Further, an effective adaptive hybrid approach is proposed to reconstruct each missing chroma pixel in the non-flat region by fusing the two reconstructed results, one from our modified NRWV-based chroma subsampling-binding and luma-guided chroma reconstruction scheme, which favors the sharp edges in SCI, as well as the other from the bicubic interpolation scheme, which favors blurred and continuous-tone textures. Further, based on the identified chroma subsampling scheme, a geometry alignment-based error compensation approach is proposed to enhance the reconstructed chroma image. Based on typical test SCIs and JCT-VC screen content videos, comprehensive experiments are carried out in HEVC-16.17 to demonstrate that in terms of quality, visual effect, and quality-bitrate tradeoff of the reconstructed SCIs, our CACR method significantly outperforms the existing state-of-the-art methods.

Adaptive Effective Wiener Filter- and Regression-based Upsampling for Asymmetric Resolution Stereoscopic Video Coding.

In asymmetric resolution stereoscopic video coding (ARSVC), a stereoscopic video consists of one full-sized leftview video sequence and the synchronized quarter-sized rightview video sequence for achieving a bitrate reduction effect by the encoder. Prior to displaying 3D scenes on the screen, it is necessary to upsample the decoded downsampled right-view video sequence at the client side. In this paper, we propose an effective adaptive upsampling method for ARSVC. First, we employ the resolution- and texture-consistency (RTC) consideration in the conventional Wiener filter-based interpolation scheme, called RTCWF, to enhance the upsampling accuracy in the spatial domain. Second, we propose a linear regression-based interview prediction (LRIP) scheme with residual compensation (RC), called LRIPRC, to increase the upsampling accuracy in the interview domain. Third, we propose an adaptive fusion-based approach to integrate RTCWF and LRIPRC, called RTCWFLRIPRC, to maximize the quality improvement of the upsampled image. Based on seven typical test stereoscopic video sequences, in 3D-HEVC, the experimental results demonstrated that in terms of six well-known quality metrics and execution time requirements, our RTCWF-LRIPRC method outperforms the state-of-the-art upsampling methods for ARSVC.

Joint Chroma Subsampling and Distortion-Minimization-Based Luma Modification for RGB Color Images With Application.

In this paper, we propose a novel and effective hybrid method, which joins the conventional chroma subsampling and the distortion-minimization-based luma modification together, to improve the quality of the reconstructed RGB full-color image. Assume the input RGB full-color image has been transformed to a YUV image, prior to compression. For each 2×2 UV block, one 4:2:0 subsampling is applied to determine the one subsampled U and V components, Us and Vs . Based on Us , Vs , and the corresponding 2×2 original RGB block, a main theorem is provided to determine the ideally modified 2×2 luma block in constant time such that the color peak signal-to-noise ratio (CPSNR) quality distortion between the original 2×2 RGB block and the reconstructed 2×2 RGB block can be minimized in a globally optimal sense. Furthermore, the proposed hybrid method and the delivered theorem are adjusted to tackle the digital time delay integration images and the Bayer mosaic images whose Bayer CFA structure has been widely used in modern commercial digital cameras. Based on the IMAX, Kodak, and screen content test image sets, the experimental results demonstrate that in high efficiency video coding, the proposed hybrid method has substantial quality improvement, in terms of the CPSNR quality, visual effect, CPSNR-bitrate trade-off, and Bjøntegaard delta PSNR performance, of the reconstructed RGB images when compared with existing chroma subsampling schemes.

Demosaicing of color filter array captured images using gradient edge detection masks and adaptive heterogeneity-projection.

Without demosaicing processing, this paper first proposes a new approach to extract more accurate gradient/edge information on mosaic images directly. Next, based on spectral-spatial correlation, a novel adaptive heterogeneity-projection with proper mask size for each pixel is presented. Combining the extracted gradient/edge information and the adaptive heterogeneity-projection values, a new edge-sensing demosaicing algorithm is presented. Based on 24 popular testing images, experimental results demonstrated that our proposed high-quality demosaicing algorithm has the best image quality performance when compared with several recently published algorithms.

Texture- and multiple-template-based algorithm for lossless compression of error-diffused images.

Recently, several efficient context-based arithmetic coding algorithms have been developed successfully for lossless compression of error-diffused images. In this paper, we first present a novel block- and texture-based approach to train the multiple-template according to the most representative texture features. Based on the trained multiple template, we next present an efficient texture- and multiple-template-based (TM-based) algorithm for lossless compression of error-diffused images. In our proposed TM-based algorithm, the input image is divided into many blocks and for each block, the best template is adaptively selected from the multiple-template based on the texture feature of that block. Under 20 testing error-diffused images and the personal computer with Intel Celeron 2.8-GHz CPU, experimental results demonstrate that with a little encoding time degradation, 0.365 s (0.901 s) on average, the compression improvement ratio of our proposed TM-based algorithm over the joint bilevel image group (JBIG) standard [over the previous block arithmetic coding for image compression (BACIC) algorithm proposed by Reavy and Boncelet is 24%] (19.4%). Under the same condition, the compression improvement ratio of our proposed algorithm over the previous algorithm by Lee and Park is 17.6% and still only has a little encoding time degradation (0.775 s on average). In addition, the encoding time required in the previous free tree-based algorithm is 109.131 s on average while our proposed algorithm takes 0.995 s; the average compression ratio of our proposed TM-based algorithm, 1.60, is quite competitive to that of the free tree-based algorithm, 1.62.

Inverse halftoning algorithm using edge-based lookup table approach.

The inverse halftoning algorithm is used to reconstruct a gray image from an input halftone image. Based on the recently published lookup table (LUT) technique, this paper presents a novel edge-based LUT method for inverse halftoning which improves the quality of the reconstructed gray image. The proposed method first uses the LUT-based inverse halftoning method as a preprocessing step to transform the given halftone image to a base gray image, and then the edges are extracted and classified from the base gray image. According to these classified edges, a novel edge-based LUT is built up to reconstruct the gray image. Based on a set of 30 real training images with both low-and high-frequency contents, experimental results demonstrated that the proposed method achieves a better image quality when compared to the currently published two methods, by Chang et al. and Mese and Vaidyanathan.

A new predictive search area approach for fast block motion estimation.

According to the observation on the distribution of motion differentials among the motion vector of any block and those of its four neighboring blocks from six real video sequences, this paper presents a new predictive search area approach for fast block motion estimation. Employing our proposed simple predictive search area approach into the full search (FS) algorithm, our improved FS algorithm leads to 93.83% average execution-time improvement ratio, but only has a small estimation accuracy degradation. We also investigate the advantages of computation and estimation accuracy of our improved FS algorithm when compared to the edge-based search algorithm of Chan and Siu; experimental results reveal that our improved FS algorithm has 74.33% average execution-time improvement ratio and has a higher estimation accuracy. Finally, we further compare the performance among our improved FS algorithm, the three-step search algorithm, and the block-based gradient descent search algorithm.