Completely Blind Quality Assessment of User Generated Video Content.

In this work, we address the challenging problem of completely blind video quality assessment (BVQA) of user generated content (UGC). The challenge is twofold since the quality prediction model is oblivious of human opinion scores, and there are no well-defined distortion models for UGC content. Our solution is inspired by a recent computational neuroscience model which hypothesizes that the human visual system (HVS) transforms a natural video input to follow a straighter temporal trajectory in the perceptual domain. A bandpass filter based computational model of the lateral geniculate nucleus (LGN) and V1 regions of the HVS was used to validate the perceptual straightening hypothesis. We hypothesize that distortions in natural videos lead to loss in straightness (or increased curvature) in their transformed representations in the HVS. We provide extensive empirical evidence to validate our hypothesis. We quantify the loss in straightness as a measure of temporal quality, and show that this measure delivers acceptable quality prediction performance on its own. Further, the temporal quality measure is combined with a state-of-the-art blind spatial (image) quality metric to design a blind video quality predictor that we call STraightness Evaluation Metric (STEM). STEM is shown to deliver state-of-the-art performance over the class of BVQA algorithms on five UGC VQA datasets including KoNViD-1K, LIVE-Qualcomm, LIVE-VQC, CVD and YouTube-UGC. Importantly, our solution is completely blind i.e., training-free, generalizes very well, is explainable, has few tunable parameters, and is simple and easy to implement.

No-Reference Video Quality Assessment Using Natural Spatiotemporal Scene Statistics.

Robust spatiotemporal representations of natural videos have several applications including quality assessment, action recognition, object tracking etc. In this paper, we propose a video representation that is based on a parameterized statistical model for the spatiotemporal statistics of mean subtracted and contrast normalized (MSCN) coefficients of natural videos. Specifically, we propose an asymmetric generalized Gaussian distribution (AGGD) to model the statistics of MSCN coefficients of natural videos and their spatiotemporal Gabor bandpass filtered outputs. We then demonstrate that the AGGD model parameters serve as good representative features for distortion discrimination. Based on this observation, we propose a supervised learning approach using support vector regression (SVR) to address the no-reference video quality assessment (NRVQA) problem. The performance of the proposed algorithm is evaluated on publicly available video quality assessment (VQA) datasets with both traditional and in-capture/authentic distortions. We show that the proposed algorithm delivers competitive performance on traditional (synthetic) distortions and acceptable performance on authentic distortions. The code for our algorithm will be released at https://www.iith.ac.in/~lfovia/downloads.html.

Study of Subjective Quality and Objective Blind Quality Prediction of Stereoscopic Videos.

We present a new subjective and objective study on full high-definition (HD) stereoscopic (3D or S3D) video quality. In subjective study, we constructed an S3D video dataset with 12 pristine and 288 test videos, and the test videos are generated by applying the H.264 and H.265 compression, blur and frame freeze artifacts. We also propose a no reference (NR) objective video quality assessment (QA) algorithm that relies on measurements of the statistical dependencies between the motion and disparity subband coefficients of S3D videos. Inspired by the Generalized Gaussian Distribution (GGD) approach in liu2011statistical, we model the joint statistical dependencies between the motion and disparity components as following a Bivariate Generalized Gaussian Distribution (BGGD). We estimate the BGGD model parameters (α,ß) and the coherence measure (Ψ) from the eigenvalues of the sample covariance matrix (M) of the BGGD. In turn, we model the BGGD parameters of pristine S3D videos using a Multivariate Gaussian (MVG) distribution. The likelihood of a test video's MVG model parameters coming from the pristine MVG model is computed and shown to play a key role in the overall quality estimation. We also estimate the global motion content of each video by averaging the SSIM scores between pairs of successive video frames. To estimate the test S3D video's spatial quality, we apply the popular 2D NR unsupervised NIQE image QA model on a frame-by-frame basis on both views. The overall quality of a test S3D video is finally computed by pooling the test S3D video's likelihood estimates, global motion strength and spatial quality scores. The proposed algorithm, which is 'completely blind' (requiring no reference videos or training on subjective scores) is called the Motion and Disparity based 3D video quality evaluator (MoDi3D). We show that MoDi3D delivers competitive performance over a wide variety of datasets including the IRCCYN dataset, the WaterlooIVC Phase I dataset, the LFOVIA dataset and our proposed LFOVIAS3DPh2 S3D video dataset.

Multiview spatial compounding using lens-based photoacoustic imaging system.

Recently, an acoustic lens has been proposed for volumetric focusing as an alternative to conventional reconstruction algorithms in Photoacoustic (PA) Imaging. Acoustic lens can significantly reduce computational complexity and facilitate the implementation of real-time and cost-effective systems. However, due to the fixed focal length of the lens, the Point Spread Function (PSF) of the imaging system varies spatially. Furthermore, the PSF is asymmetric, with the lateral resolution being lower than the axial resolution. For many medical applications, such as in vivo thyroid, breast and small animal imaging, multiple views of the target tissue at varying angles are possible. This can be exploited to reduce the asymmetry and spatial variation of system the PSF with simple spatial compounding. In this article, we present a formulation and experimental evaluation of this technique. PSF improvement in terms of resolution and Signal to Noise Ratio (SNR) with the proposed spatial compounding is evaluated through simulation. Overall image quality improvement is demonstrated with experiments on phantom and ex vivo tissue. When multiple views are not possible, an alternative residual refocusing algorithm is proposed. The performances of these two methods, both separately and in conjunction, are compared and their practical implications are discussed.

Estimating Depth-Salient Edges and Its Application to Stereoscopic Image Quality Assessment.

The human visual system pays attention to salient regions while perceiving an image. When viewing a stereoscopic 3-D (S3D) image, we hypothesize that while most of the contribution to saliency is provided by the 2-D image, a small but significant contribution is provided by the depth component. Further, we claim that only a subset of image edges contribute to depth perception while viewing an S3D image. In this paper, we propose a systematic approach for depth saliency estimation, called salient edges with respect to depth perception (SED) which localizes the depth-salient edges in an S3D image. We demonstrate the utility of SED in full reference stereoscopic image quality assessment. We consider gradient magnitude and inter-gradient maps for predicting structural similarity. A coarse quality map is estimated first by comparing the 2-D saliency and gradient maps of reference and test stereo pairs. We average this quality map to estimate luminance quality and refine this quality map using SED maps for evaluating depth quality. Finally, we combine this luminance and depth quality to obtain an overall stereo image quality. We perform a comprehensive evaluation of our metric on seven publicly available S3D IQA databases. The proposed metric shows competitive performance on all seven databases with state-of-the-art performance on three of them.

Two-sided residual refocusing for an acoustic lens-based photoacoustic imaging system.

In photoacoustic (PA) cameras, an acoustic lens-based system can form a focused image of an object plane. A real-time C-scan PA image can be formed by simply time gating the transducer response. While most of the focusing action is performed by the lens, residual refocusing is needed to image multiple depths with high resolution simultaneously. However, a refocusing algorithm for a PA camera has not been studied so far in the literature. In this work, we reformulate this residual refocusing problem for a PA camera into a two-sided wave propagation from a planar sensor array. One part of the problem deals with forward wave propagation while the other deals with time reversal. We have chosen a fast Fourier transform (FFT) based wave propagation model for the refocusing to maintain the real-time nature of the system. We have conducted point spread function (PSF) measurement experiments at multiple depths and refocused the signal using the proposed method. The full width at half maximum (FWHM), peak value and signal to noise ratio (SNR) of the refocused PSF is analyzed to quantify the effect of refocusing. We believe that using a two-dimensional transducer array combined with the proposed refocusing can lead to real-time volumetric imaging using a PA camera.

Characterization of lens based photoacoustic imaging system.

Some of the challenges in translating photoacoustic (PA) imaging to clinical applications includes limited view of the target tissue, low signal to noise ratio and the high cost of developing real-time systems. Acoustic lens based PA imaging systems, also known as PA cameras are a potential alternative to conventional imaging systems in these scenarios. The 3D focusing action of lens enables real-time C-scan imaging with a 2D transducer array. In this paper, we model the underlying physics in a PA camera in the mathematical framework of an imaging system and derive a closed form expression for the point spread function (PSF). Experimental verification follows including the details on how to design and fabricate the lens inexpensively. The system PSF is evaluated over a 3D volume that can be imaged by this PA camera. Its utility is demonstrated by imaging phantom and an ex vivo human prostate tissue sample.

An Optical Flow-Based Full Reference Video Quality Assessment Algorithm.

We present a simple yet effective optical flow-based full-reference video quality assessment (FR-VQA) algorithm for assessing the perceptual quality of natural videos. Our algorithm is based on the premise that local optical flow statistics are affected by distortions and the deviation from pristine flow statistics is proportional to the amount of distortion. We characterize the local flow statistics using the mean, the standard deviation, the coefficient of variation (CV), and the minimum eigenvalue ( λ min ) of the local flow patches. Temporal distortion is estimated as the change in the CV of the distorted flow with respect to the reference flow, and the correlation between λ min of the reference and of the distorted patches. We rely on the robust multi-scale structural similarity index for spatial quality estimation. The computed temporal and spatial distortions, thus, are then pooled using a perceptually motivated heuristic to generate a spatio-temporal quality score. The proposed method is shown to be competitive with the state-of-the-art when evaluated on the LIVE SD database, the EPFL Polimi SD database, and the LIVE Mobile HD database. The distortions considered in these databases include those due to compression, packet-loss, wireless channel errors, and rate-adaptation. Our algorithm is flexible enough to allow for any robust FR spatial distortion metric for spatial distortion estimation. In addition, the proposed method is not only parameter-free but also independent of the choice of the optical flow algorithm. Finally, we show that the replacement of the optical flow vectors in our proposed method with the much coarser block motion vectors also results in an acceptable FR-VQA algorithm. Our algorithm is called the flow similarity index.

An automated algorithm for blood vessel count and area measurement in 2-D choroidal scan images.

We present an automated algorithm for the detection of blood vessels in 2-D choroidal scan images followed by a measurement of the area of the vessels. The objective is to identify vessel parameters in the choroidal stroma that are affected by various abnormalities. The algorithm is divided into five stages. In the first stage, the image is denoised to remove sensor noise and facilitate further processing. In the second stage, the image is segmented in order to find the region of interest. In the third stage, three different contour detection methods are applied to address different challenges in vessel contour. In the fourth stage, the outputs of the three contour detection methods are combined to achieve refined vessel contour detection. In the fifth and final stage, the area of these contours are measured. The results have been evaluated by a practicing opthalmologist and performance of the algorithm relative to expert detection is reported.

A low-cost scalable solution for digitizing analog X-rays with applications to rural healthcare.

In this paper, we present a low-cost scalable solution for digitizing analog X-ray images with the goal of improving diagnostics in rural and remote areas, in addition to having potential applications in disaster healthcare. Our solution attempts to capitalize on the rapid gains made in cellular communication and mobile technologies. The proposed mobile application lets the user digitally acquire the analog X-ray image and apply enhancement operations to it. A novel nonlinear technique for X-ray image enhancement has been proposed and implemented in the application. Additionally, several standard enhancement techniques have also been implemented. A proof-of-concept of the proposed solution is demonstrated with an Android application running on a smartphone. Results from real-world data collected at a semi-urban hospital in India are presented. The Android application has been made available online at the fifth authors' homepage.

Comparison of pre-processing techniques for fluorescence microscopy images of cells labeled for actin.

Automated analysis of fluorescence microscopy images of endothelial cells labeled for actin is important for quantifying changes in the actin cytoskeleton. The current manual approach is laborious and inefficient. The goal of our work is to develop automated image analysis methods, thereby increasing cell analysis throughput. In this study, we present preliminary results on comparing different algorithms for cell segmentation and image denoising.

Rate bounds on SSIM index of quantized images.

In this paper, we derive bounds on the structural similarity (SSIM) index as a function of quantization rate for fixed-rate uniform quantization of image discrete cosine transform (DCT) coefficients under the high-rate assumption. The space domain SSIM index is first expressed in terms of the DCT coefficients of the space domain vectors. The transform domain SSIM index is then used to derive bounds on the average SSIM index as a function of quantization rate for uniform, Gaussian, and Laplacian sources. As an illustrative example, uniform quantization of the DCT coefficients of natural images is considered. We show that the SSIM index between the reference and quantized images fall within the bounds for a large set of natural images. Further, we show using a simple example that the proposed bounds could be very useful for rate allocation problems in practical image and video coding applications.

Design of linear equalizers optimized for the structural similarity index.

We propose an algorithm for designing linear equalizers that maximize the structural similarity (SSIM) index between the reference and restored signals. The SSIM index has enjoyed considerable application in the evaluation of image processing algorithms. Algorithms, however, have not been designed yet to explicitly optimize for this measure. The design of such an algorithm is nontrivial due to the nonconvex nature of the distortion measure. In this paper, we reformulate the nonconvex problem as a quasi-convex optimization problem, which admits a tractable solution. We compute the optimal solution in near closed form, with complexity of the resulting algorithm comparable to complexity of the linear minimum mean squared error (MMSE) solution, independent of the number of filter taps. To demonstrate the usefulness of the proposed algorithm, it is applied to restore images that have been blurred and corrupted with additive white gaussian noise. As a special case, we consider blur-free image denoising. In each case, its performance is compared to a locally adaptive linear MSE-optimal filter. We show that the images denoised and restored using the SSIM-optimal filter have higher SSIM index, and superior perceptual quality than those restored using the MSE-optimal adaptive linear filter. Through these results, we demonstrate that a) designing image processing algorithms, and, in particular, denoising and restoration-type algorithms, can yield significant gains over existing (in particular, linear MMSE-based) algorithms by optimizing them for perceptual distortion measures, and b) these gains may be obtained without significant increase in the computational complexity of the algorithm.