The Importance of Phase to Texture Discrimination and Similarity.

In this article, we investigate the importance of phase for texture discrimination and similarity estimation tasks. We first use two psychophysical experiments to investigate the relative importance of phase and magnitude spectra for human texture discrimination and similarity estimation. The results show that phase is more important to humans for both tasks. We further examine the ability of 51 computational feature sets to perform these two tasks. In contrast with the psychophysical experiments, it is observed that the magnitude data is more important to these computational feature sets than the phase data. We hypothesise that this inconsistency is due to the difference between the abilities of humans and the computational feature sets to utilise phase data. This motivates us to investigate the application of the 51 feature sets to phase-only images in addition to their use on the original data set. This investigation is extended to exploit Convolutional Neural Network (CNN) features. The results show that our feature fusion scheme improves the average performance of those feature sets for estimating humans' perceptual texture similarity. The superior performance should be attributed to the importance of phase to texture similarity.

Perceptual Texture Similarity Estimation: An Evaluation of Computational Features.

Estimation of texture similarity is fundamental to many material recognition tasks. This study uses fine-grained human perceptual similarity ground-truth to provide a comprehensive evaluation of 51 texture feature sets. We conduct two types of evaluation and both show that these features do not estimate similarity well when compared against human agreement rates, but that performances are improved when the features are combined using a Random Forest. Using a simple two-stage statistical model we show that few of the features capture long-range aperiodic relationships. We perform two psychophysical experiments which indicate that long-range interactions do provide humans with important cues for estimating texture similarity. This motivates an extension of the study to include Convolutional Neural Networks (CNNs) as they enable arbitrary features of large spatial extent to be learnt. Our conclusions derived from the use of two pre-trained CNNs are: that the large spatial extent exploited by the networks' top convolutional and first fully-connected layers, together with the use of large numbers of filters, confers significant advantage for estimation of perceptual texture similarity.

Perceptually Motivated Image Features Using Contours.

Dong et al. examined the ability of 51 computational feature sets to estimate human perceptual texture similarity; however, none performed well for this task. While it is well-known that the human visual system is extremely adept at exploiting longer-range aperiodic (and periodic) "contour" characteristics in images, none of the investigated feature sets exploit higher order statistics (HOS) over larger image regions ( > 19×19 pixels). We, therefore, hypothesise that long-range HOS, in the form of contour data, are useful for perceptual texture similarity estimation. We present the results of a psychophysical experiment that shows that contour data are more important, than local image patches, or global second-order data, to human observers for this task. Inspired by this finding, we propose a set of perceptually motivated image features (PMIF) that encode the long-range HOS computed from spatial and angular distributions of contour segments. We use two perceptual texture similarity estimation tasks to compare PMIF against the 51 feature sets referred to above and four commonly used contour representations. This new feature set is also examined in the context of two additional tasks: sketch-based image retrieval and natural scene recognition. The results show that the proposed feature set performs better, or at least comparably to, all the other feature sets. We attribute this promising performance to the fact that the proposed feature set exploits both short-range and long-range HOS.

Human search for a target on a textured background is consistent with a stochastic model.

Previous work has demonstrated that search for a target in noise is consistent with the predictions of the optimal search strategy, both in the spatial distribution of fixation locations and in the number of fixations observers require to find the target. In this study we describe a challenging visual-search task and compare the number of fixations required by human observers to find the target to predictions made by a stochastic search model. This model relies on a target-visibility map based on human performance in a separate detection task. If the model does not detect the target, then it selects the next saccade by randomly sampling from the distribution of saccades that human observers made. We find that a memoryless stochastic model matches human performance in this task. Furthermore, we find that the similarity in the distribution of fixation locations between human observers and the ideal observer does not replicate: Rather than making the signature doughnut-shaped distribution predicted by the ideal search strategy, the fixations made by observers are best described by a central bias. We conclude that, when searching for a target in noise, humans use an essentially random strategy, which achieves near optimal behavior due to biases in the distributions of saccades we have a tendency to make. The findings reconcile the existence of highly efficient human search performance with recent studies demonstrating clear failures of optimality in single and multiple saccade tasks.

The joint effect of mesoscale and microscale roughness on perceived gloss.

Computer simulated stimuli can provide a flexible method for creating artificial scenes in the study of visual perception of material surface properties. Previous work based on this approach reported that the properties of surface roughness and glossiness are mutually interdependent and therefore, perception of one affects the perception of the other. In this case roughness was limited to a surface property termed bumpiness. This paper reports a study into how perceived gloss varies with two model parameters related to surface roughness in computer simulations: the mesoscale roughness parameter in a surface geometry model and the microscale roughness parameter in a surface reflectance model. We used a real-world environment map to provide complex illumination and a physically-based path tracer for rendering the stimuli. Eight observers took part in a 2AFC experiment, and the results were tested against conjoint measurement models. We found that although both of the above roughness parameters significantly affect perceived gloss, the additive model does not adequately describe their mutually interactive and nonlinear influence, which is at variance with previous findings. We investigated five image properties used to quantify specular highlights, and found that perceived gloss is well predicted using a linear model. Our findings provide computational support to the 'statistical appearance models' proposed recently for material perception.

Why do rough surfaces appear glossy?

The majority of work on the perception of gloss has been performed using smooth surfaces (e.g., spheres). Previous studies that have employed more complex surfaces reported that increasing mesoscale roughness increases perceived gloss [Psychol. Sci.19, 196 (2008), J. Vis.10(9), 13 (2010), Curr. Biol.22, 1909 (2012)]. We show that the use of realistic rendering conditions is important and that, in contrast to [Psychol. Sci.19, 196 (2008), J. Vis.10(9), 13 (2010)], after a certain point increasing roughness further actually reduces glossiness. We investigate five image statistics of estimated highlights and show that for our stimuli, one in particular, which we term "percentage of highlight area," is highly correlated with perceived gloss. We investigate a simple model that explains the unimodal, nonmonotonic relationship between mesoscale roughness and percentage highlight area.

The effects of display time and eccentricity on the detection of amplitude and phase degradations in textured stimuli.

The amplitude and phase spectra of an image contain important information for perception, and a large body of work has investigated the effects of manipulating these spectra on the recognition or classification of image content. Here, we use a novel means of investigating sensitivity to amplitude and phase spectra properties, testing the ability of observers to detect degradations of the spectral content of synthetic images of textured surfaces that are broadband in the frequency domain. The effects of display time and retinal eccentricity on sensitivity to these two manipulations are compared using stimuli matched for difficulty of detection. We find no difference between the time courses for the detection of degradation in the two spectra; in both cases, accuracy rises above chance when display times are greater than 80 ms. Increasing retinal eccentricity to 8.7°, however, has a significantly stronger effect on the accuracy of detecting degradations of the amplitude spectrum than of the phase spectrum. Further, sensitivity to phase randomization that is restricted to low spatial frequencies is greater in the periphery (at 8.7° eccentricity) than in the fovea. These last two results imply that the fovea and periphery are specialized for the processing of phase spectrum information in distinct spatial frequency bands.

Modeling visual search on a rough surface.

The LNL (linear, non-linear, linear) model has previously been successfully applied to the problem of texture segmentation. In this study we investigate the extent to which a simple LNL model can simulate human performance in a search task involving a target on a textured surface. Two different classes of surface are considered: 1/f(beta)-noise and near-regular textures. We find that in both cases the search performance of the model does not differ significantly from that of people, over a wide range of task difficulties.

Perceived roughness of 1/f(beta) noise surfaces.

We report results from a new methodology for investigating the visually perceived properties of surface textures. Densely sampled two-dimensional 1/f(beta) noise processes are used to model natural looking surfaces, which are rendered using combined point-source and ambient lighting. Surfaces are shown in motion to provide rich cues to their relief. They are generated in real time to enable observers to dynamically manipulate surface parameters. A method of adjustment is employed to investigate the effects that the two surface parameters, magnitude roll-off factor and RMS height, have on perceived roughness. The results are used to develop an estimation method for perceived roughness.