Scale-invariant superiority of foveal vision in perceptual categorization.

The recognition of objects is exceedingly difficult in indirect view. This complication cannot be explained in terms of retino-cortical magnification, as size scaling fails to establish position invariance both for character recognition [Strasburger, H. & Rentschler, I. (1996) Eur. J. Neurosci., 8 1787-1791] and pattern classification [Jüttner, M. & Rentschler, I. (1996) Vision Res., 36, 1007-1021]. Thus we compared, for two tasks of discrimination learning and category learning with respect to a common set of grey-level patterns, how humans perform in foveal and extrafoveal vision. Observers learnt to discriminate (size-scaled) images equally well in foveal and extrafoveal view, whereas they displayed profound deficiencies in extrafoveal category learning for the same patterns. From the behavioural learning data, internal representations of the learning signals were reconstructed by means of computer simulations. For foveal view, these representations were found to be veridical to their physical counterparts for both learning tasks. For extrafoveal view, they were severely distorted for category learning but not for discrimination learning. A variance reduction of the pattern classes by a factor of 100 reduced the dissociation between extrafoveal categorization and discrimination but did not remove it. These observations suggest a scale-invariant superiority of foveal vision for learning object categories. This implies a high degree of space variance of visual cognition which is vastly underestimated by classical measures of visual performance, e.g. acuity, visual field and contrast sensitivity.

Object and scene analysis by saccadic eye-movements: an investigation with higher-order statistics.

Based on an information theoretical approach, we investigate feature selection processes in saccadic object and scene analysis. Saccadic eye movements of human observers are recorded for a variety of natural and artificial test images. These experimental data are used for a statistical evaluation of the fixated image regions. Analysis of second-order statistics indicates that regions with higher spatial variance have a higher probability to be fixated, but no significant differences beyond these variance effects could be found at the level of power spectra. By contrast, an investigation with higher-order statistics, as reflected in the bispectral density, yielded clear structural differences between the image regions selected by saccadic eye movements as opposed to regions selected by a random process. These results indicate that nonredundant, intrinsically two-dimensional image features like curved lines and edges, occlusions, isolated spots, etc. play an important role in the saccadic selection process which must be integrated with top-down knowledge to fully predict object and scene analysis by human observers.

Dynamics and context dependence of visual category learning.

Visual category learning by humans is observed within a paradigm of supervised learning. Mental representations for recognition are reconstructed from the observed data structures by fitting to them predicted classification data obtained from similarity-based models of recognition on the one hand and machine vision systems for image understanding on the other hand. These principles are illustrated with examples concerning the dynamics and the dependence on context of processes of category learning.

Reconstructing mental object representations: a machine vision approach to human visual recognition.

This paper introduces a new approach to assess visual representations underlying the recognition of objects. Human performance is modeled by CLARET, a machine learning and matching system, based on inductive logic programming and graph matching principles. The model is applied to data of a learning experiment addressing the role of prior experience in the ontogenesis of mental object representations. Prior experience was varied in terms of sensory modality, i.e. visual versus haptic versus visuohaptic. The analysis revealed distinct differences between the representational formats used by subjects with haptic versus those with no prior object experience. These differences suggest that prior haptic exploration stimulates the evolution of object representations which are characterized by an increased differentiation between attribute values and a pronounced structural encoding.

Innate and learned components of human visual preference.

BACKGROUND: Recent claims in neuroscience and evolutionary biology suggest that the aesthetic sense reflects preferences for image signals whose characteristics best fit innate brain mechanisms of visual recognition. RESULTS: This hypothesis was tested by behaviourally measuring, for a set of initially unfamiliar images, the effects of category learning on preference judgements by humans, and by relating the observed data to computationally reconstructed internal representations of categorical concepts. Category learning induced complex shifts in preference behaviour. Two distinct factors - complexity and bilateral symmetry - could be identified from the data as determinants of preference judgements. The effect of the complexity factor varied with object knowledge acquired through category learning. In contrast, the impact of the symmetry factor proved to be unaffected by learning experience. Computer simulations suggested that the preference for pattern complexity relies on active (top-down) mechanisms of visual recognition, whereas the preference for pattern symmetry depends on automatic (bottom-up) mechanisms. CONCLUSIONS: Human visual preferences are not fully determined by (objective) structural regularities of image stimuli but also depend on their learned (subjective) interpretation. These two aspects are reflected in distinct complementary factors underlying preference judgements, and may be related to complementary modes of visual processing in the brain.

Similarity-based models of human visual recognition.

Seven models of human visual recognition from cognitive psychology, visual psychophysics and connectionism were compared. They were used to predict psychophysical classification data obtained via supervised learning with parametrised grey-level patterns (compound Gabor signals). Four sets of learning patterns, as well as foveal and extrafoveal viewing conditions, were applied. Model performance was determined by comparing observed and predicted data with respect to root mean square deviation and to signal reconstruction via multidimensional scaling. Results show that a psychophysical theory of classification requires a similarity concept that is based both on physical signal description and on cognitive bias. The latter is less pronounced in foveal recognition, where all seven models performed almost equally well, but matters in extrafoveal recognition. Virtual prototype models (Rentschler et al. (1994), Vision Research 34, 669-687), which best accommodate stimulus- and observer-dependencies, are then of advantage. Concerning computational efficiency, a hyperBF model (Poggio and Girosi (1990), Science 247, 978) was much faster, and generalized signal detection models were much slower than the average.

Intrinsic two-dimensional features as textons.

We suggest that intrinsic two-dimensional (i2D) features, computationally defined as the outputs of nonlinear operators that model the activity of end-stopped neurons, play a role in preattentive texture discrimination. We first show that for discriminable textures with identical power spectra the predictions of traditional models depend on the type of nonlinearity and fail for energy measures. We then argue that the concept of intrinsic dimensionality, and the existence of end-stopped neurons, can help us to understand the role of the nonlinearities. Furthermore, we show examples in which models without strong i2D selectivity fail to predict the correct ranking order of perceptual segregation. Our arguments regarding the importance of i2D features resemble the arguments of Julesz and co-workers regarding textons such as terminators and crossings. However, we provide a computational framework that identifies textons with the outputs of nonlinear operators that are selective to i2D features.

Evidence-Based Pattern Classification: A Structural Approach to Human Perceptual Learning and Generalization

Models of human pattern classification have been traditionally based on implicit pattern descriptions which involve lists of continuous attribute values or discrete features. Here we propose an alternative approach which makes explicit use of pattern structure in terms of components and their unary (part-specific) and binary (part-relational) properties. Such attributes "evidence" different classes of patterns and allow one to model processes of both perceptual learning and generalization to novel instances. An object in an evidence-based system is represented by a set of rules, where each rule provides a certain amount of class-specific evidence. The accumulated class evidence over all activated rules determines the classification probability. We have examined how well this concept reflects human performance by training observers to classify compound Gabor patterns and then testing them with segmented (grey-level-transformed) versions of the patterns in the original training set. If the observers were to construct rules to define each pattern class in terms of perceived parts and their relations, then it should be expected that classification performance would generalize to these new patterns. Results confirm this hypothesis and the specific feature extraction, learning, and rule generation model used to predict performance. Copyright 1997 Academic Press

Contrast-dependent dissociation of visual recognition and detection fields.

Seeing an object 'as something' is different from simply seeing it (see Watanabe, S., 1985, Pattern Recognition: Human and Mechanical, John Wiley). This distinction between recognition and detection often goes unnoticed in physiology and clinical practice, where visual performance is characterized in terms of acuity, visual field and contrast sensitivity. The corresponding functions of stimulus detection are consistent with the neural projection properties from the retina to the striate cortex, i.e. the 'cortical magnification theory'. Yet recognition performance for characters (Strasburger, H. et al., 1994, Eur. J. Neurosci., 6, 1583-1588) and grey-level patterns (Jüttner, M. and Reutschler, I., 1996, Vision Res., 36, 1007-1022) does not fit into this scheme. Here we show that this discrepancy results in the dissociation of visual recognition and detection fields, which is dramatic at low pattern contrast. Form proper can be appreciated exclusively within the much narrower field of recognition, the window of visual intelligence. Its function is, at low contrast, probably mediated by the magnocellular pathway and at all contrasts is determined by the processing characteristics of higher stages of the ventral visual pathway.

Amblyopic quasi-blindness for image structure.

Human amblyopes display reduced contrast sensitivities, suffer from perceptual distortion, and their letter acuities are worse than is predicted from grating visibility. We sought the origin of these dysfunctions by measuring normal and amblyopic sensitivities to various forms of well-defined image distortion, namely band-limited phase quantization, phase quantization with additional amplitude modulation, and grey-scale modification. Our results prove the existence of an amblyopic quasi-blindness to image structure, that cannot be explained in terms of contrast detection. We discuss these findings within the computational scheme of image decomposition into local amplitude and local phase values. they are consistent with the assumption of amblyopic eyes beings impaired in processing local phase but having the local amplitude (or "energy", possibly at reduced gain) at their disposal. Phrased in physiological terms, we propose a scheme of complex-cells-only vision in amblyopia. We also provide a demonstration of how amblyopic eyes may see the test stimuli and natural images by generating local amplitude and phase representations at limited phase resolution.

Recognition-by-parts: a computational approach to human learning and generalization of shapes.

In this paper human pattern recognition is modeled in terms of how human observers learn to describe patterns in terms of their perceived parts, their unary (part) and binary (relational) attributes and the way in which such attribute states "evidence' different classes of shapes. This approach, originally developed in the area of computer vision, is concerned with algorithms which enable the learning of shape descriptions from examples and the classification of new data (generalization) efficiently and accurately. An object in such an "evidence-based' system is represented by a set of rules, where each rule provides a certain amount of evidence for each object class in the database. The accumulated class evidence over all activated rules can then be used to determine the classification probability. We have examined how well this model reflects human perception by training observers to classify compound Gabor patterns and then testing them with versions of such patterns which were segmented (gray-level transformed) versions of the original training set. If the observers were to construct rules to define each pattern class in terms of perceived parts and their relations, then it should be expected that classification performance would generalize to these new patterns from the original set. Results confirm this hypothesis and the specific feature extraction, learning and rule generation model used to predict performance.

Reduced perceptual dimensionality in extrafoveal vision.

The classification behaviour of human observers with respect to compound Gabor signals is tested at foveal and extrafoveal retinal positions. Classification performance is analysed in terms of a probabilistic classification model recently proposed by Rentschler, Jüttner and Caelli [(1994) Vision Research, 34, 669-687]. The analysis allows inferences about structure and dimensionality of the individual internal representations underlying the classification task and their temporal evolution during the learning process. Using this technique it is found that the internal representations of direct and eccentric viewing are intrinsically incommensurable, in the sense that extrafoveal pattern representations are characterized by a lower perceptual dimension in feature space relative to the corresponding physical input signals, whereas foveal representations are not. The observed deficits cannot be renormalized by size scaling (cortical magnification); however, they can be partially reduced by learning although the learning progress strongly depends on the observer's practice. The structural incommensurability between foveal and extrafoveal representations poses constraints on possible forms of foveal-extrafoveal interaction, which might have implications on related perceptual phenomena such as visual stability across saccadic eye movements.

Generalization of form in visual pattern classification.

Human observers were trained to criterion in classifying compound Gabor signals with symmetry relationships, and were then tested with each of 18 blob-only versions of the learning set. Generalization to dark-only and light-only blob versions of the learning signals, as well as to dark-and-light blob versions was found to be excellent, thus implying virtually perfect generalization of the ability to classify mirror-image signals. The hypothesis that the learning signals are internally represented in terms of a 'blob code' with explicit labelling of contrast polarities was tested by predicting observed generalization behaviour in terms of various types of signal representations (pixelwise, Laplacian pyramid, curvature pyramid, ON/OFF, local maxima of Laplacian and curvature operators) and a minimum-distance rule. Most representations could explain generalization for dark-only and light-only blob patterns but not for the high-thresholded versions thereof. This led to the proposal of a structure-oriented blob-code. Whether such a code could be used in conjunction with simple classifiers or should be transformed into a propositional scheme of representation operated upon by a rule-based classification process remains an open question.

Objective measurement of contrast sensitivity and visual acuity with the steady-state visual evoked potential.

Since the appearance of Campbell and Maffei's and Harter and White's reports it has been well established that the visual evoked potential (VEP) can be used to predict psychophysical contrast sensitivity and visual acuity and is thus suited as an objective technique to assess these fundamental aspects of vision. Nevertheless, the technique has not become a standard diagnostic tool, being too time-consuming to apply and suffering from variable reliability under pathological visual conditions. In addition, there are problems of reliability in normal subjects. By using an unconventional stimulus--temporally sinusoidal 16-Hz on-off modulation of sinewave gratings--we demonstrated that these problems can be alleviated in normal subjects. This stimulus avoids the low signals in the visible range that frequently occur with conventional pattern-reversal stimuli, it leads to high correspondence between normal observers, and it is much faster to apply than are transient VEPs. Initial applications of this stimulus to amblyopes yielded promising results. The steady-state VEP could consequently turn into a viable diagnostic procedure in disturbances of visual contrast perception.

Cortical magnification theory fails to predict visual recognition.

The sense of form is poor in indirect view. Yet the cortical magnification theory asserts that the disadvantage can be made up by scaling the image size according to the spatial variation in the mapping of the retina onto the cortex. It is thus assumed that all visual information passes through a functionally homogeneous neural circuitry, with the spatial sampling of input signals varying across the visual field. We challenge this notion by showing that character recognition in the visual field cannot be accommodated by any concept of sole size scaling but requires increasing both size and contrast of the target being viewed. This finding is formalized into a hyperbolic law which states that target size multiplied by log contrast is constant across the visual field. We conclude that the scalar cortical magnification theory fails for character recognition since the latter depends on multidimensional pattern representations in higher, i.e. striate and prestriate, cortical areas.

Dissociation of local and global processing in visual agnosia.

Subsequent to strokes in the right and left inferomedial occipito-temporal lobes, two patients became prosopagnosic and alexic, respectively. They also show a complementary dissociation of the analysis of handwritten text. The patient with the right posterior stroke can read it but not recognize whose handwriting it is; the patient with the left posterior stroke cannot read the text but knows who wrote it. The analysis of spatial vision revealed that the prosopagnosic patient has no problem with seeing texture elements when presented in isolation. Yet she performs poorly with Moiré and texture perception, i.e. she suffers from a selective loss of global visual perception. The alexic patient performs well with Moiré patterns but neither with (complex) texture elements nor with textures. She seemingly can locally and globally process patterns composed of simple figural elements but fails with stimuli that require the integration of features. This finding of a concomitant dissociation of local and global visual processes in the two patients supports the view that prosopagnosia as well as alexia are the most conspicuous aspects of more general alterations of visual perception.

Probabilistic analysis of human supervised learning and classification.

Probabilistic classification techniques based on Bayesian decision theory are used to analyze human supervised learning and classification. The procedure rests on the assumption that human classification behaviour is based on internal feature states which can be linked to physical feature vectors (corresponding to the system input). In the present approach, this relationship is modeled in terms of additive stochastic error signals. The corresponding random variables describe the additional degrees of bias and variance introduced by the (perceptual) process of internal feature measurement. Estimates of internal feature states are obtained by least-squares minimization. Structure and dimensionality of the resulting internal representation are displayed by plotting the configuration of internal class means, or virtual prototypes. Their temporal evolution reflects the dynamic properties of the learning process. The use of the procedure is demonstrated by analyzing the results of two experiments. First, it is shown that Minimum Distance Classifiers, such as used by Caelli, Rentschler and Scheidler [(1987) Biological Cybernetics, 57, 233-240], are suboptimal in predicting human performance. Second, it is found that extrafoveal learning is much slower than foveal learning and that extrafoveal pattern representations are severely distorted. The latter distortions reveal the existence of limitations for the generalization of supervised learning over space.

Contrast thresholds for identification of numeric characters in direct and eccentric view.

Aubert and Foerster (1857) are frequently cited for having shown that the lower visual acuity of peripheral vision can be compensated for by increasing stimulus size. This result is seemingly consistent with the concept of cortical magnification, and it has been confirmed by many subsequent authors. Yet it is rarely noted that Aubert and Foerster also observed a loss of the "quality of form." We have studied the recognition of numeric characters in foveal and eccentric vision by determining the contrast required for 67% correct identification. At each eccentricity, the lowest contrast threshold is achieved with a specific stimulus size. But the contrast thresholds for these optimal stimuli are not independent of retinal eccentricity as cortical magnification scaling would predict. With high-contrast targets, however, threshold target sizes were consistent with cortical magnification out to 6 degrees eccentricity. Beyond 6 degrees, threshold target sizes were larger than cortical magnification predicted. We also investigated recognition performance in the presence of neighboring characters (crowding phenomenon). Target character size, distance of flanking characters, and precision of focusing of attention all affect recognition. The influence of these parameters is different in the fovea and in the periphery. Our findings confirm Aubert and Foester's original observation of a qualitative difference between foveal and peripheral vision.