Dynamic decorrelation as a unifying principle for explaining a broad range of brightness phenomena.

The visual system is highly sensitive to spatial context for encoding luminance patterns. Context sensitivity inspired the proposal of many neural mechanisms for explaining the perception of luminance (brightness). Here we propose a novel computational model for estimating the brightness of many visual illusions. We hypothesize that many aspects of brightness can be explained by a dynamic filtering process that reduces the redundancy in edge representations on the one hand, while non-redundant activity is enhanced on the other. The dynamic filter is learned for each input image and implements context sensitivity. Dynamic filtering is applied to the responses of (model) complex cells in order to build a gain control map. The gain control map then acts on simple cell responses before they are used to create a brightness map via activity propagation. Our approach is successful in predicting many challenging visual illusions, including contrast effects, assimilation, and reverse contrast with the same set of model parameters.

A calcium-influx-dependent plasticity model exhibiting multiple STDP curves.

Hebbian plasticity means that if the firing of two neurons is correlated, then their connection is strengthened. Conversely, uncorrelated firing causes a decrease in synaptic strength. Spike-timing-dependent plasticity (STDP) represents one instantiation of Hebbian plasticity. Under STDP, synaptic changes depend on the relative timing of the pre- and post-synaptic firing. By inducing pre- and post-synaptic firing at different relative times the STDP curves of many neurons have been determined, and it has been found that there are different curves for different neuron types or synaptic sites. Biophysically, strengthening (long-term potentiation, LTP) or weakening (long-term depression, LTD) of glutamatergic synapses depends on the post-synaptic influx of calcium (Ca2+): weak influx leads to LTD, while strong, transient influx causes LTP. The voltage-dependent NMDA receptors are the main source of Ca2+ influx, but they will only open if a post-synaptic depolarisation coincides with pre-synaptic neurotransmitter release. Here we present a computational mechanism for Ca2+-dependent plasticity in which the interplay between the pre-synaptic neurotransmitter release and the post-synaptic membrane potential leads to distinct Ca2+ time-courses, which in turn lead to the change in synaptic strength. It is shown that the model complies with classic STDP results, as well as with results obtained with triplets of spikes. Furthermore, the model is capable of displaying different shapes of STDP curves, as observed in different experimental studies.

Luminance gradients and non-gradients as a cue for distinguishing reflectance and illumination in achromatic images: A computational approach.

The brain analyses the visual world through the luminance patterns that reach the retina. Formally, luminance (as measured by the retina) is the product of illumination and reflectance. Whereas illumination is highly variable, reflectance is a physical property that characterizes each object surface. Due to memory constraints, it seems plausible that the visual system suppresses illumination patterns before object recognition takes place. Since many combinations of reflectance and illumination can give rise to identical luminance values, finding the correct reflectance value of a surface is an ill-posed problem, and it is still an open question how it is solved by the brain. Here we propose a computational approach that first learns filter kernels ("receptive fields") for slow and fast variations in luminance, respectively, from achromatic real-world images. Distinguishing between luminance gradients (slow variations) and non-gradients (fast variations) could serve to constrain the mentioned ill-posed problem. The second stage of our approach successfully segregates luminance gradients and non-gradients from real-world images. Our approach furthermore predicts that visual illusions that contain luminance gradients (such as Adelson's checker-shadow display or grating induction) may occur as a consequence of this segregation process.

Dendritic Pooling of Noisy Threshold Processes Can Explain Many Properties of a Collision-Sensitive Visual Neuron.

Power laws describe brain functions at many levels (from biophysics to psychophysics). It is therefore possible that they are generated by similar underlying mechanisms. Previously, the response properties of a collision-sensitive neuron were reproduced by a model which used a power law for scaling its inhibitory input. A common characteristic of such neurons is that they integrate information across a large part of the visual field. Here we present a biophysically plausible model of collision-sensitive neurons with η-like response properties, in which we assume that each information channel is noisy and has a response threshold. Then, an approximative power law is obtained as a result of pooling these channels. We show that with this mechanism one can successfully predict many response characteristics of the Lobula Giant Movement Detector Neuron (LGMD). Moreover, the results depend critically on noise in the inhibitory pathway, but they are fairly robust against noise in the excitatory pathway.

The time course of estimating time-to-contact: switching between sources of information.

The different sources of information that can be used to estimate time-to-contact may have different degrees of reliability across time. For example, after a given presentation or display time, an absolute change of angular size can be more reliable than the corresponding estimation of the rate of angular expansion (e.g. motion information). One could then expect systematic biases in the observer's responses for different times of stimulus exposure. In one experiment, observers judged whether approaching objects arrived at the point of observation before or after a reference beep (1.2s) under monocular, and binocular plus monocular vision. Five display times from 0.1 to 0.9s were used. Unlike monocular viewing, where accuracy increased monotonically with display time, an interesting non-linearity occurred for objects with small size when binocular information was available. Accuracy reached maximum values for small objects with only 0.3s of vision with stereopsis. This accuracy, however, dropped significantly after 0.4s of exposure and increased again linearly with time. This is consistent with subjects switching from using binocular information to using monocular motion information when it started to become more reliable. We also explored whether monocular cues were combined differently across time by fitting a model that relates visual angle to its rate of expansion. Results show that subjects relied more on angular motion information (i.e. rate of expansion) with presentation time but interrupting this motion integration process led to a loss of accuracy in time-to-contact judgments.

Unifying time to contact estimation and collision avoidance across species.

The τ-function and the η-function are phenomenological models that are widely used in the context of timing interceptive actions and collision avoidance, respectively. Both models were previously considered to be unrelated to each other: τ is a decreasing function that provides an estimation of time-to-contact (ttc) in the early phase of an object approach; in contrast, g has a maximum before ttc. Furthermore, it is not clear how both functions could be implemented at the neuronal level in a biophysically plausible fashion. Here we propose a new framework--the corrected modified Tau function--capable of predicting both τ-type ("τ(cm)") and g-type ("t(mod)") responses. The outstanding property of our new framework is its resilience to noise. We show that t(mod) can be derived from a firing rate equation, and, as g, serves to describe the response curves of collision sensitive neurons. Furthermore, we show that tcm predicts the psychophysical performance of subjects determining ttc. Our new framework is thus validated successfully against published and novel experimental data. Within the framework, links between τ-type and η-type neurons are established. Therefore, it could possibly serve as a model for explaining the co-occurrence of such neurons in the brain.

People favour imperfect catching by assuming a stable world.

The visual angle that is projected by an object (e.g. a ball) on the retina depends on the object's size and distance. Without further information, however, the visual angle is ambiguous with respect to size and distance, because equal visual angles can be obtained from a big ball at a longer distance and a smaller one at a correspondingly shorter distance. Failure to recover the true 3D structure of the object (e.g. a ball's physical size) causing the ambiguous retinal image can lead to a timing error when catching the ball. Two opposing views are currently prevailing on how people resolve this ambiguity when estimating time to contact. One explanation challenges any inference about what causes the retinal image (i.e. the necessity to recover this 3D structure), and instead favors a direct analysis of optic flow. In contrast, the second view suggests that action timing could be rather based on obtaining an estimate of the 3D structure of the scene. With the latter, systematic errors will be predicted if our inference of the 3D structure fails to reveal the underlying cause of the retinal image. Here we show that hand closure in catching virtual balls is triggered by visual angle, using an assumption of a constant ball size. As a consequence of this assumption, hand closure starts when the ball is at similar distance across trials. From that distance on, the remaining arrival time, therefore, depends on ball's speed. In order to time the catch successfully, closing time was coupled with ball's speed during the motor phase. This strategy led to an increased precision in catching but at the cost of committing systematic errors.

"I look in your eyes, honey": internal face features induce spatial frequency preference for human face processing.

Numerous psychophysical experiments found that humans preferably rely on a narrow band of spatial frequencies for recognition of face identity. A recently conducted theoretical study by the author suggests that this frequency preference reflects an adaptation of the brain's face processing machinery to this specific stimulus class (i.e., faces). The purpose of the present study is to examine this property in greater detail and to specifically elucidate the implication of internal face features (i.e., eyes, mouth, and nose). To this end, I parameterized Gabor filters to match the spatial receptive field of contrast sensitive neurons in the primary visual cortex (simple and complex cells). Filter responses to a large number of face images were computed, aligned for internal face features, and response-equalized ("whitened"). The results demonstrate that the frequency preference is caused by internal face features. Thus, the psychophysically observed human frequency bias for face processing seems to be specifically caused by the intrinsic spatial frequency content of internal face features.

Preferred spatial frequencies for human face processing are associated with optimal class discrimination in the machine.

Psychophysical studies suggest that humans preferentially use a narrow band of low spatial frequencies for face recognition. Here we asked whether artificial face recognition systems have an improved recognition performance at the same spatial frequencies as humans. To this end, we estimated recognition performance over a large database of face images by computing three discriminability measures: Fisher Linear Discriminant Analysis, Non-Parametric Discriminant Analysis, and Mutual Information. In order to address frequency dependence, discriminabilities were measured as a function of (filtered) image size. All three measures revealed a maximum at the same image sizes, where the spatial frequency content corresponds to the psychophysical found frequencies. Our results therefore support the notion that the critical band of spatial frequencies for face recognition in humans and machines follows from inherent properties of face images, and that the use of these frequencies is associated with optimal face recognition performance.

Does face image statistics predict a preferred spatial frequency for human face processing?

Psychophysical experiments suggested a relative importance of a narrow band of spatial frequencies for recognition of face identity in humans. There exists, however, no conclusive evidence of why it is that such frequencies are preferred. To address this question, I examined the amplitude spectra of a large number of face images and observed that face spectra generally fall off more steeply with spatial frequency compared with ordinary natural images. When external face features (such as hair) are suppressed, then whitening of the corresponding mean amplitude spectra revealed higher response amplitudes at those spatial frequencies which are deemed important for processing face identity. The results presented here therefore provide support for that face processing characteristics match corresponding stimulus properties.

Gradient representations and the perception of luminosity.

The neuronal mechanisms that serve to distinguish between light emitting and light reflecting objects are largely unknown. It has been suggested that luminosity perception implements a separate pathway in the visual system, such that luminosity constitutes an independent perceptual feature. Recently, a psychophysical study was conducted to address the question whether luminosity has a feature status or not. However, the results of this study lend support to the hypothesis that luminance gradients are instead a perceptual feature. Here, I show how the perception of luminosity can emerge from a previously proposed neuronal architecture for generating representations of luminance gradients.

Gradient representation and perception in the early visual system--a novel account of Mach band formation.

Recent evidence suggests that object surfaces and their properties are represented at early stages in the visual system of primates. Most likely invariant surface properties are extracted to endow primates with robust object recognition capabilities. In real-world scenes, luminance gradients are often superimposed on surfaces. We argue that gradients should also be represented in the visual system, since they encode highly variable information, such as shading, focal blur, and penumbral blur. We present a neuronal architecture which was designed and optimized for segregating and representing luminance gradients in real-world images. Our architecture in addition provides a novel theory for Mach bands, whereby corresponding psychophysical data are predicted consistently.

Smooth gradient representations as a unifying account of Chevreul's illusion, Mach bands, and a variant of the Ehrenstein disk.

Recent evidence suggests that the primate visual system generates representations for object surfaces (where we consider representations for the surface attribute brightness). Object recognition can be expected to perform robustly if those representations are invariant despite environmental changes (e.g., in illumination). In real-world scenes, it happens, however, that surfaces are often overlaid by luminance gradients, which we define as smooth variations in intensity. Luminance gradients encode highly variable information, which may represent surface properties (curvature), nonsurface properties (e.g., specular highlights, cast shadows, illumination inhomogeneities), or information about depth relationships (cast shadows, blur). We argue, on grounds of the unpredictable nature of luminance gradients, that the visual system should establish corresponding representations, in addition to surface representations. We accordingly present a neuronal architecture, the so-called gradient system, which clarifies how spatially accurate gradient representations can be obtained by relying on only high-resolution retinal responses. Although the gradient system was designed and optimized for segregating, and generating, representations of luminance gradients with real-world luminance images, it is capable of quantitatively predicting psychophysical data on both Mach bands and Chevreul's illusion. It furthermore accounts qualitatively for a modified Ehrenstein disk.

Recovering real-world images from single-scale boundaries with a novel filling-in architecture.

Filling-in models were successful in predicting psychophysical data for brightness perception. Nevertheless, their suitability for real-world image processing has never been examined. A unified architecture for both predicting psychophysical data and real-world image processing would constitute a powerful theory for early visual information processing. As a first contribution of the present paper, we identified three principal problems with current filling-in architectures, which hamper the goal of having such a unified architecture. To overcome these problems we propose an advance to filling-in theory, called BEATS filling-in, which is based on a novel nonlinear diffusion operator. BEATS filling-in furthermore introduces novel boundary structures. We compare, by means of simulation studies with real-world images, the performance of BEATS filling-in with the recently proposed confidence-based filling-in. As a second contribution we propose a novel mechanism for encoding luminance information in contrast responses ('multiplex contrasts'), which is based on recent neurophysiological findings. Again, by simulations, we show that 'multiplex contrasts' at a single, high-resolution filter scale are sufficient for recovering absolute luminance levels. Hence, 'multiplex contrasts' represent a novel theory addressing how the brain encodes and decodes luminance information.