Qualitative perception of 3D shape from patterns of luminance curvature.

A new source of information is proposed for the perception of three-dimensional (3D) shape from shading that identifies surface concavities from the curvature of the luminance field. Two experiments measured the abilities of human observers to identify concavities on smoothly curved shaded surfaces depicted with several different patterns of illumination and several different material properties. Observers were required to identify any apparent concavities along designated cross sections of the depicted objects and to mark each concavity with an adjustable dot. To analyze the results, we computed both the surface curvature and the luminance curvature along each image cross section. The results revealed that most responses were in concave regions of the luminance profiles, although they were often shifted in phase relative to the curvature of the depicted surfaces. This pattern of performance was surprisingly robust over large changes in the pattern of illumination or surface material properties. Our analysis predicts that observers should make false alarm responses in regions where a luminance concavity does not correspond to a surface concavity, and our empirical results confirm that prediction.

Resilience of temporal processing to early and extended visual deprivation.

Early visual deprivation is known to have profound consequences on the subsequent development of spatial visual processing. However, its impact on temporal processing is not well characterized. We have examined spatial and temporal contrast sensitivity functions following treatment for early and extended bilateral visual deprivation in fifteen children born with congenital cataracts in rural India. The results reveal a marked difference in post-treatment spatial and temporal sensitivities. Whereas spatial processing in newly sighted children is significantly impaired relative to age-matched controls, temporal processing exhibits remarkable resilience and is comparable to that in the control group. This difference in spatial and temporal outcomes is especially surprising given our computational analyses of video sequences which indicate a strong linkage between the spatial and temporal spectral content of natural visual inputs. We consider possible explanations for this discrepancy.

The Veiled Virgin illustrates visual segmentation of shape by cause.

Three-dimensional (3D) shape perception is one of the most important functions of vision. It is crucial for many tasks, from object recognition to tool use, and yet how the brain represents shape remains poorly understood. Most theories focus on purely geometrical computations (e.g., estimating depths, curvatures, symmetries). Here, however, we find that shape perception also involves sophisticated inferences that parse shapes into features with distinct causal origins. Inspired by marble sculptures such as Strazza's The Veiled Virgin (1850), which vividly depict figures swathed in cloth, we created composite shapes by wrapping unfamiliar forms in textile, so that the observable surface relief was the result of complex interactions between the underlying object and overlying fabric. Making sense of such structures requires segmenting the shape based on their causes, to distinguish whether lumps and ridges are due to the shrouded object or to the ripples and folds of the overlying cloth. Three-dimensional scans of the objects with and without the textile provided ground-truth measures of the true physical surface reliefs, against which observers' judgments could be compared. In a virtual painting task, participants indicated which surface ridges appeared to be caused by the hidden object and which were due to the drapery. In another experiment, participants indicated the perceived depth profile of both surface layers. Their responses reveal that they can robustly distinguish features belonging to the textile from those due to the underlying object. Together, these findings reveal the operation of visual shape-segmentation processes that parse shapes based on their causal origin.

Effects of illumination on the categorization of shiny materials.

The present research was designed to examine how patterns of illumination influence the perceptual categorization of metal, shiny black, and shiny white materials. The stimuli depicted three possible objects that were illuminated by five possible high-dynamic-range imaging light maps, which varied in their overall distributions of illuminant directions and intensities. The surfaces included a low roughness chrome material, a shiny black material, and a shiny white material with both diffuse and specular components. Observers rated each stimulus by adjusting four sliders to indicate their confidence that the depicted material was metal, shiny black, shiny white, or something else, and these adjustments were constrained so that the sum of all four settings was always 100%. The results revealed that the metal and shiny black categories are easily confused. For example, metal materials with low intensity light maps or a narrow range of illuminant directions are often judged as shiny black, whereas shiny black materials with high intensity light maps or a wide range of illuminant directions are often judged as metal. To discover the visual information on which these judgements are based, we measured several possible image statistics, and we found two that were highly correlated with the observers' confidence ratings in appropriate contexts. We also performed a spherical harmonic analysis on the different light maps to quantitatively predict how they would bias observers' judgments of metal and shiny black surfaces.

Visual perception of shape-transforming processes: 'Shape Scission'.

Shape-deforming processes (e.g., squashing, bending, twisting) can radically alter objects' shapes. After such a transformation, some features are due to the object's original form, while others are due to the transformation, yet it is challenging to separate the two. We tested whether observers can distinguish the causal origin of different features, teasing apart the characteristics of the original shape from those imposed by transformations, a process we call 'shape scission'. Using computer graphics, we created 8 unfamiliar objects and subjected each to 8 transformations (e.g., "twisted", "inflated", "melted"). One group of participants named transformations consistently. A second group arranged cards depicting the objects into classes according to either (i) the original shape or (ii) the type of transformation. They could do this almost perfectly, suggesting that they readily distinguish the causal origin of shape features. Another group used a digital painting interface to indicate which locations on the objects appeared transformed, with responses suggesting they can localise features caused by transformations. Finally, we parametrically varied the magnitude of the transformations, and asked another group to rate the degree of transformation. Ratings correlated strongly with transformation magnitude with a tendency to overestimate small magnitudes. Responses were predicted by both the magnitude and area affected by the transformation. Together, the findings suggest that observers can scission object shapes into original shape and transformation features and access the resulting representational layers at will.

Why Does the Cortex Reorganize after Sensory Loss?

A growing body of evidence demonstrates that the brain can reorganize dramatically following sensory loss. Although the existence of such neuroplastic crossmodal changes is not in doubt, the functional significance of these changes remains unclear. The dominant belief is that reorganization is compensatory. However, results thus far do not unequivocally indicate that sensory deprivation results in markedly enhanced abilities in other senses. Here, we consider alternative reasons besides sensory compensation that might drive the brain to reorganize after sensory loss. One such possibility is that the cortex reorganizes not to confer functional benefits, but to avoid undesirable physiological consequences of sensory deafferentation. Empirical assessment of the validity of this and other possibilities defines a rich program for future research.

Haptic shape discrimination and interhemispheric communication.

In three experiments participants haptically discriminated object shape using unimanual (single hand explored two objects) and bimanual exploration (both hands were used, but each hand, left or right, explored a separate object). Such haptic exploration (one versus two hands) requires somatosensory processing in either only one or both cerebral hemispheres; previous studies related to the perception of shape/curvature found superior performance for unimanual exploration, indicating that shape comparison is more effective when only one hemisphere is utilized. The current results, obtained for naturally shaped solid objects (bell peppers, Capsicum annuum) and simple cylindrical surfaces demonstrate otherwise: bimanual haptic exploration can be as effective as unimanual exploration, showing that there is no necessary reduction in ability when haptic shape comparison requires interhemispheric communication. We found that while successive bimanual exploration produced high shape discriminability, the participants' bimanual performance deteriorated for simultaneous shape comparisons. This outcome suggests that either interhemispheric interference or the need to attend to multiple objects simultaneously reduces shape discrimination ability. The current results also reveal a significant effect of age: older adults' shape discrimination abilities are moderately reduced relative to younger adults, regardless of how objects are manipulated (left hand only, right hand only, or bimanual exploration).

Perceiving Object Shape from Specular Highlight Deformation, Boundary Contour Deformation, and Active Haptic Manipulation.

It is well known that motion facilitates the visual perception of solid object shape, particularly when surface texture or other identifiable features (e.g., corners) are present. Conventional models of structure-from-motion require the presence of texture or identifiable object features in order to recover 3-D structure. Is the facilitation in 3-D shape perception similar in magnitude when surface texture is absent? On any given trial in the current experiments, participants were presented with a single randomly-selected solid object (bell pepper or randomly-shaped "glaven") for 12 seconds and were required to indicate which of 12 (for bell peppers) or 8 (for glavens) simultaneously visible objects possessed the same shape. The initial single object's shape was defined either by boundary contours alone (i.e., presented as a silhouette), specular highlights alone, specular highlights combined with boundary contours, or texture. In addition, there was a haptic condition: in this condition, the participants haptically explored with both hands (but could not see) the initial single object for 12 seconds; they then performed the same shape-matching task used in the visual conditions. For both the visual and haptic conditions, motion (rotation in depth or active object manipulation) was present in half of the trials and was not present for the remaining trials. The effect of motion was quantitatively similar for all of the visual and haptic conditions-e.g., the participants' performance in Experiment 1 was 93.5 percent higher in the motion or active haptic manipulation conditions (when compared to the static conditions). The current results demonstrate that deforming specular highlights or boundary contours facilitate 3-D shape perception as much as the motion of objects that possess texture. The current results also indicate that the improvement with motion that occurs for haptics is similar in magnitude to that which occurs for vision.

Magically deceptive biological motion-the French Drop Sleight.

Intentional deception, as is common in the performance of magic tricks, can provide valuable insight into the mechanisms of perception and action. Much of the recent investigations into this form of deception revolve around the attention of the observer. Here, we present experiments designed to investigate the contributions of the performer to the act of deception. An experienced magician and a naïve novice performed a classic sleight known as the French Drop. Video recordings of the performance were used to measure the quality of the deception-e.g., if a non-magician observer could discriminate instances where the sleight was performed (a deceptive performance) from those where it was not (a veridical performace). During the performance we recorded the trajectory of the hands and measured muscle activity via EMG to help understand the biomechanical mechanisms of this deception. We show that expertise plays a major role in the quality of the deception and that there are significant variations in the motion and muscular behaviors between successful and unsuccessful performances. Smooth, minimal movements with an exaggerated faux-transfer of muscular tension were characteristic of better deception. This finding is consistent with anecdotal reports and the magic performance literature.

Perception of tactile graphics: embossings versus cutouts.

Graphical information, such as illustrations, graphs, and diagrams, are an essential complement to text for conveying knowledge about the world. Although graphics can be communicated well via the visual modality, conveying this information via touch has proven to be challenging. The lack of easily comprehensible tactile graphics poses a problem for the blind. In this paper, we advance a hypothesis for the limited effectiveness of tactile graphics. The hypothesis contends that conventional graphics that rely upon embossings on two-dimensional surfaces do not allow the deployment of tactile exploratory procedures that are crucial for assessing global shape. Besides potentially accounting for some of the shortcomings of current approaches, this hypothesis also serves a prescriptive purpose by suggesting a different strategy for conveying graphical information via touch, one based on cutouts. We describe experiments demonstrating the greater effectiveness of this approach for conveying shape and identity information. These results hold the potential for creating more comprehensible tactile drawings for the visually impaired while also providing insights into shape estimation processes in the tactile modality.

Is the perception of 3D shape from shading based on assumed reflectance and illumination?

The research described in the present article was designed to compare three types of image shading: one generated with a Lambertian BRDF and homogeneous illumination such that image intensity was determined entirely by local surface orientation irrespective of position; one that was textured with a linear intensity gradient, such that image intensity was determined entirely by local surface position irrespective of orientation; and another that was generated with a Lambertian BRDF and inhomogeneous illumination such that image intensity was influenced by both position and orientation. A gauge figure adjustment task was used to measure observers' perceptions of local surface orientation on the depicted surfaces, and the probe points included 60 pairs of regions that both had the same orientation. The results show clearly that observers' perceptions of these three types of stimuli were remarkably similar, and that probe regions with similar apparent orientations could have large differences in image intensity. This latter finding is incompatible with any process for computing shape from shading that assumes any plausible reflectance function combined with any possible homogeneous illumination.

Solid shape discrimination from vision and haptics: natural objects (Capsicum annuum) and Gibson's "feelies".

A set of three experiments evaluated 96 participants' ability to visually and haptically discriminate solid object shape. In the past, some researchers have found haptic shape discrimination to be substantially inferior to visual shape discrimination, while other researchers have found haptics and vision to be essentially equivalent. A primary goal of the present study was to understand these discrepant past findings and to determine the true capabilities of the haptic system. All experiments used the same task (same vs. different shape discrimination) and stimulus objects (James Gibson's "feelies" and a set of naturally shaped objects--bell peppers). However, the methodology varied across experiments. Experiment 1 used random 3-dimensional (3-D) orientations of the stimulus objects, and the conditions were full-cue (active manipulation of objects and rotation of the visual objects in depth). Experiment 2 restricted the 3-D orientations of the stimulus objects and limited the haptic and visual information available to the participants. Experiment 3 compared restricted and full-cue conditions using random 3-D orientations. We replicated both previous findings in the current study. When we restricted visual and haptic information (and placed the stimulus objects in the same orientation on every trial), the participants' visual performance was superior to that obtained for haptics (replicating the earlier findings of Davidson et al. in Percept Psychophys 15(3):539-543, 1974). When the circumstances resembled those of ordinary life (e.g., participants able to actively manipulate objects and see them from a variety of perspectives), we found no significant difference between visual and haptic solid shape discrimination.

Anticipation from biological motion: the goalkeeper problem.

People can often anticipate the outcome of another person's actions based on visual information available in the movements of the other person's body. We investigated this problem by studying how goalkeepers anticipate the direction of a penalty kick in soccer. The specific aim was to determine whether the information used to anticipate kick direction is best characterized as local to a particular body segment or distributed across multiple segments. In Experiment 1, we recorded the movements of soccer players as they kicked balls into a net. Using a novel method for analyzing motion capture data, we identified sources of local and distributed information that were reliable indicators of kick direction. In Experiments 2 and 3, subjects were presented with animations of kickers' movements prior to foot-to-ball contact and instructed to judge kick direction. Judgments were consistent with the use of distributed information, with a possible small contribution of local information.

The perception of 3D shape from planar cut contours.

A new computational analysis is described for estimating 3D shapes from orthographic images of surfaces that are textured with planar cut contours. For any given contour pattern, this model provides a family of possible interpretations that are all related by affine scaling and shearing transformations in depth, depending on the specific values of its free parameters that are used to compute the shape estimate. Two psychophysical experiments were performed in an effort to compare the model predictions with observers' judgments of 3D shape for developable and non-developable surfaces. The results reveal that observers' perceptions can be systematically distorted by affine scaling and shearing transformations in depth and that the magnitude and direction of these distortions vary systematically with the 3D orientations of the contour planes.

Fechner, information, and shape perception.

How do retinal images lead to perceived environmental objects? Vision involves a series of spatial and material transformations--from environmental objects to retinal images, to neurophysiological patterns, and finally to perceptual experience and action. A rationale for understanding functional relations among these physically different systems occurred to Gustav Fechner: Differences in sensation correspond to differences in physical stimulation. The concept of information is similar: Relationships in one system may correspond to, and thus represent, those in another. Criteria for identifying and evaluating information include (a) resolution, or the precision of correspondence; (b) uncertainty about which input (output) produced a given output (input); and (c) invariance, or the preservation of correspondence under transformations of input and output. We apply this framework to psychophysical evidence to identify visual information for perceiving surfaces. The elementary spatial structure shared by objects and images is the second-order differential structure of local surface shape. Experiments have shown that human vision is directly sensitive to this higher-order spatial information from interimage disparities (stereopsis and motion parallax), boundary contours, texture, shading, and combined variables. Psychophysical evidence contradicts other common ideas about retinal information for spatial vision and object perception.

Texture discrimination based on global feature alignments.

Three experiments are reported that examined the abilities of human observers to discriminate textures with identical distributions of orientation and spatial frequency. In Experiment 1, the stimuli consisted of low-pass filtered noise that was uniformly distributed and spatially isotropic. Observers were able to discriminate textures with identical image statistics when their frequencies were 1 cpd or less, but performance dropped off sharply for textures with higher frequencies. In Experiment 2, a novel procedure was employed with which it is possible to increase the high-frequency energy in the amplitude spectrum of a texture, while preserving the macroscopic alignments of its phase spectrum. The results reveal that this has little effect on performance, thus indicating that it is not spatial frequency per se that limits the abilities of observers to discriminate macroscopic texture patterns. When the phase spectra of these textures were randomly scrambled in Experiment 3, the frequency thresholds for discriminating textures reverted back to 1 cpd as was obtained in Experiment 1. These results provide strong evidence that human observers make use of two distinct procedures for discriminating patches of texture: One based on image statistics that is possible for all textures; and another based on macroscopic phase alignments that define features that are larger than 1°.

Fechner's aesthetics revisited.

Gustav Fechner is widely respected as a founding father of experimental psychology and psychophysics but fewer know of his interests and work in empirical aesthetics. In the later 1800s, toward the end of his career, Fechner performed experiments to empirically evaluate the beauty of rectangles, hypothesizing that the preferred shape would closely match that of the so-called 'golden rectangle'. His findings confirmed his suspicions, but in the intervening decades there has been significant evidence pointing away from that finding. Regardless of the results of this one study, Fechner ushered in the notion of using a metric to evaluate beauty in a psychophysical way. In this paper, we recreate the experiment using more naturalistic stimuli. We evaluate subjects' preferences against models that use various types of object complexity as metrics. Our findings that subjects prefer either very simple or very complex objects runs contrary to the hypothesized results, but are systematic none the less. We conclude that there are likely to be useful measures of aesthetic preference but they are likely to be complicated by the difficulty in defining some of their constituent parts.

Does monocular visual space contain planes?

The issue of the existence of planes-understood as the carriers of a nexus of straight lines-in the monocular visual space of a stationary human observer has never been addressed. The most recent empirical data apply to binocular visual space and date from the 1960s (Foley, 1964). This appears to be both the first and the last time this basic issue was addressed empirically. Yet the question is of considerable conceptual interest. Here we report on a direct empirical test of the existence of planes in monocular visual space for a group of sixteen experienced observers. For the majority of these observers monocular visual space lacks a projective structure, albeit in qualitatively different ways. This greatly reduces the set of viable geometrical models. For example, it rules out all the classical homogeneous spaces (the Cayley-Klein geometries) such as the familiar Luneburg model. The qualitatively different behavior of experienced observers implies that the generic population might well be inhomogeneous with respect to the structure of visual space.

Distortions of posterior visual space.

The study of spatial vision is a long and well traveled road (which, of course, converges to a vanishing point at the horizon). Its various distortions have been widely investigated empirically, and most concentrate, pragmatically, on the space anterior to the observer. The visual world behind the observer has received relatively less attention and it is this perspective the current experiments address. Our results show systematic perceptual distortions in the posterior visual world when viewed statically. Under static viewing conditions, observer's perceptual representation was consistently 'spread' in a hyperbolic fashion. Directions to distant, peripheral locations were consistently overestimated by about 11 degrees from the ground truth and this variability increased as the target was moved toward the center of the observer's back. The perceptual representation of posterior visual space is, no doubt, secondary to the more immediate needs of the anterior visual world. Still, it is important in some domains including certain sports, such as rowing, and in vehicular navigation.

Intercepting moving targets: a little foresight helps a lot.

Behavioral studies suggest that humans intercept moving targets by maintaining a constant bearing angle (CBA). The purely feedback-driven CBA strategy has been contrasted with the strategy of predicting the eventual time and location of the future interception point. This study considers an intermediate anticipatory strategy of moving so as to produce a CBA a short duration into the future. Subjects controlled their speed of self-motion along a linear path through a simulated environment to intercept a moving target. When targets changed speed midway through the trial in Experiment 1, subjects abandoned an ineffective CBA strategy in favor of a strategy of anticipating the most likely change in target speed. In Experiment 2, targets followed paths of varying curvature. Behavior was inconsistent with both the CBA and the purely predictive strategy. To investigate the intermediate anticipatory strategy, human performance was compared with a model of interceptive behavior that, at each time-step t, produced the velocity adjustment that would minimize the change in bearing angle at time t + Deltat, taking into account the target's behavior during that interval. Values of Deltat at which the model best fit the human data for practiced subjects varied between 0.5 and 3.5 s, suggesting that actors adopt an anticipatory strategy to keep the bearing angle constant a short time into the future.