Monocular reconstruction of shapes of natural objects from orthographic and perspective images.

Human subjects were tested in perception of shapes of 3D objects. The subjects reconstructed 3D shapes by viewing orthographic and perspective images. Perception of natural shapes was very close to veridical and was clearly better than perception of random symmetrical polyhedra. Viewing perspective images led to only slightly better performance than viewing orthographic images. In order to account for subjects' performance, we elaborated the previous computational models of 3D shape reconstruction. The previous models used as constraints mirror-symmetry and 3D compactness. The critical additional constraint was the use of a secondary mirror-symmetry that exists in most natural shapes. It is known that two planes of mirror symmetry are sufficient for a unique and veridical shape reconstruction. We also generalized the model so that it applies to both orthographic and perspective images. The results of our experiment suggest that the human visual system uses two planes of symmetry in addition to two forms of 3D compactness. Performance of the new model was highly correlated with subjects' performance with both orthographic and perspective images, which supports the claim that the most important 3D shape constraints that are used by the human visual system have been identified.

Combining contour and region for closed boundary extraction of a shape.

This study explored human ability to extract closed boundary of a target shape in the presence of noise using spatially global operations. Specifically, we investigated the contributions of contour-based processing using line edges and region-based processing using color, as well as their interaction. Performance of the subjects was reliable when the fixation was inside the shape, and it was much less reliable when the fixation was outside. With fixation inside the shape, performance was higher when both contour and color information were present compared to when only one of them was present. We propose a biologically-inspired model to emulate human boundary extraction. The model solves the shortest (least-cost) path in the log-polar representation, a representation which is a good approximation to the mapping from the retina to the visual cortex. Boundary extraction was framed as a global optimization problem with the costs of connections calculated using four features: distance of interpolation, turning angle, color similarity and color contrast. This model was tested on some of the conditions that were used in the psychophysical experiment and its performance was similar to the performance of subjects.

Testing a formal theory of perception is not easy: Comments on Yu, Todd & Petrov (2021) and Yu, Petrov & Todd (2021).

Yu, Todd, and Petrov (2021, Journal of Vision) and their follow-up study (Yu, Petrov, & Todd, 2021, i-Perception) aimed at evaluating the role of three-dimensional (3D) symmetry in binocular shape perception by comparing their experimental data to predictions they derived from our computational models. We point out in this note that their predictions were incorrect, so their studies can neither reject nor support our models of 3D shape perception. We explain (1) the role of the data and the constraints in solving ill-posed inverse problems, (2) the role of binocular depth-order, as opposed to binocular depth-intervals in shape perception, (3) the nature and the effect of 3D compactness as an a priori constraint, and (4) the implications of the separation of binocular disparity and stereoacuity in the two functional streams in the visual cortex.

When is Psychology Research Useful in Artificial Intelligence? A Case for Reducing Computational Complexity in Problem Solving.

A problem is a situation in which an agent seeks to attain a given goal without knowing how to achieve it. Human problem solving is typically studied as a search in a problem space composed of states (information about the environment) and operators (to move between states). A problem such as playing a game of chess has 10 120 $10^{120}$ possible states, and a traveling salesperson problem with as little as 82 cities already has more than 10 120 $10^{120}$ different tours (similar to chess). Biological neurons are slower than the digital switches in computers. An exhaustive search of the problem space exceeds the capacity of current computers for most interesting problems, and it is fairly clear that humans cannot in their lifetime exhaustively search even small fractions of these problem spaces. Yet, humans play chess and solve logistical problems of similar complexity on a daily basis. Even for simple problems humans do not typically engage in exploring even a small fraction of the problem space. This begs the question: How do humans solve problems on a daily basis in a fast and efficient way? Recent work suggests that humans build a problem representation and solve the represented problem-not the problem that is out there. The problem representation that is built and the process used to solve it are constrained by limits of cognitive capacity and a cost-benefit analysis discounting effort and reward. In this article, we argue that better understanding the way humans represent and solve problems using heuristics can help inform how simpler algorithms and representations can be used in artificial intelligence to lower computational complexity, reduce computation time, and facilitate real-time computation in complex problem solving.

The Concept of Symmetry and the Theory of Perception.

Perceptual constancy refers to the fact that the perceived geometrical and physical characteristics of objects remain constant despite transformations of the objects such as rigid motion. Perceptual constancy is essential in everything we do, like recognition of familiar objects and scenes, planning and executing visual navigation, visuomotor coordination, and many more. Perceptual constancy would not exist without the geometrical and physical permanence of objects: their shape, size, and weight. Formally, perceptual constancy and permanence of objects are invariants, also known in mathematics and physics as symmetries. Symmetries of the Laws of Physics received a central status due to mathematical theorems of Emmy Noether formulated and proved over 100 years ago. These theorems connected symmetries of the physical laws to conservation laws through the least-action principle. We show how Noether's theorem is applied to mirror-symmetrical objects and establishes mental shape representation (perceptual conservation) through the application of a simplicity (least-action) principle. This way, the formalism of Noether's theorem provides a computational explanation of the relation between the physical world and its mental representation.

Gestalt-like constraints produce veridical (Euclidean) percepts of 3D indoor scenes.

This study, which was influenced a lot by Gestalt ideas, extends our prior work on the role of a priori constraints in the veridical perception of 3D shapes to the perception of 3D scenes. Our experiments tested how human subjects perceive the layout of a naturally-illuminated indoor scene that contains common symmetrical 3D objects standing on a horizontal floor. In one task, the subject was asked to draw a top view of a scene that was viewed either monocularly or binocularly. The top views the subjects reconstructed were configured accurately except for their overall size. These size errors varied from trial to trial, and were shown most-likely to result from the presence of a response bias. There was little, if any, evidence of systematic distortions of the subjects' perceived visual space, the kind of distortions that have been reported in numerous experiments run under very unnatural conditions. This shown, we proceeded to use Foley's (Vision Research 12 (1972) 323-332) isosceles right triangle experiment to test the intrinsic geometry of visual space directly. This was done with natural viewing, with the impoverished viewing conditions Foley had used, as well as with a number of intermediate viewing conditions. Our subjects produced very accurate triangles when the viewing conditions were natural, but their performance deteriorated systematically as the viewing conditions were progressively impoverished. Their perception of visual space became more compressed as their natural visual environment was degraded. Once this was shown, we developed a computational model that emulated the most salient features of our psychophysical results. We concluded that human observers see 3D scenes veridically when they view natural 3D objects within natural 3D environments.

Spatially-global integration of closed, fragmented contours by finding the shortest-path in a log-polar representation.

Finding the occluding contours of objects in real 2D retinal images of natural 3D scenes is done by determining, which contour fragments are relevant, and the order in which they should be connected. We developed a model that finds the closed contour represented in the image by solving a shortest path problem that uses a log-polar representation of the image; the kind of representation known to exist in area V1 of the primate cortex. The shortest path in a log-polar representation favors the smooth, convex and closed contours in the retinal image that have the smallest number of gaps. This approach is practical because finding a globally-optimal solution to a shortest path problem is computationally easy. Our model was tested in four psychophysical experiments. In the first two experiments, the subject was presented with a fragmented convex or concave polygon target among a large number of unrelated pieces of contour (distracters). The density of these pieces of contour was uniform all over the screen to minimize spatially-local cues. The orientation of each target contour fragment was randomly perturbed by varying the levels of jitter. Subjects drew a closed contour that represented the target's contour on a screen. The subjects' performance was nearly perfect when the jitter-level was low. Their performance deteriorated as jitter-levels were increased. The performance of our model was very similar to our subjects'. In two subsequent experiments, the subject was asked to discriminate a briefly-presented egg-shaped object while maintaining fixation at several different positions relative to the closed contour of the shape. The subject's discrimination performance was affected by the fixation position in much the same way as the model's.

Philosophizing cannot substitute for experimentation: comment on Hoffman, Singh & Prakash (2014).

The perception of a 3D shape must be excluded from Hoffman et al.'s "interface theory" primarily because shape is characterized by its symmetries. When these symmetries are used as a priori constraints, 3D shapes are always recovered from 2D retinal images veridically. These facts make it clear that 3D shape perception is completely different from, as well as more important than, all other perceptions because the veridicality of our perception of 3D shapes (and 3D scenes) accounts for our successful adaptation to the natural environment.

A Bayesian model of binocular perception of 3D mirror symmetrical polyhedra.

In our previous studies, we showed that monocular perception of 3D shapes is based on a priori constraints, such as 3D symmetry and 3D compactness. The present study addresses the nature of perceptual mechanisms underlying binocular perception of 3D shapes. First, we demonstrate that binocular performance is systematically better than monocular performance, and it is close to perfect in the case of three out of four subjects. Veridical shape perception cannot be explained by conventional binocular models, in which shape was derived from depth intervals. In our new model, we use ordinal depth of points in a 3D shape provided by stereoacuity and combine it with monocular shape constraints by means of Bayesian inference. The stereoacuity threshold used by the model was estimated for each subject. This model can account for binocular shape performance of all four subjects. It can also explain the fact that when viewing distance increases, the binocular percept gradually reduces to the monocular one, which implies that monocular percept of a 3D shape is a special case of the binocular percept.

Depth cues versus the simplicity principle in 3D shape perception.

Two experiments were performed to explore the mechanisms of human 3D shape perception. In Experiment 1, the subjects' performance in a shape constancy task in the presence of several cues (edges, binocular disparity, shading and texture) was tested. The results show that edges and binocular disparity, but not shading or texture, are important in 3D shape perception. Experiment 2 tested the effect of several simplicity constraints, such as symmetry and planarity on subjects' performance in a shape constancy task. The 3D shapes were represented by edges or vertices only. The results show that performance with or without binocular disparity is at chance level, unless the 3D shape is symmetric and/or its faces are planar. In both experiments, there was a correlation between the subjects' performance with and without binocular disparity. Our study suggests that simplicity constraints, not depth cues, play the primary role in both monocular and binocular 3D shape perception. These results are consistent with our computational model of 3D shape recovery.

Human motor transfer is determined by the scaling of size and accuracy of movement.

A transfer of training design was used to examine the role of the Index of Difficulty (ID) on transfer of learning in a sequential Fitts's law task. Specifically, the role of the ratio between the accuracy and size of movement (ID) in transfer was examined. Transfer of skilled movement is better when both the size and accuracy of movement are changed by the same factor (ID is constant) than when only size or accuracy is changed. The authors infer that the size-accuracy ratio is capturing the control strategies employed during practice and thus promotes efficient transfer. Furthermore, efficient transfer is not dependent on maintaining relative timing invariance and thus the authors provide further evidence that relative timing is not an essential feature of movement control.

New approach to the perception of 3D shape based on veridicality, complexity, symmetry and volume.

This paper reviews recent progress towards understanding 3D shape perception made possible by appreciating the significant role that veridicality and complexity play in the natural visual environment. The ability to see objects as they really are "out there" is derived from the complexity inherent in the 3D object's shape. The importance of both veridicality and complexity was ignored in most prior research. Appreciating their importance made it possible to devise a computational model that recovers the 3D shape of an object from only one of its 2D images. This model uses a simplicity principle consisting of only four a priori constraints representing properties of 3D shapes, primarily their symmetry and volume. The model recovers 3D shapes from a single 2D image as well, and sometimes even better, than a human being. In the rare recoveries in which errors are observed, the errors made by the model and human subjects are very similar. The model makes no use of depth, surfaces or learning. Recent elaborations of this model include: (i) the recovery of the shapes of natural objects, including human and animal bodies with limbs in varying positions (ii) providing the model with two input images that allowed it to achieve virtually perfect shape constancy from almost all viewing directions. The review concludes with a comparison of some of the highlights of our novel, successful approach to the recovery of 3D shape from a 2D image with prior, less successful approaches.

Camouflage and visual perception.

How does an animal conceal itself from visual detection by other animals? This review paper seeks to identify general principles that may apply in this broad area. It considers mechanisms of visual encoding, of grouping and object encoding, and of search. In most cases, the evidence base comes from studies of humans or species whose vision approximates to that of humans. The effort is hampered by a relatively sparse literature on visual function in natural environments and with complex foraging tasks. However, some general constraints emerge as being potentially powerful principles in understanding concealment--a 'constraint' here means a set of simplifying assumptions. Strategies that disrupt the unambiguous encoding of discontinuities of intensity (edges), and of other key visual attributes, such as motion, are key here. Similar strategies may also defeat grouping and object-encoding mechanisms. Finally, the paper considers how we may understand the processes of search for complex targets in complex scenes. The aim is to provide a number of pointers towards issues, which may be of assistance in understanding camouflage and concealment, particularly with reference to how visual systems can detect the shape of complex, concealed objects.

A computational model that recovers the 3D shape of an object from a single 2D retinal representation.

Human beings perceive 3D shapes veridically, but the underlying mechanisms remain unknown. The problem of producing veridical shape percepts is computationally difficult because the 3D shapes have to be recovered from 2D retinal images. This paper describes a new model, based on a regularization approach, that does this very well. It uses a new simplicity principle composed of four shape constraints: viz., symmetry, planarity, maximum compactness and minimum surface. Maximum compactness and minimum surface have never been used before. The model was tested with random symmetrical polyhedra. It recovered their 3D shapes from a single randomly-chosen 2D image. Neither learning, nor depth perception, was required. The effectiveness of the maximum compactness and the minimum surface constraints were measured by how well the aspect ratio of the 3D shapes was recovered. These constraints were effective; they recovered the aspect ratio of the 3D shapes very well. Aspect ratios recovered by the model were compared to aspect ratios adjusted by four human observers. They also adjusted aspect ratios very well. In those rare cases, in which the human observers showed large errors in adjusted aspect ratios, their errors were very similar to the errors made by the model.

Binocular disparity only comes into play when everything else fails; a finding with broader implications than one might suppose.

This paper calls attention to research showing that binocular disparity, which is an effective cue to depth, plays a secondary role, at best, in the perception of 3D shape. This claim has implications both for how shape should be studied and how this unique perceptual property should be modeled. These issues are discussed from a historical perspective, which shows how the failure to appreciate the importance of the Gestalt grouping principle called 'Figure-Ground Organization' led to many unfruitful efforts. It also calls attention to how this situation can be remedied.

Detection of skewed symmetry.

This study examined the ability of human observers to discriminate between symmetric and asymmetric planar figures from perspective and orthographic images. The first experiment showed that the discrimination is reliable in the case of polygons, but not dotted patterns. The second experiment showed that the discrimination is facilitated when the projected symmetry axis or projected symmetry lines are known to the subject. A control experiment showed that the discrimination is more reliable with orthographic, than with perspective images. Based on these results, we formulated a computational model of symmetry detection. The model measures the asymmetry of the presented polygon based on its single orthographic or perspective image. Performance of the model is similar to that of the subjects.

Does contour classification precede contour grouping in perception of partially visible figures?

When a figure is only partially visible and its contours represent a small fraction of total image contours (as when there is much background clutter), a fast contour classification mechanism may filter non-figure contours in order to restrict the size of the input to subsequent contour grouping mechanisms. The results of two psychophysical experiments suggest that the human visual system can classify figure from non-figure contours on the basis of a difference in some contour property (e.g. length, orientation, curvature, etc). While certain contour properties (e.g. orientation, curvature) require only local analysis for classification, other contour properties (e.g. length) may require more global analysis of the retinal image. We constructed a pyramid-based computational model based on these observations and performed two simulations of experiment 1: one simulation with classification enabled and the other simulation with classification disabled. The classification-based simulation gave the superior account of human performance in experiment 1. When a figure is partially visible, with few contours relative to the number of non-figure contours, contour classification followed by contour grouping can be more efficient than contour grouping alone, owing to smaller input to grouping mechanisms.

Binocular shape constancy from novel views: the role of a priori constraints.

We tested shape constancy from novel views in the case of binocular viewing, using a variety of stimuli, including polyhedra, polygonal lines, and points in 3-D. The results of the psychophysical experiments show that constraints such as planarity of surface contours and symmetry are critical for reliable shape constancy. These results are consistent with the results obtained in our previous psychophysical experiments on shape constancy from novel views in the presence of a kinetic depth effect (Pizlo & Stevenson, 1999). On the basis of these results, we developed a new model of binocular shape reconstruction. The model is based on the assumption that binocular reconstruction is a difficult inverse problem, whose solution requires imposing a priori constraints on the family of possible interpretations. In the model, binocular disparity is used to correct monocularly reconstructed shape. The new model was tested on the same shapes as those used in the psychophysical experiments. The reconstructions produced by this model are substantially more reliable than the reconstructions produced by models that do not use constraints. Interestingly, monocular (but not binocular) reconstructions produced by this model correlate well with both monocular and binocular performance of human subjects. This fact suggests that binocular and monocular reconstructions of shapes in the human visual system involve similar mechanisms based on monocular shape constraints.