Dynamic estimation of task-relevant variance in movement under risk.

Humans take into account their own movement variability as well as potential consequences of different movement outcomes in planning movement trajectories. When variability increases, planned movements are altered so as to optimize expected consequences of the movement. Past research has focused on the steady-state responses to changing conditions of movement under risk. Here, we study the dynamics of such strategy adjustment in a visuomotor decision task in which subjects reach toward a display with regions that lead to rewards and penalties, under conditions of changing uncertainty. In typical reinforcement learning tasks, subjects should base subsequent strategy by computing an estimate of the mean outcome (e.g., reward) in recent trials. In contrast, in our task, strategy should be based on a dynamic estimate of recent outcome uncertainty (i.e., squared error). We find that subjects respond to increased movement uncertainty by aiming movements more conservatively with respect to penalty regions, and that the estimate of uncertainty they use is well characterized by a weighted average of recent squared errors, with higher weights given to more recent trials.

Dynamic integration of information about salience and value for saccadic eye movements.

Humans shift their gaze to a new location several times per second. It is still unclear what determines where they look next. Fixation behavior is influenced by the low-level salience of the visual stimulus, such as luminance, contrast, and color, but also by high-level task demands and prior knowledge. Under natural conditions, different sources of information might conflict with each other and have to be combined. In our paradigm, we trade off visual salience against expected value. We show that both salience and value information influence the saccadic end point within an object, but with different time courses. The relative weights of salience and value are not constant but vary from eye movement to eye movement, depending critically on the availability of the value information at the time when the saccade is programmed. Short-latency saccades are determined mainly by salience, but value information is taken into account for long-latency saccades. We present a model that describes these data by dynamically weighting and integrating detailed topographic maps of visual salience and value. These results support the notion of independent neural pathways for the processing of visual information and value.

Comparison of the distortion of probability information in decision under risk and an equivalent visual task.

Decision makers typically overweight small probabilities and underweight large probabilities. However, there are recent reports that when probability is presented in the form of relative frequencies, this typical pattern reverses. We tested this hypothesis by comparing decision making in two tasks: In one task, probability was stated numerically, and in the other task, it was conveyed through a visual representation. In the visual task, participants chose whether a "stochastic bullet" should be fired at either a large target for a small reward or a small target for a large reward. Participants' knowledge of probability in the visual task was the result of extensive practice firing bullets at targets. In the classical numerical task, participants chose between pairs of lotteries with probabilities and rewards matched to the probabilities and rewards in the visual task. We found that participants' probability-weighting functions were significantly different in the two tasks, but the pattern for the visual task was the typical, not the reversed, pattern.

Visual-haptic cue integration with spatial and temporal disparity during pointing movements.

Many perceptual cue combination studies have shown that humans can integrate sensory information across modalities as well as within a modality in a manner that is close to optimal. While the limits of sensory cue integration have been extensively studied in the context of perceptual decision tasks, the evidence obtained in the context of motor decisions provides a less consistent picture. Here, we studied the combination of visual and haptic information in the context of human arm movement control. We implemented a pointing task in which human subjects pointed at an invisible unknown target position whose vertical position varied randomly across trials. In each trial, we presented a haptic and a visual cue that provided noisy information about the target position half-way through the reach. We measured pointing accuracy as function of haptic and visual cue onset and compared pointing performance to the predictions of a multisensory decision model. Our model accounts for pointing performance by computing the maximum a posteriori estimate, assuming minimum variance combination of uncertain sensory cues. Synchronicity of cue onset has previously been demonstrated to facilitate the integration of sensory information. We tested this in trials in which visual and haptic information was presented with temporal disparity. We found that for our sensorimotor task temporal disparity between visual and haptic cue had no effect. Sensorimotor learning appears to use all available information and to apply the same near-optimal rules for cue combination that are used by perception.

Parallel visual search and rapid animal detection in natural scenes.

Human observers are capable of detecting animals within novel natural scenes with remarkable speed and accuracy. Recent studies found human response times to be as fast as 120 ms in a dual-presentation (2-AFC) setup (H. Kirchner & S. J. Thorpe, 2005). In most previous experiments, pairs of randomly chosen images were presented, frequently from very different contexts (e.g., a zebra in Africa vs. the New York Skyline). Here, we tested the effect of background size and contiguity on human performance by using a new, contiguous background image set. Individual images contained a single animal surrounded by a large, animal-free image area. The image could be positioned and cropped in such a manner that the animal could occur in one of eight evenly spaced positions on an imaginary circle (radius 10-deg visual angle). In the first (8-Choice) experiment, all eight positions were used, whereas in the second (2-Choice) and third (2-Image) experiments, the animals were only presented on the two positions to the left and right of the screen center. In the third experiment, additional rectangular frames were used to mimic the conditions of earlier studies. Average latencies on successful trials differed only slightly between conditions, indicating that the number of possible animal locations within the display does not affect decision latency. Detailed analysis of saccade targets revealed a preference toward both the head and the center of gravity of the target animal, affecting hit ratio, latency, and the number of saccades required to reach the target. These results illustrate that rapid animal detection operates scene-wide and is fast and efficient even when the animals are embedded in their natural backgrounds.

Adapting internal statistical models for interpreting visual cues to depth.

The informativeness of sensory cues depends critically on statistical regularities in the environment. However, statistical regularities vary between different object categories and environments. We asked whether and how the brain changes the prior assumptions about scene statistics used to interpret visual depth cues when stimulus statistics change. Subjects judged the slants of stereoscopically presented figures by adjusting a virtual probe perpendicular to the surface. In addition to stereoscopic disparities, the aspect ratio of the stimulus in the image provided a "figural compression" cue to slant, whose reliability depends on the distribution of aspect ratios in the world. As we manipulated this distribution from regular to random and back again, subjects' reliance on the compression cue relative to stereoscopic cues changed accordingly. When we randomly interleaved stimuli from shape categories (ellipses and diamonds) with different statistics, subjects gave less weight to the compression cue for figures from the category with more random aspect ratios. Our results demonstrate that relative cue weights vary rapidly as a function of recently experienced stimulus statistics and that the brain can use different statistical models for different object categories. We show that subjects' behavior is consistent with that of a broad class of Bayesian learning models.

Eye-hand coordination while pointing rapidly under risk.

Humans make rapid, goal-directed movements to interact with their environment. Saccadic eye movements usually accompany rapid hand movements, suggesting neural coupling, although it remains unclear what determines the strength of the coupling. Here, we present evidence that humans can alter eye-hand coordination in response to risk associated with endpoint variability. We used a paradigm in which human participants were forced to point rapidly under risk and were penalized or rewarded depending on the hand movement outcome. A separate reward schedule was employed for relative saccadic endpoint position. Participants received a monetary reward proportional to points won. We present a model that defines optimality of eye-hand coordination for this task depending on where the hand lands relative to the eye. A comparison of the results and model predictions showed that participants could optimize performance to maximize gain in some conditions, but not others. Participants produced near-optimal results when no feedback was given about relative saccade location and when negative feedback was provided for large distances between the saccade and hand. Participants were sub-optimal when given negative feedback for saccades very close to the hand endpoint. Our results suggest that eye-hand coordination is flexible when pointing rapidly under risk, but final eye position remains correlated with finger location.

Effects of salience and reward information during saccadic decisions under risk.

Previous work has demonstrated that humans select visuomotor strategies maximizing expected gain during speeded hand movements under risk; see, e.g., [Trends Cogn. Sci. 12, 291 (2008)]; [Glimcher, eds., Neuroeconomics: Decision Making and the Brain (Elsevier, 2008), p. 95]. Here we report a similar study in which we recorded saccadic eye movements in a saccadic decision task in which monetary rewards and losses were associated with the final position of the eye movement. Saccades into a color-coded target region won points; saccades into a partially overlapping or abutting penalty region could yield a loss. The points won during the experiment were converted into a small monetary bonus at the end of the experiment. We compared participants' winnings to the score of an optimal observer maximizing expected gain that was calculated based on each participant's saccadic endpoint variability, similar to a recent model of optimal movement planning under risk [J. Opt. Soc. Am. A 20, 1419 (2003)]; [Spatial Vis. 16, 255 (2003)]. We used three different experimental paradigms with different interstimulus intervals (Gap, No Gap, and Overlap) to manipulate saccadic latencies and a fourth experiment (Memory) with a prolonged 500 ms delay period. Our results show that our subjects took the reward information, as specified by the different penalties, into account when making saccades and fixated onto or very close to the target region and less into the penalty region. However, the selected strategies differed significantly from optimal strategies maximizing expected gain in conditions when the magnitude of reward or penalty was changed. Furthermore, scores were notably affected by stimulus saliency. They were higher when the target region was filled and the penalty region outlined by a thin line, as compared to conditions in which the target was indicated by a less salient stimulus. Scores were particularly poor in trials with the shortest latencies (120-140 ms) mostly obtained in the Gap paradigm. At longer latencies scores improved considerably for latencies longer than 160 ms. This was in line with an improvement in accuracy for single targets up to 160 ms. Our results indicate that processing both of reward information and of stimulus saliency affect the programming of saccades, with a dominating contribution of stimulus saliency for eye movements with faster latencies.

Visual processing, learning and feedback in the primate eye movement system.

We present an overview of recent paradigms used for studying visual information and reward processing in the human and monkey oculomotor pathways. Current evidence indicates that eye movements made during visual search tasks rely on neural computations similar to those employed when eye movements are planned and executed to obtain explicit rewards. These data suggest that human eye movements originate from the processing of (predominantly visual) sensory information, feedback about previous errors, and expectations about factors, such as reward. We conclude that these properties make the saccadic system an ideal model for studying both the behavioral and neural mechanisms for human voluntary and involuntary choice behavior.

Combination of noisy directional visual and proprioceptive information.

We present experimental and computational evidence for the estimation of visual and proprioceptive directional information during forward, visually driven arm movements. We presented noisy directional proprioceptive and visual stimuli simultaneously and in isolation midway during a pointing movement. Directional proprioceptive stimuli were created by brief force pulses, which varied in direction and were applied to the fingertip shortly after movement onset. Subjects indicated the perceived direction of the stimulus after each trial. We measured unimodal performance in trials in which we presented only the visual or only the proprioceptive stimulus. When we presented simultaneous but conflicting bimodal information, subjects' perceived direction fell in between the visual and proprioceptive directions. We find that the judged mean orientation matched the MLE predictions but did not show the expected improvement in reliability as compared to unimodal performance. We present an alternative model (probabilistic cue switching, PCS), which is consistent with our data. According to this model, subjects base their bimodal judgments on only one of two directional cues in a given trial, with relative choice probabilities proportional to the average stimulus reliability. These results suggest that subjects based their decision on a probability mixture of both modalities without integrating information across modalities.

Biases and optimality of sensory-motor and cognitive decisions.

I review a variety of behavioral studies directed at understanding how probability and value information is represented in motor and cognitive tasks. Subjects in (cognitive) decision-making tasks often misrepresent the frequency of rare events and typically fail to maximize expected gain. In contrast, subjects in mathematically equivalent movement tasks are generally found to be very good at choosing motor strategies that come close to maximizing expected gain. I discuss the differences between the sources of uncertainty for decisions in the motor and cognitive domain and summarize experimental evidence about how information about uncertainty is acquired in motor and cognitive decision tasks. Finally, I briefly review the evidence concerning the neural coding of probability, expected gain, and other information in decision-making tasks.

The role of visuohaptic experience in visually perceived depth.

Berkeley suggested that "touch educates vision," that is, haptic input may be used to calibrate visual cues to improve visual estimation of properties of the world. Here, we test whether haptic input may be used to "miseducate" vision, causing observers to rely more heavily on misleading visual cues. Human subjects compared the depth of two cylindrical bumps illuminated by light sources located at different positions relative to the surface. As in previous work using judgments of surface roughness, we find that observers judge bumps to have greater depth when the light source is located eccentric to the surface normal (i.e., when shadows are more salient). Following several sessions of visual judgments of depth, subjects then underwent visuohaptic training in which haptic feedback was artificially correlated with the "pseudocue" of shadow size and artificially decorrelated with disparity and texture. Although there were large individual differences, almost all observers demonstrated integration of haptic cues during visuohaptic training. For some observers, subsequent visual judgments of bump depth were unaffected by the training. However, for 5 of 12 observers, training significantly increased the weight given to pseudocues, causing subsequent visual estimates of shape to be less veridical. We conclude that haptic information can be used to reweight visual cues, putting more weight on misleading pseudocues, even when more trustworthy visual cues are available in the scene.

Decision making, movement planning and statistical decision theory.

We discuss behavioral studies directed at understanding how probability information is represented in motor and economic tasks. By formulating the behavioral tasks in the language of statistical decision theory, we can compare performance in equivalent tasks in different domains. Subjects in traditional economic decision-making tasks often misrepresent the probability of rare events and typically fail to maximize expected gain. By contrast, subjects in mathematically equivalent movement tasks often choose movement strategies that come close to maximizing expected gain. We discuss the implications of these different outcomes, noting the evident differences between the source of uncertainty and how information about uncertainty is acquired in motor and economic tasks.

Learning stochastic reward distributions in a speeded pointing task.

Recent studies have shown that humans effectively take into account task variance caused by intrinsic motor noise when planning fast hand movements. However, previous evidence suggests that humans have greater difficulty accounting for arbitrary forms of stochasticity in their environment, both in economic decision making and sensorimotor tasks. We hypothesized that humans can learn to optimize movement strategies when environmental randomness can be experienced and thus implicitly learned over several trials, especially if it mimics the kinds of randomness for which subjects might have generative models. We tested the hypothesis using a task in which subjects had to rapidly point at a target region partly covered by three stochastic penalty regions introduced as "defenders." At movement completion, each defender jumped to a new position drawn randomly from fixed probability distributions. Subjects earned points when they hit the target, unblocked by a defender, and lost points otherwise. Results indicate that after approximately 600 trials, subjects approached optimal behavior. We further tested whether subjects simply learned a set of stimulus-contingent motor plans or the statistics of defenders' movements by training subjects with one penalty distribution and then testing them on a new penalty distribution. Subjects immediately changed their strategy to achieve the same average reward as subjects who had trained with the second penalty distribution. These results indicate that subjects learned the parameters of the defenders' jump distributions and used this knowledge to optimally plan their hand movements under conditions involving stochastic rewards and penalties.

Visual estimation under risk.

We investigate whether observers take into account their visual uncertainty in an optimal manner in a perceptual estimation task with explicit rewards and penalties for performance. Observers judged the mean orientation of a briefly presented texture consisting of a collection of line segments. The mean and, in some experiments, the variance of the distribution of line orientations changed from trial to trial. Subjects tried to maximize the number of points won in a "bet" on the mean texture orientation. They placed their bet by rotating a visual display that indicated two ranges of orientations: a reward region and a neighboring penalty region. Subjects won 100 points if the mean texture orientation fell within the reward region, and subjects lost points (0, 100, or 500, in separate blocks) if the mean orientation fell in the penalty region. We compared each subject's performance to a decision strategy that maximizes expected gain (MEG). For the nonzero-penalty conditions, this ideal strategy predicts subjects will adjust the payoff display to shift the center of the reward region away from the perceived mean texture orientation, putting the perceived mean orientation on the opposite side of the reward region from the penalty region. This shift is predicted to be larger for (1) larger penalties, (2) penalty regions located closer to the payoff region, and (3) larger stimulus variability. While some subjects' performance was nearly optimal, other subjects displayed a variety of suboptimal strategies when stimulus variability was high and changed unpredictably from trial to trial.

Eye movements during rapid pointing under risk.

We recorded saccadic eye movements during visually-guided rapid pointing movements under risk. We intended to determine whether saccadic end points are necessarily tied to the goals of rapid pointing movements or whether, when the visual features of a display and the goals of a pointing movement are different, saccades are driven by low-level features of the visual stimulus. Subjects pointed at a stimulus configuration consisting of a target region and a penalty region. Each target hit yielded a gain of points; each penalty hit incurred a loss of points. Late responses were penalized. The luminance of either target or penalty region was indicated by a disk which differed significantly from the background in luminance, while the other region was indicated by a thin circle. In subsequent experiments, we varied the visual salience of the stimulus configuration and found that manual responses followed near-optimal strategies maximizing expected gain, independent of the salience of the target region. We suggest that the final eye position is partially pre-programmed prior to hand movement initiation. While we found that manipulations of the visual salience of the display determined the end point of the initial saccade we also found that subsequent saccades are driven by the goal of the hand movement.

Optimality of human movement under natural variations of visual-motor uncertainty.

Biological movements are prone to error. Different movements lead to different errors, and the distributions of errors depend on movement amplitude and direction. Movement planning would benefit from taking this variability into account, by applying appropriate corrections for movements associated with the different shapes and sizes of error distributions. Here we asked whether the human nervous system can do so. In a game-like task, participants performed rapid sequences of goal-directed pointing movements in different directions, toward stimulus configurations presented at different eccentricities on a slanted touch screen. The task was to accumulate rewards by hitting target regions and to minimize losses by avoiding penalty regions. The distributions of endpoint errors varied in size and degree of anisotropy across stimulus locations. Our participants adjusted their movements toward the different locations accordingly. We compared human behavior with the optimal behavior predicted by ideal movement planner maximizing expected gain. In most cases, human behavior was indistinguishable from optimal. This is evidence that human movement planning approaches statistical optimality by representing the task-relevant movement variability.

Humans rapidly estimate expected gain in movement planning.

We studied human movement planning in tasks in which subjects selected one of two goals that differed in expected gain. Each goal configuration consisted of a target circle and a partially overlapping penalty circle. Rapid hits into the target region led to a monetary bonus; accidental hits into the penalty region incurred a penalty. The outcomes assigned to target and penalty regions and the spatial arrangement of those regions were varied. Subjects preferred configurations with higher expected gain whether selection involved a rapid pointing movement or a choice by key press. Movements executed to select one of two goal configurations exhibited the same movement dynamics as pointing movements directed at a single configuration, and were executed with the same high efficiency. Our results suggest that humans choose near-optimal strategies when planning their movement, and can base their selection of strategy on a rapid judgment about the expected gain associated with possible movement goals.