A novel fMRI paradigm to dissociate the behavioral and neural components of mixed-strategy decision making from non-strategic decisions in humans.

During competitive interactions, such as predator-prey or team sports, the outcome of one's actions is dependent on both their own choices and those of their opponents. Success in these rivalries requires that individuals choose dynamically and unpredictably, often adopting a mixed strategy. Understanding the neural basis of strategic decision making is complicated by the fact that it recruits various cognitive processes that are often shared with non-strategic forms of decision making, such as value estimation, working memory, response inhibition, response selection, and reward processes. Although researchers have explored neural activity within key brain regions during mixed-strategy games, how brain activity differs in the context of strategic interactions versus non-strategic choices is not well understood. We developed a novel behavioral paradigm to dissociate choice behavior during mixed-strategy interactions from non-strategic choices, and we used task-based functional magnetic resonance imaging (fMRI) to contrast brain activation. In a block design, participants competed in the classic mixed-strategy game, "matching pennies," against a dynamic computer opponent designed to exploit predictability in players' response patterns. Results were contrasted with a non-strategic task that had comparable sensory input, motor output, and reward rate; thus, differences in behavior and brain activation reflect strategic processes. The mixed-strategy game was associated with activation of a distributed cortico-striatal network compared to the non-strategic task. We propose that choosing in mixed-strategy contexts requires additional cognitive demands present to a lesser degree during the control task, illustrating the strength of this design in probing function of cognitive systems beyond core sensory, motor, and reward processes.

On the neural implementation of the speed-accuracy trade-off.

Decisions are faster and less accurate when conditions favor speed, and are slower and more accurate when they favor accuracy. This phenomenon is referred to as the speed-accuracy trade-off (SAT). Behavioral studies of the SAT have a long history, and the data from these studies are well characterized within the framework of bounded integration. According to this framework, decision makers accumulate noisy evidence until the running total for one of the alternatives reaches a bound. Lower and higher bounds favor speed and accuracy respectively, each at the expense of the other. Studies addressing the neural implementation of these computations are a recent development in neuroscience. In this review, we describe the experimental and theoretical evidence provided by these studies. We structure the review according to the framework of bounded integration, describing evidence for (1) the modulation of the encoding of evidence under conditions favoring speed or accuracy, (2) the modulation of the integration of encoded evidence, and (3) the modulation of the amount of integrated evidence sufficient to make a choice. We discuss commonalities and differences between the proposed neural mechanisms, some of their assumptions and simplifications, and open questions for future work. We close by offering a unifying hypothesis on the present state of play in this nascent research field.

Trading speed and accuracy by coding time: a coupled-circuit cortical model.

Our actions take place in space and time, but despite the role of time in decision theory and the growing acknowledgement that the encoding of time is crucial to behaviour, few studies have considered the interactions between neural codes for objects in space and for elapsed time during perceptual decisions. The speed-accuracy trade-off (SAT) provides a window into spatiotemporal interactions. Our hypothesis is that temporal coding determines the rate at which spatial evidence is integrated, controlling the SAT by gain modulation. Here, we propose that local cortical circuits are inherently suited to the relevant spatial and temporal coding. In simulations of an interval estimation task, we use a generic local-circuit model to encode time by 'climbing' activity, seen in cortex during tasks with a timing requirement. The model is a network of simulated pyramidal cells and inhibitory interneurons, connected by conductance synapses. A simple learning rule enables the network to quickly produce new interval estimates, which show signature characteristics of estimates by experimental subjects. Analysis of network dynamics formally characterizes this generic, local-circuit timing mechanism. In simulations of a perceptual decision task, we couple two such networks. Network function is determined only by spatial selectivity and NMDA receptor conductance strength; all other parameters are identical. To trade speed and accuracy, the timing network simply learns longer or shorter intervals, driving the rate of downstream decision processing by spatially non-selective input, an established form of gain modulation. Like the timing network's interval estimates, decision times show signature characteristics of those by experimental subjects. Overall, we propose, demonstrate and analyse a generic mechanism for timing, a generic mechanism for modulation of decision processing by temporal codes, and we make predictions for experimental verification.

Evidence for a link between the experiential allocation of saccade preparation and visuospatial attention.

Whether a link exists between the two orienting processes of saccade preparation and visuospatial attention has typically been studied by using either sensory cues or predetermined rules that instruct subjects where to allocate these limited resources. In the real world, explicit instructions are not always available and presumably expectations shaped by previous experience play an important role in the allocation of these processes. Here we examined whether manipulating two experiential factors that clearly influence saccade preparation--the probability and timing of saccadic responses--also influences the allocation of visuospatial attention. Occasionally, a visual probe was presented whose spatial location and time of presentation varied relative to those of the saccade target. The proportion of erroneous saccades directed toward this probe indexed saccade preparation, and the proportion of correct discriminations of probe orientation indexed visuospatial attention. Overall, preparation and attention were significantly correlated to each other across these manipulations of saccade probability and timing. Saccade probability influenced both preparation and attention processes, whereas saccade timing influenced only preparation processes. Unexpectedly, discrimination ability was not improved in those trials in which the probe triggered an erroneous saccade despite particularly heightened levels of saccade preparation. To account for our results, we propose a conceptual dual-purpose threshold model based on neurophysiological considerations that link the processes of saccade preparation and visuospatial attention. The threshold acts both as the minimum activity level required for eliciting saccades and a maximum level for which neural activity can provide attentional benefits.

The Relationship between Saccadic Choice and Reaction Times with Manipulations of Target Value.

Choosing the option with the highest expected value (EV; reward probability × reward magnitude) maximizes the intake of reward under conditions of uncertainty. However, human economic choices indicate that our value calculation has a subjective component whereby probability and reward magnitude are not linearly weighted. Using a similar economic framework, our goal was to characterize how subjective value influences the generation of simple motor actions. Specifically, we hypothesized that attributes of saccadic eye movements could provide insight into how rhesus monkeys, a well-studied animal model in cognitive neuroscience, subjectively value potential visual targets. In the first experiment, monkeys were free to choose by directing a saccade toward one of two simultaneously displayed targets, each of which had an uncertain outcome. In this task, choices were more likely to be allocated toward the higher valued target. In the second experiment, only one of the two possible targets appeared on each trial. In this task, saccadic reaction times (SRTs) decreased toward the higher valued target. Reward magnitude had a much stronger influence on both choices and SRTs than probability, whose effect was observed only when reward magnitude was similar for both targets. Across EV blocks, a strong relationship was observed between choice preferences and SRTs. However, choices tended to maximize at skewed values whereas SRTs varied more continuously. Lastly, SRTs were unchanged when all reward magnitudes were 1×, 1.5×, and 2× their normal amount, indicating that saccade preparation was influenced by the relative value of the targets rather than the absolute value of any single-target. We conclude that value is not only an important factor( )for deliberative decision making in primates, but also for the selection and preparation of simple motor actions, such as saccadic eye movements. More precisely, our results indicate that, under conditions of uncertainty, saccade choices and reaction times are influenced by the relative expected subjective value of potential movements.

Gain modulation by an urgency signal controls the speed-accuracy trade-off in a network model of a cortical decision circuit.

The speed-accuracy trade-off (SAT) is ubiquitous in decision tasks. While the neural mechanisms underlying decisions are generally well characterized, the application of decision-theoretic methods to the SAT has been difficult to reconcile with experimental data suggesting that decision thresholds are inflexible. Using a network model of a cortical decision circuit, we demonstrate the SAT in a manner consistent with neural and behavioral data and with mathematical models that optimize speed and accuracy with respect to one another. In simulations of a reaction time task, we modulate the gain of the network with a signal encoding the urgency to respond. As the urgency signal builds up, the network progresses through a series of processing stages supporting noise filtering, integration of evidence, amplification of integrated evidence, and choice selection. Analysis of the network's dynamics formally characterizes this progression. Slower buildup of urgency increases accuracy by slowing down the progression. Faster buildup has the opposite effect. Because the network always progresses through the same stages, decision-selective firing rates are stereotyped at decision time.

Modeling the value of strategic actions in the superior colliculus.

In learning models of strategic game play, an agent constructs a valuation (action value) over possible future choices as a function of past actions and rewards. Choices are then stochastic functions of these action values. Our goal is to uncover a neural signal that correlates with the action value posited by behavioral learning models. We measured activity from neurons in the superior colliculus (SC), a midbrain region involved in planning saccadic eye movements, while monkeys performed two saccade tasks. In the strategic task, monkeys competed against a computer in a saccade version of the mixed-strategy game "matching-pennies". In the instructed task, saccades were elicited through explicit instruction rather than free choices. In both tasks neuronal activity and behavior were shaped by past actions and rewards with more recent events exerting a larger influence. Further, SC activity predicted upcoming choices during the strategic task and upcoming reaction times during the instructed task. Finally, we found that neuronal activity in both tasks correlated with an established learning model, the Experience Weighted Attraction model of action valuation (Camerer and Ho, 1999). Collectively, our results provide evidence that action values hypothesized by learning models are represented in the motor planning regions of the brain in a manner that could be used to select strategic actions.

Role of the superior colliculus in choosing mixed-strategy saccades.

Game theory outlines optimal response strategies during mixed-strategy competitions. The neural processes involved in choosing individual strategic actions, however, remain poorly understood. Here, we tested whether the superior colliculus (SC), a brain region critical for generating sensory-guided saccades, is also involved in choosing saccades under strategic conditions. Monkeys were free to choose either of two saccade targets as they competed against a computer opponent during the mixed-strategy game "matching pennies." The accuracy with which presaccadic SC activity predicted upcoming choice gradually increased in the time leading up to the saccade. Probing the SC with suprathreshold stimulation demonstrated that these evolving signals were functionally involved in preparing strategic saccades. Finally, subthreshold stimulation of the SC increased the likelihood that contralateral saccades were selected. Together, our results suggest that motor regions of the brain play an active role in choosing strategic actions rather than passively executing those prespecified by upstream executive regions.

Temporal and spatial allocation of motor preparation during a mixed-strategy game.

Adopting a mixed response strategy in competitive situations can prevent opponents from exploiting predictable play. What drives stochastic action selection is unclear given that choice patterns suggest that, on average, players are indifferent to available options during mixed-strategy equilibria. To gain insight into this stochastic selection process, we examined how motor preparation was allocated during a mixed-strategy game. If selection processes on each trial reflect a global indifference between options, then there should be no bias in motor preparation (unbiased preparation hypothesis). If, however, differences exist in the desirability of options on each trial then motor preparation should be biased toward the preferred option (biased preparation hypothesis). We tested between these alternatives by examining how saccade preparation was allocated as human subjects competed against an adaptive computer opponent in an oculomotor version of the game "matching pennies." Subjects were free to choose between two visual targets using a saccadic eye movement. Saccade preparation was probed by occasionally flashing a visual distractor at a range of times preceding target presentation. The probability that a distractor would evoke a saccade error, and when it failed to do so, the probability of choosing each of the subsequent targets quantified the temporal and spatial evolution of saccade preparation, respectively. Our results show that saccade preparation became increasingly biased as the time of target presentation approached. Specifically, the spatial locus to which saccade preparation was directed varied from trial to trial, and its time course depended on task timing.

Competitive integration of visual and preparatory signals in the superior colliculus during saccadic programming.

Efficient behavior requires that internally specified motor plans be integrated with incoming sensory information. Motor preparation and visual signals converge in the intermediate and deep layers of the superior colliculus (SC) to influence saccade planning and execution; however, the mechanism by which these sometimes conflicting signals are combined remains unclear. We studied this issue by presenting visual distractors as monkeys prepared saccades toward an upcoming target whose timing and location were fully predictable. Monkeys made more distractor-directed errors when the spatial location of visual distractors more closely coincided with the saccadic goal. Concomitant pretarget activity of SC visuomotor neurons, whose response fields were centered on the saccadic goal, was similarly increased by the presentation of nearby distractors and inhibited by the presentation of distant distractors. Finally, subthreshold microstimulation of the SC shifted the pattern of distractor-directed errors away from the saccadic goal toward that specified by the site of stimulation. Together, our results suggest that the likelihood of saccade generation is influenced by the spatial register of internal motor preparation signals and external sensory signals across the topographically organized SC map.

The influence of expected value on saccadic preparation.

Basing higher-order decisions on expected value (reward probability x reward magnitude) maximizes an agent's accruement of reward over time. The goal of this study was to determine whether the advanced preparation of simple actions reflected the expected value of the potential outcomes. Human subjects were required to direct a saccadic eye movement to a visual target that was presented either to the left or right of a central fixation point on each trial. Expected value was manipulated by adjusting the probability of presenting each target and their associated magnitude of monetary reward across 15 blocks of trials. We found that saccadic reaction times (SRTs) were negatively correlated to the relative expected value of the targets. Occasionally, an irrelevant visual distractor was presented before the target to probe the spatial allocation of saccadic preparation. Distractor-directed errors (oculomotor captures) varied as a function of the relative expected value of, and the distance of distractors from, the potential valued targets. SRTs and oculomotor captures were better correlated to the relative expected value of actions than to reward probability, reward magnitude, or overall motivation. Together, our results suggest that the level and spatial distribution of competitive dynamic neural fields representing saccadic preparation reflect the relative expected value of the potential actions.

Use of enclosures with functional vertical space by captive rhesus monkeys (Macaca mulatta) involved in biomedical research.

We assessed space use by 2 pairs of captive female rhesus monkeys recently transferred into 2 enclosures moderately larger than their former traditional research cages and providing elevated perches at or above human eye level for all monkeys. This new space did not affect the ongoing biomedical research in which these captive monkeys were involved, and we sought to determine whether they used the elevated positions preferentially, as do wild animals. The frequency and duration of visits at each of the 9 distinct regions within these enclosures was calculated during 30-min morning and evening sessions over 20 d. We found that the monkeys frequented all regions of their enclosures in a similar manner during both morning and evening sessions. However, the duration spent at each region varied significantly between morning and evening sessions, with high perches being chosen preferentially in the evenings. Overall, the monkeys spent the majority of their time at elevated positions. These results support the view that access to functional vertical space provides a preferred environment for species- specific behavior and is an option that should be considered by other research facilities.

Physiological utility theory and the neuroeconomics of choice.

Over the past half century economists have responded to the challenges of Allais [Econometrica (1953) 53], Ellsberg [Quart. J. Econ. (1961) 643] and others raised to neoclassicism either by bounding the reach of economic theory or by turning to descriptive approaches. While both of these strategies have been enormously fruitful, neither has provided a clear programmatic approach that aspires to a complete understanding of human decision making as did neoclassicism. There is, however, growing evidence that economists and neurobiologists are now beginning to reveal the physical mechanisms by which the human neuroarchitecture accomplishes decision making. Although in their infancy, these studies suggest both a single unified framework for understanding human decision making and a methodology for constraining the scope and structure of economic theory. Indeed, there is already evidence that these studies place mathematical constraints on existing economic models. This article reviews some of those constraints and suggests the outline of a neuroeconomic theory of decision.

Activity in posterior parietal cortex is correlated with the relative subjective desirability of action.

Behavioral studies suggest that making a decision involves representing the overall desirability of all available actions and then selecting that action that is most desirable. Physiological studies have proposed that neurons in the parietal cortex play a role in selecting movements for execution. To test the hypothesis that these parietal neurons encode the subjective desirability of making particular movements, we exploited Nash's game theoretic equilibrium, during which the subjective desirability of multiple actions should be equal for human players. Behavior measured during a strategic game suggests that monkeys' choices, like those of humans, are guided by subjective desirability. Under these conditions, activity in the parietal cortex was correlated with the relative subjective desirability of actions irrespective of the specific combination of reward magnitude, reward probability, and response probability associated with each action. These observations may help place many recent findings regarding the posterior parietal cortex into a common conceptual framework.

Contribution of the primate superior colliculus to inhibition of return.

The phenomenon of inhibition of return (IOR) has generated considerable interest in cognitive neuroscience because of its putative functional role in visual search, that of placing inhibitory tags on objects that have been recently inspected so as to direct further search to novel items. Many behavioral parameters of this phenomenon have been clearly delineated, and based on indirect but converging evidence, the widely held consensus is that the midbrain superior colliculus (SC) is involved in the generation of IOR. We had previously trained monkeys on a saccadic IOR task and showed that they displayed IOR in a manner similar to that observed in humans. Here we recorded the activity of single neurons in the superficial and intermediate layers of the SC while the monkeys performed this IOR task. We found that when the target was presented at a previously cued location, the stimulus-related response was attenuated and the magnitude of this response was correlated with subsequent saccadic reaction times. Surprisingly, this observed attenuation of activity during IOR was not caused by active inhibition of these neurons because (a) they were, in fact, more active following the presentation of the cue in their response field, and (b) when we repeated the same experiment while using the saccadic response time induced by electrical micro-stimulation of the SC to judge the level of excitability of the SC circuitry during the IOR task, we found faster saccades were elicited from the cued location. Our findings demonstrate that the primate SC participates in the expression of IOR; however, the SC is not the site of the inhibition. Instead, the reduced activity in the SC reflects a signal reduction that has taken place upstream.