Conditional independence as a statistical assessment of evidence integration processes.

Intuitively, combining multiple sources of evidence should lead to more accurate decisions than considering single sources of evidence individually. In practice, however, the proper computation may be difficult, or may require additional data that are inaccessible. Here, based on the concept of conditional independence, we consider expressions that can serve either as recipes for integrating evidence based on limited data, or as statistical benchmarks for characterizing evidence integration processes. Consider three events, A, B, and C. We find that, if A and B are conditionally independent with respect to C, then the probability that C occurs given that both A and B are known, P(C|A, B), can be easily calculated without the need to measure the full three-way dependency between A, B, and C. This simplified approach can be used in two general ways: to generate predictions by combining multiple (conditionally independent) sources of evidence, or to test whether separate sources of evidence are functionally independent of each other. These applications are demonstrated with four computer-simulated examples, which include detecting a disease based on repeated diagnostic testing, inferring biological age based on multiple biomarkers of aging, discriminating two spatial locations based on multiple cue stimuli (multisensory integration), and examining how behavioral performance in a visual search task depends on selection histories. Besides providing a sound prescription for predicting outcomes, this methodology may be useful for analyzing experimental data of many types.

Coupling of saccade plans to endogenous attention during urgent choices.

The neural mechanisms that willfully direct attention to specific locations in space are closely related to those for generating targeting eye movements (saccades). However, the degree to which the voluntary deployment of attention to a location is necessarily accompanied by a corresponding saccade plan remains unclear. One problem is that attention and saccades are both automatically driven by salient sensory events; another is that the underlying processes unfold within tens of milliseconds only. Here, we use an urgent task design to resolve the evolution of a visuomotor choice on a moment-by-moment basis while independently controlling the endogenous (goal-driven) and exogenous (salience-driven) contributions to performance. Human participants saw a peripheral cue and, depending on its color, either looked at it (prosaccade) or looked at a diametrically opposite, uninformative non-cue (antisaccade). By varying the luminance of the stimuli, the exogenous contributions could be cleanly dissociated from the endogenous process guiding the choice over time. According to the measured timecourses, generating a correct antisaccade requires about 30 ms more processing time than generating a correct prosaccade based on the same perceptual signal. The results indicate that saccade plans are biased toward the location where attention is endogenously deployed, but the coupling is weak and can be willfully overridden very rapidly.

Conditional independence as a statistical assessment of evidence integration processes.

Intuitively, combining multiple sources of evidence should lead to more accurate decisions than considering single sources of evidence individually. In practice, however, the proper computation may be difficult, or may require additional data that are inaccessible. Here, based on the concept of conditional independence, we consider expressions that can serve either as recipes for integrating evidence based on limited data, or as statistical benchmarks for characterizing evidence integration processes. Consider three events, A, B, and C. We find that, if A and B are conditionally independent with respect to C, then the probability that C occurs given that both A and B are known, PC|A,B, can be easily calculated without the need to measure the full three-way dependency between A, B, and C. This simplified approach can be used in two general ways: to generate predictions by combining multiple (conditionally independent) sources of evidence, or to test whether separate sources of evidence are functionally independent of each other. These applications are demonstrated with four computer-simulated examples, which include detecting a disease based on repeated diagnostic testing, inferring biological age based on multiple biomarkers of aging, discriminating two spatial locations based on multiple cue stimuli (multisensory integration), and examining how behavioral performance in a visual search task depends on selection histories. Besides providing a sound prescription for predicting outcomes, this methodology may be useful for analyzing experimental data of many types.

Stimulus salience conflicts and colludes with endogenous goals during urgent choices.

Selecting where to look next depends on both the salience of objects and current goals (what we are looking for), but discerning their relative contributions over the time frame of typical visuomotor decisions (200-250 ms) has been difficult. Here we investigate this problem using an urgent choice task with which the two contributions can be dissociated and tracked moment by moment. Behavioral data from three monkeys corresponded with model-based predictions: when salience favored the target, perceptual performance evolved rapidly and steadily toward an asymptotic level; when salience favored the distracter, many rapid errors were produced and the rise in performance took more time-effects analogous to oculomotor and attentional capture. The results show that salience has a brief (∼50 ms) but inexorable impact that leads to exogenous, involuntary capture, and this can either help or hinder performance, depending on the alignment between salience and ongoing internal goals.

A conflict between spatial selection and evidence accumulation in area LIP.

The lateral intraparietal area (LIP) contains spatially selective neurons that help guide eye movements and, according to numerous studies, do so by accumulating sensory evidence in favor of one choice (e.g., look left) or another (look right). To examine this functional link, we trained two monkeys on an urgent motion discrimination task, a task with which the evolution of both the recorded neuronal activity and the subject's choice can be tracked millisecond by millisecond. We found that while choice accuracy increased steeply with increasing sensory evidence, at the same time, the LIP selection signal became progressively weaker, as if it hindered performance. This effect was consistent with the transient deployment of spatial attention to disparate locations away from the relevant sensory cue. The results demonstrate that spatial selection in LIP is dissociable from, and may even conflict with, evidence accumulation during informed saccadic choices.

Exogenous capture accounts for fundamental differences between pro- and antisaccade performance.

To generate the next eye movement, oculomotor circuits take into consideration the physical salience of objects in view and current behavioral goals, exogenous and endogenous influences, respectively. However, the interactions between exogenous and endogenous mechanisms and their dynamic contributions to target selection have been difficult to resolve because they evolve extremely rapidly. In a recent study (Salinas et al., 2019), we achieved the necessary temporal precision using an urgent variant of the antisaccade task wherein motor plans are initiated early and choice accuracy depends sharply on when exactly the visual cue information becomes available. Empirical and modeling results indicated that the exogenous signal arrives ∼80 ms after cue onset and rapidly accelerates the (incorrect) plan toward the cue, whereas the informed endogenous signal arrives ∼25 ms later to favor the (correct) plan away from the cue. Here, we scrutinize a key mechanistic hypothesis about this dynamic, that the exogenous and endogenous signals act at different times and independently of each other. We test quantitative model predictions by comparing the performance of human participants instructed to look toward a visual cue or away from it under high urgency. We find that, indeed, the exogenous response is largely impervious to task instructions; it simply flips its sign relative to the correct choice, and this largely explains the drastic differences in psychometric performance between the two tasks. Thus, saccadic choices are strongly dictated by the alignment between salience and behavioral goals.

Under time pressure, the exogenous modulation of saccade plans is ubiquitous, intricate, and lawful.

The choice of where to look next is determined by both exogenous (bottom-up) and endogenous (top-down) factors, but details of their interaction and distinct contributions to target selection have remained elusive. Recent experiments with urgent choice tasks, in which stimuli are evaluated while motor plans are already advancing, have greatly clarified these contributions. Specifically, exogenous modulations associated with stimulus detection act rapidly and briefly (∼25 ms) to automatically halt and/or boost ongoing motor plans as per spatial congruence rules. These stereotypical modulations explain, in quantitative detail, characteristic features of many saccadic tasks (e.g. antisaccade, countermanding, saccadic-inhibition, gap, and double-step). Thus, the same low-level visuomotor interactions contribute to diverse oculomotor phenomena traditionally attributed to different neural mechanisms.

Urgent Decision Making: Resolving Visuomotor Interactions at High Temporal Resolution.

Measuring when exactly perceptual decisions are made is crucial for defining how the activation of specific neurons contributes to behavior. However, in traditional, nonurgent visuomotor tasks, the uncertainty of this temporal measurement is very large. This is a problem not only for delimiting the capacity of perception, but also for correctly interpreting the functional roles ascribed to choice-related neuronal responses. In this article, we review psychophysical, neurophysiological, and modeling work based on urgent visuomotor tasks in which this temporal uncertainty can be effectively overcome. The cornerstone of this work is a novel behavioral metric that describes the evolution of the subject's perceptual judgment moment by moment, allowing us to resolve numerous perceptual events that unfold within a few tens of milliseconds. In this framework, the neural distinction between perceptual evaluation and motor selection processes becomes particularly clear, as the conclusion of one is not contingent on that of the other.

Association Cortex Is Essential to Reverse Hemianopia by Multisensory Training.

Hemianopia induced by unilateral visual cortex lesions can be resolved by repeatedly exposing the blinded hemifield to auditory-visual stimuli. This rehabilitative "training" paradigm depends on mechanisms of multisensory plasticity that restore the lost visual responsiveness of multisensory neurons in the ipsilesional superior colliculus (SC) so that they can once again support vision in the blinded hemifield. These changes are thought to operate via the convergent visual and auditory signals relayed to the SC from association cortex (the anterior ectosylvian sulcus [AES], in cat). The present study tested this assumption by cryogenically deactivating ipsilesional AES in hemianopic, anesthetized cats during weekly multisensory training sessions. No signs of visual recovery were evident in this condition, even after providing animals with up to twice the number of training sessions required for effective rehabilitation. Subsequent training under the same conditions, but with AES active, reversed the hemianopia within the normal timeframe. These results indicate that the corticotectal circuit that is normally engaged in SC multisensory plasticity has to be operational for the brain to use visual-auditory experience to resolve hemianopia.

Multisensory Integration and the Society for Neuroscience: Then and Now.

The operation of our multiple and distinct sensory systems has long captured the interest of researchers from multiple disciplines. When the Society was founded 50 years ago to bring neuroscience research under a common banner, sensory research was largely divided along modality-specific lines. At the time, there were only a few physiological and anatomical observations of the multisensory interactions that powerfully influence our everyday perception. Since then, the neuroscientific study of multisensory integration has increased exponentially in both volume and diversity. From initial studies identifying the overlapping receptive fields of multisensory neurons, to subsequent studies of the spatial and temporal principles that govern the integration of multiple sensory cues, our understanding of this phenomenon at the single-neuron level has expanded to include a variety of dimensions. We now can appreciate how multisensory integration can alter patterns of neural activity in time, and even coordinate activity among populations of neurons across different brain areas. There is now a growing battery of sophisticated empirical and computational techniques that are being used to study this process in a number of models. These advancements have not only enhanced our understanding of this remarkable process in the normal adult brain, but also its underlying circuitry, requirements for development, susceptibility to malfunction, and how its principles may be used to mitigate malfunction.

Voluntary and involuntary contributions to perceptually guided saccadic choices resolved with millisecond precision.

In the antisaccade task, which is considered a sensitive assay of cognitive function, a salient visual cue appears and the participant must look away from it. This requires sensory, motor-planning, and cognitive neural mechanisms, but what are their unique contributions to performance, and when exactly are they engaged? Here, by manipulating task urgency, we generate a psychophysical curve that tracks the evolution of the saccadic choice process with millisecond precision, and resolve the distinct contributions of reflexive (exogenous) and voluntary (endogenous) perceptual mechanisms to antisaccade performance over time. Both progress extremely rapidly, the former driving the eyes toward the cue early on (∼100 ms after cue onset) and the latter directing them away from the cue ∼40 ms later. The behavioral and modeling results provide a detailed, dynamical characterization of attentional and oculomotor capture that is not only qualitatively consistent across participants, but also indicative of their individual perceptual capacities.

All-or-None Context Dependence Delineates Limits of FEF Visual Target Selection.

Choices of where to look are informed by perceptual judgments, which locate objects of current value or interest within the visual scene. This perceptual-motor transform is partly implemented in the frontal eye field (FEF), where visually responsive neurons appear to select behaviorally relevant visual targets and, subsequently, saccade-related neurons select the movements required to look at them. Here, we use urgent decision-making tasks to show (1) that FEF motor activity can direct accurate, visually informed choices in the complete absence of prior target-distracter discrimination by FEF visual responses and (2) that such discrimination by FEF visual cells shows an all-or-none reliance on the presence of stimulus attributes strongly associated with saliency-driven attentional allocation. The present findings suggest that FEF visual target selection is specific to visual judgments made on the basis of saliency and may not play a significant role in guiding saccadic choices informed solely by feature content.

Saccadic inhibition interrupts ongoing oculomotor activity to enable the rapid deployment of alternate movement plans.

Diverse psychophysical and neurophysiological results show that oculomotor networks are continuously active, such that plans for making the next eye movement are always ongoing. So, when new visual information arrives unexpectedly, how are those plans affected? At what point can the new information start guiding an eye movement, and how? Here, based on modeling and simulation results, we make two observations that are relevant to these questions. First, we note that many experiments, including those investigating the phenomenon known as "saccadic inhibition", are consistent with the idea that sudden-onset stimuli briefly interrupt the gradual rise in neural activity associated with the preparation of an impending saccade. And second, we show that this stimulus-driven interruption is functionally adaptive, but only if perception is fast. In that case, putting on hold an ongoing saccade plan toward location A allows the oculomotor system to initiate a concurrent, alternative plan toward location B (where a stimulus just appeared), deliberate (briefly) on the priority of each target, and determine which plan should continue. Based on physiological data, we estimate that the advantage of this strategy, relative to one in which any plan once initiated must be completed, is of several tens of milliseconds per saccade.

Saccade metrics reflect decision-making dynamics during urgent choices.

A perceptual judgment is typically characterized by constructing psychometric and chronometric functions, i.e., by mapping the accuracies and reaction times of motor choices as functions of a sensory stimulus feature dimension. Here, we show that various saccade metrics (e.g., peak velocity) are similarly modulated as functions of sensory cue viewing time during performance of an urgent-decision task. Each of the newly discovered functions reveals the dynamics of the perceptual evaluation process inherent to the underlying judgment. Remarkably, saccade peak velocity correlates with statistical decision confidence, suggesting that saccade kinematics reflect the degree of certainty with which an urgent perceptual decision is made. The data were explained by a race-to-threshold model that also replicates standard performance measures and cortical oculomotor neuronal activity in the task. The results indicate that, although largely stereotyped, saccade metrics carry subtle but reliable traces of the underlying cognitive processes that give rise to each oculomotor choice.

Motor selection dynamics in FEF explain the reaction time variance of saccades to single targets.

In studies of voluntary movement, a most elemental quantity is the reaction time (RT) between the onset of a visual stimulus and a saccade toward it. However, this RT demonstrates extremely high variability which, in spite of extensive research, remains unexplained. It is well established that, when a visual target appears, oculomotor activity gradually builds up until a critical level is reached, at which point a saccade is triggered. Here, based on computational work and single-neuron recordings from monkey frontal eye field (FEF), we show that this rise-to-threshold process starts from a dynamic initial state that already contains other incipient, internally driven motor plans, which compete with the target-driven activity to varying degrees. The ensuing conflict resolution process, which manifests in subtle covariations between baseline activity, build-up rate, and threshold, consists of fundamentally deterministic interactions, and explains the observed RT distributions while invoking only a small amount of intrinsic randomness.

When the simplest voluntary decisions appear patently suboptimal.

Rahnev & Denison (R&D) catalog numerous experiments in which performance deviates, often in subtle ways, from the theoretical ideal. We discuss an extreme case, an elementary behavior (reactive saccades to single targets) for which a simple contextual manipulation results in responses that are dramatically different from those expected based on reward maximization - and yet are highly informative and amenable to mechanistic examination.

Neural correlates of working memory development in adolescent primates.

Working memory ability matures after puberty, in parallel with structural changes in the prefrontal cortex, but little is known about how changes in prefrontal neuronal activity mediate this cognitive improvement in primates. To address this issue, we compare behavioural performance and neurophysiological activity in monkeys as they transitioned from puberty into adulthood. Here we report that monkeys perform working memory tasks reliably during puberty and show modest improvement in adulthood. The adult prefrontal cortex is characterized by increased activity during the delay period of the task but no change in the representation of stimuli. Activity evoked by distracting stimuli also decreases in the adult prefrontal cortex. The increase in delay period activity relative to the baseline activity of prefrontal neurons is the best correlate of maturation and is not merely a consequence of improved performance. Our results reveal neural correlates of the working memory improvement typical of primate adolescence.

Behavioral response inhibition and maturation of goal representation in prefrontal cortex after puberty.

Executive functions including behavioral response inhibition mature after puberty, in tandem with structural changes in the prefrontal cortex. Little is known about how activity of prefrontal neurons relates to this profound cognitive development. To examine this, we tracked neuronal responses of the prefrontal cortex in monkeys as they transitioned from puberty into adulthood and compared activity at different developmental stages. Performance of the antisaccade task greatly improved in this period. Among neural mechanisms that could facilitate it, reduction of stimulus-driven activity, increased saccadic activity, or enhanced representation of the opposing goal location, only the latter was evident in adulthood. Greatly accentuated in adults, this neural correlate of vector inversion may be a prerequisite to the formation of a motor plan to look away from the stimulus. Our results suggest that the prefrontal mechanisms that underlie mature performance on the antisaccade task are more strongly associated with forming an alternative plan of action than with suppressing the neural impact of the prepotent stimulus.

Task dependence of decision- and choice-related activity in monkey oculomotor thalamus.

Oculomotor signals circulate within putative recurrent feedback loops that include the frontal eye field (FEF) and the oculomotor thalamus (OcTh). To examine how OcTh contributes to visuomotor control, and perceptually informed saccadic choices in particular, neural correlates of perceptual judgment and motor selection in OcTh were evaluated and compared with those previously reported for FEF in the same subjects. Monkeys performed three tasks: a choice task in which perceptual decisions are urgent, a choice task in which identical decisions are made without time pressure, and a single-target, delayed saccade task. The OcTh yielded far fewer task-responsive neurons than the FEF, but across responsive pools, similar neuron types were found, ranging from purely visual to purely saccade related. Across such types, the impact of the perceptual information relevant to saccadic choices was qualitatively the same in FEF and OcTh. However, distinct from that in FEF, activity in OcTh was strongly task dependent, typically being most vigorous in the urgent task, less so in the easier choice task, and least in the single-target task. This was true for responsive and nonresponsive cells alike. Neurons with exclusively motor-related activity showed strong task dependence, fired less, and differed most patently from their FEF counterparts, whereas those that combined visual and motor activity fired most similarly to their FEF counterparts. The results suggest that OcTh activity is more distantly related to saccade production per se, because its degree of commitment to a motor choice varies markedly as a function of ongoing cognitive or behavioral demands.

Vision: a moving hill for spatial updating on the fly.

A recent study reveals a dynamic neural map that provides a continuous representation of remembered visual stimulus locations with respect to constantly changing gaze. This finding suggests a new mechanistic framework for understanding the spatiotemporal dynamics of goal-directed action.