Behavioral and ERP evidence that object-based attention utilizes fine-grained spatial mechanisms.

Valdes-Sosa, Cobo, and Pinilla (1998) introduced a transparent-motion design that provided evidence of object-based attention whereby attention embraces all features of an attentionally cued perceptual object including new unpredictable features such as a brief translation. Subsequent studies using variants of that design appeared to provide further behavioral, electrophysiological, and brain imaging evidence of object-based attention. Stoner and Blanc (2010) observed, however, that these previous results could potentially be explained by feature-based competition/normalization models of attention. To distinguish between the object-based and feature-based accounts, they introduced "feature swaps" into a delayed-onset variant of the transparent-motion design (Reynolds, Alborzian, & Stoner, 2003). Whereas the object-based attention account predicted that the effect of cueing would survive these feature swaps, the motion-competition account predicted that the effect of cueing would be reversed by these feature swaps. The behavioral results of Stoner and Blanc (2010) supported the object-based account, and in doing so, provided evidence that the attentional advantage in this design is spatially selective at the scale of the intermixed texture elements (i.e., dots) of the overlapping and moving dot fields. In the present study, we used the design of Stoner and Blanc (2010) to investigate both psychophysical performance and evoked activities under different cueing and feature swapping conditions. We confirmed that the behavioral effects of attentional cueing survived feature swaps and found event-related potential (ERP) correlates of those effects in the N1 component range over occipital and parieto-occipital scalp sites. These modulations of the neural activity were, moreover, significantly associated with variation in behavioral performance values across the different conditions. Our findings thus provide the first evidence of the role of the N1 component in object-based attention in this transparent-motion design under conditions that rule out feature-based mechanisms and that reveal selective processing at a fine spatial scale.

Functionally specific optogenetic modulation in primate visual cortex.

In primates, visual perception is mediated by brain circuits composed of submillimeter nodes linked together in specific networks that process different types of information, such as eye specificity and contour orientation. We hypothesized that optogenetic stimulation targeted to cortical nodes could selectively activate such cortical networks. We used viral transfection methods to confer light sensitivity to neurons in monkey primary visual cortex. Using intrinsic signal optical imaging and single-unit electrophysiology to assess effects of targeted optogenetic stimulation, we found that (i) optogenetic stimulation of single ocular dominance columns (eye-specific nodes) revealed preferential activation of nearby same-eye columns but not opposite-eye columns, and (ii) optogenetic stimulation of single orientation domains increased visual response of matching orientation domains and relatively suppressed nonmatching orientation selectivity. These findings demonstrate that optical stimulation of single nodes leads to modulation of functionally specific cortical networks related to underlying neural architecture.

Auditory modulation of spiking activity and local field potentials in area MT does not appear to underlie an audiovisual temporal illusion.

The timing of brief stationary sounds has been shown to alter the perceived speed of visual apparent motion (AM), presumably by altering the perceived timing of the individual frames of the AM stimuli and/or the duration of the interstimulus intervals (ISIs) between those frames. To investigate the neural correlates of this "temporal ventriloquism" illusion, we recorded spiking and local field potential (LFP) activity from the middle temporal area (area MT) in awake, fixating macaques. We found that the spiking activity of most MT neurons (but not the LFP) was tuned for the ISI/speed (these parameters covaried) of our AM stimuli but that auditory timing had no effect on that tuning. We next asked whether the predicted changes in perceived timing were reflected in the timing of neuronal responses to the individual frames of the AM stimuli. Although spiking dynamics were significantly, if weakly, affected by auditory timing in a minority of neurons, the timing of spike responses did not systematically mirror the predicted perception of stimuli. Conversely, the duration of LFP responses in ß- and γ-frequency bands was qualitatively consistent with human perceptual reports. We discovered, however, that LFP responses to auditory stimuli presented alone were robust and that responses to audiovisual stimuli were predicted by the linear sum of responses to auditory and visual stimuli presented individually. In conclusion, we find evidence of auditory input into area MT but not of the nonlinear audiovisual interactions we had hypothesized to underlie the illusion. NEW & NOTEWORTHY We utilized a set of audiovisual stimuli that elicit an illusion demonstrating "temporal ventriloquism" in visual motion and that have spatiotemporal intervals for which neurons within the middle temporal area are selective. We found evidence of auditory input into the middle temporal area but not of the nonlinear audiovisual interactions underlying this illusion. Our findings suggest that either the illusion was absent in our nonhuman primate subjects or the neuronal correlates of this illusion lie within other areas.

Nonhuman primate model of schizophrenia using a noninvasive EEG method.

There is growing evidence that impaired sensory-processing significantly contributes to the cognitive deficits found in schizophrenia. For example, the mismatch negativity (MMN) and P3a event-related potentials (ERPs), neurophysiological indices of sensory and cognitive function, are reduced in schizophrenia patients and may be used as biomarkers of the disease. In agreement with glutamatergic theories of schizophrenia, NMDA antagonists, such as ketamine, elicit many symptoms of schizophrenia when administered to normal subjects, including reductions in the MMN and the P3a. We sought to develop a nonhuman primate (NHP) model of schizophrenia based on NMDA-receptor blockade using subanesthetic administration of ketamine. This provided neurophysiological measures of sensory and cognitive function that were directly comparable to those recorded from humans. We first developed methods that allowed recording of ERPs from humans and rhesus macaques and found homologous MMN and P3a ERPs during an auditory oddball paradigm. We then investigated the effect of ketamine on these ERPs in macaques. As found in humans with schizophrenia, as well as in normal subjects given ketamine, we observed a significant decrease in amplitude of both ERPs. Our findings suggest the potential of a pharmacologically induced model of schizophrenia in NHPs that can pave the way for EEG-guided investigations into cellular mechanisms and therapies. Furthermore, given the established link between these ERPs, the glutamatergic system, and deficits in other neuropsychiatric disorders, our model can be used to investigate a wide range of pathologies.

Stimulus-dependent variability and noise correlations in cortical MT neurons.

Population codes assume that neural systems represent sensory inputs through the firing rates of populations of differently tuned neurons. However, trial-by-trial variability and noise correlations are known to affect the information capacity of neural codes. Although recent studies have shown that stimulus presentation reduces both variability and rate correlations with respect to their spontaneous level, possibly improving the encoding accuracy, whether these second order statistics are tuned is unknown. If so, second-order statistics could themselves carry information, rather than being invariably detrimental. Here we show that rate variability and noise correlation vary systematically with stimulus direction in directionally selective middle temporal (MT) neurons, leading to characteristic tuning curves. We show that such tuning emerges in a stochastic recurrent network, for a set of connectivity parameters that overlaps with a single-state scenario and multistability. Information theoretic analysis shows that second-order statistics carry information that can improve the accuracy of the population code.

Optogenetics through windows on the brain in the nonhuman primate.

Optogenetics combines optics and genetics to control neuronal activity with cell-type specificity and millisecond temporal precision. Its use in model organisms such as rodents, Drosophila, and Caenorhabditis elegans is now well-established. However, application of this technology in nonhuman primates (NHPs) has been slow to develop. One key challenge has been the delivery of viruses and light to the brain through the thick dura mater of NHPs, which can only be penetrated with large-diameter devices that damage the brain. The opacity of the NHP dura prevents visualization of the underlying cortex, limiting the spatial precision of virus injections, electrophysiological recordings, and photostimulation. Here, we describe a new optogenetics approach in which the native dura is replaced with an optically transparent artificial dura. This artificial dura can be penetrated with fine glass micropipettes, enabling precisely targeted injections of virus into brain tissue with minimal damage to cortex. The expression of optogenetic agents can be monitored visually over time. Most critically, this optical window permits targeted, noninvasive photostimulation and concomitant measurements of neuronal activity via intrinsic signal imaging and electrophysiological recordings. We present results from both anesthetized-paralyzed (optical imaging) and awake-behaving NHPs (electrophysiology). The improvements over current methods made possible by the artificial dura should enable the widespread use of optogenetic tools in NHP research, a key step toward the development of therapies for neuropsychiatric and neurological diseases in humans.

Static sound timing alters sensitivity to low-level visual motion.

Visual motion processing is essential to survival in a dynamic world and is probably the best-studied facet of visual perception. It has been recently discovered that the timing of brief static sounds can bias visual motion perception, an effect attributed to "temporal ventriloquism" whereby the timing of the sounds "captures" the timing of the visual events. To determine whether this cross-modal interaction is dependent on the involvement of higher-order attentive tracking mechanisms, we used near-threshold motion stimuli that isolated low-level pre-attentive visual motion processing. We found that the timing of brief sounds altered sensitivity to these visual motion stimuli in a manner that paralleled changes in the timing of the visual stimuli. Our findings indicate that auditory timing impacts visual motion processing very early in the processing hierarchy and without the involvement of higher-order attentional and/or position tracking mechanisms.

Object-based attention to one of two superimposed surfaces alters responses in human early visual cortex.

Faced with an overwhelming amount of sensory information, we are able to prioritize the processing of select spatial locations and visual features. The neuronal mechanisms underlying such spatial and feature-based selection have been studied in considerable detail. More recent work shows that attention can also be allocated to objects, even spatially superimposed objects composed of dynamically changing features that must be integrated to create a coherent object representation. Much less is known about the mechanisms underlying such object-based selection. Our goal was to investigate behavioral and neuronal responses when attention was directed to one of two objects, specifically one of two superimposed transparent surfaces, in a task designed to preclude space-based and feature-based selection. We used functional magnetic resonance imaging (fMRI) to measure changes in blood oxygen level-dependent (BOLD) signals when attention was deployed to one or the other surface. We found that visual areas V1, V2, V3, V3A, and MT+ showed enhanced BOLD responses to translations of an attended relative to an unattended surface. These results reveal that visual areas as early as V1 can be modulated by attending to objects, even objects defined by dynamically changing elements. This provides definitive evidence in humans that early visual areas are involved in a seemingly high-order process. Furthermore, our results suggest that these early visual areas may participate in object-specific feature "binding," a process that seemingly must occur for an object or a surface to be the unit of attentional selection.

Auditory modulation of visual apparent motion with short spatial and temporal intervals.

Recently, E. Freeman and J. Driver (2008) reported a cross-modal temporal interaction in which brief sounds drive the perceived direction of visual apparent-motion, an effect they attributed to "temporal capture" of the visual stimuli by the sounds (S. Morein-Zamir, S. Soto-Faraco, & A. Kingstone, 2003). Freeman and Driver used "long-range" visual motion stimuli, which travel over long spatial and temporal intervals and engage high-order cortical areas (K. G. Claeys, D. T. Lindsey, E. De Schutter, & G. A. Orban, 2003; Y. Zhuo et al., 2003). We asked whether Freeman and Driver's temporal effects extended to the short-range apparent-motion stimuli that engage cortical area MT, a lower-order area with well-established spatiotemporal selectivity for visual motion (e.g. A. Mikami, 1991, 1992; A. Mikami, W. T. Newsome, & R. H. Wurtz, 1986a, 1986b; W. T. Newsome, A. Mikami, & R. H. Wurtz, 1986). Consistent with a temporal-capture account, we found that static sounds bias the perception of both the direction (Experiment 1) and the speed (Experiment 2) of short-range motion. Our results suggest that auditory timing may interact with visual spatiotemporal processing as early as cortical area MT. Examination of the neuronal responses of this well-studied area to the stimuli used in this study would provide a test and might provide insight into the neuronal representation of time.

Exploring the mechanisms underlying surface-based stimulus selection.

Valdes-Sosa, et al. (2000) introduced a transparent-motion design that provides evidence of surface-based processing of visual motion. We show that this design suffers from a motion-duration confound that admits an alternative explanation based on neuronal adaptation and competition. We tested this explanation by reversing the relationship between motion duration and which perceptual surface was "cued". We also examined the role of color duration. Our findings support the surface-based account and, more specifically, demonstrate that this type of surface-based selection involves selective spatial processing at the scale of the texture elements that define the transparent surfaces.

Stimulus dependency and mechanisms of surround modulation in cortical area MT.

The perceptual interpretation of a given visual feature depends on the surrounding context. To explore the neural mechanisms underlying such contextual interactions in the motion domain, we studied responses of neurons in the middle temporal area (MT) of macaque monkeys while presenting a variety of center-surround stimuli that stimulated both the classical receptive visual field (CRF) and the receptive field surround. In human psychophysical experiments, the perceptual impact of the surround stimulus on the center stimulus varied from motion capture ("integration") to motion contrast ("segmentation"). In our neurophysiological experiments, the directional tuning of surround modulation with these stimuli ranged from antagonistic (consistent with motion contrast) to integrative (consistent with motion capture) and agreed qualitatively with perception under some but not all conditions. Most strikingly, for a stimulus that elicited perceptual motion contrast, surround modulation was integrative if the CRF stimulus was ambiguous due to the aperture problem. In addition, we found that surround modulation was linked to response magnitude: stimuli eliciting the largest responses yielded the strongest antagonism and those eliciting the smallest responses yielded the strongest integration. We developed a neural network model that accounts for this finding as well as a previous finding that surround suppression in area MT is contrast-dependent. Our findings suggest that changes in MT surround modulation result from shifts in the balance between directionally tuned excitation and inhibition mediated by changes in input strength. We speculate that input strength is, in turn, linked with the ambiguity of the motion present within the CRF.

Occlusion and the solution to visual motion ambiguity: Looking beyond the aperture problem.

A horizontally moving grating viewed within a diamond-shaped aperture can be made to appear to move obliquely by introducing appropriate depth-ordering cues (R. O. Duncan, T. D. Albright, & G. R. Stoner, 2000). It is commonly assumed that the depth cues in such displays determine which line terminators are seen as intrinsic to the grating and which are seen as resulting from occlusion and hence extrinsic to the grating. The ambiguous motion of the grating (arising from the aperture problem) is then supposed to be overcome by selectively pooling motion signals arising from the intrinsic terminators with those arising from the grating while discounting the motion of the extrinsic terminators. In our first experiment, we tested the sufficiency of this explanation. Observers reported the direction of motion of ambiguously moving random dots viewed through a diamond-shaped aperture defined by four panels. Binocular disparity was used to simulate occlusion: two panels occluded the virtual surface upon which the dots were positioned and two panels were occluded by that surface. Reports were significantly biased toward the direction of the occluding panels. Since none of the moving features abutted the surrounding panels, none should have been classified as extrinsic and hence this result cannot have relied on terminator classification. In a second experiment, we tested the hypothesis that depth-ordering cues selectively gate the propagation of motion signals so that the representation of the moving surface extends behind the occluders. This was tested by asking observers to report the direction of moving dots viewed through a briefly "opened" probe window within either occluding or occluded panels. Consistent with our hypothesis, evidence of motion propagation was only found for probe windows within occluding panels. Surprisingly, however, this propagation was only observed when the dots in the inducer window moved away from the probe window, suggesting a "pull," and not a "push" mechanism.

Adaptive surround modulation in cortical area MT.

Visual motion perception relies on two opposing operations: integration and segmentation. Integration overcomes motion ambiguity in the visual image by spatial pooling of motion signals, whereas segmentation identifies differences between adjacent moving objects. For visual motion area MT, previous investigations have reported that stimuli in the receptive field surround, which do not elicit a response when presented alone, can nevertheless modulate responses to stimuli in the receptive field center. The directional tuning of this "surround modulation" has been found to be mainly antagonistic and hence consistent with segmentation. Here, we report that surround modulation in area MT can be either antagonistic or integrative depending upon the visual stimulus. Both types of modulation were delayed relative to response onset. Our results suggest that the dominance of antagonistic modulation in previous MT studies was due to stimulus choice and that segmentation and integration are achieved, in part, via adaptive surround modulation.

Stimulus-specific competitive selection in macaque extrastriate visual area V4.

Macaque visual area V4 has been implicated in the selective processing of stimuli. Prior studies of selection in area V4 have used spatially separate stimuli, thus confounding selection of retinotopic location with selection of the stimulus at that location. We asked whether V4 neurons can selectively respond to one of two differently colored stimuli even when they are spatially superimposed. We find that delaying one of the two stimuli leads to selective processing of the delayed stimulus by area V4 neurons. This selective processing persists when the stimuli move together across the visual field, thereby successively activating different populations of neurons. We also find that this effect is not a spatially global form of feature-based selection. We conclude that selective processing in area V4 is neither exclusively spatial nor feature-based and may thus be surface- or object-based.

Adaptation of gender derived from biological motion.

Human observers adapted to complex biological motions that distinguish males from females: viewing the gait of one gender biased judgments of subsequent gaits toward the opposite gender. This adaptation was not simply due to local features of the stimuli but instead relied upon the global motion of the figures. These results suggest the existence of neurons selective for gender and demonstrate that gender-from-motion judgments are not fixed but depend upon recent viewing history.

Interacting competitive selection in attention and binocular rivalry.

Visuomotor processing is selective - only a small subset of stimuli that impinge on the retinae reach perceptual awareness and/or elicit behavioral responses. Both binocular rivalry and attention involve visual selection, but affect perception quite differently. During rivalry, awareness alternates between different stimuli presented to the two eyes. In contrast, attending to one of the two stimuli impairs discrimination of the ignored stimulus, but without causing it to perceptually disappear. We review experiments demonstrating that, despite their phenomenological differences, attention and rivalry depend upon shared competitive selection mechanisms. These experiments, moreover, reveal stimulus selection that is surface-based and requires coordination between the different neuronal populations that respond as a surface changes its attributes (type of motion) over time. This surface-based selection, in turn biases interocular competition, favoring the eye whose image is consistent with the selected surface. The review ends with speculation about the role of the thalamus in mediating this dynamic coordination, as well as thoughts about what underlies the differences in the phenomenology of selective attention and rivalry.

Second-order motion conveys depth-order information.

Psychophysical and neurophysiological studies have revealed that the visual system is sensitive to both "first-order" motion, in which moving features are defined by luminance cues, and "second-order" motion, in which motion is defined by nonluminance cues, such as contrast or flicker. Here we show psychophysically that common types of second-order stimuli provide potent cues to depth order. Although motion defined exclusively by nonluminance cues may be relatively rare in natural scenes, the depth-order cues offered by second-order stimuli arise ubiquitously as a result of occlusion of one moving object by another. Our results thus shed new light on the ecological importance of second-order motion. Furthermore, our results imply that visual cortical areas that have been shown to be responsive to second-order motion may be extracting information not just about object motion as has been assumed, but also about the relative depth of objects.

Object-based attention determines dominance in binocular rivalry.

A question of long-standing interest to philosophers, psychologists and neuroscientists is how the brain selects which signals enter consciousness. Binocular rivalry and attention both involve selection of visual stimuli, but affect perception quite differently. During binocular rivalry, awareness alternates between two different stimuli presented to the two eyes. In contrast, attending to one of two different stimuli impairs discrimination of the ignored stimulus, but without causing it to disappear from consciousness. Here we show that despite this difference, attention and rivalry rely on shared object-based selection mechanisms. We cued attention to one of two superimposed transparent surfaces and then deleted the image of one surface from each eye, resulting in rivalry. Observers usually reported seeing only the cued surface. They were also less accurate in judging unpredictable changes in the features of the uncued surface. Our design ensured that selection of the cued surface could not have resulted from spatial, ocular or feature-based mechanisms. Rather, attention was drawn to one surface, and this caused the other surface to be perceptually suppressed during rivalry. These results raise the question of how object representations compete during these two forms of perceptual selection, even as the features of those objects change unpredictably over time.

Attentional selection of superimposed surfaces cannot be explained by modulation of the gain of color channels.

When two differently colored, superimposed patterns of dots rotate in opposite directions, this yields the percept of two superimposed transparent surfaces. If observers are cued to attend to one set of dots, they are impaired in making judgments about the other set. Since the two sets of dots are overlapping, the cueing effect cannot be explained by spatial attention. This has led to the interpretation that the impairment reflects surface-based attentional selection. However, recent single-unit recording studies in monkeys have found that attention can modulate the gain of neurons tuned for features such as color. Thus, rather than reflecting the selection of a surface, the behavioral effects might simply reflect a reduction in the gain of color channels selective for the color of the uncued set of dots (feature-based attention), as if viewing the surfaces through a colored filter. If so, then the impairment should be eliminated when the two surfaces are made the same color. Instead, we find that the impairment persists with no reduction in strength. Our findings thus rule out the color gain explanation.