Image-dependence of the detectability of optogenetic stimulation in macaque inferotemporal cortex.

Artificial activation of neurons in early visual areas induces perception of simple visual flashes.1,2 Accordingly, stimulation in high-level visual cortices is expected to induce perception of complex features.3,4 However, results from studies in human patients challenge this expectation. Stimulation rarely induces any detectable visual event, and never a complex one, in human subjects with closed eyes.2 Stimulation of the face-selective cortex in a human patient led to remarkable hallucinations only while the subject was looking at faces.5 In contrast, stimulations of color- and face-selective sites evoke notable hallucinations independent of the object being viewed.6 These anecdotal observations suggest that stimulation of high-level visual cortex can evoke perception of complex visual features, but these effects depend on the availability and content of visual input. In this study, we introduce a novel psychophysical task to systematically investigate characteristics of the perceptual events evoked by optogenetic stimulation of macaque inferior temporal (IT) cortex. We trained macaque monkeys to detect and report optogenetic impulses delivered to their IT cortices7,8,9 while holding fixation on object images. In a series of experiments, we show that detection of cortical stimulation is highly dependent on the choice of images presented to the eyes and it is most difficult when fixating on a blank screen. These findings suggest that optogenetic stimulation of high-level visual cortex results in easily detectable distortions of the concurrent contents of vision.

Behavioral detectability of optogenetic stimulation of inferior temporal cortex varies with the size of concurrently viewed objects.

We have previously demonstrated that macaque monkeys can behaviorally detect a subtle optogenetic impulse delivered to their inferior temporal (IT) cortex. We have also shown that the ability to detect the cortical stimulation impulse varies depending on some characteristics of the visual images viewed at the time of brain stimulation, revealing the visual nature of the perceptual events induced by stimulation of the IT cortex. Here we systematically studied the effect of the size of viewed objects on behavioral detectability of optogenetic stimulation of the central IT cortex. Surprisingly, we found that behavioral detection of the same optogenetic impulse highly varies with the size of the viewed object images. Reduction of the object size in four steps from 8 to 1 degree of visual angle significantly decreased detection performance. These results show that identical stimulation impulses delivered to the same neural population induce variable perceptual events depending on the mere size of the objects viewed at the time of brain stimulation.

Paradoxical impact of memory on color appearance of faces.

What is color vision for? Here we compared the extent to which memory modulates color appearance of objects and faces. Participants matched the colors of stimuli illuminated by low-pressure sodium light, which renders scenes monochromatic. Matches for fruit were not predicted by stimulus identity. In contrast, matches for faces were predictable, but surprising: faces appeared green and looked sick. The paradoxical face-color percept could be explained by a Bayesian observer model constrained by efficient coding. The color-matching data suggest that the face-color prior is established by visual signals arising from the recently evolved L-M cone system, not the older S-cone channel. Taken together, the results show that when retinal mechanisms of color vision are impaired, the impact of memory on color perception is greatest for face color, supporting the idea that trichromatic color plays an important role in social communication.

Facephenes and rainbows: Causal evidence for functional and anatomical specificity of face and color processing in the human brain.

Neuroscientists have long debated whether some regions of the human brain are exclusively engaged in a single specific mental process. Consistent with this view, fMRI has revealed cortical regions that respond selectively to certain stimulus classes such as faces. However, results from multivoxel pattern analyses (MVPA) challenge this view by demonstrating that category-selective regions often contain information about "nonpreferred" stimulus dimensions. But is this nonpreferred information causally relevant to behavior? Here we report a rare opportunity to test this question in a neurosurgical patient implanted for clinical reasons with strips of electrodes along his fusiform gyri. Broadband gamma electrocorticographic responses in multiple adjacent electrodes showed strong selectivity for faces in a region corresponding to the fusiform face area (FFA), and preferential responses to color in a nearby site, replicating earlier reports. To test the causal role of these regions in the perception of nonpreferred dimensions, we then electrically stimulated individual sites while the patient viewed various objects. When stimulated in the FFA, the patient reported seeing an illusory face (or "facephene"), independent of the object viewed. Similarly, stimulation of color-preferring sites produced illusory "rainbows." Crucially, the patient reported no change in the object viewed, apart from the facephenes and rainbows apparently superimposed on them. The functional and anatomical specificity of these effects indicate that some cortical regions are exclusively causally engaged in a single specific mental process, and prompt caution about the widespread assumption that any information scientists can decode from the brain is causally relevant to behavior.

#TheDress: Categorical perception of an ambiguous color image.

We present a full analysis of data from our preliminary report (Lafer-Sousa, Hermann, & Conway, 2015) and test whether #TheDress image is multistable. A multistable image must give rise to more than one mutually exclusive percept, typically within single individuals. Clustering algorithms of color-matching data showed that the dress was seen categorically, as white/gold (W/G) or blue/black (B/K), with a blue/brown transition state. Multinomial regression predicted categorical labels. Consistent with our prior hypothesis, W/G observers inferred a cool illuminant, whereas B/K observers inferred a warm illuminant; moreover, subjects could use skin color alone to infer the illuminant. The data provide some, albeit weak, support for our hypothesis that day larks see the dress as W/G and night owls see it as B/K. About half of observers who were previously familiar with the image reported switching categories at least once. Switching probability increased with professional art experience. Priming with an image that disambiguated the dress as B/K biased reports toward B/K (priming with W/G had negligible impact); furthermore, knowledge of the dress's true colors and any prior exposure to the image shifted the population toward B/K. These results show that some people have switched their perception of the dress. Finally, consistent with a role of attention and local image statistics in determining how multistable images are seen, we found that observers tended to discount as achromatic the dress component that they did not attend to: B/K reporters focused on a blue region, whereas W/G reporters focused on a golden region.

Color-Biased Regions of the Ventral Visual Pathway Lie between Face- and Place-Selective Regions in Humans, as in Macaques.

The existence of color-processing regions in the human ventral visual pathway (VVP) has long been known from patient and imaging studies, but their location in the cortex relative to other regions, their selectivity for color compared with other properties (shape and object category), and their relationship to color-processing regions found in nonhuman primates remain unclear. We addressed these questions by scanning 13 subjects with fMRI while they viewed two versions of movie clips (colored, achromatic) of five different object classes (faces, scenes, bodies, objects, scrambled objects). We identified regions in each subject that were selective for color, faces, places, and object shape, and measured responses within these regions to the 10 conditions in independently acquired data. We report two key findings. First, the three previously reported color-biased regions (located within a band running posterior-anterior along the VVP, present in most of our subjects) were sandwiched between face-selective cortex and place-selective cortex, forming parallel bands of face, color, and place selectivity that tracked the fusiform gyrus/collateral sulcus. Second, the posterior color-biased regions showed little or no selectivity for object shape or for particular stimulus categories and showed no interaction of color preference with stimulus category, suggesting that they code color independently of shape or stimulus category; moreover, the shape-biased lateral occipital region showed no significant color bias. These observations mirror results in macaque inferior temporal cortex (Lafer-Sousa and Conway, 2013), and taken together, these results suggest a homology in which the entire tripartite face/color/place system of primates migrated onto the ventral surface in humans over the course of evolution. SIGNIFICANCE STATEMENT: Here we report that color-biased cortex is sandwiched between face-selective and place-selective cortex on the bottom surface of the brain in humans. This face/color/place organization mirrors that seen on the lateral surface of the temporal lobe in macaques, suggesting that the entire tripartite system is homologous between species. This result validates the use of macaques as a model for human vision, making possible more powerful investigations into the connectivity, precise neural codes, and development of this part of the brain. In addition, we find substantial segregation of color from shape selectivity in posterior regions, as observed in macaques, indicating a considerable dissociation of the processing of shape and color in both species.

Striking individual differences in color perception uncovered by 'the dress' photograph.

'The dress' is a peculiar photograph: by themselves the dress' pixels are brown and blue, colors associated with natural illuminants, but popular accounts (#TheDress) suggest the dress appears either white/gold or blue/black. Could the purported categorical perception arise because the original social-media question was an alternative-forced-choice? In a free-response survey (N = 1401), we found that most people, including those naïve to the image, reported white/gold or blue/black, but some said blue/brown. Reports of white/gold over blue/black were higher among older people and women. On re-test, some subjects reported a switch in perception, showing the image can be multistable. In a language-independent measure of perception, we asked subjects to identify the dress' colors from a complete color gamut. The results showed three peaks corresponding to the main descriptive categories, providing additional evidence that the brain resolves the image into one of three stable percepts. We hypothesize that these reflect different internal priors: some people favor a cool illuminant (blue sky), discount shorter wavelengths, and perceive white/gold; others favor a warm illuminant (incandescent light), discount longer wavelengths, and see blue/black. The remaining subjects may assume a neutral illuminant, and see blue/brown. We show that by introducing overt cues to the illumination, we can flip the dress color.

Binocular stereoscopy in visual areas V-2, V-3, and V-3A of the macaque monkey.

Over 40 years ago, Hubel and Wiesel gave a preliminary report of the first account of cells in monkey cerebral cortex selective for binocular disparity. The cells were located outside of V-1 within a region referred to then as "area 18." A full-length manuscript never followed, because the demarcation of the visual areas within this region had not been fully worked out. Here, we provide a full description of the physiological experiments and identify the locations of the recorded neurons using a contemporary atlas generated by functional magnetic resonance imaging; we also perform an independent analysis of the location of the neurons relative to an anatomical landmark (the base of the lunate sulcus) that is often coincident with the border between V-2 and V-3. Disparity-tuned cells resided not only in V-2, the area now synonymous with area 18, but also in V-3 and probably within V-3A. The recordings showed that the disparity-tuned cells were biased for near disparities, tended to prefer vertical orientations, clustered by disparity preference, and often required stimulation of both eyes to elicit responses, features strongly suggesting a role in stereoscopic depth perception.

Color-detection thresholds in rhesus macaque monkeys and humans.

Macaque monkeys are a model of human color vision. To facilitate linking physiology in monkeys with psychophysics in humans, we directly compared color-detection thresholds in humans and rhesus monkeys. Colors were defined by an equiluminant plane of cone-opponent color space. All subjects were tested on an identical apparatus with a four-alternative forced-choice task. Targets were 2° square, centered 2° from fixation, embedded in luminance noise. Across all subjects, the change in detection thresholds from initial testing to plateau performance ("learning") was similar for +L − M (red) colors and +M − L (bluish-green) colors. But the extent of learning was higher for +S (lavender) than for −S (yellow-lime); moreover, at plateau performance, the cone contrast at the detection threshold was higher for +S than for −S. These asymmetries may reflect differences in retinal circuitry for S-ON and S-OFF. At plateau performance, the two species also had similar detection thresholds for all colors, although monkeys had shorter reaction times than humans and slightly lower thresholds for colors that modulated L/M cones. We discuss whether these observations, together with previous work showing that monkeys have lower spatial acuity than humans, could be accounted for by selective pressures driving higher chromatic sensitivity at the cost of spatial acuity amongst monkeys, specifically for the more recently evolved L − M mechanism.

Parallel, multi-stage processing of colors, faces and shapes in macaque inferior temporal cortex.

Visual-object processing culminates in inferior temporal cortex (IT). To assess the organization of IT, we measured functional magnetic resonance imaging responses in alert monkeys to achromatic images (faces, fruit, bodies and places) and colored gratings. IT contained multiple color-biased regions, which were typically ventral to face patches and yoked to them, spaced regularly at four locations predicted by known anatomy. Color and face selectivity increased for more anterior regions, indicative of a broad hierarchical arrangement. Responses to non-face shapes were found across IT, but were stronger outside color-biased regions and face patches, consistent with multiple parallel streams. IT also contained multiple coarse eccentricity maps: face patches overlapped central representations, color-biased regions spanned mid-peripheral representations and place-biased regions overlapped peripheral representations. These results show that IT comprises parallel, multi-stage processing networks subject to one organizing principle.

Psychophysical chromatic mechanisms in macaque monkey.

Chromatic mechanisms have been studied extensively with psychophysical techniques in humans, but the number and nature of the mechanisms are still controversial. Appeals to monkey neurophysiology are often used to sort out the competing claims and to test hypotheses arising from the experiments in humans, but psychophysical chromatic mechanisms have never been assessed in monkeys. Here we address this issue by measuring color-detection thresholds in monkeys before and after chromatic adaptation, employing a standard approach used to determine chromatic mechanisms in humans. We conducted separate experiments using adaptation configured as either flickering full-field colors or heterochromatic gratings. Full-field colors would favor activity within the visual system at or before the arrival of retinal signals to V1, before the spatial transformation of color signals by the cortex. Conversely, gratings would favor activity within the cortex where neurons are often sensitive to spatial chromatic structure. Detection thresholds were selectively elevated for the colors of full-field adaptation when it modulated along either of the two cardinal chromatic axes that define cone-opponent color space [L vs M or S vs (L + M)], providing evidence for two privileged cardinal chromatic mechanisms implemented early in the visual-processing hierarchy. Adaptation with gratings produced elevated thresholds for colors of the adaptation regardless of its chromatic makeup, suggesting a cortical representation comprised of multiple higher-order mechanisms each selective for a different direction in color space. The results suggest that color is represented by two cardinal channels early in the processing hierarchy and many chromatic channels in brain regions closer to perceptual readout.

Color tuning in alert macaque V1 assessed with fMRI and single-unit recording shows a bias toward daylight colors.

Colors defined by the two intermediate directions in color space, "orange-cyan" and "lime-magenta," elicit the same spatiotemporal average response from the two cardinal chromatic channels in the lateral geniculate nucleus (LGN). While we found LGN functional magnetic resonance imaging (fMRI) responses to these pairs of colors were statistically indistinguishable, primary visual cortex (V1) fMRI responses were stronger to orange-cyan. Moreover, linear combinations of single-cell responses to cone-isolating stimuli of V1 cone-opponent cells also yielded stronger predicted responses to orange-cyan over lime-magenta, suggesting these neurons underlie the fMRI result. These observations are consistent with the hypothesis that V1 recombines LGN signals into "higher-order" mechanisms tuned to noncardinal color directions. In light of work showing that natural images and daylight samples are biased toward orange-cyan, our findings further suggest that V1 is adapted to daylight. V1, especially double-opponent cells, may function to extract spatial information from color boundaries correlated with scene-structure cues, such as shadows lit by ambient blue sky juxtaposed with surfaces reflecting sunshine.

Vision and art: an interdisciplinary approach to neuroscience education.

Undergraduate institutions are increasingly adopting neuroscience within their curricula, although it is unclear how best to implement this material given the interdisciplinary nature of the field, which requires knowledge of basic physics, chemistry, biology and psychology. This difficulty is compounded by declines over recent decades in the amount of physics education that students receive in high school, which hinders students' ability to grasp basic principles of neuroscience. Here we discuss our experiences as teacher (BRC) and student (RLS) with an undergraduate course in Vision and Art. The course capitalizes on students' prior interest in visual art to motivate an understanding of the physiological and computational neural processes that underlie vision; our aim is that the learning strategies that students acquire as a result of the format and interdisciplinary approach of the course will increase students' critical thinking skills and benefit them as they pursue other domains of inquiry. The course includes both expert lectures on central themes of vision along with a problem-based learning (PBL) laboratory component that directly engages the students as empirical scientists. We outline the syllabus, the motivation for using PBL, and describe a number of hands-on laboratory exercises, many of which require only inexpensive and readily available equipment. We have developed a website that we hope will facilitate student-driven inquiry beyond the classroom and foster inter-institutional collaboration in this endeavor. We conclude the paper with a discussion of the potential limitations of the course and how to evaluate the success of the course and the website.