Colour information biases facial age estimation and reduces inter-observer variability.

Age estimation is a perceptual task that people perform automatically and effortlessly on a daily basis. Colour has been identified as one of the facial cues that contributes to age perception. To investigate further the role of colour in age perception, we manipulated the chromatic content of facial images holistically. In Experiment 1, images were shown in colour or grey scale; in Experiment 2, images were shown with red-green contrast increased or decreased; in Experiment 3, images were shown with modified yellow-blue contrast. We examined whether the presence of chromatic information biases the perception of age and/or affects inter-observer variability in age judgements, and whether specific chromatic information affects the perception of age. We found that the same face tended to be judged as younger with increased red-green contrast compared to decreased red-green contrast, suggesting that red-green contrast directly affects age perception. Inter-observer variability in age ratings was significantly lower when participants were asked to rate colour compared with grey scale versions of images. This finding indicates that colour carries information useful cues for age estimation.

Magnetoencephalography contrast adaptation reflects perceptual adaptation.

Contrast adaptation is a fundamental visual process that has been extensively investigated and used to infer the selectivity of visual cortex. We recently reported an apparent disconnect between the effects of contrast adaptation on perception and functional magnetic resonance imaging BOLD response adaptation, in which adaptation between chromatic and achromatic stimuli measured psychophysically showed greater selectivity than adaptation measured using BOLD signals. Here we used magnetoencephalography (MEG) recordings of neural responses to the same chromatic and achromatic adaptation conditions to characterize the neural effects of contrast adaptation and to determine whether BOLD adaptation or MEG better reflect the measured perceptual effects. Participants viewed achromatic, L-M isolating, or S-cone isolating radial sinusoids before adaptation and after adaptation to each of the three contrast directions. We measured adaptation-related changes in the neural response to a range of stimulus contrast amplitudes using two measures of the MEG response: the overall response amplitude, and a novel time-resolved measure of the contrast response function, derived from a classification analysis combined with multidimensional scaling. Within-stimulus adaptation effects on the contrast response functions in each case showed a pattern of contrast-gain or a combination of contrast-gain and response-gain effects. Cross-stimulus adaptation conditions showed that adaptation effects were highly stimulus selective across early, ventral, and dorsal visual cortical areas, consistent with the perceptual effects.

Exploring Information Flow from Posteromedial Cortex during Visuospatial Working Memory: A Magnetoencephalography Study.

The posteromedial cortex (PMC) is a major hub of the brain's default mode network, and is implicated in a broad range of internally driven cognitions, including visuospatial working memory. However, its precise contribution to these cognitive processes remains unclear. Using MEG, we measured PMC activity in healthy human participants (young adults of both sexes) while they performed a visuospatial working memory task. Multivariate pattern classification analyses revealed stimulus-related information during encoding and retrieval in a set of a priori defined cortical ROIs, including prefrontal, occipital, and ventrotemporal cortices, in addition to PMC. We measured the extent to which this stimulus information was exchanged between areas in an information flow analysis, measuring Granger-causal relationships between areas over time. Consistent with the visual nature of the task, information from occipital cortex shaped other regions across most epochs. However, the PMC shaped object representations in occipital and prefrontal cortices during visuospatial working memory, influencing occipital cortex during retrieval and PFC across all task epochs. Our findings are consistent with a proposed role for the PMC in the representation of internal content, including remembered information, and in the comparison of external stimuli with remembered material.SIGNIFICANCE STATEMENT The human brain operates as a collection of highly interconnected regions. Mapping the function of this interconnectivity, as well as the specializations within different regions, is central to understanding the neural processes underlying cognition. The posteromedial cortex (PMC) is a highly connected cortical region, implicated in visuospatial working memory, although its precise contribution remains unclear. We measured the activity of PMC during a visuospatial working memory task, testing how different regions represented the stimuli, and whether these representations were driven by other cortical regions. We found that PMC influenced stimulus information in other regions across all task phases, suggesting that PMC plays a key role in shaping stimulus representations during visuospatial working memory.

Face detection from patterns of shading and shadows: The role of overhead illumination in generating the familiar appearance of the human face.

Face detection in human vision relies on a stereotypical pattern of visual features common to different faces. How are these visual features generated in the environment? Here we investigate how characteristic patterns of shading and shadows that occur across the face act as a cue for face detection. We use 3D rendering to isolate facial shading under simulated lighting conditions, comparing the broad patterns of contrast that occur across the face when light arrives from different angles. We find that human performance in discriminating faces from non-face objects using these contrast patterns depends strongly on the lighting direction. In particular, light arriving from above the brow tends to facilitate face detection - consistent with the statistics of real-world lighting environments, in which light commonly arrives more strongly from above. Indeed, in a further experiment, we find that asymmetries in lighting that occur in complex and naturalistic lighting environments produce contrast patterns across the face that facilitate face detection. These effects occurred independent of the lighting direction relative to the viewer, suggesting that cues to face detection emerge from the interaction between face morphology and vertical asymmetries in lighting direction, independent of the viewer's knowledge or expectations about lighting direction. Comparison with the performance of an image classifier suggests that the effects of lighting direction partly reflect differences in image information that result from the interaction between shape and illumination, as well as face detection in human observers being better-tuned to the pattern of shading and shadows that occurs across an upright face that is lit from overhead.

Neural Coding of Visual Objects Rapidly Reconfigures to Reflect Subtrial Shifts in Attentional Focus.

Every day, we respond to the dynamic world around us by choosing actions to meet our goals. Flexible neural populations are thought to support this process by adapting to prioritize task-relevant information, driving coding in specialized brain regions toward stimuli and actions that are currently most important. Accordingly, human fMRI shows that activity patterns in frontoparietal cortex contain more information about visual features when they are task-relevant. However, if this preferential coding drives momentary focus, for example, to solve each part of a task in turn, it must reconfigure more quickly than we can observe with fMRI. Here, we used multivariate pattern analysis of magnetoencephalography data to test for rapid reconfiguration of stimulus information when a new feature becomes relevant within a trial. Participants saw two displays on each trial. They attended to the shape of a first target then the color of a second, or vice versa, and reported the attended features at a choice display. We found evidence of preferential coding for the relevant features in both trial phases, even as participants shifted attention mid-trial, commensurate with fast subtrial reconfiguration. However, we only found this pattern of results when the stimulus displays contained multiple objects and not in a simpler task with the same structure. The data suggest that adaptive coding in humans can operate on a fast, subtrial timescale, suitable for supporting periods of momentary focus when complex tasks are broken down into simpler ones, but may not always do so.

Spatial and Feature-selective Attention Have Distinct, Interacting Effects on Population-level Tuning.

Attention can be deployed in different ways: When searching for a taxi in New York City, we can decide where to attend (e.g., to the street) and what to attend to (e.g., yellow cars). Although we use the same word to describe both processes, nonhuman primate data suggest that these produce distinct effects on neural tuning. This has been challenging to assess in humans, but here we used an opportunity afforded by multivariate decoding of MEG data. We found that attending to an object at a particular location and attending to a particular object feature produced effects that interacted multiplicatively. The two types of attention induced distinct patterns of enhancement in occipital cortex, with feature-selective attention producing relatively more enhancement of small feature differences and spatial attention producing relatively larger effects for larger feature differences. An information flow analysis further showed that stimulus representations in occipital cortex were Granger-caused by coding in frontal cortices earlier in time and that the timing of this feedback matched the onset of attention effects. The data suggest that spatial and feature-selective attention rely on distinct neural mechanisms that arise from frontal-occipital information exchange, interacting multiplicatively to selectively enhance task-relevant information.

Attention selectively enhances stimulus information for surround over foveal stimulus representations in occipital cortex.

By attending to part of a visual scene, we can prioritize processing of the most relevant visual information and so use our limited resources effectively. Previous functional magnetic resonance imaging (fMRI) work has shown that attention can increase overall blood-oxygen-level-dependent (BOLD) signal responsiveness but also enhances the stimulus information in terms of classifier performance. Here, we investigate how these effects vary across the visual field. We compare attention-enhanced fMRI-BOLD amplitude responses and classifier accuracy in fovea and surrounding stimulus regions using a set of four simple stimuli subdivided into a foveal region (1.4° diameter) and a surround region (15° diameter). We found dissociations between the effects of attention on average response and in enhancing stimulus information. In early visual cortex, we found that attention increased the amplitude of responses to both foveal and surround parts of the stimuli and increased classifier performance only for the surround stimulus. Conversely, ventral visual areas showed less change in average response but greater changes in decoding. Unlike for early visual cortex, in the ventral visual cortex attention produced similar changes in decoding for center and surround stimuli.

Reaction times predict dynamic brain representations measured with MEG for only some object categorisation tasks.

Behavioural categorisation reaction times (RTs) provide a useful way to link behaviour to brain representations measured with neuroimaging. In this framework, objects are assumed to be represented in a multidimensional activation space, with the distances between object representations indicating their degree of neural similarity. Faster RTs have been reported to correlate with greater distances from a classification decision boundary for animacy. Objects inherently belong to more than one category, yet it is not known whether the RT-distance relationship, and its evolution over the time-course of the neural response, is similar across different categories. Here we used magnetoencephalography (MEG) to address this question. Our stimuli included typically animate and inanimate objects, as well as more ambiguous examples (i.e., robots and toys). We conducted four semantic categorisation tasks on the same stimulus set assessing animacy, living, moving, and human-similarity concepts, and linked the categorisation RTs to MEG time-series decoding data. Our results show a sustained RT-distance relationship throughout the time course of object processing for not only animacy, but also categorisation according to human-similarity. Interestingly, this sustained RT-distance relationship was not observed for the living and moving category organisations, despite comparable classification accuracy of the MEG data across all four category organisations. Our findings show that behavioural RTs predict representational distance for an organisational principle other than animacy, however further research is needed to determine why this relationship is observed only for some category organisations and not others.

A humanness dimension to visual object coding in the brain.

Neuroimaging studies investigating human object recognition have primarily focused on a relatively small number of object categories, in particular, faces, bodies, scenes, and vehicles. More recent studies have taken a broader focus, investigating hypothesized dichotomies, for example, animate versus inanimate, and continuous feature dimensions, such as biologically similarity. These studies typically have used stimuli that are identified as animate or inanimate, neglecting objects that may not fit into this dichotomy. We generated a novel stimulus set including standard objects and objects that blur the animate-inanimate dichotomy, for example, robots and toy animals. We used MEG time-series decoding to study the brain's emerging representation of these objects. Our analysis examined contemporary models of object coding such as dichotomous animacy, as well as several new higher order models that take into account an object's capacity for agency (i.e. its ability to move voluntarily) and capacity to experience the world. We show that early (0-200 â€‹ms) responses are predicted by the stimulus shape, assessed using a retinotopic model and shape similarity computed from human judgments. Thereafter, higher order models of agency/experience provided a better explanation of the brain's representation of the stimuli. Strikingly, a model of human similarity provided the best account for the brain's representation after an initial perceptual processing phase. Our findings provide evidence for a new dimension of object coding in the human brain - one that has a "human-centric" focus.

fMRI representational similarity analysis reveals graded preferences for chromatic and achromatic stimulus contrast across human visual cortex.

Human visual cortex is partitioned into different functional areas that, from lower to higher, become increasingly selective and responsive to complex feature dimensions. Here we use a Representational Similarity Analysis (RSA) of fMRI-BOLD signals to make quantitative comparisons across LGN and multiple visual areas of the low-level stimulus information encoded in the patterns of voxel responses. Our stimulus set was picked to target the four functionally distinct subcortical channels that input visual cortex from the LGN: two achromatic sinewave stimuli that favor the responses of the high-temporal magnocellular and high-spatial parvocellular pathways, respectively, and two chromatic stimuli isolating the L/M-cone opponent and S-cone opponent pathways, respectively. Each stimulus type had three spatial extents to sample both foveal and para-central visual field. With the RSA, we compare quantitatively the response specializations for individual stimuli and combinations of stimuli in each area and how these change across visual cortex. First, our results replicate the known response preferences for motion/flicker in the dorsal visual areas. In addition, we identify two distinct gradients along the ventral visual stream. In the early visual areas (V1-V3), the strongest differential representation is for the achromatic high spatial frequency stimuli, suitable for form vision, and a very weak differentiation of chromatic versus achromatic contrast. Emerging in ventral occipital areas (V4, VO1 and VO2), however, is an increasingly strong separation of the responses to chromatic versus achromatic contrast and a decline in the high spatial frequency representation. These gradients provide new insight into how visual information is transformed across the visual cortex.

Long-Range Interocular Suppression in Adults with Strabismic Amblyopia: A Pilot fMRI Study.

Interocular suppression plays an important role in the visual deficits experienced by individuals with amblyopia. Most neurophysiological and functional MRI studies of suppression in amblyopia have used dichoptic stimuli that overlap within the visual field. However, suppression of the amblyopic eye also occurs when the dichoptic stimuli do not overlap, a phenomenon we refer to as long-range suppression. We used functional MRI to test the hypothesis that long-range suppression reduces neural activity in V1, V2 and V3 in adults with amblyopia, indicative of an early, active inhibition mechanism. Five adults with amblyopia and five controls viewed monocular and dichoptic quadrant stimuli during fMRI. Three of five participants with amblyopia experienced complete perceptual suppression of the quadrants presented to their amblyopic eye under dichoptic viewing. The blood oxygen level dependant (BOLD) responses within retinotopic regions corresponding to amblyopic and fellow eye stimuli were analyzed for response magnitude, time to peak, effective connectivity and stimulus classification. Dichoptic viewing slightly reduced the BOLD response magnitude in amblyopic eye retinotopic regions in V1 and reduced the time to peak response; however, the same effects were also present in the non-dominant eye of controls. Effective connectivity was unaffected by suppression, and the results of a classification analysis did not differ significantly between the control and amblyopia groups. Overall, we did not observe a neural signature of long-range amblyopic eye suppression in V1, V2 or V3 using functional MRI in this initial study. This type of suppression may involve higher level processing areas within the brain.

Color contrast adaptation: fMRI fails to predict behavioral adaptation.

fMRI-adaptation is a valuable tool for inferring the selectivity of neural responses. Here we use it in human color vision to test the selectivity of responses to S-cone opponent (blue-yellow), L/M-cone opponent (red-green), and achromatic (Ach) contrast across nine regions of interest in visual cortex. We measure psychophysical adaptation, using comparable stimuli to the fMRI-adaptation, and find significant selective adaptation for all three stimulus types, implying separable visual responses to each. For fMRI-adaptation, we find robust adaptation but, surprisingly, much less selectivity due to high levels of cross-stimulus adaptation in all conditions. For all BY and Ach test/adaptor pairs, selectivity is absent across all ROIs. For RG/Ach stimulus pairs, this paradigm has previously shown selectivity for RG in ventral areas and for Ach in dorsal areas. For chromatic stimulus pairs (RG/BY), we find a trend for selectivity in ventral areas. In conclusion, we find an overall lack of correspondence between BOLD and behavioral adaptation suggesting they reflect different aspects of the underlying neural processes. For example, raised cross-stimulus adaptation in fMRI may reflect adaptation of the broadly-tuned normalization pool. Finally, we also identify a longer-timescale adaptation (1h) in both BOLD and behavioral data. This is greater for chromatic than achromatic contrast. The longer-timescale BOLD effect was more evident in the higher ventral areas than in V1, consistent with increasing windows of temporal integration for higher-order areas.

Reevaluating hMT+ and hV4 functional specialization for motion and static contrast using fMRI-guided repetitive transcranial magnetic stimulation.

Although visual areas hMT+ and hV4 are considered to have segregated functions for the processing of motion and form within dorsal and ventral streams, respectively, more recent evidence favors some functional overlap. Here we use fMRI-guided online repetitive transcranial magnetic stimulation (rTMS) to test two associated hypotheses: that area hV4 is causally involved in the perception of motion and hMT+ in the perception of static form. We use variations of a common global stimulus to test two dynamic motion-based tasks and two static form-based tasks in ipsilateral and contralateral visual fields. We find that rTMS to both hMT+ and hV4 significantly impairs direction discrimination and causes a perceptual slowing of motion, implicating hV4 in motion perception. Stimulation of hMT+ impairs motion in both visual fields, implying that disruption to one hMT+ disrupts the other with both needed for optimal performance. For the second hypothesis, we find the novel result that hV4 stimulation markedly reduces perceived contrast of a static stimulus. hMT+ stimulation also produces an effect, implicating it in static contrast perception. Our findings are the first to show that rTMS of hV4 can produce a large perceptual effect and, taken together, suggest a less rigid functional segregation between hMT+ and hV4 than previously thought.

Ghosts in machine learning for cognitive neuroscience: Moving from data to theory.

The application of machine learning methods to neuroimaging data has fundamentally altered the field of cognitive neuroscience. Future progress in understanding brain function using these methods will require addressing a number of key methodological and interpretive challenges. Because these challenges often remain unseen and metaphorically "haunt" our efforts to use these methods to understand the brain, we refer to them as "ghosts". In this paper, we describe three such ghosts, situate them within a more general framework from philosophy of science, and then describe steps to address them. The first ghost arises from difficulties in determining what information machine learning classifiers use for decoding. The second ghost arises from the interplay of experimental design and the structure of information in the brain - that is, our methods embody implicit assumptions about information processing in the brain, and it is often difficult to determine if those assumptions are satisfied. The third ghost emerges from our limited ability to distinguish information that is merely decodable from the brain from information that is represented and used by the brain. Each of the three ghosts place limits on the interpretability of decoding research in cognitive neuroscience. There are no easy solutions, but facing these issues squarely will provide a clearer path to understanding the nature of representation and computation in the human brain.

Interpreting the dimensions of neural feature representations revealed by dimensionality reduction.

Recent progress in understanding the structure of neural representations in the cerebral cortex has centred around the application of multivariate classification analyses to measurements of brain activity. These analyses have proved a sensitive test of whether given brain regions provide information about specific perceptual or cognitive processes. An exciting extension of this approach is to infer the structure of this information, thereby drawing conclusions about the underlying neural representational space. These approaches rely on exploratory data-driven dimensionality reduction to extract the natural dimensions of neural spaces, including natural visual object and scene representations, semantic and conceptual knowledge, and working memory. However, the efficacy of these exploratory methods is unknown, because they have only been applied to representations in brain areas for which we have little or no secondary knowledge. One of the best-understood areas of the cerebral cortex is area MT of primate visual cortex, which is known to be important in motion analysis. To assess the effectiveness of dimensionality reduction for recovering neural representational space we applied several dimensionality reduction methods to multielectrode measurements of spiking activity obtained from area MT of marmoset monkeys, made while systematically varying the motion direction and speed of moving stimuli. Despite robust tuning at individual electrodes, and high classifier performance, dimensionality reduction rarely revealed dimensions for direction and speed. We use this example to illustrate important limitations of these analyses, and suggest a framework for how to best apply such methods to data where the structure of the neural representation is unknown.

Dynamic population codes of multiplexed stimulus features in primate area MT.

The middle-temporal area (MT) of primate visual cortex is critical in the analysis of visual motion. Single-unit studies suggest that the response dynamics of neurons within area MT depend on stimulus features, but how these dynamics emerge at the population level, and how feature representations interact, is not clear. Here, we used multivariate classification analysis to study how stimulus features are represented in the spiking activity of populations of neurons in area MT of marmoset monkey. Using representational similarity analysis we distinguished the emerging representations of moving grating and dot field stimuli. We show that representations of stimulus orientation, spatial frequency, and speed are evident near the onset of the population response, while the representation of stimulus direction is slower to emerge and sustained throughout the stimulus-evoked response. We further found a spatiotemporal asymmetry in the emergence of direction representations. Representations for high spatial frequencies and low temporal frequencies are initially orientation dependent, while those for high temporal frequencies and low spatial frequencies are more sensitive to motion direction. Our analyses reveal a complex interplay of feature representations in area MT population response that may explain the stimulus-dependent dynamics of motion vision.NEW & NOTEWORTHY Simultaneous multielectrode recordings can measure population-level codes that previously were only inferred from single-electrode recordings. However, many multielectrode recordings are analyzed using univariate single-electrode analysis approaches, which fail to fully utilize the population-level information. Here, we overcome these limitations by applying multivariate pattern classification analysis and representational similarity analysis to large-scale recordings from middle-temporal area (MT) in marmoset monkeys. Our analyses reveal a dynamic interplay of feature representations in area MT population response.

A step toward understanding the human ventral visual pathway.

The human ventral visual pathway is implicated in higher order form processing, but the organizational principles within this region are not yet well understood. Recently, Lafer-Sousa, Conway, and Kanwisher (J Neurosci 36: 1682-1697, 2016) used functional magnetic resonance imaging to demonstrate that functional responses in the human ventral visual pathway share a broad homology with the those in macaque inferior temporal cortex, providing new evidence supporting the validity of the macaque as a model of the human visual system in this region. In addition, these results give new clues for understanding the organizational principles within the ventral visual pathway and the processing of higher order color and form, suggesting new avenues for research into this cortical region.

Representational dynamics of object recognition: Feedforward and feedback information flows.

Object perception involves a range of visual and cognitive processes, and is known to include both a feedfoward flow of information from early visual cortical areas to higher cortical areas, along with feedback from areas such as prefrontal cortex. Previous studies have found that low and high spatial frequency information regarding object identity may be processed over different timescales. Here we used the high temporal resolution of magnetoencephalography (MEG) combined with multivariate pattern analysis to measure information specifically related to object identity in peri-frontal and peri-occipital areas. Using stimuli closely matched in their low-level visual content, we found that activity in peri-occipital cortex could be used to decode object identity from ~80ms post stimulus onset, and activity in peri-frontal cortex could also be used to decode object identity from a later time (~265ms post stimulus onset). Low spatial frequency information related to object identity was present in the MEG signal at an earlier time than high spatial frequency information for peri-occipital cortex, but not for peri-frontal cortex. We additionally used Granger causality analysis to compare feedforward and feedback influences on representational content, and found evidence of both an early feedfoward flow and later feedback flow of information related to object identity. We discuss our findings in relation to existing theories of object processing and propose how the methods we use here could be used to address further questions of the neural substrates underlying object perception.

A new type of change blindness: smooth, isoluminant color changes are monitored on a coarse spatial scale.

Attending selectively to changes in our visual environment may help filter less important, unchanging information within a scene. Here, we demonstrate that color changes can go unnoticed even when they occur throughout an otherwise static image. The novelty of this demonstration is that it does not rely upon masking by a visual disruption or stimulus motion, nor does it require the change to be very gradual and restricted to a small section of the image. Using a two-interval, forced-choice change-detection task and an odd-one-out localization task, we showed that subjects were slowest to respond and least accurate (implying that change was hardest to detect) when the color changes were isoluminant, smoothly varying, and asynchronous with one another. This profound change blindness offers new constraints for theories of visual change detection, implying that, in the absence of transient signals, changes in color are typically monitored at a coarse spatial scale.

Color responsiveness argues against a dorsal component of human V4.

The retinotopic organization, position, and functional responsiveness of some early visual cortical areas in human and non-human primates are consistent with their being homologous structures. The organization of other areas remains controversial. A critical debate concerns the potential human homologue of macaque area V4, an area very responsive to colored images: specifically, whether human V4 is divided between ventral and dorsal components, as in the macaque, or whether human V4 is confined to one ventral area. We used fMRI to define these areas retinotopically in human and to test the impact of image color on their responsivity. We found a robust preference for full-color movie segments over a luminance-matched achromatic version in ventral V4 but little or no preference in the vicinity of the putative dorsal counterpart. Contrary to previous reports that visual field coverage in the ventral part of V4 is deficient without the dorsal part, we found that coverage in ventral V4 extended to the lower vertical meridian, including the entire contralateral hemifield. Together these results provide evidence against a dorsal component of human V4. Instead, they are consistent with human V4 being a single, ventral region that is sensitive to the chromatic components of images.