Visual context affects the perceived timing of tactile sensations elicited through intra-cortical microstimulation.

Intra-cortical microstimulation (ICMS) is a technique to provide tactile sensations for a somatosensory brain-machine interface (BMI). A viable BMI must function within the rich, multisensory environment of the real world, but how ICMS is integrated with other sensory modalities is poorly understood. To investigate how ICMS percepts are integrated with visual information, ICMS and visual stimuli were delivered at varying times relative to one another. Both visual context and ICMS current amplitude were found to bias the qualitative experience of ICMS. In two tetraplegic participants, ICMS and visual stimuli were more likely to be experienced as occurring simultaneously when visual stimuli were more realistic, demonstrating an effect of visual context on the temporal binding window. The peak of the temporal binding window varied but was consistently offset from zero, suggesting that multisensory integration with ICMS can suffer from temporal misalignment. Recordings from primary somatosensory cortex (S1) during catch trials where visual stimuli were delivered without ICMS demonstrated that S1 represents visual information related to ICMS across visual contexts.

S1 represents multisensory contexts and somatotopic locations within and outside the bounds of the cortical homunculus.

Recent literature suggests that tactile events are represented in the primary somatosensory cortex (S1) beyond its long-established topography; in addition, the extent to which S1 is modulated by vision remains unclear. To better characterize S1, human electrophysiological data were recorded during touches to the forearm or finger. Conditions included visually observed physical touches, physical touches without vision, and visual touches without physical contact. Two major findings emerge from this dataset. First, vision strongly modulates S1 area 1, but only if there is a physical element to the touch, suggesting that passive touch observation is insufficient to elicit neural responses. Second, despite recording in a putative arm area of S1, neural activity represents both arm and finger stimuli during physical touches. Arm touches are encoded more strongly and specifically, supporting the idea that S1 encodes tactile events primarily through its topographic organization but also more generally, encompassing other areas of the body.

Temporal dynamics of the neural representation of hue and luminance polarity.

Hue and luminance contrast are basic visual features. Here we use multivariate analyses of magnetoencephalography data to investigate the timing of the neural computations that extract them, and whether they depend on common neural circuits. We show that hue and luminance-contrast polarity can be decoded from MEG data and, with lower accuracy, both features can be decoded across changes in the other feature. These results are consistent with the existence of both common and separable neural mechanisms. The decoding time course is earlier and more temporally precise for luminance polarity than hue, a result that does not depend on task, suggesting that luminance contrast is an updating signal that separates visual events. Meanwhile, cross-temporal generalization is slightly greater for representations of hue compared to luminance polarity, providing a neural correlate of the preeminence of hue in perceptual grouping and memory. Finally, decoding of luminance polarity varies depending on the hues used to obtain training and testing data. The pattern of results is consistent with observations that luminance contrast is mediated by both L-M and S cone sub-cortical mechanisms.

Color Space Geometry Uncovered with Magnetoencephalography.

The geometry that describes the relationship among colors, and the neural mechanisms that support color vision, are unsettled. Here, we use multivariate analyses of measurements of brain activity obtained with magnetoencephalography to reverse-engineer a geometry of the neural representation of color space. The analyses depend upon determining similarity relationships among the spatial patterns of neural responses to different colors and assessing how these relationships change in time. We evaluate the approach by relating the results to universal patterns in color naming. Two prominent patterns of color naming could be accounted for by the decoding results: the greater precision in naming warm colors compared to cool colors evident by an interaction of hue and lightness, and the preeminence among colors of reddish hues. Additional experiments showed that classifiers trained on responses to color words could decode color from data obtained using colored stimuli, but only at relatively long delays after stimulus onset. These results provide evidence that perceptual representations can give rise to semantic representations, but not the reverse. Taken together, the results uncover a dynamic geometry that provides neural correlates for color appearance and generates new hypotheses about the structure of color space.

Deconstructing Theory-of-Mind Impairment in High-Functioning Adults with Autism.

Inferring the beliefs, desires, and intentions of other people ("theory of mind," ToM) requires specialized psychological processes that represent the minds of others as distinct from our own [1-3]. ToM is engaged ubiquitously in our everyday social behavior and features a specific developmental trajectory [4] that is notably delayed in children with autism spectrum disorder (ASD) [5, 6]. In healthy individuals, model-based analyses of social learning and decision-making have successfully elucidated specific computational components of ToM processing [7-11]. However, the use of this approach to study ToM impairment in ASD has been extremely limited [10, 12]. To better characterize specific ToM impairment in ASD, we developed a novel learning task and applied model-based analyses in high-functioning adults with ASD and matched healthy controls. After completing a charitable donation task, participants performed a "mentalizer" task in which they observed another person (the agent) complete the same charity task. The mentalizer task probed the participants' ability to acquire and use ToM representations. To accurately predict agent behavior, participants needed to dynamically track the agent's beliefs (true or false) about an experimental context that varied over time and use that information to infer the agent's intentions from their actions. ASD participants were specifically impaired at using their estimates of agent belief to learn agent intentions, though their ability to track agent belief was intact and their reasoning about belief and intentions was rational. Furthermore, model parameters correlated with aspects of social functioning, e.g., ADOS severity scores [13]. Together, these results identify novel, and more specific, targets for future research.