Distinct auditory and visual tool regions with multisensory response properties in human parietal cortex.

Left parietal cortex has been associated with the human-specific ability of sophisticated tool use. Yet, it is unclear how tool information is represented across senses. Here, we compared auditory and visual tool-specific activations within healthy human subjects to probe the relation of tool-specific networks, uni- and multisensory response properties, and functional and structural connectivity using functional and diffusion-weighted MRI. In each subject, we identified an auditory tool network with regions in left anterior inferior parietal cortex (aud-aIPL), bilateral posterior lateral sulcus, and left inferior precentral sulcus, and a visual tool network with regions in left aIPL (vis-aIPL) and bilateral inferior temporal gyrus. Aud-aIPL was largely separate and anterior/inferior from vis-aIPL, with varying degrees of overlap across subjects. Both regions displayed a strong preference for tools versus other stimuli presented within the same modality. Despite their modality preference, aud-aIPL and vis-aIPL and a region in left inferior precentral sulcus displayed multisensory response properties, as revealed in multivariate analyses. Thus, two largely separate tool networks are engaged by the visual and auditory modalities with nodes in parietal and prefrontal cortex potentially integrating information across senses. The diversification of tool processing in human parietal cortex underpins its critical role in complex object processing.

Task-Modulated Cortical Representations of Natural Sound Source Categories.

In everyday sound environments, we recognize sound sources and events by attending to relevant aspects of an acoustic input. Evidence about the cortical mechanisms involved in extracting relevant category information from natural sounds is, however, limited to speech. Here, we used functional MRI to measure cortical response patterns while human listeners categorized real-world sounds created by objects of different solid materials (glass, metal, wood) manipulated by different sound-producing actions (striking, rattling, dropping). In different sessions, subjects had to identify either material or action categories in the same sound stimuli. The sound-producing action and the material of the sound source could be decoded from multivoxel activity patterns in auditory cortex, including Heschl's gyrus and planum temporale. Importantly, decoding success depended on task relevance and category discriminability. Action categories were more accurately decoded in auditory cortex when subjects identified action information. Conversely, the material of the same sound sources was decoded with higher accuracy in the inferior frontal cortex during material identification. Representational similarity analyses indicated that both early and higher-order auditory cortex selectively enhanced spectrotemporal features relevant to the target category. Together, the results indicate a cortical selection mechanism that favors task-relevant information in the processing of nonvocal sound categories.

Short-term plasticity of visuo-haptic object recognition.

Functional magnetic resonance imaging (fMRI) studies have provided ample evidence for the involvement of the lateral occipital cortex (LO), fusiform gyrus (FG), and intraparietal sulcus (IPS) in visuo-haptic object integration. Here we applied 30 min of sham (non-effective) or real offline 1 Hz repetitive transcranial magnetic stimulation (rTMS) to perturb neural processing in left LO immediately before subjects performed a visuo-haptic delayed-match-to-sample task during fMRI. In this task, subjects had to match sample (S1) and target (S2) objects presented sequentially within or across vision and/or haptics in both directions (visual-haptic or haptic-visual) and decide whether or not S1 and S2 were the same objects. Real rTMS transiently decreased activity at the site of stimulation and remote regions such as the right LO and bilateral FG during haptic S1 processing. Without affecting behavior, the same stimulation gave rise to relative increases in activation during S2 processing in the right LO, left FG, bilateral IPS, and other regions previously associated with object recognition. Critically, the modality of S2 determined which regions were recruited after rTMS. Relative to sham rTMS, real rTMS induced increased activations during crossmodal congruent matching in the left FG for haptic S2 and the temporal pole for visual S2. In addition, we found stronger activations for incongruent than congruent matching in the right anterior parahippocampus and middle frontal gyrus for crossmodal matching of haptic S2 and in the left FG and bilateral IPS for unimodal matching of visual S2, only after real but not sham rTMS. The results imply that a focal perturbation of the left LO triggers modality-specific interactions between the stimulated left LO and other key regions of object processing possibly to maintain unimpaired object recognition. This suggests that visual and haptic processing engage partially distinct brain networks during visuo-haptic object matching.

Multisensory interactions between auditory and haptic object recognition.

Object manipulation produces characteristic sounds and causes specific haptic sensations that facilitate the recognition of the manipulated object. To identify the neural correlates of audio-haptic binding of object features, healthy volunteers underwent functional magnetic resonance imaging while they matched a target object to a sample object within and across audition and touch. By introducing a delay between the presentation of sample and target stimuli, it was possible to dissociate haptic-to-auditory and auditory-to-haptic matching. We hypothesized that only semantically coherent auditory and haptic object features activate cortical regions that host unified conceptual object representations. The left fusiform gyrus (FG) and posterior superior temporal sulcus (pSTS) showed increased activation during crossmodal matching of semantically congruent but not incongruent object stimuli. In the FG, this effect was found for haptic-to-auditory and auditory-to-haptic matching, whereas the pSTS only displayed a crossmodal matching effect for congruent auditory targets. Auditory and somatosensory association cortices showed increased activity during crossmodal object matching which was, however, independent of semantic congruency. Together, the results show multisensory interactions at different hierarchical stages of auditory and haptic object processing. Object-specific crossmodal interactions culminate in the left FG, which may provide a higher order convergence zone for conceptual object knowledge.

Vision holds a greater share in visuo-haptic object recognition than touch.

The integration of visual and haptic input can facilitate object recognition. Yet, vision might dominate visuo-haptic interactions as it is more effective than haptics in processing several object features in parallel and recognizing objects outside of reaching space. The maximum likelihood approach of multisensory integration would predict that haptics as the less efficient sense for object recognition gains more from integrating additional visual information than vice versa. To test for asymmetries between vision and touch in visuo-haptic interactions, we measured regional changes in brain activity using functional magnetic resonance imaging while healthy individuals performed a delayed-match-to-sample task. We manipulated identity matching of sample and target objects: We hypothesized that only coherent visual and haptic object features would activate unified object representations. The bilateral object-specific lateral occipital cortex, fusiform gyrus, and intraparietal sulcus showed increased activation to crossmodal compared to unimodal matching but only for congruent object pairs. Critically, the visuo-haptic interaction effects in these regions depended on the sensory modality which processed the target object, being more pronounced for haptic than visual targets. This preferential response of visuo-haptic regions indicates a modality-specific asymmetry in crossmodal matching of visual and haptic object features, suggesting a functional primacy of vision over touch in visuo-haptic object recognition.

The cortical eye proprioceptive signal modulates neural activity in higher-order visual cortex as predicted by the variation in visual sensitivity.

Whereas the links between eye movements and the shifts in visual attention are well established, less is known about how eye position affects the prioritization of visual space. It was recently observed that visual sensitivity varies with the direction of gaze and the level of excitability in the eye proprioceptive representation in human left somatosensory cortex (S1(EYE)), so that after 1Hz repetitive transcranial magnetic stimulation (rTMS) over S1(EYE), targets presented nearer the center of the orbit are detected more accurately. Here we used whole-brain functional magnetic resonance imaging to map areas where S1(EYE)-rTMS affects the neural response evoked by retinally identical stimuli depending on the direction of rotation of the right eye. After S1(EYE)-rTMS, a single area in the left cuneus outside Brodmann Areas 17/18 showed an increased neuronal response to a right hemifield target when the right eye was rotated leftwards as compared with when it was rotated rightwards. This effect was larger after S1(EYE)-rTMS than after rTMS of a control area in the motor cortex. The neural response to retinally identical stimuli in this area could be predicted from the changes in visual detectability observed previously, but not from the location of the visual targets relative to the body. These results strongly argue for a modulatory connection from the eye proprioceptive area in the somatosensory cortex to the higher-order visual cortex. This connection may contribute to flexibly allocate priorities for visual perception depending on the proprioceptively signaled direction of gaze.

The left fusiform gyrus hosts trisensory representations of manipulable objects.

During object manipulation the brain integrates the visual, auditory, and haptic experience of an object into a unified percept. Previous brain imaging studies have implicated for instance the dorsal part of the lateral occipital complex in visuo-tactile and the posterior superior temporal sulcus in audio-visual integration of object-related inputs (Amedi et al., 2005). Yet it is still unclear which brain regions represent object-specific information of all three sensory modalities. To address this question, we performed two complementary functional magnetic resonance imaging experiments. In the first experiment, we identified brain regions which were consistently activated by unimodal visual, auditory, and haptic processing of manipulable objects relative to non-object control stimuli presented in the same modality. In the second experiment, we assessed regional brain activations when participants had to match object-related information that was presented simultaneously in two or all three modalities. Only a well-defined region in left fusiform gyrus (FG) showed an object-specific activation during unisensory processing in the visual, auditory, and tactile modalities. The same region was also consistently activated during multisensory matching of object-related information across all three senses. Taken together, our results suggest that this region is central to the recognition of manipulable objects. A putative role of this FG region is to unify object-specific information provided by the visual, auditory, and tactile modalities into trisensory object representations.

How does transcranial magnetic stimulation modify neuronal activity in the brain? Implications for studies of cognition.

Transcranial magnetic stimulation (TMS) uses a magnetic field to "carry" a short lasting electrical current pulse into the brain where it stimulates neurones, particularly in superficial regions of cerebral cortex. TMS can interfere with cognitive functions in two ways. A high intensity TMS pulse causes a synchronised high frequency burst of discharge in a relatively large population of neurones that is terminated by a long lasting GABAergic inhibition. The combination of artificial synchronisation of activity followed by depression effectively disrupts perceptual, motor and cognitive processes in the human brain. This transient neurodisruption has been termed a "virtual lesion". Smaller intensities of stimulation produce less activity; in such cases, cognitive operations can probably continue but are disrupted because of the added noisy input from the TMS pulse. It is usually argued that if a TMS pulse affects performance, then the area stimulated must provide an essential contribution to behaviour being studied. However, there is one exception to this: the pulse could be applied to an area that is not involved in the task but which has projections to the critical site. Activation of outputs from the site of stimulation could potentially disrupt processing at the distant site, interfering with behaviour without having any involvement in the task. A final important feature of the response to TMS is "context dependency", which indicates that the response depends on how excitable the cortex is at the time the stimulus is applied: if many neurones are close to firing threshold then the more of them are recruited by the pulse than at rest. Many studies have noted this context-dependent modulation. However, it is often assumed that the excitability of an area has a simple relationship to activity in that area. We argue that this is not necessarily the case. Awareness of the problem may help resolve some apparent anomalies in the literature.