Early visual experience and the recognition of basic facial expressions: involvement of the middle temporal and inferior frontal gyri during haptic identification by the early blind.

Face perception is critical for social communication. Given its fundamental importance in the course of evolution, the innate neural mechanisms can anticipate the computations necessary for representing faces. However, the effect of visual deprivation on the formation of neural mechanisms that underlie face perception is largely unknown. We previously showed that sighted individuals can recognize basic facial expressions by haptics surprisingly well. Moreover, the inferior frontal gyrus (IFG) and posterior superior temporal sulcus (pSTS) in the sighted subjects are involved in haptic and visual recognition of facial expressions. Here, we conducted both psychophysical and functional magnetic-resonance imaging (fMRI) experiments to determine the nature of the neural representation that subserves the recognition of basic facial expressions in early blind individuals. In a psychophysical experiment, both early blind and sighted subjects haptically identified basic facial expressions at levels well above chance. In the subsequent fMRI experiment, both groups haptically identified facial expressions and shoe types (control). The sighted subjects then completed the same task visually. Within brain regions activated by the visual and haptic identification of facial expressions (relative to that of shoes) in the sighted group, corresponding haptic identification in the early blind activated regions in the inferior frontal and middle temporal gyri. These results suggest that the neural system that underlies the recognition of basic facial expressions develops supramodally even in the absence of early visual experience.

Tactile perception of nonpainful unpleasantness in relation to perceived roughness: effects of inter-element spacing and speed of relative motion of rigid 2-D raised-dot patterns at two body loci.

Rigid surfaces consisting of spatially jittered 2-D raised-dot patterns with different inter-element spacings were moved back and forth across the skin at three different speeds (10-fold range). Within each psychophysical experiment, participants numerically estimated the perceived magnitude of either unpleasantness (nonpainful) or roughness of 2-D raised-dot surfaces applied to two stationary body sites (experiment 1: fingers; experiment 2: forearm). The psychophysical functions for the two types of perceptual judgment were highly similar at both body loci; more specifically, the perceived magnitude of unpleasantness and roughness both increased monotonically as a power function of increasing inter-element spacing, with the rate of growth declining at the upper end of the continuum. These results suggest that inter-element spacing is a critical determinant of the perceived magnitude of unpleasantness (nonpainful), as well as of roughness. Each perceptual judgment also increased as a function of increasing relative speed at both body loci. However, the magnitude of this effect was significantly greater for perceived unpleasantness than for perceived roughness; conversely, the speed effect was significantly greater on the forearm than on the fingers. Several possible explanations for these findings are considered.

Haptic object perception: spatial dimensionality and relation to vision.

Enabled by the remarkable dexterity of the human hand, specialized haptic exploration is a hallmark of object perception by touch. Haptic exploration normally takes place in a spatial world that is three-dimensional; nevertheless, stimuli of reduced spatial dimensionality are also used to display spatial information. This paper examines the consequences of full (three-dimensional) versus reduced (two-dimensional) spatial dimensionality for object processing by touch, particularly in comparison with vision. We begin with perceptual recognition of common human-made artefacts, then extend our discussion of spatial dimensionality in touch and vision to include faces, drawing from research on haptic recognition of facial identity and emotional expressions. Faces have often been characterized as constituting a specialized input for human perception. We find that contrary to vision, haptic processing of common objects is impaired by reduced spatial dimensionality, whereas haptic face processing is not. We interpret these results in terms of fundamental differences in object perception across the modalities, particularly the special role of manual exploration in extracting a three-dimensional structure.

Irrelevant visual faces influence haptic identification of facial expressions of emotion.

This study demonstrates that when people attempt to identify a facial expression of emotion (FEE) by haptically exploring a 3D facemask, they are affected by viewing a simultaneous, task-irrelevant visual FEE portrayed by another person. In comparison to a control condition, where visual noise was presented, the visual FEE facilitated haptic identification when congruent (visual and haptic FEEs same category). When the visual and haptic FEEs were incongruent, haptic identification was impaired, and error responses shifted toward the visually depicted emotion. In contrast, visual emotion labels that matched or mismatched the haptic FEE category produced no such effects. The findings indicate that vision and touch interact in FEE recognition at a level where featural invariants of the emotional category (cf. precise facial geometry or general concepts) are processed, even when the visual and haptic FEEs are not attributable to a common source. Processing mechanisms behind these effects are considered.

Learning and generalization in haptic classification of 2-D raised-line drawings of facial expressions of emotion by sighted and adventitiously blind observers.

Sighted blindfolded individuals can successfully classify basic facial expressions of emotion (FEEs) by manually exploring simple 2-D raised-line drawings (Lederman et al 2008, IEEE Transactions on Haptics 1 27-38). The effect of training on classification accuracy was assessed by sixty sighted blindfolded participants (experiment 1) and by three adventitiously blind participants (experiment 2). We further investigated whether the underlying learning process(es) constituted token-specific learning and/or generalization. A hybrid learning paradigm comprising pre/post and old/new test comparisons was used. For both participant groups, classification accuracy for old (ie trained) drawings markedly increased over study trials (mean improvement --76%, and 88%, respectively). Additionally, RT decreased by a mean of 30% for the sighted, and 31% for the adventitiously blind. Learning was mostly token-specific, but some generalization was also observed for both groups. The sighted classified novel drawings of all six FEEs faster with training (mean RT decrease = 20%). Accuracy also improved significantly (mean improvement = 20%), but this improvement was restricted to two FEEs (anger and sadness). Two of three adventitiously blind participants classified new drawings more accurately (mean improvement = 30%); however, RTs for this group did not reflect generalization. Based on a limited number of blind subjects, our results tentatively suggest that adventitiously blind individuals learn to haptically classify FEEs as well as, or even better than, sighted persons.

Representing human hands haptically or visually from first-person versus third-person perspectives.

Humans can recognise human body parts haptically as well as visually. We employed a mental-rotation task to determine whether participants could adopt a third-person perspective when judging the laterality of life-like human hands. Female participants adopted either a first-person or a third-person perspective using vision (experiment 1) or haptics (experiment 2), with hands presented at various orientations within a horizontal plane. In the first-person perspective task, most participants responded more slowly as hand orientation increasingly deviated from the participant's upright orientation, regardless of modality. In the visual third-person perspective task, most participants responded more slowly as hand orientation increasingly deviated from the experimenter's upright orientation; in contrast, less than half of the participants produced this same inverted U-shaped response-time function haptically. In experiment 3, participants were explicitly instructed to adopt a third-person perspective haptically by mentally rotating the rubber hand to the experimenter's upright orientation. Most participants produced an inverted U-shaped function. Collectively, these results suggest that humans can accurately assume a third-person perspective when hands are explored haptically or visually. With less explicit instructions, however, the canonical orientation for hand representation may be more strongly influenced haptically than visually by body-based heuristics, and less easily modified by perspective instructions.

Brain networks involved in haptic and visual identification of facial expressions of emotion: an fMRI study.

Previous neurophysiological and neuroimaging studies have shown that a cortical network involving the inferior frontal gyrus (IFG), inferior parietal lobe (IPL) and cortical areas in and around the posterior superior temporal sulcus (pSTS) region is employed in action understanding by vision and audition. However, the brain regions that are involved in action understanding by touch are unknown. Lederman et al. (2007) recently demonstrated that humans can haptically recognize facial expressions of emotion (FEE) surprisingly well. Here, we report a functional magnetic resonance imaging (fMRI) study in which we test the hypothesis that the IFG, IPL and pSTS regions are involved in haptic, as well as visual, FEE identification. Twenty subjects haptically or visually identified facemasks with three different FEEs (disgust, neutral and happiness) and casts of shoes (shoes) of three different types. The left posterior middle temporal gyrus, IPL, IFG and bilateral precentral gyrus were activated by FEE identification relative to that of shoes, regardless of sensory modality. By contrast, an inferomedial part of the left superior parietal lobule was activated by haptic, but not visual, FEE identification. Other brain regions, including the lingual gyrus and superior frontal gyrus, were activated by visual identification of FEEs, relative to haptic identification of FEEs. These results suggest that haptic and visual FEE identification rely on distinct but overlapping neural substrates including the IFG, IPL and pSTS region.

Functional specialization and convergence in the occipito-temporal cortex supporting haptic and visual identification of human faces and body parts: an fMRI study.

Humans can recognize common objects by touch extremely well whenever vision is unavailable. Despite its importance to a thorough understanding of human object recognition, the neuroscientific study of this topic has been relatively neglected. To date, the few published studies have addressed the haptic recognition of nonbiological objects. We now focus on haptic recognition of the human body, a particularly salient object category for touch. Neuroimaging studies demonstrate that regions of the occipito-temporal cortex are specialized for visual perception of faces (fusiform face area, FFA) and other body parts (extrastriate body area, EBA). Are the same category-sensitive regions activated when these components of the body are recognized haptically? Here, we use fMRI to compare brain organization for haptic and visual recognition of human body parts. Sixteen subjects identified exemplars of faces, hands, feet, and nonbiological control objects using vision and haptics separately. We identified two discrete regions within the fusiform gyrus (FFA and the haptic face region) that were each sensitive to both haptically and visually presented faces; however, these two regions differed significantly in their response patterns. Similarly, two regions within the lateral occipito-temporal area (EBA and the haptic body region) were each sensitive to body parts in both modalities, although the response patterns differed. Thus, although the fusiform gyrus and the lateral occipito-temporal cortex appear to exhibit modality-independent, category-sensitive activity, our results also indicate a degree of functional specialization related to sensory modality within these structures.

Assessing subclinical tactual deficits in the hand function of diabetic blind persons at risk for peripheral neuropathy.

OBJECTIVE: To assess subclinical impairments in tactual hand function produced by diabetes mellitus in late-blind adults with diabetic retinopathy. DESIGN: The survey compares diabetic blind with nondiabetic blind and blindfolded sighted controls in terms of their performance on a battery of tests that assess tactual hand function. SETTING: Subjects were evaluated at their rehabilitation program center in Madrid. PARTICIPANTS: Nine (referred) diabetic blind subjects affected by diabetic retinopathy versus 10 (referred) nondiabetic blind subjects versus 10 blindfolded sighted volunteers, all right-handed and matched for age. Subjects were referred by the training professionals of the rehabilitation program center and asked to volunteer. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Cutaneous force and spatial resolution thresholds, haptic psychophysical functions for perceived roughness, weight, and size, and both accuracy and response times for haptic classification of 3-dimensional common objects. Measures of joint mobility, muscular strength, and motor dexterity were also included. RESULTS: The diabetic blind performed significantly poorer than the controls in terms of force sensitivity (distal and proximal finger pads, and palm), spatial resolution (distal finger pad only), motor dexterity, perceived roughness, and finally, haptic object classification response times for texture-diagnostic objects. CONCLUSIONS: Subclinical disturbances in the tactual hand function of the diabetic blind subjects were only documented in perceptual and motor tasks for which cutaneous, as opposed to kinesthetic, information was particularly relevant.

Haptic face processing.

We present an overview of a new multidisciplinary research program that focuses on haptic processing of human facial identity and facial expressions of emotion. A series of perceptual and neuroscience experiments with live faces and/or rigid three-dimensional facemasks is outlined. To date, several converging methodologies have been adopted: behavioural experimental studies with neurologically intact participants, neuropsychological behavioural research with prosopagnosic individuals, and neuroimaging studies using fMRI techniques. In each case, we have asked what would happen if the hands were substituted for the eyes. We confirm that humans can haptically determine both identity and facial expressions of emotion in facial displays at levels well above chance. Clearly, face processing is a bimodal phenomenon. The processes and representations that underlie such patterns of behaviour are also considered.

Haptic roughness perception of linear gratings via bare finger or rigid probe.

The magnitude of perceived roughness was haptically estimated as subjects freely explored linear gratings with either the bare finger or a rigid stylus-shaped probe. A considerably expanded range of ridge and groove width was investigated, relative to the extant literature. The four experiments collectively indicate that, for both finger and probe-end effectors, the variance in the estimates of perceived roughness was predominantly predicted by a single parameter: groove width. The functions relating perceived roughness to groove width increased over a narrow band relative to the full range of values, then flattened. These data have archival values for models of roughness perception involving both direct and indirect touch.

A haptic face-inversion effect.

We examined whether a face-inversion effect occurs when participants explore faces by touch. We used a haptic version of the inversion paradigm with 3-D clay facemasks and non-face control objects (teapots) moulded from real objects. Young, neurologically intact, blindfolded participants performed a temporally unconstrained haptic same/different task in each of four stimulus conditions: upright facemasks, inverted facemasks, upright teapots, and inverted teapots. There was a significant inversion effect for faces in terms of accuracy, but none for teapots. The results are considered in terms of the consequences of sequential manual exploration for haptic face processing.

Multisensory activation of the intraparietal area when classifying grating orientation: a functional magnetic resonance imaging study.

Humans can judge grating orientation by touch. Previous studies indicate that the extrastriate cortex is involved in tactile orientation judgments, suggesting that this area is related to visual imagery. However, it has been unclear which neural mechanisms are crucial for the tactile processing of orientation, because visual imagery is not always required for tactile spatial tasks. We expect that such neural mechanisms involve multisensory areas, because our perception of space is highly integrated across modalities. The current study uses functional magnetic resonance imaging during the classification of grating orientations to evaluate the neural substrates responsible for the multisensory spatial processing of orientation. We hypothesized that a region within the intraparietal sulcus (IPS) would be engaged in orientation processing, regardless of the sensory modality. Sixteen human subjects classified the orientations of passively touched gratings and performed two control tasks with both the right and left hands. Tactile orientation classification activated regions around the right postcentral sulcus and IPS, regardless of the hand used, when contrasted with roughness classification of the same stimuli. Right-lateralized activation was confirmed in these regions by evaluating the hemispheric effects of tactile spatial processing with both hands. In contrast, visual orientation classification activated the left middle occipital gyrus when contrasted with color classification of the same stimuli. Furthermore, visual orientation classification activated a part of the right IPS that was also activated by the tactile orientation task. Thus, we suggest that a part of the right IPS is engaged in the multisensory spatial processing of grating orientation.

Haptic face identification activates ventral occipital and temporal areas: an fMRI study.

Many studies in visual face recognition have supported a special role for the right fusiform gyrus. Despite the fact that faces can also be recognized haptically, little is known about the neural correlates of haptic face recognition. In the current fMRI study, neurologically intact participants were intensively trained to identify specific facemasks (molded from live faces) and specific control objects. When these stimuli were presented in the scanner, facemasks activated left fusiform and right hippocampal/parahippocampal areas (and other regions) more than control objects, whereas the latter produced no activity greater than the facemasks. We conclude that these ventral occipital and temporal areas may play an important role in the haptic identification of faces at the subordinate level. We further speculate that left fusiform gyrus may be recruited more for facemasks than for control objects because of the increased need for sequential processing by the haptic system.

Imagining material versus geometric properties of objects: an fMRI study.

Two experiments are reported that used fMRI to compare the brain activation during the imagery of material and geometric object features. In the first experiment, participants were to mentally evaluate objects along either a material dimension (roughness, hardness and temperature; e.g., Which is harder, a potato or a mushroom?) or a geometric dimension (size and shape; e.g., Which is larger, a pumpkin or a cucumber?). In the second experiment, when given the name of an object and either a material (roughness and hardness) or geometric (size and shape) property participants rated the object on a scale from 1 to 4. Both experiments were designed to examine the underlying neural substrate that supports the processing of material object properties with respect to geometric properties. Considering the relative amount of activation across the two types of object properties, we found that (1) the interrogation of geometric features differentially evokes visual imagery which involves the region in and around the intraparietal sulcus, (2) the interrogation of material features differentially evokes the processing of semantic object representations which involves the inferior extrastriate region, and (3) the lateral occipital cortex (LOC) responds to shape processing regardless of whether the feature being queried is a material or geometric feature.

Tactile estimation of the roughness of gratings yields a graded response in the human brain: an fMRI study.

Human subjects can tactually estimate the magnitude of surface roughness. Although many psychophysical and neurophysiological experiments have elucidated the peripheral neural mechanisms that underlie tactile roughness estimation, the associated cortical mechanisms are not well understood. To identify the brain regions responsible for the tactile estimation of surface roughness, we used functional magnetic resonance imaging (fMRI). We utilized a combination of categorical (subtraction) and parametric factorial approaches wherein roughness was varied during both the task and its control. Fourteen human subjects performed a tactile roughness-estimation task and received the identical tactile stimulation without estimation (no-estimation task). The bilateral parietal operculum (PO), insula and right lateral prefrontal cortex showed roughness-related activation. The bilateral PO and insula showed activation during the no-estimation task, and hence might represent the sensory-based processing during roughness estimation. By contrast, the right prefrontal cortex is more related to the cognitive processing, as there was activation during the estimation task compared with the no-estimation task, but little activation was observed during the no-estimation task in comparison with rest. The lateral prefrontal area might play an important cognitive role in tactile estimation of surface roughness, whereas the PO and insula might be involved in the sensory processing that is important for estimating surface roughness.

Haptic identification of common objects: effects of constraining the manual exploration process.

In this article, we address the effects on haptic recognition of common objects when manual exploration is constrained by using two kinds of rigid links--sheaths (Experiment 1A) and probes (Experiments 1B and 2). The collective effects of five different constraints are considered, including three from previous research (i.e., reducing the number of end effectors, wearing a compliant finger cover, and splinting the fingers; Klatzky, Loomis, Lederman, Wake, & Fujita, 1993) and from two current constraints (i.e., wearing a rigid finger sheath and using a rigid probe). The resulting impairments are interpreted in terms of the loss of somatosensory information from cutaneous and/or kinesthetic inputs. In addition, we relate the results to the design of haptic interfaces for teleoperation and virtual environments, which share some of the same reduction of sensory cues that we have produced experimentally.

Haptic processing of the location of a known property: does knowing what you've touched tell you where it is?

The relationship between knowing where a haptic property is located and knowing what it is was investigated using a haptic-search paradigm. Across trials, from one to six stimuli were presented simultaneously to varying combinations of the middle three fingertips of both hands. Participants reported the presence/absence of a target or its location for four perceptual dimensions: rough/smooth, edge/no edge, relative position (right/left), and relative orientation (right/left). Reaction time data were plotted as a function of set size. The slope data indicated no difference in processing load for location as compared to identity processing. However, the intercept data did reveal a cost associated with processing location information. Location information was not obtained for "free" when identity was processed. The data also supported a critical distinction between material and edge dimensions versus geometric dimensions, as the size of the cost associated with processing location was larger for spatial than for intensive stimuli.

Haptic face recognition and prosopagnosia.

Cases of cross-modal influence have been observed since the beginning of psychological science. Yet some abilities like face recognition are traditionally only investigated in the visual domain. People with normal visual face-recognition capacities identify inverted faces more poorly than upright faces. An abnormal pattern of performance with inverted faces by prosopagnosic individuals is characteristically interpreted as evidence for a deficit in configural processing essential for normal face recognition. We investigated whether such problems are unique to vision by examining face processing by hand in a prosopagnosic individual. We used the haptic equivalent of the visual-inversion paradigm to investigate haptic face recognition. If face processing is specific to vision, our participant should not show difficulty processing faces haptically and should perform with the same ease as normal controls. Instead, we show that a prosopagnosic individual cannot haptically recognize faces. Moreover, he shows similar abnormal inversion effects by hand and eye. These results suggest that face-processing deficits can be found across different input modalities. Our findings also extend the notion of configural processing to haptic face and object recognition.

Watching a cursor distorts haptically guided reproduction of mouse movement.

Participants moved a mouse along a force-feedback-defined linear path, either without vision or while watching a cursor set to 1 of 3 levels of visual:haptic gain (all >1:1). They attempted to haptically reproduce the movement without visual feedback. Errors increased with gain, reaching 70% overestimation at the highest gain. Forewarning participants about gain variability did not eliminate this effect. The gain level was potentially cued during the movement by the mismatch between visual feedback and kinesthetic feedback. Moreover, because participants did not achieve cursor-speed constancy across gain levels, visual speed was another cue to gain. Collectively, these cues failed to prevent visual distortion of movement reproduction.