Early blindness modulates haptic object recognition.

Haptic object recognition is usually an efficient process although slower and less accurate than its visual counterpart. The early loss of vision imposes a greater reliance on haptic perception for recognition compared to the sighted. Therefore, we may expect that congenitally blind persons could recognize objects through touch more quickly and accurately than late blind or sighted people. However, the literature provided mixed results. Furthermore, most of the studies on haptic object recognition focused on performance, devoting little attention to the exploration procedures that conducted to that performance. In this study, we used iCube, an instrumented cube recording its orientation in space as well as the location of the points of contact on its faces. Three groups of congenitally blind, late blind and age and gender-matched blindfolded sighted participants were asked to explore the cube faces where little pins were positioned in varying number. Participants were required to explore the cube twice, reporting whether the cube was the same or it differed in pins disposition. Results showed that recognition accuracy was not modulated by the level of visual ability. However, congenitally blind touched more cells simultaneously while exploring the faces and changed more the pattern of touched cells from one recording sample to the next than late blind and sighted. Furthermore, the number of simultaneously touched cells negatively correlated with exploration duration. These findings indicate that early blindness shapes haptic exploration of objects that can be held in hands.

Combined spatial and frequency encoding for electrotactile feedback of myoelectric signals.

Electrotactile stimulation has been commonly used in human-machine interfaces to provide feedback to the user, thereby closing the control loop and improving performance. The encoding approach, which defines the mapping of the feedback information into stimulation profiles, is a critical component of an electrotactile interface. Ideally, the encoding will provide a high-fidelity representation of the feedback variable while being easy to perceive and interpret by the subject. In the present study, we performed a closed-loop experiment wherein discrete and continuous coding schemes are combined to exploit the benefits of both techniques. Subjects performed a muscle activation-matching task relying solely on electrotactile feedback representing the generated myoelectric signal (EMG). In particular, we investigated the performance of two different coding schemes (spatial and spatial combined with frequency) at two feedback resolutions (low: 3 and high: 5 intervals). In both schemes, the stimulation electrodes were placed circumferentially around the upper arm. The magnitude of the normalized EMG was divided into intervals, and each electrode was associated with one interval. When the generated EMG entered one of the intervals, the associated electrode started stimulating. In the combined encoding, the additional frequency modulation of the active electrode also indicated the momentary magnitude of the signal within the interval. The results showed that combined coding decreased the undershooting rate, variability and absolute deviation when the resolution was low but not when the resolution was high, where it actually worsened the performance. This demonstrates that combined coding can improve the effectiveness of EMG feedback, but that this effect is limited by the intrinsic variability of myoelectric control. Our findings, therefore, provide important insights as well as elucidate limitations of the information encoding methods when using electrotactile stimulation to convey a feedback signal characterized by high variability (EMG biofeedback).

Mental Rotation Skill Shapes Haptic Exploration Strategies.

Haptic exploration strategies have been traditionally studied focusing on hand movements and neglecting how objects are moved in space. However, in daily life situations touch and movement cannot be disentangled. Furthermore, the relation between object manipulation as well as performance in haptic tasks and spatial skill is still little understood. In this study, we used iCube, a sensorized cube recording its orientation in space as well as the location of the points of contact on its faces. Participants had to explore the cube faces where little pins were positioned in varying number and count the number of pins on the faces with either even or odd number of pins. At the end of this task, they also completed a standard visual mental rotation test (MRT). Results showed that higher MRT scores were associated with better performance in the task with iCube both in term of accuracy and exploration speed and exploration strategies associated with better performance were identified. High performers tended to rotate the cube so that the explored face had the same spatial orientation (i.e., they preferentially explored the upward face and rotated iCube to explore the next face in the same orientation). They also explored less often twice the same face and were faster and more systematic in moving from one face to the next. These findings indicate that iCube could be used to infer subjects' spatial skill in a more natural and unobtrusive fashion than with standard MRTs.

Non-informative vision improves spatial tactile discrimination on the shoulder but does not influence detection sensitivity.

Vision of the body has been reported to improve tactile acuity even when vision is not informative about the actual tactile stimulation. However, it is currently unclear whether this effect is limited to body parts such as hand, forearm or foot that can be normally viewed, or it also generalizes to body locations, such as the shoulder, that are rarely before our own eyes. In this study, subjects consecutively performed a detection threshold task and a numerosity judgment task of tactile stimuli on the shoulder. Meanwhile, they watched either a real-time video showing their shoulder or simply a fixation cross as control condition. We show that non-informative vision improves tactile numerosity judgment which might involve tactile acuity, but not tactile sensitivity. Furthermore, the improvement in tactile accuracy modulated by vision seems to be due to an enhanced ability in discriminating the number of adjacent active electrodes. These results are consistent with the view that bimodal visuotactile neurons sharp tactile receptive fields in an early somatosensory map, probably via top-down modulation of lateral inhibition.

Temporal Asynchrony but Not Total Energy Nor Duration Improves the Judgment of Numerosity in Electrotactile Stimulation.

Stroke patients suffer from impairments of both motor and somatosensory functions. The functional recovery of upper extremities is one of the primary goals of rehabilitation programs. Additional somatosensory deficits limit sensorimotor function and significantly affect its recovery after the neuromotor injury. Sensory substitution systems, providing tactile feedback, might facilitate manipulation capability, and improve patient's dexterity during grasping movements. As a first step toward this aim, we evaluated the ability of healthy subjects in exploiting electrotactile feedback on the shoulder to determine the number of perceived stimuli in numerosity judgment tasks. During the experiment, we compared four different stimulation patterns (two simultaneous: short and long, intermittent and sequential) differing in total duration, total energy, or temporal synchrony. The experiment confirmed that the subject ability to enumerate electrotactile stimuli decreased with increasing the number of active electrodes. Furthermore, we found that, in electrotactile stimulation, the temporal coding schemes, and not total energy or duration modulated the accuracy in numerosity judgment. More precisely, the sequential condition resulted in significantly better numerosity discrimination than intermittent and simultaneous stimulation. These findings, together with the fact that the shoulder appeared to be a feasible stimulation site to communicate tactile information via electrotactile feedback, can serve as a guide to deliver tactile feedback to proximal areas in stroke survivors who lack sensory integrity in distal areas of their affected arm, but retain motor skills.

Enhancing general spatial skills of young visually impaired people with a programmable distance discrimination training: a case control study.

BACKGROUND: The estimation of relative distance is a perceptual task used extensively in everyday life. This important skill suffers from biases that may be more pronounced when estimation is based on haptics. This is especially true for the blind and visually impaired, for which haptic estimation of distances is paramount but not systematically trained. We investigated whether a programmable tactile display, used autonomously, can improve distance discrimination ability in blind and severely visually impaired youngsters between 7 and 22 years-old. METHODS: Training consisted of four weekly sessions in which participants were asked to haptically find, on the programmable tactile display, the pairs of squares which were separated by the shortest and longest distance in tactile images with multiple squares. A battery of haptic tests with raised-line drawings was administered before and after training, and scores were compared to those of a control group that did only the haptic battery, without doing the distance discrimination training on the tactile display. RESULTS: Both blind and severely impaired youngsters became more accurate and faster at the task during training. In haptic battery results, blind and severely impaired youngsters who used the programmable display improved in three and two tests, respectively. In contrast, in the control groups, the blind control group improved in only one test, and the severely visually impaired in no tests. CONCLUSIONS: Distance discrimination skills can be trained equally well in both blind and severely impaired participants. More importantly, autonomous training with the programmable tactile display had generalized effects beyond the trained task. Participants improved not only in the size discrimination test but also in memory span tests. Our study shows that tactile stimulation training that requires minimal human assistance can effectively improve generic spatial skills.

Improving spatial working memory in blind and sighted youngsters using programmable tactile displays.

OBJECTIVE: To investigate whether training with tactile matrices displayed with a programmable tactile display improves recalling performance of spatial images in blind, low-vision and sighted youngsters. To code and understand the behavioral underpinnings of learning two-dimensional tactile dispositions, in terms of spontaneous exploration strategies. METHODS: Three groups of blind, low-vision and sighted youngsters between 6 and 18 years old performed four training sessions with a weekly schedule in which they were asked to memorize single or double spatial layouts, featured as two-dimensional matrices. RESULTS: Results showed that all groups of participants significantly improved their recall performance compared to the first session baseline in the single-matrix task. No statistical difference in performance between groups emerged in this task. Instead, the learning effect in visually impaired participants is reduced in the double-matrix task, whereas it is still robust in blindfolded sighted controls. We also coded tactile exploration strategies in both tasks and their correlation with performance. Sighted youngsters, in particular, favored a proprioceptive exploration strategy. Finally, performance in the double-matrix task negatively correlated with using one hand and positively correlated with a proprioceptive strategy. CONCLUSION: The results of our study indicate that blind persons do not easily process two separate spatial layouts. However, rehabilitation programs promoting bi-manual and proprioceptive approaches to tactile exploration might help improve spatial abilities. Finally, programmable tactile displays are an effective way to make spatial and graphical configurations accessible to visually impaired youngsters and they can be profitably exploited in rehabilitation.

Updated Tactile Feedback with a Pin Array Matrix Helps Blind People to Reduce Self-Location Errors.

Autonomous navigation in novel environments still represents a challenge for people with visual impairment (VI). Pin array matrices (PAM) are an effective way to display spatial information to VI people in educative/rehabilitative contexts, as they provide high flexibility and versatility. Here, we tested the effectiveness of a PAM in VI participants in an orientation and mobility task. They haptically explored a map showing a scaled representation of a real room on the PAM. The map further included a symbol indicating a virtual target position. Then, participants entered the room and attempted to reach the target three times. While a control group only reviewed the same, unchanged map on the PAM between trials, an experimental group also received an updated map representing, in addition, the position they previously reached in the room. The experimental group significantly improved across trials by having both reduced self-location errors and reduced completion time, unlike the control group. We found that learning spatial layouts through updated tactile feedback on programmable displays outperforms conventional procedures on static tactile maps. This could represent a powerful tool for navigation, both in rehabilitation and everyday life contexts, improving spatial abilities and promoting independent living for VI people.

The Effect of Programmable Tactile Displays on Spatial Learning Skills in Children and Adolescents of Different Visual Disability.

Vision loss has severe impacts on physical, social and emotional well-being. The education of blind children poses issues as many scholar disciplines (e.g., geometry, mathematics) are normally taught by heavily relying on vision. Touch-based assistive technologies are potential tools to provide graphical contents to blind users, improving learning possibilities and social inclusion. Raised-lines drawings are still the golden standard, but stimuli cannot be reconfigured or adapted and the blind person constantly requires assistance. Although much research concerns technological development, little work concerned the assessment of programmable tactile graphics, in educative and rehabilitative contexts. Here we designed, on programmable tactile displays, tests aimed at assessing spatial memory skills and shapes recognition abilities. Tests involved a group of blind and a group of low vision children and adolescents in a four-week longitudinal schedule. After establishing subject-specific difficulty levels, we observed a significant enhancement of performance across sessions and for both groups. Learning effects were comparable to raised paper control tests: however, our setup required minimal external assistance. Overall, our results demonstrate that programmable maps are an effective way to display graphical contents in educative/rehabilitative contexts. They can be at least as effective as traditional paper tests yet providing superior flexibility and versatility.

Conditioned sounds enhance visual processing.

This psychophysics study investigated whether prior auditory conditioning influences how a sound interacts with visual perception. In the conditioning phase, subjects were presented with three pure tones ( =  conditioned stimuli, CS) that were paired with positive, negative or neutral unconditioned stimuli. As unconditioned reinforcers we employed pictures (highly pleasant, unpleasant and neutral) or monetary outcomes (+50 euro cents, -50 cents, 0 cents). In the subsequent visual selective attention paradigm, subjects were presented with near-threshold Gabors displayed in their left or right hemifield. Critically, the Gabors were presented in synchrony with one of the conditioned sounds. Subjects discriminated whether the Gabors were presented in their left or right hemifields. Participants determined the location more accurately when the Gabors were presented in synchrony with positive relative to neutral sounds irrespective of reinforcer type. Thus, previously rewarded relative to neutral sounds increased the bottom-up salience of the visual Gabors. Our results are the first demonstration that prior auditory conditioning is a potent mechanism to modulate the effect of sounds on visual perception.

Looming sounds enhance orientation sensitivity for visual stimuli on the same side as such sounds.

Several recent multisensory studies show that sounds can influence visual processing. Some visual judgments can be enhanced for visual stimuli near a sound occurring around the same time. A recent TMS study (Romei et al. 2009) indicates looming sounds might influence visual cortex particularly strongly. But unlike most previous behavioral studies of possible audio-visual exogenous effects, TMS phosphene thresholds rather than judgments of external visual stimuli were measured. Moreover, the visual hemifield assessed relative to the hemifield of the sound was not varied. Here, we compared the impact of looming sounds to receding or "static" sounds, using auditory stimuli adapted from Romei et al. (2009), but now assessing any influence on visual orientation discrimination for Gabor patches (well-known to involve early visual cortex) when appearing in the same hemifield as the sound or on the opposite side. The looming sounds that were effective in Romei et al. (2009) enhanced visual orientation sensitivity (d') here on the side of the sound, but not for the opposite hemifield. This crossmodal, spatially specific effect was stronger for looming than receding or static sounds. Similarly to Romei et al. (2009), the differential effect for looming sounds was eliminated when using white noise rather than structured sounds. Our new results show that looming structured sounds can specifically benefit visual orientation sensitivity in the hemifield of the sound, even when the sound provides no information about visual orientation itself.

Independent mechanisms for ventriloquism and multisensory integration as revealed by theta-burst stimulation.

The visual and auditory systems often concur to create a unified perceptual experience and to determine the localization of objects in the external world. Co-occurring auditory and visual stimuli in spatial coincidence are known to enhance performance of auditory localization due to the integration of stimuli from different sensory channels (i.e. multisensory integration). However, auditory localization of audiovisual stimuli presented at spatial disparity might also induce a mislocalization of the sound towards the visual stimulus (i.e. ventriloquism effect). Using repetitive transcranial magnetic stimulation we tested the role of right temporoparietal (rTPC), right occipital (rOC) and right posterior parietal (rPPC) cortex in an auditory localization task in which indices of ventriloquism and multisensory integration were computed. We found that suppression of rTPC excitability by means of continuous theta-burst stimulation (cTBS) reduced multisensory integration. No similar effect was found for cTBS over rOC. Moreover, inhibition of rOC, but not of rTPC, suppressed the visual bias in the contralateral hemifield. In contrast, cTBS over rPPC did not produce any modulation of ventriloquism or integrative effects. The double dissociation found in the present study suggests that ventriloquism and audiovisual multisensory integration are functionally independent phenomena and may be underpinned by partially different neural circuits.

Temporo-nasal asymmetry in multisensory integration mediated by the Superior Colliculus.

Temporo-nasal asymmetry in visual responses has been observed in many behavioural studies. These observations have typically been attributed to the anatomical asymmetry of fibres projecting to the Superior Colliculus (SC), even though this attribution is debated. The present study investigates temporo-nasal asymmetry in multisensory integration, and, by exploiting the absence of S-cone input to the SC, measures a behavioural response dependent strictly on the activity of the SC itself. We used a redundant signal paradigm for simple reaction times, with visual stimuli (red or purple) presented in either the temporal or the nasal hemifield. Participants responded more quickly to concurrent audio-visual (AV) stimuli than to either an auditory or a visual stimulus alone, an established phenomenon known as the Redundant Target Effect (RTE). The nature of this effect was dependent on the colour of the visual stimuli, suggesting its modulation by collicular circuits. When spatially-coincident audio-visual stimuli were visible to the SC (i.e. red stimuli), the RTE depended on a neural coactivation mechanism, suggesting an integration of multisensory information. When using stimuli invisible to the SC (i.e. purple stimuli), the RTE depended only on a simple statistical facilitation effect, in which the two sensory stimuli were processed by independent channels. Finally, we demonstrate that the multisensory integration effect was stronger for stimuli presented to the temporal hemifield than to the nasal hemifield. Taken together, these findings suggested that multisensory stimulation can be differentially effective depending on specific stimulus parameters.

Cross-modal localization in hemianopia: new insights on multisensory integration.

The superior colliculus (SC) has been implicated in the mediation of residual visual function in hemianopic patients, and has been shown to be capable of using multiple sensory cues to facilitate its localization functions. The aim of the present study was to examine the possibility that the SC could effect covert visual processes, via multisensory integration of auditory and visual stimuli in patients with visual field loss. To this aim hard-to-localize auditory targets were presented alone (unimodal condition) or with a visual stimulus (cross-modal condition) in either hemifield and at various spatial (0 degree, 16 degrees, 32 degrees) and temporal (0 ms, 500 ms) disparities. The results showed substantial field-specific differences. As expected, a visual stimulus in the intact hemifield induced a strong visual bias in auditory localization independent of the spatial disparities, and did so even when the two stimuli were temporally offset. In these spatially disparate conditions, the localization accuracy was markedly reduced. In the blind hemifield, however, the visual stimulus affected auditory localization only when it was coincident with that target in both space and time. In this circumstance auditory localization performance was markedly enhanced. This result strongly suggests that covert visual processes remain active in hemianopia, though they differ from those in the normal hemifield. A likely explanation of these differences is that enhancement and visual bias depend on different neural pathways: with the former dependent on circuits involving the superior colliculus, a structure involved in the integration of cues from multiple senses to facilitate orientation/localization; and the latter dependent on geniculo-striate circuits that facilitate more detailed analyses of the visual scene. Overall the present results not only enhance our understanding of the impact of covert visual processes in hemianopic patients, but also enhance our knowledge of how different brain regions areas contribute to processing cross-modal information.

Multisensory integration for orienting responses in humans requires the activation of the superior colliculus.

Animal studies have shown that the superior colliculus (SC) is important for synthesising information from multiple senses into a unified map of space. Here, we tested whether the SC is a critical neural substrate for multisensory spatial integration in humans. To do so, we took advantage of neurophysiological findings revealing that the SC does not receive direct projections from short-wavelength-sensitive S cones. In a simple reaction-time task, participants responded more quickly to concurrent peripheral (extra-foveal) audiovisual (AV) stimuli than to an auditory or visual stimulus alone, a phenomenon known as the redundant target effect (RTE). We show that the nature of this RTE was dependent on the colour of the visual stimulus. When using purple short-wavelength stimuli, to which the SC is blind, RTE was simply explained by probability summation, indicating that the redundant auditory and visual channels are independent. Conversely, with red long-wavelength stimuli, visible to the SC, the RTE was related to nonlinear neural summation, which constitutes evidence of integration of different sensory information. We also demonstrate that when AV stimuli were presented at fixation, so that the spatial orienting component of the task was reduced, neural summation was possible regardless of stimulus colour. Together, these findings provide support for a pivotal role of the SC in mediating multisensory spatial integration in humans, when behaviour involves spatial orienting responses.

The contribution of prefrontal cortex to global perception.

Recent research suggests a role of top-down modulatory signals on perceptual processing, particularly for the integration of local elementary information to form a global holistic percept. In this study we investigated whether prefrontal cortex may be instrumental in this top-down modulation in humans. We measured detection thresholds for perceiving a circle defined by a closed chain of grating patches in 6 patients with prefrontal lesions, 4 control patients with temporal lesions and 17 healthy control subjects. Performance of patients with prefrontal lesions was worse than that of patients with temporal lesions and normal controls when the patterns were sparse, requiring integration across relatively extensive regions of space, but similar to the control groups for denser patterns. The results clearly implicate the prefrontal cortex in the process of integrating elementary features into a holistic global percept, when the elements do not form a "pop-out" display.

Multisensory-mediated auditory localization.

Multisensory integration is a powerful mechanism for maximizing sensitivity to sensory events. We examined its effects on auditory localization in healthy human subjects. The specific objective was to test whether the relative intensity and location of a seemingly irrelevant visual stimulus would influence auditory localization in accordance with the inverse effectiveness and spatial rules of multisensory integration that have been developed from neurophysiological studies with animals [Stein and Meredith, 1993 The Merging of the Senses (Cambridge, MA: MIT Press)]. Subjects were asked to localize a sound in one condition in which a neutral visual stimulus was either above threshold (supra-threshold) or at threshold. In both cases the spatial disparity of the visual and auditory stimuli was systematically varied. The results reveal that stimulus salience is a critical factor in determining the effect of a neutral visual cue on auditory localization. Visual bias and, hence, perceptual translocation of the auditory stimulus appeared when the visual stimulus was supra-threshold, regardless of its location. However, this was not the case when the visual stimulus was at threshold. In this case, the influence of the visual cue was apparent only when the two cues were spatially coincident and resulted in an enhancement of stimulus localization. These data suggest that the brain uses multiple strategies to integrate multisensory information.