Biomimetic sensory feedback through peripheral nerve stimulation improves dexterous use of a bionic hand.

We describe use of a bidirectional neuromyoelectric prosthetic hand that conveys biomimetic sensory feedback. Electromyographic recordings from residual arm muscles were decoded to provide independent and proportional control of a six-DOF prosthetic hand and wrist-the DEKA LUKE arm. Activation of contact sensors on the prosthesis resulted in intraneural microstimulation of residual sensory nerve fibers through chronically implanted Utah Slanted Electrode Arrays, thereby evoking tactile percepts on the phantom hand. With sensory feedback enabled, the participant exhibited greater precision in grip force and was better able to handle fragile objects. With active exploration, the participant was also able to distinguish between small and large objects and between soft and hard ones. When the sensory feedback was biomimetic-designed to mimic natural sensory signals-the participant was able to identify the objects significantly faster than with the use of traditional encoding algorithms that depended on only the present stimulus intensity. Thus, artificial touch can be sculpted by patterning the sensory feedback, and biologically inspired patterns elicit more interpretable and useful percepts.

Vision is superior to touch in shape perception even with equivalent peripheral input.

Results from previous studies suggest that two-dimensional spatial patterns are processed similarly in vision and touch when the patterns are equated for effective size or when visual stimuli are blurred to mimic the spatial filtering of the skin. In the present study, we measured subjects' ability to perceive the shape of familiar and unfamiliar visual and tactile patterns to compare form processing in the two modalities. As had been previously done, the two-dimensional tactile and visual patterns were adjusted in size to stimulate an equivalent number of receptors in the two modalities. We also distorted the visual patterns, using a filter that accurately mimics the spatial filtering effected by the skin to further equate the peripheral images in the two modalities. We found that vision consistently outperformed touch regardless of the precise perceptual task and of how familiar the patterns were. Based on an examination of both the earlier and present data, we conclude that visual processing of both familiar and unfamiliar two-dimensional patterns is superior to its tactile counterpart except under very restricted conditions.

Behavioral demonstration of a somatosensory neuroprosthesis.

Tactile sensation is critical for effective object manipulation, but current prosthetic upper limbs make no provision for delivering somesthetic feedback to the user. For individuals who require use of prosthetic limbs, this lack of feedback transforms a mundane task into one that requires extreme concentration and effort. Although vibrotactile motors and sensory substitution devices can be used to convey gross sensations, a direct neural interface is required to provide detailed and intuitive sensory feedback. In light of this, we describe the implementation of a somatosensory prosthesis with which we elicit, through intracortical microstimulation (ICMS), percepts whose magnitude is graded according to the force exerted on the prosthetic finger. Specifically, the prosthesis consists of a sensorized finger, the force output of which is converted into a regime of ICMS delivered to primary somatosensory cortex through chronically implanted multi-electrode arrays. We show that the performance of animals (Rhesus macaques) on a tactile task is equivalent whether stimuli are delivered to the native finger or to the prosthetic finger.

Real-time implementation of biofidelic SA1 model for tactile feedback.

In order for the functionality of an upper-limb prosthesis to approach that of a real limb it must be able to, accurately and intuitively, convey sensory feedback to the limb user. This paper presents results of the real-time implementation of a 'biofidelic' model that describes mechanotransduction in Slowly Adapting Type 1 (SA1) afferent fibers. The model accurately predicts the timing of action potentials for arbitrary force or displacement stimuli and its output can be used as stimulation times for peripheral nerve stimulation by a neuroprosthetic device. The model performance was verified by comparing the predicted action potential (or spike) outputs against measured spike outputs for different vibratory stimuli. Furthermore experiments were conducted to show that, like real SA1 fibers, the model's spike rate varies according to input pressure and that a periodic 'tapping' stimulus evokes periodic spike outputs.

Conveying tactile feedback in sensorized hand neuroprostheses using a biofidelic model of mechanotransduction.

One approach to conveying tactile feedback from sensorized neural prostheses is to characterize the neural signals that would normally be produced in an intact limb and reproduce them through electrical stimulation of the residual peripheral nerves. Toward this end, we have developed a model that accurately replicates the neural activity evoked by any dynamic stimulus in the three types of mechanoreceptive afferents that innervate the glabrous skin of the hand. The model takes as input the position of the stimulus as a function of time, along with its first (velocity), second (acceleration), and third (jerk) derivatives. This input is filtered and passed through an integrate-and-fire mechanism to generate a train of spikes as output. The major conclusion of this study is that the timing of individual spikes evoked in mechanoreceptive fibers innervating the hand can be accurately predicted by this model. We discuss how this model can be integrated in a sensorized prosthesis and show that the activity in a population of simulated afferents conveys information about the location, timing, and magnitude of contact between the hand and an object.

The tactile integration of local motion cues is analogous to its visual counterpart.

The visual and somatosensory systems have been shown to process spatial information similarly. Here we investigate tactile motion processing using stimuli whose perceptual properties have been well established in vision research, namely superimposed gratings (plaids), barber poles, and bar fields. In both modalities, information about stimulus motion (speed and direction) conveyed by neurons at low levels of sensory processing is ambiguous, a conundrum known as the aperture problem. Our results suggest that the tactile perception of motion, analogous to its visual counterpart, operates in multiple stages: first, the perceived direction of motion is determined by a majority vote from local motion detectors, which are subject to the aperture problem. As in vision, the conflict between the cues from terminators and other local motion cues is gradually resolved over time so that the perceived direction approaches the veridical direction of motion.

The tactile perception of stimulus orientation.

Studies of the visual system suggest that, at an early stage of form processing, a stimulus is represented as a set of contours and that a critical feature of these local contours is their orientation. Here, we characterize the ability of human observers to identify or discriminate the orientation of bars and edges presented to the distal fingerpad. The experiments were performed using a 400-probe stimulator that allowed us to flexibly deliver stimuli across a wide range of conditions. Orientation thresholds, approximately 20 degrees on average, varied only slightly across modes of stimulus presentation (scanned or indented), stimulus amplitudes, scanning speeds, and different stimulus types (bars or edges). The tactile orientation acuity was found to be poorer than its visual counterpart for stimuli of similar aspect ratio, contrast, and size. This result stands in contrast to the equivalent spatial acuity of the two systems (at the limit set by peripheral innervation density) and to the results of studies of tactile and visual letter recognition, which show that the two modalities yield comparable performance when stimuli are scaled appropriately.

Texture perception through direct and indirect touch: an analysis of perceptual space for tactile textures in two modes of exploration.

Considerable information about the texture of objects can be perceived remotely through a probe. It is not clear, however, how texture perception with a probe compares with texture perception with the bare finger. Here we investigate the perception of a variety of textured surfaces encountered daily (e.g., corduroy, paper, and rubber) using the two scanning modes - direct touch through the finger and indirect touch through a probe held in the hand - in two tasks. In the first task, subjects rated the overall pair-wise dissimilarity of the textures. In the second task, subjects rated each texture along three continua, namely, perceived roughness, hardness, and stickiness of the surfaces, shown previously as the primary dimensions of texture perception in direct touch. From the dissimilarity judgment experiment, we found that the texture percept is similar though not identical in the two scanning modes. From the adjective rating experiments, we found that while roughness ratings are similar, hardness and stickiness ratings tend to differ between scanning conditions. These differences between the two modes of scanning are apparent in perceptual space for tactile textures based on multidimensional scaling (MDS) analysis. Finally, we demonstrate that three physical quantities, vibratory power, compliance, and friction carry roughness, hardness, and stickiness information, predicting perceived dissimilarity of texture pairs with indirect touch. Given that different types of texture information are processed by separate groups of neurons across direct and indirect touch, we propose that the neural mechanisms underlying texture perception differ between scanning modes.

Influence of visual motion on tactile motion perception.

Subjects were presented with pairs of tactile drifting sinusoids and made speed discrimination judgments. On some trials, a visual drifting sinusoid, which subjects were instructed to ignore, was presented simultaneously with one of the two tactile stimuli. When the visual and tactile gratings drifted in the same direction (i.e., from left to right), the visual distractors were found to increase the perceived speed of the tactile gratings. The effect of the visual distractors was proportional to their temporal frequency but not to their perceived speed. When the visual and tactile gratings drifted in opposite directions, the distracting effect of the visual distractors was either substantially reduced or, in some cases, reversed (i.e., the distractors slowed the perceived speed of the tactile gratings). This result suggests that the observed visual-tactile interaction is dependent on motion and not simply on the oscillations inherent in drifting sinusoids. Finally, we find that disrupting the temporal synchrony between the visual and tactile stimuli eliminates the distracting effect of the visual stimulus. We interpret this latter finding as evidence that the observed visual-tactile interaction operates at the sensory level and does not simply reflect a response bias.

Temporal factors in tactile spatial acuity: evidence for RA interference in fine spatial processing.

We investigated the extent to which subjects' ability to perceive the fine spatial structure of a stimulus depends on its temporal properties (namely the frequency at which it vibrates). Subjects were presented with static or vibrating gratings that varied in spatial period (1-8 mm) and vibratory frequency (5-80 Hz) and judged the orientation of the gratings, presented either parallel or perpendicular to the long axis of the finger. We found that the grating orientation threshold (GOT)-the spatial period at which subjects can reliably discriminate the orientation of the grating-increased as the vibratory frequency of the gratings increased. As the spatial modulation of SA1 and RA afferent fibers has been found to be independent of vibratory frequency, the frequency dependence of spatial acuity cannot be attributed to changes in the quality of the peripheral signal. Furthermore, we found GOTs to be relatively independent of stimulus amplitude, so the low spatial acuity at high flutter frequencies does not appear to be due to an inadequacy in the strength of the afferent response at those frequencies. We hypothesized that the RA signal, the strength of which increases with vibratory frequency, interfered with the spatially modulated signal conveyed by SA1 fibers. Consistent with this hypothesis, we found that adapting RA afferent fibers improved spatial acuity, as gauged by GOTs, at the high flutter frequencies.

SA1 and RA afferent responses to static and vibrating gratings.

SA1 and RA afferent fibers differ both in their ability to convey information about the fine spatial structure of tactile stimuli and in their frequency sensitivity profiles. In the present study, we investigated the extent to which the spatial resolution of the signal conveyed by SA1 and RA fibers depends on the temporal properties of the stimulus. To that end, we recorded the responses evoked in SA1 and RA fibers of macaques by static and vibrating gratings that varied in spatial period, vibratory frequency, and amplitude. Gratings were oriented either parallel to the long axis of the finger (vertical) or perpendicular to it (horizontal). We examined the degree to which afferent responses were dependent on the spatial period, vibratory frequency, amplitude, and orientation of the gratings. We found that the spatial modulation of the afferent responses increased as the spatial period of the gratings increased; the spatial modulation was the same for static and vibrating gratings, despite large differences in evoked spike rates; the spatial modulation in SA1 responses was independent of stimulus amplitude over the range of amplitudes tested, whereas RA modulation decreased slightly as the stimulus amplitude increased; vertical gratings evoked stronger and more highly modulated responses than horizontal gratings; the modulation in SA1 responses was higher than that in RA responses at all frequencies and amplitudes. The behavioral consequences of these neurophysiological findings are examined in a companion paper.

Time-course of vibratory adaptation and recovery in cutaneous mechanoreceptive afferents.

Extended suprathreshold vibratory stimulation applied to the skin results in a desensitization of cutaneous mechanoreceptive afferents. In a companion paper, we describe the dependence of the threshold shift on the parameters of the adapting stimulus and discuss neural mechanisms underlying afferent adaptation. Here we describe the time-course of afferent adaptation and recovery. We found that absolute and entrainment thresholds rise and fall exponentially during adaptation and recovery with time constants that vary with fiber type. slowly adapting type I (SA1) afferents adapt most rapidly, and pacinian (PC) afferents adapt most slowly, whereas rapidly adapting (RA) afferents exhibit intermediate rates of adaptation; SA1 fibers also recover more rapidly from adaptation than RA and PC fibers. We also showed that threshold adaptation is accompanied by a shift in the timing of the spikes within individual cycles of the adapting stimulus (i.e., a shift in the impulse phase). We invoked an integrate-and-fire model to explore possible mechanisms underlying afferent adaptation. Finally, we found that the time-course of afferent adaptation is more rapid than that of its psychophysical counterpart, as is the time-course of recovery from adaptation, suggesting that central factors play a role in the psychophysical phenomenon.

Vibratory adaptation of cutaneous mechanoreceptive afferents.

The objective of this study was to investigate the effects of extended suprathreshold vibratory stimulation on the sensitivity of slowly adapting type 1 (SA1), rapidly adapting (RA), and Pacinian (PC) afferents. To that end, an algorithm was developed to track afferent absolute (I0) and entrainment (I1) thresholds as they change over time. We recorded afferent responses to periliminal vibratory test stimuli, which were interleaved with intense vibratory conditioning stimuli during the adaptation period of each experimental run. From these measurements, the algorithm allowed us to infer changes in the afferents' sensitivity. We investigated the stimulus parameters that affect adaptation by assessing the degree to which adaptation depends on the amplitude and frequency of the adapting stimulus. For all three afferent types, I0 and I1 increased with increasing adaptation frequency and amplitude. The degree of adaptation seems to be independent of the firing rate evoked in the afferent by the conditioning stimulus. In the analysis, we distinguished between additive adaptation (in which I0 and I1 shift equally) and multiplicative effects (in which the ratio I1/I0 remains constant). RA threshold shifts are almost perfectly additive. SA1 threshold shifts are close to additive and far from multiplicative (I1 threshold shifts are twice the I0 shifts). PC shifts are more difficult to classify. We used an integrate-and-fire model to study the possible neural mechanisms. A change in transducer gain predicts a multiplicative change in I0 and I1 and is thus ruled out as a mechanism underlying SA1 and RA adaptation. A change in the resting action potential threshold predicts equal, additive change in I0 and I1 and thus accounts well for RA adaptation. A change in the degree of refractoriness during the relative refractory period predicts an additional change in I1 such as that observed for SA1 fibers. We infer that adaptation is caused by an increase in spiking thresholds produced by ion flow through transducer channels in the receptor membrane. In a companion paper, we describe the time-course of vibratory adaptation and recovery for SA1, RA, and PC fibers.

Vibrotaction and texture perception.

Several recent studies support Katz's hypothesis that vibrotaction plays a role in the perception of tactile textures with elements too small and closely spaced to be processed spatially. For example, eliminating vibration by preventing movement of a stimulus surface across the skin compromises psychophysical scaling and discrimination of fine, but not coarse, textures. Fine-texture discrimination is also impaired when vibrotactile channels are desensitized by adaptation. A role for vibrotaction in texture perception is plausible, given the keenness of this submodality: the sensory qualities produced by a sinusoidal vibration uniquely specify its frequency and amplitude, and subjects can distinguish some complex vibrations that differ in waveform but have the same spectral components. Finally, imposed vibration can modify the perceived texture of a haptically-examined surface. Taken together, these lines of evidence support the view that vibrotaction is both necessary and sufficient for the perception of fine tactile textures.

Vibrotactile adaptation impairs discrimination of fine, but not coarse, textures.

The effect of vibrotactile adaptation on the ability to discriminate textured surfaces was examined in three experiments. The surfaces were rectilinear arrays of pyramids produced by etching of silicon wafers. Adaptation to 100-Hz vibration severely hampered discrimination of surfaces with spatial periods below 100 microm (Experiment 1), but had little effect on the discrimination of coarser textures (Experiment 2). To determine which vibrotactile channel--Rapidly Adapting or Pacinian--plays the larger role in mediating the discrimination of fine textures, widely separated adapting frequencies (10 and 250 Hz) were used in Experiment 3. The fact that high- but not low-frequency adaptation interfered with discrimination suggests that the Pacinian system contributes importantly to this ability. Taken as a whole, the results of this study strongly support the duplex theory of tactile texture perception, according to which different mechanisms--spatial and vibrotactile--mediate the perception of coarse and fine textures, respectively.

Complex tactile waveform discrimination.

Complex vibrotactile waveforms consisting of two superimposed sinusoids at varying phases were presented to the fingertip, and observers made "same-different" judgments. It was found that the low-frequency (10Hz+30Hz) waveforms were discriminable from one another while discrimination of the high-frequency (100Hz+300Hz) vibrations was poor. High-frequency adaptation did not impair discrimination of the low-frequency waveforms, suggesting that the RA channel mediated discrimination. Low-frequency adaptation impaired discrimination of the high-frequency stimuli, suggesting that the RA channel likewise mediated the modest level of performance observed in the absence of an adapting stimulus. The results indicate that this channel encodes complex waveforms temporally. A simple model for low-frequency waveform discrimination is proposed. The results obtained with the high-frequency complex waveforms are compatible with the hypothesis that the PC channel integrates stimulus energy over time.