The Single-Pitch Texel: A flexible and practical texture-rendering algorithm.

As the number of applications for tactile feedback technology rapidly increases, so too does the need for efficient, flexible, and extensible representations of virtual textures. The previously introduced Single-Pitch Texel rendering algorithm offers designers the ability to produce textures with perceptually wide-band spectral characteristics while requiring very few input parameters. This paper expands on the capabilities of the rendering algorithm. Diverse families of fine textures, with widely varied spectral characteristics, were shown to be rendered reliably using the Texel algorithm. Furthermore, by leveraging an assistive algorithm, subjects were shown to consistently navigate the Texel parameter space in a matching task. Finally, a psychophysical study was conducted to demonstrate the rendering algorithm's resilience to spectral quantization, further reducing the data required to represent a virtual texture.

PixeLite: A Thin and Wearable High Bandwidth Electroadhesive Haptic Array.

We present PixeLite, a novel haptic device that produces distributed lateral forces on the fingerpad. PixeLite is 0.15 mm thick, weighs 1.00 g, and consists of a 4×4 array of electroadhesive brakes ("pucks") that are each 1.5 mm in diameter and spaced 2.5 mm apart. The array is worn on the fingertip and slid across an electrically grounded countersurface. It can produce perceivable excitation up to 500 Hz. When a puck is activated at 150 V at 5 Hz, friction variation against the countersurface causes displacements of 627 ± 59 µm. The displacement amplitude decreases as frequency increases, and at 150 Hz is 47 ± 6 µm. The stiffness of the finger, however, causes a substantial amount of mechanical puck-to-puck coupling, which limits the ability of the array to create spatially localized and distributed effects. A first psychophysical experiment showed that PixeLite's sensations can be localized to an area of about 30% of the total array area. A second experiment, however, showed that exciting neighboring pucks out of phase with one another in a checkerboard pattern did not generate perceived relative motion. Instead, mechanical coupling dominates the motion, resulting in a single frequency felt by the bulk of the finger.

A Low-Parameter Rendering Algorithm for Fine Textures.

This paper introduces a novel rendering algorithm for virtual textures, specifically those with characteristic length scales below 1 mm. By leveraging the relatively lossy mode of human tactile perception at this length scale, a virtual texture with wide-band spectral characteristics can be reduced to a spatial sequence of single-frequency texels, where each frequency is pulled stochastically from a distribution. A psychophysical study was conducted to demonstrate that, below a limiting physical texel length, virtual textures defined by identical frequency distributions are perceptually indiscriminable. Additionally, an exploratory study mapped the distribution parameters of the texel-based rendering to spectral characteristics of perceptually similar multi-frequency virtual textures.

Building a Navigable Fine Texture Design Space.

Friction modulation technology enables the creation of textural effects on flat haptic displays. However, an intuitive and manageably small design space for construction of such haptic textures remains an unfulfilled goal for user interface designers. In this paper, we explore perceptually relevant features of fine texture for use in texture construction and modification. Beginning with simple sinusoidal patterns of friction force that vary in frequency and amplitude, we define irregularity, essentially a variable amount of introduced noise, as a third building block of a texture pattern. We demonstrate using multidimensional scaling that all three parameters are scalable features perceptually distinct from each other. Additionally, participants' verbal descriptions of this 3-dimensional design space provide insight into their intuitive interpretation of the physical parameter changes.

Comparison of Wide-Band Vibrotactile and Friction Modulation Surface Gratings.

This article seeks to understand conditions under which virtual gratings produced via vibrotaction and friction modulation are perceived as similar and to find physical origins in the results. To accomplish this, we developed two single-axis devices, one based on electroadhesion and one based on out-of-plane vibration. The two devices had identical touch surfaces, and the vibrotactile device used a novel closed-loop controller to achieve precise control of out-of-plane plate displacement under varying load conditions across a wide ranget of frequencies. A first study measured the perceptual intensity equivalence curve of gratings generated under electroadhesion and vibrotaction across the 20-400 Hz frequency range. A second study assessed the perceptual similarity between two forms of skin excitation given the same driving frequency and same perceived intensity. Our results indicate that it is largely the out-of-plane velocity that predicts vibrotactile intensity relative to shear forces generated by friction modulation. A high degree of perceptual similarity between gratings generated through friction modulation and through vibrotaction is apparent and tends to scale with actuation frequency suggesting perceptual indifference to the manner of fingerpad actuation in the upper frequency range.

How the Mechanical Properties and Thickness of Glass Affect TPaD Performance.

One well-known class of surface haptic devices that we have called Tactile Pattern Displays (TPaDs) uses ultrasonic transverse vibrations of a touch surface to modulate fingertip friction. This article addresses the power consumption of glass TPaDs, which is an important consideration in the context of mobile touchscreens. In particular, based on existing ultrasonic friction reduction models, we consider how the mechanical properties (density and Young's modulus) and thickness of commonly-used glass formulations affect TPaD performance, namely the relation between its friction reduction ability and its real power consumption. Experiments performed with eight types of TPaDs and an electromechanical model for the fingertip-TPaD system indicate: 1) TPaD performance decreases as glass thickness increases; 2) TPaD performance increases as the Young's modulus and density of glass decrease; and 3) real power consumption of a TPaD decreases as the contact force increases. Proper applications of these results can lead to significant increases in TPaD performance.

Localizable Button Click Rendering via Active Lateral Force Feedback.

In this article, we have developed a novel button click rendering mechanism based on active lateral force feedback. The effect can be localized because electroadhesion between a finger and a surface can be localized. Psychophysical experiments were conducted to evaluate the quality of a rendered button click, which subjects judged to be acceptable. Both the experiment results and the subjects' comments confirm that this button click rendering mechanism has the ability to generate a range of realistic button click sensations that could match subjects' different preferences. We can, thus, generate a button click on a flat surface without macroscopic motion of the surface in the lateral or normal direction, and can localize this haptic effect to an individual finger.

UltraShiver: Lateral Force Feedback on a Bare Fingertip via Ultrasonic Oscillation and Electroadhesion.

We propose a new lateral force feedback device, the UltraShiver, which employs a combination of in-plane ultrasonic oscillation (around 30 kHz) and out-of-plane electroadhesion. It can achieve a strong active lateral force (400 mN) on the bare fingertip while operating silently. The lateral force is a function of pressing force, lateral vibration velocity, and electroadhesive voltage, as well as the relative phase between the velocity and voltage. In this paper, we perform experiments to investigate characteristics of the UltraShiver and their influence on lateral force. A lumped-parameter model is developed to understand the physical underpinnings of these influences. The model with frequency-weighted electroadhesion forces shows good agreement with experimental results. In addition, a Gaussian-like potential well is rendered as an application of the UltraShiver.

The Application of Tactile, Audible, and Ultrasonic Forces to Human Fingertips Using Broadband Electroadhesion.

We report an electroadhesive approach to controlling friction forces on sliding fingertips which is capable of producing vibrations across an exceedingly broad range of tactile, audible, and ultrasonic frequencies. Vibrations on the skin can be felt directly, and vibrations in the air can be heard emanating from the finger. Additionally, we report evidence from an investigation of the electrical dynamics of the system suggesting that an air gap at the skin/surface interface is primarily responsible for the induced electrostatic attraction underlying the electroadhesion effect. We developed an experimental apparatus capable of recording friction forces up to a frequency of 6 kHz, and used it to characterize two different electroadhesive systems, both of which exhibit flat force magnitude responses throughout the measurement range. These systems use custom electrical hardware to modulate a high frequency current and apply surprisingly low distortion, broadband forces to the skin. Recordings of skin vibrations with a laser Doppler vibrometer demonstrate the tactile capabilities of the system, while recordings of vibrations in the air with a MEMS microphone quantify the audible response and reveal the existence of ultrasonic forces applied to the skin via electronic friction modulation. Implications for surface haptic and audio-haptic displays are briefly discussed.

eShiver: Lateral Force Feedback on Fingertips through Oscillatory Motion of an Electroadhesive Surface.

We describe a new haptic force feedback device capable of creating lateral shear force on a bare fingertip-the eShiver. The eShiver creates a net lateral force from in-plane oscillatory motion of a surface synchronized with a "friction switch" based on Johnsen-Rahbek electroadhesion. Using an artificial finger, a maximum net lateral force of ±300 mN is achieved at 55 Hz lateral oscillation frequency, and net force is shown to be a function of velocity and applied voltage, as well as the phase between them. A second set of experiments is carried out on a human finger, and a lateral force of up to ±450 mN is achieved at a lateral oscillation frequency of 1,000 Hz. This force is reached at a peak lateral surface velocity of 400 mm/s and a peak applied voltage of 400 V. We develop a simple lumped parameter model of the eShiver, and a time domain simulation of the artificial finger is shown to agree with the experimental results. Three distinct zones of operation are found, which predict the limitations of force generation and which may be used for optimization. The human finger is found to be similar to the artificial finger in its dependence on actuation parameters, suggesting that the same lumped parameter model may be applied, albeit with different parameters. Curiously, the friction force due to Johnsen-Rahbek electroadhesion is found to increase substantially over time as the finger remains in contact with the surface. Considerations for optimizing the performance of the eShiver are discussed.

Human-in-the-loop active electrosense.

Active electrosense is a non-visual, short range sensing system used by weakly electric fish, enabling such fish to locate and identify objects in total darkness. Here we report initial findings from the use of active electrosense for object localization during underwater teleoperation with a virtual reality (VR) head-mounted display (HMD). The advantage of electrolocating with a VR system is that it naturally allows for aspects of the task that are difficult for a person to perform to be allocated to the computer. However, interpreting weak and incomplete patterns in the incoming data is something that people are typically far better at than computers. To achieve human-computer synergy, we integrated an active electrosense underwater robot with the Oculus Rift HMD. The virtual environment contains a visualization of the electric images of the objects surrounding the robot as well as various virtual fixtures that guide users to regions of higher information value. Initial user testing shows that these fixtures significantly reduce the time taken to localize an object, but may not increase the accuracy of the position estimate. Our results highlight the advantages of translating the unintuitive physics of electrolocation to an intuitive visual representation for accomplishing tasks in environments where imaging systems fail, such as in dark or turbid water.

Enhanced detection performance in electrosense through capacitive sensing.

Weakly electric fish emit an AC electric field into the water and use thousands of sensors on the skin to detect field perturbations due to surrounding objects. The fish's active electrosensory system allows them to navigate and hunt, using separate neural pathways and receptors for resistive and capacitive perturbations. We have previously developed a sensing method inspired by the weakly electric fish to detect resistive perturbations and now report on an extension of this system to detect capacitive perturbations as well. In our method, an external object is probed by an AC field over multiple frequencies. We present a quantitative framework that relates the response of a capacitive object at multiple frequencies to the object's composition and internal structure, and we validate this framework with an electrosense robot that implements our capacitive sensing method. We define a metric for comparing the electrosensory range of different underwater electrosense systems. For detecting non-conductive objects, we show that capacitive sensing performs better than resistive sensing by almost an order of magnitude using this measure, while for conductive objects there is a four-fold increase in performance. Capacitive sensing could therefore provide electric fish with extended sensing range for capacitive objects such as prey, and gives artificial electrolocation systems enhanced range for targets that are capacitive.

Multiple Fingers - One Gestalt.

The Gestalt theory of perception offered principles by which distributed visual sensations are combined into a structured experience ("Gestalt"). We demonstrate conditions whereby haptic sensations at two fingertips are integrated in the perception of a single object. When virtual bumps were presented simultaneously to the right hand's thumb and index finger during lateral arm movements, participants reported perceiving a single bump. A discrimination task measured the bump's perceived location and perceptual reliability (assessed by differential thresholds) for four finger configurations, which varied in their adherence to the Gestalt principles of proximity (small versus large finger separation) and synchrony (virtual spring to link movements of the two fingers versus no spring). According to models of integration, reliability should increase with the degree to which multi-finger cues integrate into a unified percept. Differential thresholds were smaller in the virtual-spring condition (synchrony) than when fingers were unlinked. Additionally, in the condition with reduced synchrony, greater proximity led to lower differential thresholds. Thus, with greater adherence to Gestalt principles, thresholds approached values predicted for optimal integration. We conclude that the Gestalt principles of synchrony and proximity apply to haptic perception of surface properties and that these principles can interact to promote multi-finger integration.

Coincidence avoidance principle in surface haptic interpretation.

When multiple fingertips experience force sensations, how does the brain interpret the combined sensation? In particular, under what conditions are the sensations perceived as separate or, alternatively, as an integrated whole? In this work, we used a custom force-feedback device to display force signals to two fingertips (index finger and thumb) as they traveled along collinear paths. Each finger experienced a pattern of forces that, taken individually, produced illusory virtual bumps, and subjects reported whether they felt zero, one, or two bumps. We varied the spatial separation between these bump-like force-feedback regions, from being much greater than the finger span to nearly exactly the finger span. When the bump spacing was the same as the finger span, subjects tended to report only one bump. We found that the results are consistent with a quantitative model of perception in which the brain selects a structural interpretation of force signals that relies on minimizing coincidence stemming from accidental alignments between fingertips and inferred surface structures.

Search efficiency for tactile features rendered by surface haptic displays.

Haptic interfaces controlled by a single fingertip or hand-held probe tend to display surface features individually, requiring serial search for multiple features. Novel surface haptic devices, however, have the potential to provide displays to multiple fingertips simultaneously, affording the possibility of parallel search. Using variable-friction surface haptic devices, we investigated the ability of participants to detect a target feature among a set of distractors in parallel across the fingers. We found that searches for a material property (slipperiness) and an illusory shape (virtual hole) were significantly impaired by distractors, while search for an abrupt discontinuity (virtual edge) was not. The efficiency of search for edges rendered by surface haptics suggests that they engage primitive detectors in the haptic perceptual system.

Development of a mechatronic platform and validation of methods for estimating ankle stiffness during the stance phase of walking.

The mechanical properties of human joints (i.e., impedance) are constantly modulated to precisely govern human interaction with the environment. The estimation of these properties requires the displacement of the joint from its intended motion and a subsequent analysis to determine the relationship between the imposed perturbation and the resultant joint torque. There has been much investigation into the estimation of upper-extremity joint impedance during dynamic activities, yet the estimation of ankle impedance during walking has remained a challenge. This estimation is important for understanding how the mechanical properties of the human ankle are modulated during locomotion, and how those properties can be replicated in artificial prostheses designed to restore natural movement control. Here, we introduce a mechatronic platform designed to address the challenge of estimating the stiffness component of ankle impedance during walking, where stiffness denotes the static component of impedance. The system consists of a single degree of freedom mechatronic platform that is capable of perturbing the ankle during the stance phase of walking and measuring the response torque. Additionally, we estimate the platform's intrinsic inertial impedance using parallel linear filters and present a set of methods for estimating the impedance of the ankle from walking data. The methods were validated by comparing the experimentally determined estimates for the stiffness of a prosthetic foot to those measured from an independent testing machine. The parallel filters accurately estimated the mechatronic platform's inertial impedance, accounting for 96% of the variance, when averaged across channels and trials. Furthermore, our measurement system was found to yield reliable estimates of stiffness, which had an average error of only 5.4% (standard deviation: 0.7%) when measured at three time points within the stance phase of locomotion, and compared to the independently determined stiffness values of the prosthetic foot. The mechatronic system and methods proposed in this study are capable of accurately estimating ankle stiffness during the foot-flat region of stance phase. Future work will focus on the implementation of this validated system in estimating human ankle impedance during the stance phase of walking.

Inertia compensation control of a one-degree-of-freedom exoskeleton for lower-limb assistance: initial experiments.

A new method of lower-limb exoskeleton control aimed at improving the agility of leg-swing motion is presented. In the absence of control, an exoskeleton's mechanism usually hinders agility by adding mechanical impedance to the legs. The uncompensated inertia of the exoskeleton will reduce the natural frequency of leg swing, probably leading to lower step frequency during walking as well as increased metabolic energy consumption. The proposed controller emulates inertia compensation by adding a feedback loop consisting of low-pass filtered angular acceleration multiplied by a negative gain. This gain simulates negative inertia in the low-frequency range. The resulting controller combines two assistive effects: increasing the natural frequency of the lower limbs and performing net work per swing cycle. The controller was tested on a statically mounted exoskeleton that assists knee flexion and extension. Subjects performed movement sequences, first unassisted and then using the exoskeleton, in the context of a computer-based task resembling a race. In the exoskeleton's baseline state, the frequency of leg swing and the mean angular velocity were consistently reduced. The addition of inertia compensation enabled subjects to recover their normal frequency and increase their selected angular velocity. The work performed by the exoskeleton was evidenced by catch trials in the protocol.

Robotic touch shifts perception of embodiment to a prosthesis in targeted reinnervation amputees.

Existing prosthetic limbs do not provide amputees with cutaneous feedback. Tactile feedback is essential to intuitive control of a prosthetic limb and it is now clear that the sense of body self-identification is also linked to cutaneous touch. Here we have created an artificial sense of touch for a prosthetic limb by coupling a pressure sensor on the hand through a robotic stimulator to surgically redirected cutaneous sensory nerves (targeted reinnervation) that once served the lost limb. We hypothesize that providing physiologically relevant cutaneous touch feedback may help an amputee incorporate an artificial limb into his or her self image. To investigate this we used a robotic touch interface coupled with a prosthetic limb and tested it with two targeted reinnervation amputees in a series of experiments fashioned after the Rubber Hand Illusion. Results from both subjective (self-reported) and objective (physiological) measures of embodiment (questionnaires, psychophysical temporal order judgements and residual limb temperature measurements) indicate that returning physiologically appropriate cutaneous feedback from a prosthetic limb drives a perceptual shift towards embodiment of the device for these amputees. Measurements provide evidence that the illusion created is vivid. We suggest that this may help amputees to more effectively incorporate an artificial limb into their self image, providing the possibility that a prosthesis becomes not only a tool, but also an integrated body part.

Development of a model osseo-magnetic link for intuitive rotational control of upper-limb prostheses.

The lack of proprioceptive feedback is a serious deficiency of current prosthetic control systems. The Osseo-Magnetic Link (OML) is a novel humeral or wrist rotation control system that could preserve proprioception. It utilizes a magnet implanted within the residual bone and sensors mounted in the prosthetic socket to detect magnetic field vectors and determine the bone's orientation. This allows the use of volitional bone rotation to control a prosthetic rotator. We evaluated the performance of the OML using a physical model of a transhumeral residual limb. A small Neodymium-Iron-Boron magnet was placed in a model humerus, inside a model upper arm. Four three-axis Hall-effect sensors were mounted on a ring 3 cm distal to the magnet. An optimization algorithm based on Newton's method determined the position and orientation of the magnet within the model humerus under various conditions, including bone translations, interference, and magnet misalignment. The orientation of the model humerus was determined within 3° for rotations centered in the arm; an additional 6° error was found for translations 20 mm from center. Adjustments in sensor placement may reduce these errors. The results demonstrate that the OML is a feasible solution for providing prosthesis rotation control while preserving rotational proprioception.

Design and validation of a platform robot for determination of ankle impedance during ambulation.

In order to provide natural, biomimetic control to recently developed powered ankle prostheses, we must characterize the impedance of the ankle during ambulation tasks. To this end, a platform robot was developed that can apply an angular perturbation to the ankle during ambulation and simultaneously acquire ground reaction force data. In this study, we detail the design of the platform robot and characterize the impedance of the ankle during quiet standing. Subjects were perturbed by a 3° dorsiflexive ramp perturbation with a length of 150 ms. The impedance was defined parametrically, using a second order model to map joint angle to the torque response. The torque was determined using the inverted pendulum assumption, and impedance was identified by the least squares best estimate, yielding an average damping coefficient of 0.03 ± 0.01 Nms/° and an average stiffness coefficient of 3.1 ± 1.2 Nm/°. The estimates obtained by the proposed platform robot compare favorably to those published in the literature. Future work will investigate the impedance of the ankle during ambulation for powered prosthesis controller development.