Development and Evaluation of a Learning-based Model for Real-time Haptic Texture Rendering.

Current Virtual Reality (VR) environments lack the haptic signals that humans experience during real-life interactions, such as the sensation of texture during lateral movement on a surface. Adding realistic haptic textures to VR environments requires a model that generalizes to variations of a user's interaction and to the wide variety of existing textures in the world. Current methodologies for haptic texture rendering exist, but they usually develop one model per texture, resulting in low scalability. We present a deep learning-based action-conditional model for haptic texture rendering and evaluate its perceptual performance in rendering realistic texture vibrations through a multi-part human user study. This model is unified over all materials and uses data from a vision-based tactile sensor (GelSight) to render the appropriate surface conditioned on the user's action in real-time. For rendering texture, we use a high-bandwidth vibrotactile transducer attached to a 3D Systems Touch device. The results of our user study shows that our learning-based method creates high-frequency texture renderings with comparable or better quality than state-of-the-art methods without the need to learn a separate model per texture. Furthermore, we show that the method is capable of rendering previously unseen textures using a single GelSight image of their surface.

Design and Evaluation of Haptic Guidance in Ultrasound-Based Needle-Insertion Procedures.

OBJECTIVE: This article presents two haptic guidance systems designed to help a clinician keep an ultrasound probe steady when completing ultrasound-assisted needle insertion tasks. These procedures demand spatial reasoning and hand-eye coordination because the clinician must align a needle with the ultrasound probe and extrapolate the needle trajectory using only a 2D ultrasound image. Past research has shown that visual guidance helps the clinician align the needle, but does not help the clinician keep the ultrasound probe steady, sometimes resulting in a failed procedure. METHODS: We created two separate haptic guidance systems to provide feedback if the user tilts the ultrasound probe away from the desired setpoint using (1) vibrotactile stimulation provided by a voice coil motor or (2) distributed tactile pressure provided by a pneumatic mechanism. RESULTS: Both systems significantly reduced probe deviation and correction time to errors during a needle insertion task. We also tested the two feedback systems in a more clinically relevant setup and showed that the perceptibility of the feedback was not affected by the addition of a sterile bag placed over the actuators and gloves worn by the user. CONCLUSION: These studies show that both types of haptic feedback are promising for helping the user keep the ultrasound probe steady during ultrasound-assisted needle insertion tasks. Survey results indicated that users preferred the pneumatic system over the vibrotactile system. SIGNIFICANCE: Haptic feedback may improve user performance in ultrasound-based needle-insertion procedures and shows promise in training for needle-insertion tasks and other medical procedures where guidance is required.

Electrochemistry of Uranyl Peroxide Solutions during Electrospray Ionization.

The ionization of uranyl triperoxide monomer, [(UO2)(O2)3]4- (UT), and uranyl peroxide cage cluster, [(UO2)28(O2)42 - x(OH)2x]28- (U28), was studied with electrospray ionization mass spectrometry (ESI-MS). Experiments including tandem mass spectrometry with collision-induced dissociation (MS/CID/MS), use of natural water and D2O as solvent, and use of N2 and SF6 as nebulizer gases, provide insight into the mechanisms of ionization. The U28 nanocluster under MS/CID/MS with collision energies ranging from 0 to 25 eV produced the monomeric units UOx- (x = 3-8) and UOxHy- (x = 4-8, y = 1, 2). UT under ESI conditions yielded the gas-phase ions UOx- (x = 4-6) and UOxHy- (x = 4-8, y = 1-3). Mechanisms that produce the observed anions in the UT and U28 systems are: (a) gas-phase combinations of uranyl monomers in the collision cell upon fragmentation of U28, (b) reduction-oxidation resulting from the electrospray process, and (c) ionization of surrounding analytes, creating reactive oxygen species that then coordinate to uranyl ions. The electronic structures of anions UOx- (x = 6-8) were investigated using density functional theory (DFT).

Preference-Driven Texture Modeling Through Interactive Generation and Search.

Data-driven texture modeling and rendering has pushed the limit of realism in haptics. However, the lack of haptic texture databases, difficulties of model interpolation and expansion, and the complexity of real textures prevent data-driven methods from capturing a large variety of textures and from customizing models to suit specific output hardware or user needs. This work proposes an interactive texture generation and search framework driven by user input. We design a GAN-based texture model generator, which can create a wide range of texture models using Auto-Regressive processes. Our interactive texture search method, which we call "preference-driven," follows an evolutionary strategy given guidance from user's preferred feedback within a set of generated texture models. We implemented this framework on a 3D haptic device and conducted a two-phase user study to evaluate the efficiency and accuracy of our method for previously unmodeled textures. The results showed that by comparing the feel of real and generated virtual textures, users can follow an evolutionary process to efficiently find a virtual texture model that matched or exceeded the realism of a data-driven model. Furthermore, for 4 out of 5 real textures, ≥ 80% of the preference-driven models from participants were rated comparable to the data-driven models.

Effects of Physical Hardness on the Perception of Rendered Stiffness in an Encountered-Type Haptic Display.

Rendering stable hard surfaces is an important problem in haptics; current haptic devices cannot render hard objects and free space together. In our previous work, we addressed these limitations using an encountered-type haptic display system, which showed significant improvements compared to traditional rendering methods. In our approach, we attach a plate with the desired hardness to the kinesthetic device's end-effector, which the user interacts with using an untethered stylus. This method allows us to directly change the hardness of the end-effector based on the rendered object. In this paper, we evaluate how changing the hardness of the end-effector can mask the device's stiffness and affect the user's perception of the interaction. Our human subject experiment results indicate that when the end-effector is made of a hard material, it is difficult for users to perceive the stiffness change rendered by the device. On the other hand, this stiffness change is easily distinguished when the end-effector is made of a soft material. These results show promise for our combined hardness-stiffness display in avoiding the limitations of haptic devices when rendering hard surfaces.

Data-Driven Sparse Skin Stimulation Can Convey Social Touch Information to Humans.

During social interactions, people use auditory, visual, and haptic cues to convey their thoughts, emotions, and intentions. Due to weight, energy, and other hardware constraints, it is difficult to create devices that completely capture the complexity of human touch. Here we explore whether a sparse representation of human touch is sufficient to convey social touch signals. To test this we collected a dataset of social touch interactions using a soft wearable pressure sensor array, developed an algorithm to map recorded data to an array of actuators, then applied our algorithm to create signals that drive an array of normal indentation actuators placed on the arm. Using this wearable, low-resolution, low-force device, we find that users are able to distinguish the intended social meaning, and compare performance to results based on direct human touch. As online communication becomes more prevalent, such systems to convey haptic signals could allow for improved distant socializing and empathetic remote human-human interaction.

Towards Multisensory Perception: Modeling and Rendering Sounds of Tool-Surface Interactions.

Touch-produced sounds in tool-surface interactions convey rich information about textured surface properties and provide direct feedback about how users interact with the surface. This article presents a statistical learning-based approach for modeling and rendering touch-produced sounds in real time. We apply a data-driven modeling method, which recreates highly realistic sounds using audio signals recorded from unconstrained tool-surface interactions. The recorded sound is segmented, and each segment is labeled with the average velocity during that time. We model each segment with wavelet tree models using a moving window approach. Each window is analyzed by fast wavelet transform and is then broken down into a tree structure. During rendering, we use the user's current velocity to select tree models and synthesize new sounds by breadth-first search and inverse wavelet transform. We conducted a user study to evaluate the realism of our virtual sounds and their effect on human's perception of the texture dimensions in the presence of simultaneous real haptic cues. The results showed that in the presence of haptic cues, the virtual sound can more completely capture the texture's roughness and hardness than haptic cues alone. However, the perception on slipperiness depended mainly on touch.

Understanding Continuous and Pleasant Linear Sensations on the Forearm From a Sequential Discrete Lateral Skin-Slip Haptic Device.

A continuous stroking sensation on the skin can convey messages or emotion cues. We seek to induce this sensation using a combination of illusory motion and lateral stroking via a haptic device. Our system provides discrete lateral skin-slip on the forearm with rotating tactors, which independently provide lateral skin-slip in a timed sequence. We vary the sensation by changing the angular velocity and delay between adjacent tactors, such that the apparent speed of the perceived stroke ranges from 2.5 to 48.2 cm/s. We investigated which actuation parameters create the most pleasant and continuous sensations through a user study with 16 participants. On average, the sensations were rated by participants as both continuous and pleasant. The most continuous and pleasant sensations were created by apparent speeds of 7.7 and 5.1 cm/s, respectively. We also investigated the effect of spacing between contact points on the pleasantness and continuity of the stroking sensation, and found that the users experience a pleasant and continuous linear sensation even when the space between contact points is relatively large (40 mm). Understanding how sequential discrete lateral skin-slip creates continuous linear sensations can influence the design and control of future wearable haptic devices.

Resonant Frequency Skin Stretch for Wearable Haptics.

Resonant frequency skin stretch uses cyclic lateral skin stretches matching the skin's resonant frequency to create highly noticeable stimuli, signifying a new approach for wearable haptic stimulation. Four experiments were performed to explore biomechanical and perceptual aspects of resonant frequency skin stretch. In the first experiment, effective skin resonant frequencies were quantified at the forearm, shank, and foot. In the second experiment, perceived haptic stimuli were characterized for skin stretch actuations across a spectrum of frequencies. In the third experiment, perceived haptic stimuli were characterized by different actuator masses. In the fourth experiment, haptic classification ability was determined as subjects differentiated haptic stimulation cues while sitting, walking, and jogging. Results showed that subjects perceived stimulations at, above, and below the skin's resonant frequency differently: stimulations lower than the skin resonant frequency felt like distinct impacts, stimulations at the skin resonant frequency felt like cyclic skin stretches, and stimulations higher than the skin resonant frequency felt like standard vibrations. Subjects successfully classified stimulations while sitting, walking, and jogging, perceived haptic stimuli was affected by actuator mass, and classification accuracy decreased with increasing speed, especially for stimulations at the shank. This paper could facilitate more widespread use of wearable skin stretch. Potential applications include gaming, medical simulation, and surgical augmentation, and for training to reduce injury risk or improve sports performance.

Haptic Orientation Guidance Using Two Parallel Double-Gimbal Control Moment Gyroscopes.

This paper presents a system of two double-gimbal control moment gyroscopes (CMGs) for providing ungrounded kinesthetic haptic feedback. By spinning a second flywheel opposite the first, and rotating them through opposite trajectories, undesired gyroscopic effects can be eliminated, isolating a single torque axis. This produces a moment pulse proportional to the flywheel spin speed and rotation speed. Rotating the CMG gimbals quickly in one direction, then resetting them more slowly generates repeated torque pulses indicating a clear direction cue. We present the mathematical model for moments produced by this system and verify that the performance of our device matches this model. Using these asymmetric moment pulses, we provide haptic cues to participants in two studies. In the first study, users simply identify the direction of torque cues. In the second study, we use the torque pulses to guide users to target orientations. Performance in both studies shows that this system has the potential to provide useful guidance for applications where ungrounded haptic feedback is desired.

Importance of Matching Physical Friction, Hardness, and Texture in Creating Realistic Haptic Virtual Surfaces.

Interacting with physical objects through a tool elicits tactile and kinesthetic sensations that comprise your haptic impression of the object. These cues, however, are largely missing from interactions with virtual objects, yielding an unrealistic user experience. This article evaluates the realism of virtual surfaces rendered using haptic models constructed from data recorded during interactions with real surfaces. The models include three components: surface friction, tapping transients, and texture vibrations. We render the virtual surfaces on a SensAble Phantom Omni haptic interface augmented with a Tactile Labs Haptuator for vibration output. We conducted a human-subject study to assess the realism of these virtual surfaces and the importance of the three model components. Following a perceptual discrepancy paradigm, subjects compared each of 15 real surfaces to a full rendering of the same surface plus versions missing each model component. The realism improvement achieved by including friction, tapping, or texture in the rendering was found to directly relate to the intensity of the surface's property in that domain (slipperiness, hardness, or roughness). A subsequent analysis of forces and vibrations measured during interactions with virtual surfaces indicated that the Omni's inherent mechanical properties corrupted the user's haptic experience, decreasing realism of the virtual surface.

Modeling and rendering realistic textures from unconstrained tool-surface interactions.

Texture gives real objects an important perceptual dimension that is largely missing from virtual haptic interactions due to limitations of standard modeling and rendering approaches. This paper presents a set of methods for creating a haptic texture model from tool-surface interaction data recorded by a human in a natural and unconstrained manner. The recorded high-frequency tool acceleration signal, which varies as a function of normal force and scanning speed, is segmented and modeled as a piecewise autoregressive (AR) model. Each AR model is labeled with the source segment's median force and speed values and stored in a Delaunay triangulation to create a model set for a given texture. We use these texture model sets to render synthetic vibration signals in real time as a user interacts with our TexturePad system, which includes a Wacom tablet and a stylus augmented with a Haptuator. We ran a human-subject study with two sets of ten participants to evaluate the realism of our virtual textures and the strengths and weaknesses of this approach. The results indicated that our virtual textures accurately capture and recreate the roughness of real textures, but other modeling and rendering approaches are required to completely match surface hardness and slipperiness.

Biomedical evaluation of free-ranging ring-tailed lemurs (Lemur catta) in three habitats at the Beza Mahafaly Special Reserve, Madagascar.

Complete physical examinations and biomedical sample collection were performed on 70 free-ranging ring-tailed lemurs (Lemur catta) from three different habitats in the Beza Mahfaly Special Reserve (BMSR), in southern Madagascar, to assess the impact of humans and habitat on lemur health. Lemurs were chemically immobilized with ketamine and diazepam administered via blow darts for concurrent biomedical, morphometric, and behavioral studies. Subsets of the animals had blood analyzed for hematology, serum chemistry, micronutrients, fat-soluble vitamins (vitamins A, D, and E), measures of iron metabolism, and polymerase chain reaction assays (PCR) for Toxoplasma gondii, Hemoplasma spp., Bartonella spp., Ehrlichia spp., Anaplasma phagocytophilum, and Neorickettsia risticii. Results were compared on the basis of gender and the habitats at the study site: reserve (intact gallery forest), degraded (human inhabited and altered), and marginal (dry didieracea forest with heavy grazing and tree cutting). Levels of vitamin D, triglycerides, and cholesterol, and measures of iron metabolism for BMSR lemurs were greater than those previously reported for a free-ranging lemur population (Tsimanampetsotsa Strict Nature Reserve, Madagascar) with less access to foods of anthropogenic origin. BMSR ring-tailed lemurs from a habitat with less water (marginal) had higher sodium (P = 0.051), chloride (P = 0.045), osmolality (P = 0.010), and amylase (P = 0.05) levels than lemurs from other BMSR habitats, suggesting that these lemurs were less hydrated. Vitamin D levels of male lemurs were higher (P = 0.011) than those of females at BMSR, possibly because of differences in sunning behavior or differential selection of food items. The biological significance is uncertain for other parameters with statistically significant differences. All samples tested (n = 20) were negative for the pathogens tested using PCR assays. Continued concurrent biomedical and ecological research is needed at BMSR to confirm these results and determine their association with population mortality and fecundity rates.