It Sounds Cool: Exploring Sonification of Mid-Air Haptic Textures Exploration on Texture Judgments, Body Perception, and Motor Behaviour.

Ultrasonic mid-air haptic technology allows for the perceptual rendering of textured surfaces onto the user's hand. Unlike real textured surfaces, however, mid-air haptic feedback lacks implicit multisensory cues needed to reliably infer a texture's attributes (e.g., its roughness). In this paper, we combined mid-air haptic textures with congruent sound feedback to investigate how sonification could influence people's (1) explicit judgment of the texture attributes, (2) explicit sensations of their own hand, and (3) implicit motor behavior during haptic exploration. Our results showed that audio cues (presented solely or combined with haptics) influenced participants' judgment of the texture attributes (roughness, hardness, moisture and viscosity), produced some hand sensations (the feeling of having a hand smoother, softer, looser, more flexible, colder, wetter and more natural), and changed participants' speed (moving faster or slower) while exploring the texture. We then conducted a principal component analysis to better understand and visualize the found results and conclude with a short discussion on how audio-haptic associations can be used to create embodied experiences in emerging application scenarios in the metaverse.

An Interactive System for Fine Motor Rehabilitation.

PURPOSE: One of the most important aspects in neuromotor rehabilitation is the need of feedback for patients. The rehabilitation system's efficiency relies on the therapist's judgment; the therapist tells the patient whether he/she is performing the exercises correctly. This process may be quite subjective, because it depends on the therapist's personal opinion. On the other hand, recent studies have shown that vibrotactile biofeedback can improve the effectiveness of interaction as it is a very helpful tool in the physiological process of neuromotor rehabilitation. DESIGN: We designed an interactive system focused on rehabilitation of the upper limbs using active markers and image processing, which consists of drawing activities in both augment and virtual reality. METHODS: The system gives the user a correction through multimodal stimuli feedback (vibrotactile, visual, and sound stimulus) and force measurement to let the patients know if they are not achieving the tasks' goals. FINDINGS: The developed system could be used by nursing assistants to better help patients. The purpose of this system was assisting patients with injuries in shoulders, elbows, or wrists, providing an audio-vibrotactile feedback as a factor of correction in the movements of the patient. To examine our system, 11 participants were asked to participate in an experiment where they performed activities focused to strengthen their fine motor movements. CONCLUSIONS AND CLINICAL RELEVANCE: Results showed show that patients' fine motor skills improved 10% on average by comparing their error rates throughout the sessions.