Acoustic cues of keyboard mechanics enable auditory localization of upright piano tones.

Piano tone localization at the performer's listening point is a multisensory process involving audition, vision, and upper limb proprioception. The consequent representation of the auditory scene, especially in experienced pianists, is likely also influenced by their memory about the instrument keyboard. Disambiguating such components is not obvious, and first requires an analysis of the acoustic tone localization process to assess the role of auditory feedback in forming this scene. This analysis is complicated by the acoustic behavior of the piano, which does not guarantee the activation of the auditory precedence effect during a tone attack, nor can it provide robust interaural differences during the subsequent free evolution of the sound. In a tone localization task using a Disklavier upright piano (which can be operated remotely and configured to have its hammers hit a damper instead of producing a tone), twenty-three expert musicians, including pianists, successfully recognized the angular position of seven evenly distributed notes across the keyboard. The experiment involved listening to either full piano tones or just the key mechanical noise, with no additional feedback from other senses. This result suggests that the key mechanical noise alone activated the localization process without support from vision and/or limb proprioception. Since the same noise is present in the onset of the full tones, the key mechanics of our piano created a touch precursor in such tones that may be responsible of their correct angular localization by means of the auditory precedence effect. However, the significance of pitch cues arriving at a listener after the touch precursor was not measured when full tones were presented. As these cues characterize a note and, hence, the corresponding key position comprehensively, an open question remains regarding the contribution of pianists' spatial memory of the instrument keyboard to tone localization.

Effects of Vibration Direction and Pressing Force on Finger Vibrotactile Perception and Force Control.

This paper reports about the effects of vibration direction and finger-pressing force on vibrotactile perception, with the goal of improving the effectiveness of haptic feedback on interactive surfaces. An experiment was conducted to assess the sensitivity to normal or tangential vibration at 250 Hz of a finger exerting constant pressing forces of 0.5 or 4.9 N. Results show that perception thresholds for normal vibration depend on the applied pressing force, significantly decreasing for the stronger force level. Conversely, perception thresholds for tangential vibrations are independent of the applied force, and approximately equal the lowest thresholds measured for normal vibration.

An Open-Source Robotic Tool for the Simulation of Quasi-Static Finger Pressing on Stationary and Vibrating Surfaces.

The Bogus Finger is a remote-controllable tool for simulating vertical pressing forces of various magnitude as exerted by a human finger. Its main application is the characterization of haptic devices under realistic active touch conditions. The device is released as an open-source hardware and software DIY project that can be easily built using off-the-shelf components. We report the characterization of the quasi-static properties of the device, and validate its dynamic response to pressing on a vibrating surface by comparison with human fingers. The present prototype configuration accurately reproduces the mechanical impedance of the human finger in the frequency range 200-400 Hz.

Interactive Vibrotactile Feedback Enhances the Perceived Quality of a Surface for Musical Expression and the Playing Experience.

An advanced multi-touch sensor surface aimed at musical expression was recently equipped by the authors with interactive multi-point localized vibrotactile feedback. Using such interface, a subjective assessment was conducted that measured how the presence and type of vibration affect the perceived quality of the device and various attributes related to the playing experience. Two clearly distinct sound settings each with three vibrotactile feedback strategies were tested. At each trial, the task was to play freely while comparing two related setups which used the same sound setting and differed only in the presence/absence of vibration. Independent of the sound setting, as compared to the respective non-vibrating setups, vibrations conveying frequency and amplitude dynamics cues coherent with the player's gesture and/or sonic feedback had the most positive effect. Vibrotactile feedback especially improved the enjoyment of playing and the perceived potential for musical expressivity.

Path Following in Non-Visual Conditions.

Path-following tasks have been investigated mostly under visual conditions, that is when subjects are able to see both the path and the tool, or limb, used for navigation. Moreover, only basic path shapes are usually adopted. In the present experiment, participants must rely exclusively on continuous, non-speech, and ecological auditory and vibrotactile cues to follow a path on a flat surface. Two different, asymmetric path shapes were tested. Participants navigated by moving their index finger over a surface sensing position and force. Results show that the different non-visual feedback modes did not affect the task's accuracy, yet they affected its speed, with vibrotactile feedback causing the slowest gestures. Also, vibrotactile feedback caused participants to exert more force over the surface. Finally, the shape of the path was relevant to the accuracy, and participants tended to prefer audio over vibrotactile and audio-tactile feedback.

Detection of keyboard vibrations and effects on perceived piano quality.

Two experiments were conducted on an upright and a grand piano, both either producing string vibrations or conversely being silent after the initial keypress, while pianists were listening to the feedback from a synthesizer through insulating headphones. In a quality experiment, participants unaware of the silent mode were asked to play freely and then rate the instrument according to a set of attributes and general preference. Participants preferred the vibrating over the silent setup, and preference ratings were associated to auditory attributes of richness and naturalness in the low and middle ranges. Another experiment on the same setup measured the detection of vibrations at the keyboard, while pianists played notes and chords of varying dynamics and duration. Sensitivity to string vibrations was highest in the lowest register and gradually decreased up to note D5. After the percussive transient, the tactile stimuli exhibited spectral peaks of acceleration whose perceptibility was demonstrated by tests conducted in active touch conditions. The two experiments confirm that piano performers perceive vibratory cues of strings mediated by spectral and spatial summations occurring in the Pacinian system in their fingertips, and suggest that such cues play a role in the evaluation of quality of the musical instrument.

Vibrotactile Sensitivity in Active Touch: Effect of Pressing Force.

An experiment was conducted to study the effects of force produced by active touch on vibrotactile perceptual thresholds. The task consisted in pressing the fingertip against a flat rigid surface that provided either sinusoidal or broadband vibration. Three force levels were considered, ranging from light touch to hard press. Finger contact areas were measured during the experiment, showing positive correlation with the respective applied forces. Significant effects on thresholds were found for vibration type and force level. Moreover, possibly due to the concurrent effect of large (unconstrained) finger contact areas, active pressing forces, and long duration stimuli, the measured perceptual thresholds are considerably lower than what previously reported in the literature.

Might as well jump: sound affects muscle activation in skateboarding.

The aim of the study is to reveal the role of sound in action anticipation and performance, and to test whether the level of precision in action planning and execution is related to the level of sensorimotor skills and experience that listeners possess about a specific action. Individuals ranging from 18 to 75 years of age--some of them without any skills in skateboarding and others experts in this sport--were compared in their ability to anticipate and simulate a skateboarding jump by listening to the sound it produces. Only skaters were able to modulate the forces underfoot and to apply muscle synergies that closely resembled the ones that a skater would use if actually jumping on a skateboard. More importantly we showed that only skaters were able to plan the action by activating anticipatory postural adjustments about 200 ms after the jump event. We conclude that expert patterns are guided by auditory events that trigger proper anticipations of the corresponding patterns of movements.