The high/low frequency balance drives tactile perception of noisy vibrations.

Noisy vibrotactile signals transmitted during tactile explorations of an object provide precious information on the nature of its surface. Understanding the link between signal properties and how they are interpreted by the tactile sensory system remains challenging. In this paper, we investigated human perception of broadband, stationary vibrations recorded during exploration of textures and reproduced using a vibrotactile actuator. Since intensity is a well-established perceptual attribute, we here focused on the relevance of the spectral content. The stimuli were first equalized in perceived intensity and subsequently used to identify the most salient spectral features using dissimilarity estimations between pairs of successive vibration. Based on dimensionally reduced spectral representations, models of dissimilarity ratings showed that the balance between low and high frequencies was the most important cue. Formal validation of this result was achieved through a Mushra experiment, in which participants assessed the fidelity of resynthesized vibrations with various distorted frequency balances. These findings offer valuable insights into human vibrotactile perception and establish a computational framework for analyzing vibrations as humans do. Moreover, they pave the way for signal synthesis and compression based on sparse representations, holding significance for applications involving complex vibratory feedback.

Hearing as adaptive cascaded envelope interpolation.

The human auditory system is designed to capture and encode sounds from our surroundings and conspecifics. However, the precise mechanisms by which it adaptively extracts the most important spectro-temporal information from sounds are still not fully understood. Previous auditory models have explained sound encoding at the cochlear level using static filter banks, but this vision is incompatible with the nonlinear and adaptive properties of the auditory system. Here we propose an approach that considers the cochlear processes as envelope interpolations inspired by cochlear physiology. It unifies linear and nonlinear adaptive behaviors into a single comprehensive framework that provides a data-driven understanding of auditory coding. It allows simulating a broad range of psychophysical phenomena from virtual pitches and combination tones to consonance and dissonance of harmonic sounds. It further predicts the properties of the cochlear filters such as frequency selectivity. Here we propose a possible link between the parameters of the model and the density of hair cells on the basilar membrane. Cascaded Envelope Interpolation may lead to improvements in sound processing for hearing aids by providing a non-linear, data-driven, way to preprocessing of acoustic signals consistent with peripheral processes.

Hearing elliptic movements reveals the imprint of action on prototypical geometries.

Within certain categories of geometric shapes, prototypical exemplars that best characterize the category have been evidenced. These geometric prototypes are classically identified through the visual and haptic perception or motor production and are usually characterized by their spatial dimension. However, whether prototypes can be recalled through the auditory channel has not been formally investigated. Here we address this question by using auditory cues issued from timbre-modulated friction sounds evoking human drawing elliptic movements. Since non-spatial auditory cues were previously found useful for discriminating distinct geometric shapes such as circles or ellipses, it is hypothesized that sound dynamics alone can evoke shapes such as an exemplary ellipse. Four experiments were conducted and altogether revealed that a common elliptic prototype emerges from auditory, visual, and motor modalities. This finding supports the hypothesis of a common coding of geometric shapes according to biological rules with a prominent role of sensory-motor contingencies in the emergence of such prototypical geometry.

Tactile perception of auditory roughness.

Auditory roughness resulting from fast temporal beatings is often studied by summing two pure tones with close frequencies. Interestingly, the tactile counterpart of auditory roughness can be provided through touch with vibrotactile actuators. However, whether auditory roughness could also be perceived through touch and whether it exhibits similar characteristics are unclear. Here, auditory roughness perception and its tactile counterpart were evaluated using pairs of pure tone stimuli. Results revealed similar roughness curves in both modalities, suggesting similar sensory processing. This study attests to the relevance of such a paradigm for investigating auditory and tactile roughness in a multisensory fashion.

Rhythm perception is shared between audio and haptics.

A surface texture is perceived through both the sound and vibrations produced while being explored by our fingers. Because of their common origin, both modalities have a strong influence on each other, particularly at above 60 Hz for which vibrotactile perception and pitch perception share common neural processes. However, whether the sensation of rhythm is shared between audio and haptic perception is still an open question. In this study, we show striking similarities between the audio and haptic perception of rhythmic changes, and demonstrate the interaction of both modalities below 60 Hz. Using a new surface-haptic device to synthesize arbitrary audio-haptic textures, psychophysical experiments demonstrate that the perception threshold curves of audio and haptic rhythmic gradients are the same. Moreover, multimodal integration occurs when audio and haptic rhythmic gradients are congruent. We propose a multimodal model of rhythm perception to explain these observations. These findings suggest that audio and haptic signals are likely to be processed by common neural mechanisms also for the perception of rhythm. They provide a framework for audio-haptic stimulus generation that is beneficial for nonverbal communication or modern human-machine interfaces.

Cellists' sound quality is shaped by their primary postural behavior.

During the last 20 years, the role of musicians' body movements has emerged as a central question in instrument practice: Why do musicians make so many postural movements, for instance, with their torsos and heads, while playing musical instruments? The musical significance of such ancillary gestures is still an enigma and therefore remains a major pedagogical challenge, since one does not know if these movements should be considered essential embodied skills that improve musical expressivity. Although previous studies established clear connections between musicians' body movements and musical structures (particularly for clarinet, piano or violin performances), no evidence of direct relationships between body movements and the quality of the produced timbre has ever been found. In this study, focusing on the area of bowed-string instruments, we address the problem by showing that cellists use a set of primary postural directions to develop fluid kinematic bow features (velocity, acceleration) that prevent the production of poor quality (i.e., harsh, shrill, whistling) sounds. By comparing the body-related angles between normal and posturally constrained playing situations, our results reveal that the chest rotation and vertical inclination made by cellists act as coordinative support for the kinematics of the bowing gesture. These findings support the experimental works of Alexander, especially those that showed the role of head movements with respect to the upper torso (the so-called primary control) in ensuring the smooth transmission of fine motor control in musicians all the way to the produced sound. More generally, our research highlights the importance of focusing on this fundamental postural sense to improve the quality of human activities across different domains (music, dance, sports, rehabilitation, working positions, etc.).

Exploring sound perception through vocal imitations.

Understanding how sounds are perceived and interpreted is an important challenge for researchers dealing with auditory perception. The ecological approach to perception suggests that the salient perceptual information that enables an auditor to recognize events through sounds is contained in specific structures called invariants. Identifying such invariants is of interest from a fundamental point of view to better understand auditory perception and it is also useful to include perceptual considerations to model and control sounds. Among the different approaches used to identify perceptually relevant sound structures, vocal imitations are believed to bring a fresh perspective to the field. The main goal of this paper is to better understand how invariants are transmitted through vocal imitations. A sound corpus containing different types of known invariants obtained from an existing synthesizer was established. Participants took part in a test where they were asked to imitate the sound corpus. A continuous and sparse model adapted to the specificities of the vocal imitations was then developed and used to analyze the imitations. Results show that participants were able to highlight salient elements of the sounds that partially correspond to the invariants used in the sound corpus. This study also confirms that vocal imitations reveal how these invariants are transmitted through perception and offers promising perspectives on auditory investigations.

Detection of Friction-Modulated Textures is Limited by Vibrotactile Sensitivity.

Modulation of the frictional force of a fingertip sliding over a surface-haptic device can produce compelling sensations of texture and relief. The virtual sensation is particularly apparent and feel as fixed in space if the stimulus is rigorously correlated with the displacement of the finger. While frictional textures tactually resemble their real counterparts, some exploratory conditions under which the sharpness of the texture declines exist. We postulate that this decline in sharpness is caused by the perceptual limitation of the attempt to interpret the variation in friction as an out-of-plane sinusoidal topography. To investigate these questions, we measured the detection thresholds of sinusoidal friction-modulated gratings for a wide range of spatial periods explored at two different speeds. We compared the results with the detection thresholds, reported in the literature, of real gratings and vibrotactile stimuli. We found that the detection of spatial friction-modulated textures does not follow the same trend as that of real textures but is more similar to the vibrotactile rendering, which is strongly influenced by the exploratory speed. This article provides a better understanding of the perception of friction-modulated textures and provides insight into how to design impactful stimuli on surface-haptic devices.

Chimpanzees use tree species with a resonant timbre for accumulative stone throwing.

Animals use tools for communication relatively rarely compared to tool use for extractive foraging. We investigated the tool-use behaviour accumulative stone throwing (AST) in wild chimpanzees, who regularly throw rocks at trees, producing impact sounds and resulting in the aggregations of rocks. The function of AST remains unknown but appears to be communication-related. We conducted field experiments to test whether impact sounds produced by throwing rocks at trees varied according to the tree's properties. Specifically, we compared impact sounds of AST and non-AST tree species. We measured three acoustic descriptors related to intrinsic timbre quality, and found that AST tree species produced impact sounds that were less damped, with spectral energy concentrated at lower frequencies compared to non-AST tree species. Buttress roots in particular produced timbres with low-frequency energy (low spectral centroid) and slower signal onset (longer attack time). In summary, chimpanzees use tree species capable of producing more resonant sounds for AST compared to other tree species available.

Exploring the perceived harshness of cello sounds by morphing and synthesis techniques.

Cello bowing requires a very fine control of the musicians' gestures to ensure the quality of the perceived sound. When the interaction between the bow hair and the string is optimal, the sound is perceived as broad and round. On the other hand, when the gestural control becomes more approximate, the sound quality deteriorates and often becomes harsh, shrill, and quavering. In this study, such a timbre degradation, often described by French cellists as harshness (décharnement), is investigated from both signal and perceptual perspectives. Harsh sounds were obtained from experienced cellists subjected to a postural constraint. A signal approach based on Gabor masks enabled us to capture the main dissimilarities between round and harsh sounds. Two complementary methods perceptually validated these signal features: First, a predictive regression model of the perceived harshness was built from sound continua obtained by a morphing technique. Next, the signal structures identified by the model were validated within a perceptual timbre space, obtained by multidimensional scaling analysis on pairs of synthesized stimuli controlled in harshness. The results revealed that the perceived harshness was due to a combination between a more chaotic harmonic behavior, a formantic emergence, and a weaker attack slope.

Auditory Time-Frequency Masking for Spectrally and Temporally Maximally-Compact Stimuli.

Many audio applications perform perception-based time-frequency (TF) analysis by decomposing sounds into a set of functions with good TF localization (i.e. with a small essential support in the TF domain) using TF transforms and applying psychoacoustic models of auditory masking to the transform coefficients. To accurately predict masking interactions between coefficients, the TF properties of the model should match those of the transform. This involves having masking data for stimuli with good TF localization. However, little is known about TF masking for mathematically well-localized signals. Most existing masking studies used stimuli that are broad in time and/or frequency and few studies involved TF conditions. Consequently, the present study had two goals. The first was to collect TF masking data for well-localized stimuli in humans. Masker and target were 10-ms Gaussian-shaped sinusoids with a bandwidth of approximately one critical band. The overall pattern of results is qualitatively similar to existing data for long maskers. To facilitate implementation in audio processing algorithms, a dataset provides the measured TF masking function. The second goal was to assess the potential effect of auditory efferents on TF masking using a modeling approach. The temporal window model of masking was used to predict present and existing data in two configurations: (1) with standard model parameters (i.e. without efferents), (2) with cochlear gain reduction to simulate the activation of efferents. The ability of the model to predict the present data was quite good with the standard configuration but highly degraded with gain reduction. Conversely, the ability of the model to predict existing data for long maskers was better with than without gain reduction. Overall, the model predictions suggest that TF masking can be affected by efferent (or other) effects that reduce cochlear gain. Such effects were avoided in the experiment of this study by using maximally-compact stimuli.

Seeing Circles and Drawing Ellipses: When Sound Biases Reproduction of Visual Motion.

The perception and production of biological movements is characterized by the 1/3 power law, a relation linking the curvature and the velocity of an intended action. In particular, motions are perceived and reproduced distorted when their kinematics deviate from this biological law. Whereas most studies dealing with this perceptual-motor relation focused on visual or kinaesthetic modalities in a unimodal context, in this paper we show that auditory dynamics strikingly biases visuomotor processes. Biologically consistent or inconsistent circular visual motions were used in combination with circular or elliptical auditory motions. Auditory motions were synthesized friction sounds mimicking those produced by the friction of the pen on a paper when someone is drawing. Sounds were presented diotically and the auditory motion velocity was evoked through the friction sound timbre variations without any spatial cues. Remarkably, when subjects were asked to reproduce circular visual motion while listening to sounds that evoked elliptical kinematics without seeing their hand, they drew elliptical shapes. Moreover, distortion induced by inconsistent elliptical kinematics in both visual and auditory modalities added up linearly. These results bring to light the substantial role of auditory dynamics in the visuo-motor coupling in a multisensory context.

When eyes drive hand: Influence of non-biological motion on visuo-motor coupling.

Many studies stressed that the human movement execution but also the perception of motion are constrained by specific kinematics. For instance, it has been shown that the visuo-manual tracking of a spotlight was optimal when the spotlight motion complies with biological rules such as the so-called 1/3 power law, establishing the co-variation between the velocity and the trajectory curvature of the movement. The visual or kinesthetic perception of a geometry induced by motion has also been shown to be constrained by such biological rules. In the present study, we investigated whether the geometry induced by the visuo-motor coupling of biological movements was also constrained by the 1/3 power law under visual open loop control, i.e. without visual feedback of arm displacement. We showed that when someone was asked to synchronize a drawing movement with a visual spotlight following a circular shape, the geometry of the reproduced shape was fooled by visual kinematics that did not respect the 1/3 power law. In particular, elliptical shapes were reproduced when the circle is trailed with a kinematics corresponding to an ellipse. Moreover, the distortions observed here were larger than in the perceptual tasks stressing the role of motor attractors in such a visuo-motor coupling. Finally, by investigating the direct influence of visual kinematics on the motor reproduction, our result conciliates previous knowledge on sensorimotor coupling of biological motions with external stimuli and gives evidence to the amodal encoding of biological motion.

"Let Me Hear Your Handwriting!" Evaluating the Movement Fluency from Its Sonification.

The quality of handwriting is evaluated from the visual inspection of its legibility and not from the movement that generates the trace. Although handwriting is achieved in silence, adding sounds to handwriting movement might help towards its perception, provided that these sounds are meaningful. This study evaluated the ability to judge handwriting quality from the auditory perception of the underlying sonified movement, without seeing the written trace. In a first experiment, samples of a word written by children with dysgraphia, proficient children writers, and proficient adult writers were collected with a graphic tablet. Then, the pen velocity, the fluency, and the axial pen pressure were sonified in order to create forty-five audio files. In a second experiment, these files were presented to 48 adult listeners who had to mark the underlying unseen handwriting. In order to evaluate the relevance of the sonification strategy, two experimental conditions were compared. In a first 'implicit' condition, the listeners made their judgment without any knowledge of the mapping between the sounds and the handwriting variables. In a second 'explicit' condition, they knew what the sonified variables corresponded to and the evaluation criteria. Results showed that, under the implicit condition, two thirds of the listeners marked the three groups of writers differently. In the explicit condition, all listeners marked the dysgraphic handwriting lower than that of the two other groups. In a third experiment, the scores given from the auditory evaluation were compared to the scores given by 16 other adults from the visual evaluation of the trace. Results revealed that auditory evaluation was more relevant than the visual evaluation for evaluating a dysgraphic handwriting. Handwriting sonification might therefore be a relevant tool allowing a therapist to complete the visual assessment of the written trace by an auditory control of the handwriting movement quality.

Effects of clozapine on perceptual abnormalities and sensory gating: a preliminary cross-sectional study in schizophrenia.

The aim of the present study was to investigate the effect of second-generation antipsychotics (clozapine or another second-generation antipsychotic) on perceptual abnormalities related to sensory gating deficit. Although clozapine is known to improve sensory gating assessed neurophysiologically, we hypothesized that patients with schizophrenia treated with clozapine would report less perceptual abnormalities related to sensory gating deficit than patients treated with other second-generation antipsychotics do. Forty patients with a diagnosis of schizophrenia were investigated (10 patients treated with clozapine and 30 patients treated with another second-generation antipsychotic drug). Perceptual abnormalities were assessed with the Sensory Gating Inventory. Sensory gating was assessed through electroencephalogram with the auditory event-related potential method by measuring P50 amplitude changes in a dual click conditioning-testing procedure. Patients treated with clozapine present normal sensory gating and report less perceptual abnormalities related to sensory gating than patients treated with other second-generation antipsychotics do. Although the cross-sectional design of this study is limited because causal inferences cannot be clearly concluded, the present study suggests clinical and neurophysiological advantages of clozapine compared with other second-generation antipsychotics and provides a basis for future investigations on the effect of this treatment on perceptual abnormalities related to sensory gating deficit in patients with schizophrenia.

The effect of real-time auditory feedback on learning new characters.

The present study investigated the effect of handwriting sonification on graphomotor learning. Thirty-two adults, distributed in two groups, learned four new characters with their non-dominant hand. The experimental design included a pre-test, a training session, and two post-tests, one just after the training sessions and another 24h later. Two characters were learned with and two without real-time auditory feedback (FB). The first group first learned the two non-sonified characters and then the two sonified characters whereas the reverse order was adopted for the second group. Results revealed that auditory FB improved the speed and fluency of handwriting movements but reduced, in the short-term only, the spatial accuracy of the trace. Transforming kinematic variables into sounds allows the writer to perceive his/her movement in addition to the written trace and this might facilitate handwriting learning. However, there were no differential effects of auditory FB, neither long-term nor short-term for the subjects who first learned the characters with auditory FB. We hypothesize that the positive effect on the handwriting kinematics was transferred to characters learned without FB. This transfer effect of the auditory FB is discussed in light of the Theory of Event Coding.

Validation of the French sensory gating inventory: a confirmatory factor analysis.

The Sensory Gating Inventory (SGI) is an instrument investigating daily experiences of sensory gating deficit developed for English speaking schizophrenia patients. The purpose of this study is to design and validate a French version of the SGI. A forward-backward translation of the SGI was performed. The psychometric properties of the French SGI version were analyzed. A confirmatory factor analysis (CFA) was carried out to determine whether factor structure of the French version is similar to the original English version. In a sample of 363 healthy subjects (mean age=31.8 years, S.D.=12.2 years) the validation process revealed satisfactory psychometric properties: the internal consistency reliability was confirmed for each dimension; each item achieved the 0.40 standard threshold for item-internal consistency; each item was more highly correlated with its contributive dimension than with the other dimensions; and based on a CFA, we found a 4-factor structure for the French version of the SGI similar to the original instrument. Test-retest reliability was not determined. The French version of the SGI is a psychometrically sound self-report for measuring phenomenological sensory gating experiences.

From acoustic descriptors to evoked quality of car door sounds.

This article describes the first part of a study aiming at adapting the mechanical car door construction to the drivers' expectancies in terms of perceived quality of cars deduced from car door sounds. A perceptual cartography of car door sounds is obtained from various listening tests aiming at revealing both ecological and analytical properties linked to evoked car quality. In the first test naive listeners performed absolute evaluations of five ecological properties (i.e., solidity, quality, weight, closure energy, and success of closure). Then experts in the area of automobile doors categorized the sounds according to organic constituents (lock, joints, door panel), in particular whether or not the lock mechanism could be perceived. Further, a sensory panel of naive listeners identified sensory descriptors such as classical descriptors or onomatopoeia that characterize the sounds, hereby providing an analytic description of the sounds. Finally, acoustic descriptors were calculated after decomposition of the signal into a lock and a closure component by the Empirical Mode Decomposition (EMD) method. A statistical relationship between the acoustic descriptors and the perceptual evaluations of the car door sounds could then be obtained through linear regression analysis.

Do schizophrenia patients with low P50-suppression report more perceptual anomalies with the sensory gating inventory?

BACKGROUND: P50 amplitude changes in dual click conditioning-testing procedure might be a neurophysiological marker of deficient sensory gating in schizophrenia. However, the relationship between abnormalities in the neurophysiological and phenomenological dimensions of sensory gating in schizophrenia remains unclear. The aim of the present study was to determine if patients with low P50-suppression (below 50%) report more perceptual anomalies. METHODS: Three groups were compared: twenty-nine schizophrenia patients with high P50-suppression (above 50% amplitude suppression), twenty-three schizophrenia patients with low P50-suppression (below 50%) and twenty-six healthy subjects. The Sensory Gating Inventory (SGI), a four-factor self-report questionnaire, was used to measure perceptual anomalies related to sensory gating. A comparison of demographic and clinical data was also carried out. RESULTS: Patients with low P50-suppression presented: i) significantly higher scores on the SGI (for the overall SGI score and for each of the 4 factors) and ii) significantly larger P50 amplitude at the second click, than both patients with high P50-suppression and healthy subjects. There were no group differences in the most of demographic and clinical data. DISCUSSION: The finding offers support for conceptual models wherein abnormal neurophysiologic responses to repetitive stimuli give rise to clinically relevant perceptions of being inundated and overwhelmed by external sensory stimuli. Further studies are needed to explore the contributions of clinical symptoms, medication and neuropsychological functions to the relationship between P50-suppression and the SGI, and the role of sensory "gating in" versus "gating out".

From sound to shape: auditory perception of drawing movements.

This study investigates the human ability to perceive biological movements through friction sounds produced by drawings and, furthermore, the ability to recover drawn shapes from the friction sounds generated. In a first experiment, friction sounds, real-time synthesized and modulated by the velocity profile of the drawing gesture, revealed that subjects associated a biological movement to those sounds whose timbre variations were generated by velocity profiles following the 1/3 power law. This finding demonstrates that sounds can adequately inform about human movements if their acoustic characteristics are in accordance with the kinematic rule governing actual movements. Further investigations of our ability to recognize drawn shapes were carried out in 2 association tasks in which both recorded and synthesized sounds had to be associated to both distinct and similar visual shapes. Results revealed that, for both synthesized and recorded sounds, subjects made correct associations for distinct shapes, although some confusion was observed for similar shapes. The comparisons made between recorded and synthesized sounds lead to conclude that the timbre variations induced by the velocity profile enabled the shape recognition. The results are discussed in the context of the ecological and ideomotor frameworks.