Evaluating speech-in-speech perception via a humanoid robot.

Introduction: Underlying mechanisms of speech perception masked by background speakers, a common daily listening condition, are often investigated using various and lengthy psychophysical tests. The presence of a social agent, such as an interactive humanoid NAO robot, may help maintain engagement and attention. However, such robots potentially have limited sound quality or processing speed. Methods: As a first step toward the use of NAO in psychophysical testing of speech- in-speech perception, we compared normal-hearing young adults' performance when using the standard computer interface to that when using a NAO robot to introduce the test and present all corresponding stimuli. Target sentences were presented with colour and number keywords in the presence of competing masker speech at varying target-to-masker ratios. Sentences were produced by the same speaker, but voice differences between the target and masker were introduced using speech synthesis methods. To assess test performance, speech intelligibility and data collection duration were compared between the computer and NAO setups. Human-robot interaction was assessed using the Negative Attitude Toward Robot Scale (NARS) and quantification of behavioural cues (backchannels). Results: Speech intelligibility results showed functional similarity between the computer and NAO setups. Data collection durations were longer when using NAO. NARS results showed participants had a relatively positive attitude toward "situations of interactions" with robots prior to the experiment, but otherwise showed neutral attitudes toward the "social influence" of and "emotions in interaction" with robots. The presence of more positive backchannels when using NAO suggest higher engagement with the robot in comparison to the computer. Discussion: Overall, the study presents the potential of the NAO for presenting speech materials and collecting psychophysical measurements for speech-in-speech perception.

Perception of voice cues in school-age children with hearing aids.

The just-noticeable differences (JNDs) of the voice cues of voice pitch (F0) and vocal-tract length (VTL) were measured in school-aged children with bilateral hearing aids and children and adults with normal hearing. The JNDs were larger for hearing-aided than normal-hearing children up to the age of 12 for F0 and into adulthood for all ages for VTL. Age was a significant factor for both groups for F0 JNDs, but only for the hearing-aided group for VTL JNDs. Age of maturation was later for F0 than VTL. Individual JNDs of the two groups largely overlapped for F0, but little for VTL. Hearing thresholds (unaided or aided, 500-400 Hz, overlapping with mid-range speech frequencies) did not correlate with the JNDs. However, extended low-frequency hearing thresholds (unaided, 125-250 Hz, overlapping with voice F0 ranges) correlated with the F0 JNDs. Hence, age and hearing status differentially interact with F0 and VTL perception, and VTL perception seems challenging for hearing-aided children. On the other hand, even children with profound hearing loss could do the task, indicating a hearing aid benefit for voice perception. Given the significant age effect and that for F0 the hearing-aided children seem to be catching up with age-typical development, voice cue perception may continue developing in hearing-aided children.

Use of a humanoid robot for auditory psychophysical testing.

Tasks in psychophysical tests can at times be repetitive and cause individuals to lose engagement during the test. To facilitate engagement, we propose the use of a humanoid NAO robot, named Sam, as an alternative interface for conducting psychophysical tests. Specifically, we aim to evaluate the performance of Sam as an auditory testing interface, given its potential limitations and technical differences, in comparison to the current laptop interface. We examine the results and durations of two voice perception tests, voice cue sensitivity and voice gender categorisation, obtained from both the conventionally used laptop interface and Sam. Both tests investigate the perception and use of two speaker-specific voice cues, fundamental frequency (F0) and vocal tract length (VTL), important for characterising voice gender. Responses are logged on the laptop using a connected mouse, and on Sam using the tactile sensors. Comparison of test results from both interfaces shows functional similarity between the interfaces and replicates findings from previous studies with similar tests. Comparison of test durations shows longer testing times with Sam, primarily due to longer processing times in comparison to the laptop, as well as other design limitations due to the implementation of the test on the robot. Despite the inherent constraints of the NAO robot, such as in sound quality, relatively long processing and testing times, and different methods of response logging, the NAO interface appears to facilitate collecting similar data to the current laptop interface, confirming its potential as an alternative psychophysical test interface for auditory perception tests.

Algorithmic voice transformations reveal the phonological basis of language-familiarity effects in cross-cultural emotion judgments.

People have a well-described advantage in identifying individuals and emotions in their own culture, a phenomenon also known as the other-race and language-familiarity effect. However, it is unclear whether native-language advantages arise from genuinely enhanced capacities to extract relevant cues in familiar speech or, more simply, from cultural differences in emotional expressions. Here, to rule out production differences, we use algorithmic voice transformations to create French and Japanese stimulus pairs that differed by exactly the same acoustical characteristics. In two cross-cultural experiments, participants performed better in their native language when categorizing vocal emotional cues and detecting non-emotional pitch changes. This advantage persisted over three types of stimulus degradation (jabberwocky, shuffled and reversed sentences), which disturbed semantics, syntax, and supra-segmental patterns, respectively. These results provide evidence that production differences are not the sole drivers of the language-familiarity effect in cross-cultural emotion perception. Listeners' unfamiliarity with the phonology of another language, rather than with its syntax or semantics, impairs the detection of pitch prosodic cues and, in turn, the recognition of expressive prosody.

Phonological effects on the perceptual weighting of voice cues for voice gender categorization.

Voice perception and speaker identification interact with linguistic processing. This study investigated whether lexicality and/or phonological effects alter the perceptual weighting of voice pitch (F0) and vocal-tract length (VTL) cues for perceived voice gender categorization. F0 and VTL of forward words and nonwords (for lexicality effect), and time-reversed nonwords (for phonological effect through phonetic alterations) were manipulated. Participants provided binary "man"/"woman" judgements of the different voice conditions. Cue weights for time-reversed nonwords were significantly lower than cue weights for both forward words and nonwords, but there was no significant difference between forward words and nonwords. Hence, voice cue utilization for voice gender judgements seems to be affected by phonological, rather than lexicality effects.

Happy you, happy me: expressive changes on a stranger's voice recruit faster implicit processes than self-produced expressions.

In social interactions, people have to pay attention both to the 'what' and 'who'. In particular, expressive changes heard on speech signals have to be integrated with speaker identity, differentiating e.g. self- and other-produced signals. While previous research has shown that self-related visual information processing is facilitated compared to non-self stimuli, evidence in the auditory modality remains mixed. Here, we compared electroencephalography (EEG) responses to expressive changes in sequence of self- or other-produced speech sounds using a mismatch negativity (MMN) passive oddball paradigm. Critically, to control for speaker differences, we used programmable acoustic transformations to create voice deviants that differed from standards in exactly the same manner, making EEG responses to such deviations comparable between sequences. Our results indicate that expressive changes on a stranger's voice are highly prioritized in auditory processing compared to identical changes on the self-voice. Other-voice deviants generate earlier MMN onset responses and involve stronger cortical activations in a left motor and somatosensory network suggestive of an increased recruitment of resources for less internally predictable, and therefore perhaps more socially relevant, signals.

DAVID: An open-source platform for real-time transformation of infra-segmental emotional cues in running speech.

We present an open-source software platform that transforms emotional cues expressed by speech signals using audio effects like pitch shifting, inflection, vibrato, and filtering. The emotional transformations can be applied to any audio file, but can also run in real time, using live input from a microphone, with less than 20-ms latency. We anticipate that this tool will be useful for the study of emotions in psychology and neuroscience, because it enables a high level of control over the acoustical and emotional content of experimental stimuli in a variety of laboratory situations, including real-time social situations. We present here results of a series of validation experiments aiming to position the tool against several methodological requirements: that transformed emotions be recognized at above-chance levels, valid in several languages (French, English, Swedish, and Japanese) and with a naturalness comparable to natural speech.

Enhancing aesthetic appreciation by priming canvases with actions that match the artist's painting style.

The creation of an artwork requires motor activity. To what extent is art appreciation divorced from that activity and to what extent is it linked to it? That is the question which we set out to answer. We presented participants with pointillist-style paintings featuring discernible brushstrokes and asked them to rate their liking of each canvas when it was preceded by images priming a motor act either compatible or incompatible with the simulation of the artist's movements. We show that action priming, when congruent with the artist's painting style, enhanced aesthetic preference. These results support the hypothesis that involuntary covert painting simulation contributes to aesthetic appreciation during passive observation of artwork.