ABSTRACT
To achieve human-like behaviour during speech interactions, it is necessary for a humanoid robot to estimate the location of a human talker. Here, we present a method to optimize the parameters used for the direction of arrival (DOA) estimation, while also considering real-time applications for human-robot interaction scenarios. This method is applied to binaural sound source localization framework on a humanoid robotic head. Real data is collected and annotated for this work. Optimizations are performed via a brute force method and a Bayesian model based method, results are validated and discussed, and effects on latency for real-time use are also explored.
Subject(s)
Robotics , Sound Localization , Humans , Bayes Theorem , Acoustics , GravitationABSTRACT
OBJECTIVE: The aim of this study was to investigate whether consumer-grade mobile audio equipment can be reliably used as a platform for the notched-noise test, including when the test is conducted outside the laboratory. DESIGN: Two studies were conducted: Study 1 was a notched-noise masking experiment with three different setups: in a psychoacoustic test booth with a standard laboratory PC; in a psychoacoustic test booth with a mobile device; and in a quiet office room with a mobile device. Study 2 employed the same task as Study 1, but compared circumaural headphones to insert earphones. STUDY SAMPLE: Nine and ten young, normal-hearing participants completed studies 1 and 2, respectively. RESULTS: The test-retest accuracy of the notched-noise test on the mobile implementation did not differ from that for the laboratory setup. A possible effect of the earphone design was identified in Study 1, which was corroborated by Study 2, where test-retest variability was smallest when comparing results from experiments conducted using identical acoustic transducers. CONCLUSIONS: Results and test-retest repeatability comparable to standard laboratory settings for the notched-noise test can be obtained with mobile equipment outside the laboratory.
Subject(s)
Acoustics , Noise , Humans , Noise/adverse effects , Psychoacoustics , Computers, Handheld , TransducersABSTRACT
There is a long-standing tradition to assess hearing-aid benefits using lab-based speech intelligibility tests. Towards a more everyday-like scenario, the current study investigated the effects of hearing-aid amplification and noise on face-to-face communication between two conversational partners. Eleven pairs, consisting of a younger normal-hearing (NH) and an older hearing-impaired (HI) participant, solved spot-the-difference tasks while their conversations were recorded. In a two-block randomized design, the tasks were solved in quiet or noise, both with and without the HI participant receiving hearing-aid amplification with active occlusion cancellation. In the presence of 70 dB SPL babble noise, participants had fewer, slower, and less well-timed turn-starts, while speaking louder with longer inter-pausal units (IPUs, stretches of continuous speech surrounded by silence) and reducing their articulation rates. All these changes are indicative of increased communication effort. The timing of turn-starts by the HI participants exhibited more variability than that of their NH conversational partners. In the presence of background noise, the timing of turn-starts by the HI participants became even more variable, and their NH partners spoke louder. When the HI participants were provided with hearing-aid amplification, their timing of turn-starts became faster, they increased their articulation rate, and they produced shorter IPUs, all indicating reduced communication effort. In conclusion, measures of the conversational dynamics showed that background noise increased the communication effort, especially for the HI participants, and that providing hearing-aid amplification caused the HI participant to behave more like their NH conversational partner, especially in quiet situations.
Subject(s)
Hearing Aids , Hearing Loss, Sensorineural , Hearing Loss , Speech Perception , Hearing , Hearing Loss/diagnosis , Humans , Speech IntelligibilityABSTRACT
This study provides a framework for measuring conversational dynamics between conversational partners (interlocutors). Conversations from 20 pairs of young, normal-hearing, native-Danish talkers were recorded when speaking in both quiet and noise (70 dBA sound pressure level [SPL]) and in Danish and English. Previous studies investigating the intervals from when one talker stops talking to when the next one starts, termed floor-transfer offsets (FTOs), suggest that typical turn-taking requires interlocutors to predict when the current talker will finish their turn. We hypothesized that adding noise and/or speaking in a second language (L2) would increase the communication difficulty and result in longer and more variable FTOs. The median and interquartile range of FTOs increased slightly in noise, and in L2, there was a small increase in interquartile range but a small decrease in the median of FTO durations. It took the participants longer to complete the task in both L2 and noise, indicating increased communication difficulty. The average duration of interpausal units, that is, units of connected speech surrounded by silences of 180 ms or more, increased by 18% in noise and 8% in L2. These findings suggest that talkers held their turn for longer, allowing more time for speech understanding and planning. In L2, participants spoke slower, and in both L2 and noise, they took fewer turns. These changes in behavior may have offset some of the increased difficulty when communicating in noise or L2. We speculate that talkers prioritize the maintenance of turn-taking timing over other speech measures.
Subject(s)
Language , Speech Perception , Alpha-Ketoglutarate-Dependent Dioxygenase FTO , Communication , Hearing Tests , Humans , NoiseABSTRACT
While an audiogram is a useful method of characterizing hearing loss, it has been suggested that including a complementary, suprathreshold measure, for example, a measure of the status of the cochlear active mechanism, could lead to improved diagnostics and improved hearing-aid fitting in individual listeners. While several behavioral and physiological methods have been proposed to measure the cochlear-nonlinearity characteristics, evidence of a good correspondence between them is lacking, at least in the case of hearing-impaired listeners. If this lack of correspondence is due to, for example, limited reliability of one of such measures, it might be a reason for limited evidence of the benefit of measuring peripheral compression. The aim of this study was to investigate the relation between measures of the peripheral-nonlinearity status estimated using two psychoacoustical methods (based on the notched-noise and temporal-masking curve methods) and otoacoustic emissions, on a large sample of hearing-impaired listeners. While the relation between the estimates from the notched-noise and the otoacoustic emissions experiments was found to be stronger than predicted by the audiogram alone, the relations between the two measures and the temporal-masking based measure did not show the same pattern, that is, the variance shared by any of the two measures with the temporal-masking curve-based measure was also shared with the audiogram.
Subject(s)
Hearing Loss, Sensorineural , Perceptual Masking , Auditory Threshold , Cochlea , Humans , Reproducibility of ResultsABSTRACT
Linearized encoding models are increasingly employed to model cortical responses to running speech. Recent extensions to subcortical responses suggest clinical perspectives, potentially complementing auditory brainstem responses (ABRs) or frequency-following responses (FFRs) that are current clinical standards. However, while it is well-known that the auditory brainstem responds both to transient amplitude variations and the stimulus periodicity that gives rise to pitch, these features co-vary in running speech. Here, we discuss challenges in disentangling the features that drive the subcortical response to running speech. Cortical and subcortical electroencephalographic (EEG) responses to running speech from 19 normal-hearing listeners (12 female) were analyzed. Using forward regression models, we confirm that responses to the rectified broadband speech signal yield temporal response functions consistent with wave V of the ABR, as shown in previous work. Peak latency and amplitude of the speech-evoked brainstem response were correlated with standard click-evoked ABRs recorded at the vertex electrode (Cz). Similar responses could be obtained using the fundamental frequency (F0) of the speech signal as model predictor. However, simulations indicated that dissociating responses to temporal fine structure at the F0 from broadband amplitude variations is not possible given the high co-variance of the features and the poor signal-to-noise ratio (SNR) of subcortical EEG responses. In cortex, both simulations and data replicated previous findings indicating that envelope tracking on frontal electrodes can be dissociated from responses to slow variations in F0 (relative pitch). Yet, no association between subcortical F0-tracking and cortical responses to relative pitch could be detected. These results indicate that while subcortical speech responses are comparable to click-evoked ABRs, dissociating pitch-related processing in the auditory brainstem may be challenging with natural speech stimuli.
ABSTRACT
There is conflicting evidence about the relative benefit of slow- and fast-acting compression for speech intelligibility. It has been hypothesized that fast-acting compression improves audibility at low signal-to-noise ratios (SNRs) but may distort the speech envelope at higher SNRs. The present study investigated the effects of compression with a nearly instantaneous attack time but either fast (10 ms) or slow (500 ms) release times on consonant identification in hearing-impaired listeners. Consonant-vowel speech tokens were presented at a range of presentation levels in two conditions: in the presence of interrupted noise and in quiet (with the compressor "shadow-controlled" by the corresponding mixture of speech and noise). These conditions were chosen to disentangle the effects of consonant audibility and noise-induced forward masking on speech intelligibility. A small but systematic intelligibility benefit of fast-acting compression was found in both the quiet and the noisy conditions for the lower speech levels. No detrimental effects of fast-acting compression were observed when the speech level exceeded the level of the noise. These findings suggest that fast-acting compression provides an audibility benefit in fluctuating interferers when compared with slow-acting compression while not substantially affecting the perception of consonants at higher SNRs.
Subject(s)
Acoustic Stimulation/methods , Hearing Aids , Hearing Loss, Sensorineural/rehabilitation , Sound Spectrography/methods , Speech Intelligibility/physiology , Adult , Aged , Case-Control Studies , Female , Hearing Loss, Sensorineural/diagnosis , Humans , Male , Phonetics , Prosthesis Design , Reference Values , Signal-To-Noise Ratio , Speech Reception Threshold Test , Young AdultABSTRACT
Purpose: Studying the variations in speech levels with changing background noise level and talker-to-listener distance for talkers wearing hearing protection devices (HPDs) can aid in understanding communication in background noise. Method: Speech was recorded using an intra-aural HPD from 12 different talkers at 5 different distances in 3 different noise conditions and 2 quiet conditions. Results: This article proposes models that can predict the difference in speech level as a function of background noise level and talker-to-listener distance for occluded talkers. The proposed model complements the existing model presented by Pelegrín-García, Smits, Brunskog, and Jeong (2011) and expands on it by taking into account the effects of occlusion and background noise level on changes in speech sound level. Conclusions: Three models of the relationship between vocal effort, background noise level, and talker-to-listener distance for talkers wearing HPDs are presented. The model with the best prediction intervals is a talker-dependent model that requires the users' unoccluded speech level at 10 m as a reference. A model describing the relationship between speech level, talker-to-listener distance, and background noise level for occluded talkers could eventually be incorporated with radio protocols to transmit verbal communication only to an intended set of listeners within a given spatial range-this range being dependent on the changes in speech level and background noise level.
Subject(s)
Distance Perception , Ear Protective Devices/adverse effects , Noise , Speech Acoustics , Speech Perception , Adult , Female , Healthy Volunteers , Humans , Male , Perceptual Masking , Phonetics , Young AdultABSTRACT
It is well known that pure-tone audiometry does not sufficiently describe individual hearing loss (HL) and that additional measures beyond pure-tone sensitivity might improve the diagnostics of hearing deficits. Specifically, forward masking experiments to estimate basilar-membrane (BM) input-output (I/O) function have been proposed. However, such measures are very time consuming. The present study investigated possible modifications of the temporal masking curve (TMC) paradigm to improve time and measurement efficiency. In experiment 1, estimates of knee point (KP) and compression ratio (CR) of individual BM I/Os were derived without considering the corresponding individual "off-frequency" TMC. While accurate estimation of KPs was possible, it is difficult to ensure that the tested dynamic range is sufficient. Therefore, in experiment 2, a TMC-based paradigm, referred to as the "gap method", was tested. In contrast to the standard TMC paradigm, the maker level was kept fixed and the "gap threshold" was obtained, such that the masker just masks a low-level (12 dB sensation level) signal. It is argued that this modification allows for better control of the tested stimulus level range, which appears to be the main drawback of the conventional TMC method. The results from the present study were consistent with the literature when estimating KP levels, but showed some limitations regarding the estimation of the CR values. Perspectives and limitations of both approaches are discussed.
Subject(s)
Audiometry, Pure-Tone/methods , Basilar Membrane/physiopathology , Hearing Loss, Sensorineural/physiopathology , Perceptual Masking/physiology , Acoustic Stimulation/methods , Adult , Auditory Threshold , Basilar Membrane/physiology , Female , Hearing Loss, Sensorineural/diagnosis , Humans , Male , Middle Aged , Reproducibility of Results , Sensitivity and Specificity , Time FactorsABSTRACT
This study investigated the relationship between speech perception performance in spatially complex, lateralized listening scenarios and temporal fine-structure (TFS) coding at low frequencies. Young normal-hearing (NH) and two groups of elderly hearing-impaired (HI) listeners with mild or moderate hearing loss above 1.5 kHz participated in the study. Speech reception thresholds (SRTs) were estimated in the presence of either speech-shaped noise, two-, four-, or eight-talker babble played reversed, or a nonreversed two-talker masker. Target audibility was ensured by applying individualized linear gains to the stimuli, which were presented over headphones. The target and masker streams were lateralized to the same or to opposite sides of the head by introducing 0.7-ms interaural time differences between the ears. TFS coding was assessed by measuring frequency discrimination thresholds and interaural phase difference thresholds at 250 Hz. NH listeners had clearly better SRTs than the HI listeners. However, when maskers were spatially separated from the target, the amount of SRT benefit due to binaural unmasking differed only slightly between the groups. Neither the frequency discrimination threshold nor the interaural phase difference threshold tasks showed a correlation with the SRTs or with the amount of masking release due to binaural unmasking, respectively. The results suggest that, although HI listeners with normal hearing thresholds below 1.5 kHz experienced difficulties with speech understanding in spatially complex environments, these limitations were unrelated to TFS coding abilities and were only weakly associated with a reduction in binaural-unmasking benefit for spatially separated competing sources.
Subject(s)
Auditory Threshold , Hearing Loss , Speech Perception , Hearing Tests , Humans , Noise , Perceptual MaskingABSTRACT
This study proposes a binaural extension to the multi-resolution speech-based envelope power spectrum model (mr-sEPSM) [Jørgensen, Ewert, and Dau (2013). J. Acoust. Soc. Am. 134, 436-446]. It consists of a combination of better-ear (BE) and binaural unmasking processes, implemented as two monaural realizations of the mr-sEPSM combined with a short-term equalization-cancellation process, and uses the signal-to-noise ratio in the envelope domain (SNRenv) as the decision metric. The model requires only two parameters to be fitted per speech material and does not require an explicit frequency weighting. The model was validated against three data sets from the literature, which covered the following effects: the number of maskers, the masker types [speech-shaped noise (SSN), speech-modulated SSN, babble, and reversed speech], the masker(s) azimuths, reverberation on the target and masker, and the interaural time difference of the target and masker. The Pearson correlation coefficient between the simulated speech reception thresholds and the data across all experiments was 0.91. A model version that considered only BE processing performed similarly (correlation coefficient of 0.86) to the complete model, suggesting that BE processing could be considered sufficient to predict intelligibility in most realistic conditions.
Subject(s)
Perceptual Masking/physiology , Signal-To-Noise Ratio , Speech Intelligibility , Acoustic Stimulation , Algorithms , Auditory Threshold , Female , Humans , Models, Neurological , Speech PerceptionABSTRACT
OBJECTIVE: Speech production in noise with varying talker-to-listener distance has been well studied for the open ear condition. However, occluding the ear canal can affect the auditory feedback and cause deviations from the models presented for the open-ear condition. Communication is a main concern for people wearing hearing protection devices (HPD). Although practical, radio communication is cumbersome, as it does not distinguish designated receivers. A smarter radio communication protocol must be developed to alleviate this problem. Thus, it is necessary to model speech production in noise while wearing HPDs. Such a model opens the door to radio communication systems that distinguish receivers and offer more efficient communication between persons wearing HPDs. DESIGN: This paper presents the results of a pilot study aimed to investigate the effects of occluding the ear on changes in voice level and fundamental frequency in noise and with varying talker-to-listener distance. STUDY SAMPLE: Twelve participants with a mean age of 28 participated in this study. RESULTS: Compared to existing data, results show a trend similar to the open ear condition with the exception of the occluded quiet condition. CONCLUSIONS: This implies that a model can be developed to better understand speech production for the occluded ear.
Subject(s)
Ear Protective Devices/adverse effects , Noise/adverse effects , Speech Acoustics , Speech Production Measurement/methods , Voice , Adult , Female , Healthy Volunteers , Humans , Male , Pilot ProjectsABSTRACT
OBJECTIVES: Previous work has shown that individuals with lower working memory demonstrate reduced intelligibility for speech processed with fast-acting compression amplification. This relationship has been noted in fluctuating noise, but the extent of noise modulation that must be present to elicit such an effect is unknown. This study expanded on previous study by exploring the effect of background noise modulations in relation to compression speed and working memory ability, using a range of signal to noise ratios. DESIGN: Twenty-six older participants between ages 61 and 90 years were grouped by high or low working memory according to their performance on a reading span test. Speech intelligibility was measured for low-context sentences presented in background noise, where the noise varied in the extent of amplitude modulation. Simulated fast- or slow-acting compression amplification combined with individual frequency-gain shaping was applied to compensate for the individual's hearing loss. RESULTS: Better speech intelligibility scores were observed for participants with high working memory when fast compression was applied than when slow compression was applied. The low working memory group behaved in the opposite way and performed better under slow compression compared with fast compression. There was also a significant effect of the extent of amplitude modulation in the background noise, such that the magnitude of the score difference (fast versus slow compression) depended on the number of talkers in the background noise. The presented signal to noise ratios were not a significant factor on the measured intelligibility performance. CONCLUSION: In agreement with earlier research, high working memory allowed better speech intelligibility when fast compression was applied in modulated background noise. In the present experiment, that effect was present regardless of the extent of background noise modulation.
Subject(s)
Hearing Aids , Hearing Loss, Sensorineural/rehabilitation , Memory, Short-Term , Noise , Speech Perception , Aged , Aged, 80 and over , Data Compression/methods , Female , Hearing Loss, Sensorineural/psychology , Humans , Male , Middle Aged , Speech IntelligibilityABSTRACT
Past studies have shown that speakers spontaneously adjust their speech acoustics in response to their auditory feedback perturbed in real time. In the case of formant perturbation, the majority of studies have examined speaker's compensatory production using the English vowel /É/ as in the word "head." Consistent behavioral observations have been reported, and there is lively discussion as to how the production system integrates auditory versus somatosensory feedback to control vowel production. However, different vowels have different oral sensation and proprioceptive information due to differences in the degree of lingual contact or jaw openness. This may in turn influence the ways in which speakers compensate for auditory feedback. The aim of the current study was to examine speakers' compensatory behavior with six English monophthongs. Specifically, the current study tested to see if "closed vowels" would show less compensatory production than "open vowels" because closed vowels' strong lingual sensation may richly specify production via somatosensory feedback. Results showed that, indeed, speakers exhibited less compensatory production with the closed vowels. Thus sensorimotor control of vowels is not fixed across all vowels; instead it exerts different influences across different vowels.
Subject(s)
Feedback, Sensory/physiology , Phonation/physiology , Phonetics , Speech Acoustics , Adolescent , Adult , Canada , Female , Humans , Language , United States/ethnology , Young AdultABSTRACT
We perceive identity, expression and speech from faces. While perception of identity and expression depends crucially on the configuration of facial features it is less clear whether this holds for visual speech perception. Facial configuration is poorly perceived for upside-down faces as demonstrated by the Thatcher illusion in which the orientation of the eyes and mouth with respect to the face is inverted (Thatcherization). This gives the face a grotesque appearance but this is only seen when the face is upright. Thatcherization can likewise disrupt visual speech perception but only when the face is upright indicating that facial configuration can be important for visual speech perception. This effect can propagate to auditory speech perception through audiovisual integration so that Thatcherization disrupts the McGurk illusion in which visual speech perception alters perception of an incongruent acoustic phoneme. This is known as the McThatcher effect. Here we show that the McThatcher effect is reflected in the McGurk mismatch negativity (MMN). The MMN is an event-related potential elicited by a change in auditory perception. The McGurk-MMN can be elicited by a change in auditory perception due to the McGurk illusion without any change in the acoustic stimulus. We found that Thatcherization disrupted a strong McGurk illusion and a correspondingly strong McGurk-MMN only for upright faces. This confirms that facial configuration can be important for audiovisual speech perception. For inverted faces we found a weaker McGurk illusion but, surprisingly, no MMN. We also found no correlation between the strength of the McGurk illusion and the amplitude of the McGurk-MMN. We suggest that this may be due to a threshold effect so that a strong McGurk illusion is required to elicit the McGurk-MMN.
Subject(s)
Brain/physiology , Facial Expression , Pattern Recognition, Visual/physiology , Recognition, Psychology/physiology , Speech Perception/physiology , Adolescent , Adult , Electroencephalography , Evoked Potentials, Auditory , Female , Humans , Illusions/physiology , Male , Young AdultABSTRACT
Previous research employing a real-time auditory perturbation paradigm has shown that talkers monitor their own speech attributes such as fundamental frequency, vowel intensity, vowel formants, and fricative noise as part of speech motor control. In the case of vowel formants or fricative noise, what was manipulated is spectral information about the filter function of the vocal tract. However, segments can be contrasted by parameters other than spectral configuration. It is possible that the feedback system monitors phonation timing in the way it does spectral information. This study examined whether talkers exhibit a compensatory behavior when manipulating information about voicing. When talkers received feedback of the cognate of the intended voicing category (saying "tipper" while hearing "dipper" or vice versa), they changed the voice onset time and in some cases the following vowel.
Subject(s)
Feedback, Psychological/physiology , Feedback, Sensory/physiology , Perceptual Distortion/physiology , Phonation/physiology , Speech Perception/physiology , Adaptation, Physiological/physiology , Adolescent , Computer Systems , Female , Humans , Motor Skills/physiology , Noise , Psychoacoustics , Speech Production Measurement , Time Factors , Young AdultABSTRACT
Masking release (MR) is the improvement in speech intelligibility for a fluctuating interferer compared to stationary noise. Reduction in MR due to vocoder processing is usually linked to distortions in the temporal fine structure of the stimuli and a corresponding reduction in the fundamental frequency (F0) cues. However, it is unclear if envelope periodicity related to F0, produced by the interaction between unresolved harmonics, contributes to MR. In the present study, MR was determined from speech reception thresholds measured in the presence of stationary speech-shaped noise and a competing talker. Two types of processing were applied to the stimuli: (1) An amplitude- and frequency-modulated vocoder attenuated the envelope periodicity and (2) high-pass (HP) filtering (cutoff = 500 Hz) reduced the influence of F0-related information from low-order resolved harmonics. When applied individually, MR was unaffected by HP filtering, but slightly reduced when envelope periodicity was attenuated. When both were applied, MR was strongly reduced. Thus, the results indicate that F0-related information is crucial for MR, but that it is less important whether the F0-related information is conveyed by low-order resolved harmonics or by envelope periodicity as a result of unresolved harmonics. Further, envelope periodicity contributes substantially to MR.
Subject(s)
Cues , Noise/adverse effects , Perceptual Masking , Periodicity , Speech Acoustics , Speech Intelligibility , Speech Perception , Time Perception , Acoustic Stimulation , Adult , Auditory Threshold , Humans , Male , Sound Spectrography , Speech Reception Threshold Test , Time Factors , Young AdultABSTRACT
The everyday act of speaking involves the complex processes of speech motor control. An important component of control is monitoring, detection, and processing of errors when auditory feedback does not correspond to the intended motor gesture. Here we show, using fMRI and converging operations within a multivoxel pattern analysis framework, that this sensorimotor process is supported by functionally differentiated brain networks. During scanning, a real-time speech-tracking system was used to deliver two acoustically different types of distorted auditory feedback or unaltered feedback while human participants were vocalizing monosyllabic words, and to present the same auditory stimuli while participants were passively listening. Whole-brain analysis of neural-pattern similarity revealed three functional networks that were differentially sensitive to distorted auditory feedback during vocalization, compared with during passive listening. One network of regions appears to encode an "error signal" regardless of acoustic features of the error: this network, including right angular gyrus, right supplementary motor area, and bilateral cerebellum, yielded consistent neural patterns across acoustically different, distorted feedback types, only during articulation (not during passive listening). In contrast, a frontotemporal network appears sensitive to the speech features of auditory stimuli during passive listening; this preference for speech features was diminished when the same stimuli were presented as auditory concomitants of vocalization. A third network, showing a distinct functional pattern from the other two, appears to capture aspects of both neural response profiles. Together, our findings suggest that auditory feedback processing during speech motor control may rely on multiple, interactive, functionally differentiated neural systems.