Evaluation of adjustment behaviour in a semi-supervised self-adjustment fine-tuning procedure for hearing aids.

OBJECTIVE: This study investigated the adjustment behaviour of hearing aid (HA) users participating in a semi-supervised self-adjustment fine-tuning procedure for HAs. The aim was to link behaviour with the reproducibility and duration of the adjustments. DESIGN: Participants used a two-dimensional user interface to identify their HA gain preferences while listening to realistic sound scenes presented in a laboratory environment. The interface allowed participants to adjust amplitude (vertical axis) and spectral slope (horizontal axis) simultaneously. Participants were clustered according to their interaction with the user interface, and their search directions were analysed. STUDY SAMPLE: Twenty older experienced HA users were invited to participate in this study. RESULTS: We identified four different archetypes of adjustment behaviour (curious, cautious, semi-browsing, and full-on browsing) by analysing the trace points of all measurements for each participant. Furthermore, participants used predominantly horizontal or vertical paths when searching for their preference. Neither the archetype, nor the search directions, nor the participants' technology commitment was predictive of the reproducibility or the adjustment duration. CONCLUSIONS: The findings suggest that enforcement of a specific adjustment behaviour or search direction is not necessary to obtain fast, reliable self-adjustments. Furthermore, no strict requirements with respect to technology commitment are necessary.

Evaluation of a semi-supervised self-adjustment fine-tuning procedure for hearing aids.

OBJECTIVE: This study investigated the effects of different adjustment criteria and sound scenes on self-adjusted hearing-aid gain settings. Self-adjusted settings were evaluated for speech recognition in noise, perceived listening effort, and preference. DESIGN: This study evaluated a semi-supervised self-adjustment fine-tuning procedure that presents realistic everyday sound scenes in a laboratory environment, using a two-dimensional user interface, and enabling simultaneous changes in amplitude and spectral slope. While exploring the two-dimensional space of parameter settings, the hearing-aid users were instructed to optimise either listening comfort or speech understanding. STUDY SAMPLE: Twenty experienced hearing aid users (median age 69.5 years) were invited to participate in this study. RESULTS: Adjustment criterion and sound scenes had a significant effect on preferred gain settings. No differences in signal-to-noise ratios required for 50% speech intelligibility or in the perceived listening effort were observed between the adjusted settings of the two adjustment criteria. There was a preference for the self-adjusted settings over the prescriptive first fit. CONCLUSIONS: Listeners could reliably select their preferred gains to the two adjustment criteria and for different speech stimuli.

Spatio-temporal Integration of Speech Reflections in Hearing-Impaired Listeners.

Speech recognition in rooms requires the temporal integration of reflections which arrive with a certain delay after the direct sound. It is commonly assumed that there is a certain temporal window of about 50-100 ms, during which reflections can be integrated with the direct sound, while later reflections are detrimental to speech intelligibility. This concept was challenged in a recent study by employing binaural room impulse responses (RIRs) with systematically varied interaural phase differences (IPDs) and amplitude of the direct sound and a variable number of reflections delayed by up to 200 ms. When amplitude or IPD favored late RIR components, normal-hearing (NH) listeners appeared to be capable of focusing on these components rather than on the precedent direct sound, which contrasted with the common concept of considering early RIR components as useful and late components as detrimental. The present study investigated speech intelligibility in the same conditions in hearing-impaired (HI) listeners. The data indicate that HI listeners were generally less able to "ignore" the direct sound than NH listeners, when the most useful information was confined to late RIR components. Some HI listeners showed a remarkable inability to integrate across multiple reflections and to optimally "shift" their temporal integration window, which was quite dissimilar to NH listeners. This effect was most pronounced in conditions requiring spatial and temporal integration and could provide new challenges for individual prediction models of binaural speech intelligibility.

Automated Speech Audiometry for Integrated Voice Over Internet Protocol Communication Services.

PURPOSE: Problems in speech recognition are often apparent in telecommunication situations. For ecologically valid assessments of such conditions, it is important to quantify the impact of real environments including acoustic conditions at a far-end communication device and all paths of transmission degradation. This study presents an automated matrix sentence test procedure based on automatic speech recognition (ASR) integrated in a Voice over Internet Protocol (VoIP) infrastructure and compares the individual effects of transmission degradations with results from laboratory measurements. METHOD: Speech recognition thresholds (SRTs) were measured in 16 normal-hearing subjects in four test conditions: (a) a laboratory condition guided by a human experimenter, (b) a laboratory condition with reduced bandwidth and (c) additionally reduced headset quality to simulate typical communication systems, and (d) an automated, ASR-controlled adaptive test procedure over a real VoIP infrastructure. Errors of the ASR system were analyzed to show possible effects on measurement outcome Results: Measured SRTs showed a highly significant correlation (r = .93) between the fully automatic and "laboratory" conditions, with a constant bias of about 1 dB indicating a linear shift of the data without affecting the distribution around the mean. The individual impact of the different system degradations on SRTs could be quantified Conclusions: This study provides a proof of concept for automated ASR-based SRT measurements over VoIP systems for speech audiometric testing in real communication systems, as it produced results comparable to traditional laboratory settings for this group of 16 normal-hearing subjects. This makes VoIP services a promising candidate for speech audiometric testing in real communication systems.

A joint framework for blind prediction of binaural speech intelligibility and perceived listening effort.

Speech perception is strongly affected by noise and reverberation in the listening room, and binaural processing can substantially facilitate speech perception in conditions when target speech and maskers originate from different directions. Most studies and proposed models for predicting spatial unmasking have focused on speech intelligibility. The present study introduces a model framework that predicts both speech intelligibility and perceived listening effort from the same output measure. The framework is based on a combination of a blind binaural processing stage employing a blind equalization cancelation (EC) mechanism, and a blind backend based on phoneme probability classification. Neither frontend nor backend require any additional information, such as the source directions, the signal-to-noise ratio (SNR), or the number of sources, allowing for a fully blind perceptual assessment of binaural input signals consisting of target speech mixed with noise. The model is validated against a recent data set in which speech intelligibility and perceived listening effort were measured for a range of acoustic conditions differing in reverberation and binaural cues [Rennies and Kidd (2018), J. Acoust. Soc. Am. 144, 2147-2159]. Predictions of the proposed model are compared with a non-blind binaural model consisting of a non-blind EC stage and a backend based on the speech intelligibility index. The analyses indicated that all main trends observed in the experiments were correctly predicted by the blind model. The overall proportion of variance explained by the model (R² = 0.94) for speech intelligibility was slightly worse than for the non-blind model (R² = 0.98). For listening effort predictions, both models showed lower prediction accuracy, but still explained significant proportions of the observed variance (R² = 0.88 and R² = 0.71 for the non-blind and blind model, respectively). Closer inspection showed that the differences between data and predictions were largest for binaural conditions at high SNRs, where the perceived listening effort of human listeners tended to be underestimated by the models, specifically by the blind version.

Relation between hearing abilities and preferred playback settings for speech perception in complex listening conditions.

OBJECTIVE: This study investigated if individual preferences with respect to the trade-off between a good signal-to-noise ratio and a distortion-free speech target were stable across different masking conditions and if simple adjustment methods could be used to identify subjects as either "noise haters" or "distortions haters". DESIGN: In each masking condition, subjects could adjust the target speech level according to their preferences by employing (i) linear gain or gain at the cost of (ii) clipping distortions or (iii) compression distortions. The comparison of these processing conditions allowed investigating the preferred trade-off between distortions and noise disturbance. STUDY SAMPLE: Thirty subjects differing widely in hearing status (normal-hearing to moderately impaired) and age (23-85 years). RESULTS: High test-retest stability of individual preferences was found for all modification schemes. The preference adjustments suggested that subjects could be consistently categorised along a scale from "noise haters" to "distortion haters", and this preference trait remained stable through all maskers, spatial conditions, and types of distortions. CONCLUSIONS: Employing quick self-adjustment to collect listening preferences in complex listening conditions revealed a stable preference trait along the "noise vs. distortions" tolerance dimension. This could potentially help in fitting modern hearing aid algorithms to the individual user.

Prediction of individual speech recognition performance in complex listening conditions.

This study examined how well individual speech recognition thresholds in complex listening scenarios could be predicted by a current binaural speech intelligibility model. Model predictions were compared with experimental data measured for seven normal-hearing and 23 hearing-impaired listeners who differed widely in their degree of hearing loss, age, as well as performance in clinical speech tests. The experimental conditions included two masker types (multi-talker or two-talker maskers), and two spatial conditions (maskers co-located with the frontal target or symmetrically separated from the target). The results showed that interindividual variability could not be well predicted by a model including only individual audiograms. Predictions improved when an additional individual "proficiency factor" was derived from one of the experimental conditions or a standard speech test. Overall, the current model can predict individual performance relatively well (except in conditions high in informational masking), but the inclusion of age-related factors may lead to even further improvements.

The importance of a broad bandwidth for understanding "glimpsed" speech.

When a target talker speaks in the presence of competing talkers, the listener must not only segregate the voices but also understand the target message based on a limited set of spectrotemporal regions ("glimpses") in which the target voice dominates the acoustic mixture. Here, the hypothesis that a broad audible bandwidth is more critical for these sparse representations of speech than it is for intact speech is tested. Listeners with normal hearing were presented with sentences that were either intact, or progressively "glimpsed" according to a competing two-talker masker presented at various levels. This was achieved by using an ideal binary mask to exclude time-frequency units in the target that would be dominated by the masker in the natural mixture. In each glimpsed condition, speech intelligibility was measured for a range of low-pass conditions (cutoff frequencies from 500 to 8000 Hz). Intelligibility was poorer for sparser speech, and the bandwidth required for optimal intelligibility increased with the sparseness of the speech. The combined effects of glimpsing and bandwidth reduction were well captured by a simple metric based on the proportion of audible target glimpses retained. The findings may be relevant for understanding the impact of high-frequency hearing loss on everyday speech communication.

Measurement and Prediction of Binaural-Temporal Integration of Speech Reflections.

For speech intelligibility in rooms, the temporal integration of speech reflections is typically modeled by separating the room impulse response (RIR) into an early (assumed beneficial for speech intelligibility) and a late part (assumed detrimental). This concept was challenged in this study by employing binaural RIRs with systematically varied interaural phase differences (IPDs) and amplitude of the direct sound and a variable number of reflections delayed by up to 200 ms. Speech recognition thresholds in stationary noise were measured in normal-hearing listeners for 86 conditions. The data showed that direct sound and one or several early speech reflections could be perfectly integrated when they had the same IPD. Early reflections with the same IPD as the noise (but not as the direct sound) could not be perfectly integrated with the direct sound. All conditions in which the dominant speech information was within the early RIR components could be well predicted by a binaural speech intelligibility model using classic early/late separation. In contrast, when amplitude or IPD favored late RIR components, listeners appeared to be capable of focusing on these components rather than on the precedent direct sound. This could not be modeled by an early/late separation window but required a temporal integration window that can be flexibly shifted along the RIR.

Energetic and Informational Components of Speech-on-Speech Masking in Binaural Speech Intelligibility and Perceived Listening Effort.

Speech perception in complex sound fields can greatly benefit from different unmasking cues to segregate the target from interfering voices. This study investigated the role of three unmasking cues (spatial separation, gender differences, and masker time reversal) on speech intelligibility and perceived listening effort in normal-hearing listeners. Speech intelligibility and categorically scaled listening effort were measured for a female target talker masked by two competing talkers with no unmasking cues or one to three unmasking cues. In addition to natural stimuli, all measurements were also conducted with glimpsed speech-which was created by removing the time-frequency tiles of the speech mixture in which the maskers dominated the mixture-to estimate the relative amounts of informational and energetic masking as well as the effort associated with source segregation. The results showed that all unmasking cues as well as glimpsing improved intelligibility and reduced listening effort and that providing more than one cue was beneficial in overcoming informational masking. The reduction in listening effort due to glimpsing corresponded to increases in signal-to-noise ratio of 8 to 18 dB, indicating that a significant amount of listening effort was devoted to segregating the target from the maskers. Furthermore, the benefit in listening effort for all unmasking cues extended well into the range of positive signal-to-noise ratios at which speech intelligibility was at ceiling, suggesting that listening effort is a useful tool for evaluating speech-on-speech masking conditions at typical conversational levels.

Benefit of binaural listening as revealed by speech intelligibility and listening effort.

In contrast to the well-known benefits for speech intelligibility, the advantage afforded by binaural stimulus presentation for reducing listening effort has not been thoroughly examined. This study investigated spatial release of listening effort and its relation to binaural speech intelligibility in listeners with normal hearing. Psychometric functions for speech intelligibility of a frontal target talker masked by a stationary speech-shaped noise were estimated for several different noise azimuths, different degrees of reverberation, and by maintaining only interaural level or time differences. For each of these conditions, listening effort was measured using a categorical scaling procedure. The results revealed that listening effort was significantly reduced when target and masker were spatially separated in anechoic conditions. This effect extended well into the range of signal-to-noise ratios (SNRs) in which speech intelligibility was at ceiling, and disappeared only at the highest SNRs. In reverberant conditions, spatial release from listening effort was observed for high, but not low, direct-to-reverberant ratios. The findings suggest that listening effort assessment can be a useful method for revealing the benefits of spatial separation of sources under realistic listening conditions comprising favorable SNRs and low reverberation, which typically are not apparent by other means.

Extension and evaluation of a near-end listening enhancement algorithm for listeners with normal and impaired hearing.

In many applications in which speech is played back via a sound reinforcement system such as public address systems and mobile phones, speech intelligibility is degraded by additive environmental noise. A possible solution to maintain high intelligibility in noise is to pre-process the speech signal based on the estimated noise power at the position of the listener. The previously proposed AdaptDRC algorithm [Schepker, Rennies, and Doclo (2015). J. Acoust. Soc. Am. 138, 2692-2706] applies both frequency shaping and dynamic range compression under an equal-power constraint, where the processing is adaptively controlled by short-term estimates of the speech intelligibility index. Previous evaluations of the algorithm have focused on normal-hearing listeners. In this study, the algorithm was extended with an adaptive gain stage under an equal-peak-power constraint, and evaluated with eleven normal-hearing and ten mildly to moderately hearing-impaired listeners. For normal-hearing listeners, average improvements in speech reception thresholds of about 4 and 8 dB compared to the unprocessed reference condition were measured for the original algorithm and its extension, respectively. For hearing-impaired listeners, the average improvements were about 2 and 6 dB, indicating that the relative improvement due to the proposed adaptive gain stage was larger for these listeners than the benefit of the original processing stages.

Perceived listening effort and speech intelligibility in reverberation and noise for hearing-impaired listeners.

OBJECTIVE: The purpose of this study was to assess perceived listening effort and speech intelligibility in reverberant and noisy conditions for hearing-impaired listeners for conditions that are similar according to the speech transmission index (STI). DESIGN: Scaled listening effort was measured in four different conditions at five different STI generated using various relative contributions of noise and reverberant interferences. Intelligibility was measured for a subset of conditions. STUDY SAMPLE: Twenty mildly to moderately hearing-impaired listeners. RESULTS: In general, listening effort decreased and speech intelligibility increased with increasing STI. For simulated impulse responses consisting of white Gaussian noise exponentially decaying in time, a good agreement between conditions of different relative contributions of noise and reverberation was found. For real impulse responses, the STI slightly overestimated the effect of reverberation on the perceived listening effort and underestimated its effect on speech intelligibility. Including the average hearing loss in the calculation of the STI led to a better agreement between STI predictions and subjective data. CONCLUSION: Speech intelligibility and listening effort provide complementary tools to evaluate speech perception over a broad range of acoustic scenarios. In addition, when incorporating hearing loss information the STI provides a rough prediction of listening effort in these acoustic scenarios.

Speech-in-noise enhancement using amplification and dynamic range compression controlled by the speech intelligibility index.

In many speech communication applications, such as public address systems, speech is degraded by additive noise, leading to reduced speech intelligibility. In this paper a pre-processing algorithm is proposed that is capable of increasing speech intelligibility under an equal-power constraint. The proposed AdaptDRC algorithm comprises two time- and frequency-dependent stages, i.e., an amplification stage and a dynamic range compression stage that are both dependent on the Speech Intelligibility Index (SII). Experiments using two objective measures, namely, the extended SII and the short-time objective intelligibility measure (STOI), and a formal listening test were conducted to compare the AdaptDRC algorithm with a modified version of a recently proposed algorithm in three different noise conditions (stationary car noise and speech-shaped noise and non-stationary cafeteria noise). While the objective measures indicate a similar performance for both algorithms, results from the formal listening test indicate that for the two stationary noises both algorithms lead to statistically significant improvements in speech intelligibility and for the non-stationary cafeteria noise only the proposed AdaptDRC algorithm leads to statistically significant improvements. A comparison of both objective measures and results from the listening test shows high correlations, although, in general, the performance of both algorithms is overestimated.

Listening effort and speech intelligibility in listening situations affected by noise and reverberation.

This study compared the combined effect of noise and reverberation on listening effort and speech intelligibility to predictions of the speech transmission index (STI). Listening effort was measured in normal-hearing subjects using a scaling procedure. Speech intelligibility scores were measured in the same subjects and conditions: (a) Speech-shaped noise as the only interfering factor, (b) + (c) fixed signal-to-noise ratios (SNRs) of 0 or 7 dB and reverberation as detrimental factors, and (d) reverberation as the only detrimental factor. In each condition, SNR and reverberation were combined to produce STI values of 0.17, 0.30, 0.43, 0.57, and 0.70, respectively. Listening effort always decreased with increasing STI, thus enabling a rough prediction, but a significant bias was observed indicating that listening effort was lower in reverberation only than in noise only at the same STI for one type of impulse responses. Accordingly, speech intelligibility increased with increasing STI and was significantly better in reverberation only than in noise only at the same STI. Further analyses showed that the broadband reverberation time is not always a good estimate of speech degradation in reverberation and that different speech materials may differ in their robustness toward detrimental effects of reverberation.

Modeling the effects of a single reflection on binaural speech intelligibility.

Recently the influence of delay and azimuth of a single speech reflection on speech reception thresholds (SRTs) was systematically investigated using frontal, diffuse, and lateral noise [Warzybok et al. (2013). J. Acoust. Soc. Am. 133, 269-282]. The experiments showed that the benefit of an early reflection was independent of its azimuth and mostly independent of noise type, but that the detrimental effect of a late reflection depended on its direction relative to the noise. This study tests if different extensions of a binaural speech intelligibility model can predict these data. The extensions differ in the order in which binaural processing and temporal integration of early reflections take place. Models employing a correction for the detrimental effects of reverberation on speech intelligibility after performing the binaural processing predict SRTs in symmetric masking conditions (frontal, diffuse), but cannot predict the measured interaction of temporal and spatial integration. In contrast, a model extension accounting for the distinction between useful and detrimental reflections before the binaural processing stage predicts the data with an overall R(2) of 0.95. This indicates that any model framework predicting speech intelligibility in rooms should incorporate an interaction between binaural and temporal integration of reflections at a comparatively early stage.

Effects of spatial and temporal integration of a single early reflection on speech intelligibility.

In order to study the interaction between the intelligibility advantage in rooms due to the presence of early reflections and due to binaural unmasking, a series of speech reception threshold experiments was performed employing a single reflection of the frontal target speech source as a function of its delay ranging from 0 to 200 ms. The direction of the reflection and the spatial characteristic of the interfering noise (diotic, diffuse, or laterally localized) were varied in the experiments. For the frontal reflection, full temporal integration was observed for all three noise types up to a delay of at least 25 ms followed by gradual intelligibility decay at longer delays. At 200 ms delay the reflection introduced additional intelligibility deterioration. For short delays, intelligibility was not reduced when the reflection was spatially separated from the direct sound in the diffuse and lateral noise conditions. A release from the deterioration effect at 200 ms delay was found for all spatially separated reflections. The suppression of a detrimental reflection was symmetrical in diffuse noise, but azimuth-dependent in lateral noise. This indicates an interaction of spatial and temporal processing of speech reflections which challenges existing binaural speech intelligibility models.

Spectro-temporal weighting of loudness.

Real-world sounds like speech or traffic noise typically exhibit spectro-temporal variability because the energy in different spectral regions evolves differently as a sound unfolds in time. However, it is currently not well understood how the energy in different spectral and temporal portions contributes to loudness. This study investigated how listeners weight different temporal and spectral components of a sound when judging its overall loudness. Spectral weights were measured for the combination of three loudness-matched narrowband noises with different center frequencies. To measure temporal weights, 1,020-ms stimuli were presented, which randomly changed in level every 100 ms. Temporal weights were measured for each narrowband noise separately, and for a broadband noise containing the combination of the three noise bands. Finally, spectro-temporal weights were measured with stimuli where the level of the three narrowband noises randomly and independently changed every 100 ms. The data consistently showed that (i) the first 300 ms of the sounds had a greater influence on overall loudness perception than later temporal portions (primacy effect), and (ii) the lowest noise band contributed significantly more to overall loudness than the higher bands. The temporal weights did not differ between the three frequency bands. Notably, the spectral weights and temporal weights estimated from the conditions with only spectral or only temporal variability were very similar to the corresponding weights estimated in the spectro-temporal condition. The results indicate that the temporal and the spectral weighting of the loudness of a time-varying sound are independent processes. The spectral weights remain constant across time, and the temporal weights do not change across frequency. The results are discussed in the context of current loudness models.

Prediction of the influence of reverberation on binaural speech intelligibility in noise and in quiet.

Reverberation usually degrades speech intelligibility for spatially separated speech and noise sources since spatial unmasking is reduced and late reflections decrease the fidelity of the received speech signal. The latter effect could not satisfactorily be predicted by a recently presented binaural speech intelligibility model [Beutelmann et al. (2010). J. Acoust. Soc. Am. 127, 2479-2497]. This study therefore evaluated three extensions of the model to improve its predictions: (1) an extension of the speech intelligibility index based on modulation transfer functions, (2) a correction factor based on the room acoustical quantity "definition," and (3) a separation of the speech signal into useful and detrimental parts. The predictions were compared to results of two experiments in which speech reception thresholds were measured in a reverberant room in quiet and in the presence of a noise source for listeners with normal hearing. All extensions yielded better predictions than the original model when the influence of reverberation was strong, while predictions were similar for conditions with less reverberation. Although model (3) differed substantially in the assumed interaction of binaural processing and early reflections, its predictions were very similar to model (2) that achieved the best fit to the data.

Spectral loudness summation of nonsimultaneous tone pulses.

The level of broadband signals is usually lower than that of equally loud narrow-band signals. This effect, referred to as spectral loudness summation, is commonly measured for broadband signals where all frequency components are presented simultaneously. The present study investigated to what extent spectral loudness summation also occurs for nonsimultaneously presented frequency components. Spectral loudness summation was measured in normal-hearing listeners with an adaptive forced-choice procedure for sequences of short tone pulses with varying frequencies, randomly chosen from a set of five frequencies. In addition, spectral loudness summation was measured for the simultaneous presentation of all five frequencies. The comparison stimulus consisted of tone pulses with the same frequency for all tone pulses of the sequence and the same repetition rate and overall duration as the test signal. The pulse duration was 10, 20, 50, or 100 ms and the inter-pulse interval ranged from 0 to 390 ms. In general, a considerable nonsimultaneous spectral loudness summation was found for short pulse durations and inter-pulse intervals, but a residual effect was also observed for the largest inter-pulse interval. The data are discussed in the light of repetition-rate dependent spectral loudness summation and effects of persistence of specific loudness after tone-pulse offset.