Evidence for proactive and retroactive temporal pattern analysis in simultaneous maskinga).

Amplitude modulation (AM) of a masker reduces its masking on a simultaneously presented unmodulated pure-tone target, which likely involves dip listening. This study tested the idea that dip-listening efficiency may depend on stimulus context, i.e., the match in AM peakedness (AMP) between the masker and a precursor or postcursor stimulus, assuming a form of temporal pattern analysis process. Masked thresholds were measured in normal-hearing listeners using Schroeder-phase harmonic complexes as maskers and precursors or postcursors. Experiment 1 showed threshold elevation (i.e., interference) when a flat cursor preceded or followed a peaked masker, suggesting proactive and retroactive temporal pattern analysis. Threshold decline (facilitation) was observed when the masker AMP was matched to the precursor, irrespective of stimulus AMP, suggesting only proactive processing. Subsequent experiments showed that both interference and facilitation (1) remained robust when a temporal gap was inserted between masker and cursor, (2) disappeared when an F0-difference was introduced between masker and precursor, and (3) decreased when the presentation level was reduced. These results suggest an important role of envelope regularity in dip listening, especially when masker and cursor are F0-matched and, therefore, form one perceptual stream. The reported effects seem to represent a time-domain variant of comodulation masking release.

Pitch discrimination in electric hearing with inconsistent and consistent amplitude-modulation and inter-pulse rate cues.

Users of cochlear implants (CIs) struggle in situations that require selective hearing to focus on a target source while ignoring other sources. One major reason for that is the limited access to timing cues such as temporal pitch or interaural time differences (ITDs). Various approaches to improve timing-cue sensitivity while maintaining speech understanding have been proposed, among them inserting extra pulses with short inter-pulse intervals (SIPIs) into amplitude-modulated (AM) high-rate pulse trains. Indeed, SIPI rates matching the naturally occurring AM rates improve pitch discrimination. For ITD, however, low SIPI rates are required, potentially mismatching the naturally occurring AM rates and thus creating unknown pitch effects. In this study, we investigated the perceptual contribution of AM and SIPI rate to pitch discrimination in five CI listeners and with two AM depths (0.1 and 0.5). Our results show that the SIPI-rate cue generally dominated the percept for both consistent and inconsistent cues. When tested with inconsistent cues, also the AM rate contributed, however, at the large AM depth only. These findings have implications when aiming at jointly improving temporal-pitch and ITD sensitivity in a future mixed-rate stimulation approach.

Contextual Lateralization Based on Interaural Level Differences Is Preshaped by the Auditory Periphery and Predominantly Immune Against Sequential Segregation.

The perceived azimuth of a target sound is determined by the interaural time difference and the interaural level difference (ILD) and is subject to contextual effects from precursor sounds. This study characterized ILD-based precursor effects (PEs) for high-frequency stimuli in a total of seven normal-hearing listeners. In Experiment 1, precursor and target were band-pass-filtered noises approximately centered at 4 kHz (1.2- and 1-octave bandwidth, respectively) separated by a 10-ms gap. The effects of precursor location (ipsilateral, contralateral, and central) on the perceived target azimuth were measured using a head-pointing task. Relative to control trials without a precursor, ipsilateral precursors biased the perceived target azimuth toward midline (medial bias) and contralateral precursors biased it contralaterally (lateral bias). Central precursors caused a symmetric lateral bias. An auditory periphery model that determines the "internal" ILD at the auditory nerve level, including either realistic efferent compression control or auditory nerve adaptation, explained about 50% of the variance in the PEs. These within-trial PEs were accompanied by an across-trial PE, inducing medial bias. Experiment 2 studied the role of sequential segregation in the within-trial PE by introducing a pitch difference between precursor and target. Segregation conditions caused increased PE for ipsilateral, no effect for contralateral, and either no effect or reduced PE for central precursors. Overall, the ILD-based within-trial PE appears to be preshaped already in the auditory periphery and the mechanism underlying at least the ipsilateral PE appears to be immune against sequential segregation.

Binaural-cue Weighting and Training-Induced Reweighting Across Frequencies.

During sound lateralization, the information provided by interaural differences in time (ITD) and level (ILD) is weighted, with ITDs and ILDs dominating for low and high frequencies, respectively. For mid frequencies, the weighting between these binaural cues can be changed via training. The present study investigated whether binaural-cue weights change gradually with increasing frequency region, whether they can be changed in various frequency regions, and whether such binaural-cue reweighting generalizes to untrained frequencies. In two experiments, a total of 39 participants lateralized 500-ms, 1/3-octave-wide noise bursts containing various ITD/ILD combinations in a virtual audio-visual environment. Binaural-cue weights were measured before and after a 2-session training in which, depending on the group, either ITDs or ILDs were visually reinforced. In experiment 1, four frequency bands (centered at 1000, 1587, 2520, and 4000 Hz) and a multiband stimulus comprising all four bands were presented during weight measurements. During training, only the 1000-, 2520-, and 4000-Hz bands were presented. In experiment 2, the weight measurements only included the two mid-frequency bands, while the training only included the 1587-Hz band. ILD weights increased gradually from low- to high-frequency bands. When ILDs were reinforced during training, they increased for the 4000- (experiment 1) and 2520-Hz band (experiment 2). When ITDs were reinforced, ITD weights increased only for the 1587-Hz band (at specific azimuths). This suggests that ILD reweighting requires high, and ITD reweighting requires low frequencies without including frequency regions providing fine-structure ITD cues. The changes in binaural-cue weights were independent of the trained bands, suggesting some generalization of binaural-cue reweighting.

Temporal integration contributes to the masking release by amplitude modulation.

Modulated maskers produce less amount of masking than unmodulated maskers, an effect referred to as masking release (MR). Both listening in the temporal dips and fast cochlear compression have been suggested as underlying mechanisms. We addressed the role of dip listening by measuring temporal integration in simultaneous masking using Schroeder-phase harmonic complexes (SPHC) with various phase curvatures. In an experiment with six normal-hearing listeners, SPHC masker and pure-tone target stimuli were covaried in duration at a high masker level. The MR increased with stimulus duration, suggesting integration of target information across multiple masker dips. The duration dependence of the MR was predicted by a physiology-inspired model based on the temporal envelope modulation strength in the auditory periphery. The modeling analysis suggested that listeners detect the presence of the target by a reduction in fluctuation strength that results primarily from a decline of F0-based response peaks, an effect known as synchrony capture. The detailed pattern of masked thresholds across various masker phase curvatures was not predicted by the model, suggesting that its phase response does not well fit the human phase response. Overall, temporal integration across neural envelope features associated with the masker dips seems to contribute to the MR with SPHCs.

Reweighting of Binaural Localization Cues in Bilateral Cochlear-Implant Listeners.

Normal-hearing (NH) listeners rely on two binaural cues, the interaural time (ITD) and level difference (ILD), for azimuthal sound localization. Cochlear-implant (CI) listeners, however, rely almost entirely on ILDs. One reason is that present-day clinical CI stimulation strategies do not convey salient ITD cues. But even when presenting ITDs under optimal conditions using a research interface, ITD sensitivity is lower in CI compared to NH listeners. Since it has recently been shown that NH listeners change their ITD/ILD weighting when only one of the cues is consistent with visual information, such reweighting might add to CI listeners' low perceptual contribution of ITDs, given their daily exposure to reliable ILDs but unreliable ITDs. Six bilateral CI listeners completed a multi-day lateralization training visually reinforcing ITDs, flanked by a pre- and post-measurement of ITD/ILD weights without visual reinforcement. Using direct electric stimulation, we presented 100- and 300-pps pulse trains at a single interaurally place-matched electrode pair, conveying ITDs and ILDs in various spatially consistent and inconsistent combinations. The listeners' task was to lateralize the stimuli in a virtual environment. Additionally, ITD and ILD thresholds were measured before and after training. For 100-pps stimuli, the lateralization training increased the contribution of ITDs slightly, but significantly. Thresholds were neither affected by the training nor correlated with weights. For 300-pps stimuli, ITD weights were lower and ITD thresholds larger, but there was no effect of training. On average across test sessions, adding azimuth-dependent ITDs to stimuli containing ILDs increased the extent of lateralization for both 100- and 300-pps stimuli. The results suggest that low-rate ITD cues, robustly encoded with future CI systems, may be better exploitable for sound localization after increasing their perceptual weight via training.

Reweighting of Binaural Localization Cues Induced by Lateralization Training.

Normal-hearing listeners adapt to alterations in sound localization cues. This adaptation can result from the establishment of a new spatial map of the altered cues or from a stronger relative weighting of unaltered compared to altered cues. Such reweighting has been shown for monaural vs. binaural cues. However, studies attempting to reweight the two binaural cues, interaural differences in time (ITD) and level (ILD), yielded inconclusive results. This study investigated whether binaural-cue reweighting can be induced by lateralization training in a virtual audio-visual environment. Twenty normal-hearing participants, divided into two groups, completed the experiment consisting of 7 days of lateralization training, preceded and followed by a test measuring the binaural-cue weights. Participants' task was to lateralize 500-ms bandpass-filtered (2-4 kHz) noise bursts containing various combinations of spatially consistent and inconsistent binaural cues. During training, additional visual cues reinforced the azimuth corresponding to ITDs in one group and ILDs in the other group and the azimuthal ranges of the binaural cues were manipulated group-specifically. Both groups showed a significant increase of the reinforced-cue weight from pre- to posttest, suggesting that participants reweighted the binaural cues in the expected direction. This reweighting occurred within the first training session. The results are relevant as binaural-cue reweighting likely occurs when normal-hearing listeners adapt to new acoustic environments. Reweighting might also be a factor underlying the low contribution of ITDs to sound localization of cochlear-implant listeners as they typically do not experience reliable ITD cues with clinical devices.

Temporal-pitch sensitivity in electric hearing with amplitude modulation and inserted pulses with short inter-pulse intervals.

Listeners with cochlear implants (CIs) typically show poor sensitivity to the temporal-envelope pitch of high-rate pulse trains. Sensitivity to interaural time differences improves when adding pulses with short inter-pulse intervals (SIPIs) to high-rate pulse trains. In the current study, monaural temporal-pitch sensitivity with SIPI pulses was investigated for six CI listeners. Amplitude-modulated single-electrode stimuli, representing the coding of the fundamental frequency (F0) in the envelope of a high-rate carrier, were used. Two SIPI-insertion approaches, five modulation depths, two typical speech-F0s, and two carrier rates were tested. SIPI pulses were inserted either in every amplitude-modulation period (full-rate SIPI) to support the F0 cue or in every other amplitude-modulation period (half-rate SIPI) to circumvent a potential rate limitation at higher F0s. The results demonstrate that full-rate SIPI pulses improve temporal-pitch sensitivity across F0s and particularly at low modulation depths where envelope-pitch cues are weak. The half-rate SIPI pulses did not circumvent the limitation and further increased variability across listeners. Further, no effect of the carrier rate was found. Thus, the SIPI approach appears to be a promising approach to enhance CI listeners' access to temporal-envelope pitch cues at pulse rates used clinically.

Improving Interaural Time Difference Sensitivity Using Short Inter-pulse Intervals with Amplitude-Modulated Pulse Trains in Bilateral Cochlear Implants.

Interaural time differences (ITDs) at low frequencies are important for sound localization and spatial speech unmasking. These ITD cues are not encoded in commonly used envelope-based stimulation strategies for cochlear implants (CIs) using high pulse rates. However, ITD sensitivity can be improved by adding extra pulses with short inter-pulse intervals (SIPIs) in unmodulated high-rate trains. Here, we investigated whether this improvement also applies to amplitude-modulated (AM) high-rate pulse trains. To this end, we systematically varied the temporal position of SIPI pulses within the envelope cycle (SIPI phase), the fundamental frequency (F0) of AM (125 Hz and 250 Hz), and AM depth (from 0.1 to 0.9). Stimuli were presented at an interaurally place-matched electrode pair at a reference pulse rate of 1000 pulses/s. Participants performed an ITD-based left/right discrimination task. SIPI insertion resulted in improved ITD sensitivity throughout the range of modulation depths and for both male and female F0s. The improvements were largest for insertion at and around the envelope peak. These results are promising for conveying salient ITD cues at high pulse rates commonly used to encode speech information.

Introducing Short Interpulse Intervals in High-Rate Pulse Trains Enhances Binaural Timing Sensitivity in Electric Hearing.

Common envelope-based stimulation strategies for cochlear implants (CIs) use relatively high carrier rates in order to properly encode the speech envelope. For such rates, CI listeners show poor sensitivity to interaural time differences (ITDs), which are important for horizontal-plane sound localization and spatial unmasking of speech. Based on the findings from previous studies, we predicted that ITD sensitivity can be enhanced by including pulses with short interpulse intervals (SIPIs), to a 1000-pulses-per-second (pps) reference pulse train. We measured the sensitivity of eight bilateral CI listeners to ITD while systematically varying both the rate at which SIPIs are introduced ("SIPI rate") and the time interval between the two pulses forming a SIPI ("SIPI fraction"). Results showed largely enhanced ITD sensitivity relative to the reference condition, with the size of the improvement increasing with decreasing SIPI rate and decreasing SIPI fraction. For the lowest SIPI fraction, insertion of extra pulses brought ITD sensitivity to the level measured for low-rate pulse trains with rates matching the SIPI rates. The results appear promising for the goal of enhancing ITD sensitivity with envelope-based CI strategies by inserting SIPI pulses at strategic times in speech stimuli.

Temporal effects in interaural and sequential level difference perception.

Temporal effects in interaural level difference (ILD) perception are not well understood. While it is often assumed that ILD sensitivity is independent of the temporal stimulus properties, a reduction of ILD sensitivity for stimuli with a high modulation rate has been reported (known under the term binaural adaptation). Experiment 1 compared ILD thresholds and sequential-level-difference (SLD) thresholds using 300-ms bandpass-filtered pulse trains (centered at 4 kHz) with rates of 100, 400, and 800 pulses per second (pps). In contrast to the SLD thresholds, ILD thresholds were elevated at 800 pps, consistent with literature data that had previously been attributed to binaural adaptation. Experiment 2 showed better ILD sensitivity for pulse trains than for pure tones, suggesting that amplitude modulation enhances ILD sensitivity. The present ILD data and binaural adaptation data from the literature were predicted by a model combining well-established auditory periphery front-ends with an interaural comparison stage. The model also accounted for other published ILD data, including target ILD thresholds in diotic forward and backward fringes and ILD thresholds with different amounts of interaural correlation. Overall, a variety of temporal effects in ILD perception, including binaural adaptation, appear to be largely attributable to monaural peripheral auditory processing.

Psychophysical and modeling approaches towards determining the cochlear phase response based on interaural time differences.

The cochlear phase response is often estimated by measuring masking of a tonal target by harmonic complexes with various phase curvatures. Maskers yielding most modulated internal envelope representations after passing the cochlear filter are thought to produce minimum masking, with fast-acting cochlear compression as the main contributor to that effect. Thus, in hearing-impaired (HI) listeners, reduced cochlear compression hampers estimation of the phase response using the masking method. This study proposes an alternative approach, based on the effect of the envelope modulation strength on the sensitivity to interaural time differences (ITDs). To evaluate the general approach, ITD thresholds were measured in seven normal-hearing listeners using 300-ms Schroeder-phase harmonic complexes with nine different phase curvatures. ITD thresholds tended to be lowest for phase curvatures roughly similar to those previously shown to produce minimum masking. However, an unexpected ITD threshold peak was consistently observed for a particular negative phase curvature. An auditory-nerve based ITD model predicted the general pattern of ITD thresholds except for the threshold peak, as well as published envelope ITD data. Model predictions simulating outer hair cell loss support the feasibility of the ITD-based approach to estimate the phase response in HI listeners.

Binaural timing information in electric hearing at low rates: Effects of inaccurate encoding and loudness.

Stimulation strategies for cochlear implants potentially impose timing limitations that may hinder the correct encoding and representation of interaural time differences (ITDs) in realistic bilateral signals. This study aimed to specify the tolerable room for inaccurate encoding of ITDs at low rates by investigating the perceptual degradation due to the removal of individual pulses at various levels of loudness. Unmodulated, 100-pulses-per-second pulse trains were presented at a single, interaurally pitch-matched electrode pair. In experiment I, ITD thresholds were measured applying different degrees of bilateral, interaurally-uncorrelated pulse removal. The ITD sensitivity deteriorated with increasing degree of pulse removal, with significant deterioration for degrees of 16% or greater. In experiment II, the interaction between loudness and pulse removal was investigated. Louder stimuli yielded better ITD sensitivity, however, no further improvement was found for stimuli louder than "medium." When removing 8% of the pulses, the ITD sensitivity deteriorated significantly across the entire loudness range tested. A loudness-induced compensation for the deterioration of ITD sensitivity due to pulse removal seems to be feasible for soft stimuli but not for medium or loud stimuli. Overall, our findings suggest that the degree of pulse removal employed in low-rate channels within coding strategies should not exceed 8%.

Auditory Time-Frequency Masking for Spectrally and Temporally Maximally-Compact Stimuli.

Many audio applications perform perception-based time-frequency (TF) analysis by decomposing sounds into a set of functions with good TF localization (i.e. with a small essential support in the TF domain) using TF transforms and applying psychoacoustic models of auditory masking to the transform coefficients. To accurately predict masking interactions between coefficients, the TF properties of the model should match those of the transform. This involves having masking data for stimuli with good TF localization. However, little is known about TF masking for mathematically well-localized signals. Most existing masking studies used stimuli that are broad in time and/or frequency and few studies involved TF conditions. Consequently, the present study had two goals. The first was to collect TF masking data for well-localized stimuli in humans. Masker and target were 10-ms Gaussian-shaped sinusoids with a bandwidth of approximately one critical band. The overall pattern of results is qualitatively similar to existing data for long maskers. To facilitate implementation in audio processing algorithms, a dataset provides the measured TF masking function. The second goal was to assess the potential effect of auditory efferents on TF masking using a modeling approach. The temporal window model of masking was used to predict present and existing data in two configurations: (1) with standard model parameters (i.e. without efferents), (2) with cochlear gain reduction to simulate the activation of efferents. The ability of the model to predict the present data was quite good with the standard configuration but highly degraded with gain reduction. Conversely, the ability of the model to predict existing data for long maskers was better with than without gain reduction. Overall, the model predictions suggest that TF masking can be affected by efferent (or other) effects that reduce cochlear gain. Such effects were avoided in the experiment of this study by using maximally-compact stimuli.

The role of compression in the simultaneous masker phase effect.

Peripheral compression is believed to play a major role in the masker phase effect (MPE). While compression is almost instantaneous, activation of the efferent system reduces compression in a temporally evolving manner. To study the role of efferent-controlled compression in the MPE, in experiment 1, simultaneous masking of a 30-ms 4-kHz tone by 40-ms Schroeder-phase harmonic complexes was measured with on- and off-frequency precursors as a function of masker phase curvature for two masker levels (60 and 90 dB sound pressure level). The MPE was quantified by the threshold range [min/max difference (MMD)] across the phase curvatures. For the 60-dB condition, the presence of on-frequency precursor decreased the MMD from 10 to 5 dB. Experiment 2 studied the role of the precursor on the auditory filter's bandwidth. The on-frequency precursor was found to increase the bandwidth, an effect incorporated in the subsequent modeling. A model of the auditory periphery including cochlear filtering and basilar membrane compression generally underestimated the MMDs. A model based on two-step compression, including compression of inner hair cells, accounted for the MMDs across precursor and level conditions. Overall, the observed precursor effects and the model predictions suggest an important role of compression in the simultaneous MPE.

Erratum: Modeling sound-source localization in sagittal planes for human listeners [J. Acoust. Soc. Am. 136, 791-802 (2014)].

This erratum concerns Eq. (4) of the original article, which defines the distance metric of the comparison process of the sagittal-plane sound localization model. The distance metric was actually implemented as a mean absolute difference but was erroneously described as a L1-norm difference.

Modeling the Effects of Sensorineural Hearing Loss on Sound Localization in the Median Plane.

Listeners use monaural spectral cues to localize sound sources in sagittal planes (along the up-down and front-back directions). How sensorineural hearing loss affects the salience of monaural spectral cues is unclear. To simulate the effects of outer-hair-cell (OHC) dysfunction and the contribution of different auditory-nerve fiber types on localization performance, we incorporated a nonlinear model of the auditory periphery into a model of sagittal-plane sound localization for normal-hearing listeners. The localization model was first evaluated in its ability to predict the effects of spectral cue modifications for normal-hearing listeners. Then, we used it to simulate various degrees of OHC dysfunction applied to different types of auditory-nerve fibers. Predicted localization performance was hardly affected by mild OHC dysfunction but was strongly degraded in conditions involving severe and complete OHC dysfunction. These predictions resemble the usually observed degradation in localization performance induced by sensorineural hearing loss. Predicted localization performance was best when preserving fibers with medium spontaneous rates, which is particularly important in view of noise-induced hearing loss associated with degeneration of this fiber type. On average across listeners, predicted localization performance was strongly related to level discrimination sensitivity of auditory-nerve fibers, indicating an essential role of this coding property for localization accuracy in sagittal planes.

Unconscious cross-modal priming of auditory sound localization by visual words.

Little is known about the cross-modal integration of unconscious and conscious information. In the current study, we therefore tested whether the spatial meaning of an unconscious visual word, such as up, influences the perceived location of a subsequently presented auditory target. Although cross-modal integration of unconscious information is generally rare, unconscious meaning stemming from only 1 particular modality could, in principle, be available for other modalities. Also, on the basis of known influences and dependencies of meaning on sensory information processing, such an unconscious meaning-based effect could impact sensory processing in a different modality. In 3 experiments, this prediction was confirmed. We found that an unconscious spatial word, such as up, facilitated position discrimination of a spatially congruent sound (here, a sound from above) as compared to a spatially incongruent sound (here, from below). This was found even though participants did not recognize the meaning of the primes. The results show that unconscious processing extends to semantic-sensory connections between different modalities. (PsycINFO Database Record

Channel Interaction and Current Level Affect Across-Electrode Integration of Interaural Time Differences in Bilateral Cochlear-Implant Listeners.

Sensitivity to interaural time differences (ITDs) is important for sound localization. Normal-hearing listeners benefit from across-frequency processing, as seen with improved ITD thresholds when consistent ITD cues are presented over a range of frequency channels compared with when ITD information is only presented in a single frequency channel. This study aimed to clarify whether cochlear-implant (CI) listeners can make use of similar processing when being stimulated with multiple interaural electrode pairs transmitting consistent ITD information. ITD thresholds for unmodulated, 100-pulse-per-second pulse trains were measured in seven bilateral CI listeners using research interfaces. Consistent ITDs were presented at either one or two electrode pairs at different current levels, allowing for comparisons at either constant level per component electrode or equal overall loudness. Different tonotopic distances between the pairs were tested in order to clarify the potential influence of channel interaction. Comparison of ITD thresholds between double pairs and the respective single pairs revealed systematic effects of tonotopic separation and current level. At constant levels, performance with double-pair stimulation improved compared with single-pair stimulation but only for large tonotopic separation. Comparisons at equal overall loudness revealed no benefit from presenting ITD information at two electrode pairs for any tonotopic spacing. Irrespective of electrode-pair configuration, ITD sensitivity improved with increasing current level. Hence, the improved ITD sensitivity for double pairs found for a large tonotopic separation and constant current levels seems to be due to increased loudness. The overall data suggest that CI listeners can benefit from combining consistent ITD information across multiple electrodes, provided sufficient stimulus levels and that stimulating electrode pairs are widely spaced.

Perception and coding of interaural time differences with bilateral cochlear implants.

Bilateral cochlear implantation is increasingly becoming the standard in the clinical treatment of bilateral deafness. The main motivation is to provide users of bilateral cochlear implants (CIs) access to binaural cues essential for localizing sound sources and understanding speech in environments of interfering sounds. One of those cues, interaural level differences, can be perceived well by CI users to allow some basic left versus right localization. However, interaural time differences (ITDs) which are important for localization of low-frequency sounds and spatial release from masking are not adequately represented by clinical envelope-based CI systems. Here, we first review the basic ITD sensitivity of CI users, particularly their dependence on stimulation parameters like stimulation rate and place, modulation rate, and envelope shape in single-electrode stimulation, as well as stimulation level, electrode spacing, and monaural across-electrode timing in multiple-electrode stimulation. Then, we discuss factors involved in ITD perception in electric hearing including the match between highly phase-locked electric auditory nerve response properties and binaural cell properties, the restricted stimulation of apical tonotopic pathways, channel interactions in multiple-electrode stimulation, and the onset age of binaural auditory input. Finally, we present clinically available CI stimulation strategies and experimental strategies aiming at improving listeners' access to ITD cues. This article is part of a Special Issue entitled .