Cortical Entrainment to Speech Produced by Cochlear Implant Users and Normal-Hearing Talkers.

The perceived sound quality of speech produced by hard-of-hearing individuals greatly depends on the degree and configuration of their hearing loss. A cochlear implant (CI) may provide some compensation and auditory feedback to monitor/control speech production. However, to date, the speech produced by CI users is still different in quality from that produced by normal-hearing (NH) talkers. In this study, we attempted to address this difference by examining the cortical activity of NH listeners when listening to continuous speech produced by 8 CI talkers and 8 NH talkers. We utilized a discriminative model to decode and reconstruct the speech envelope from the single-trial electroencephalogram (EEG) recorded from scalp electrode in NH listeners when listening to continuous speech. The correlation coefficient between the reconstructed envelope and original speech envelope was computed as a metric to quantify the difference in response to the speech produced by CI and NH talkers. The same listeners were asked to rate the perceived sound quality of the speech as a behavioral sound quality assessment. Both behavioral perceived sound quality ratings and the cortical entrainment to speech envelope were higher for the speech set produced by NH talkers than for the speech set produced by CI talkers.

Cross-Frequency Coupling in Cortical Processing of Speech.

This study examines power-power cross-frequency coupling (CFC) between different frequency bands of cortical activity in normal-hearing (NH) listeners and its association to the processing temporal envelope (ENV) and temporal fine structure (TFS) of speech. CFC between alpha and theta bands and between gamma and theta bands was investigated when only ENV or TFS or the original speech itself were processed. Comparing the cortical activity in response to ENV and original speech, there was an increase in alpha-theta CFC and in gamma-theta CFC when listening to ENV alone. However, when comparing the response when to listening TFS alone, there was a reduction in gamma-theta CFC compared to the original speech and the alpha-theta CFC was comparable to the equivalent observed with original speech. The increase in CFC may suggest that there is more synchrony across different bands of cortical activity in processing ENV than TFS. These measures can serve as indicators when either ENV or TFS is perceived.

Cortical entrainment to speech produced by cochlear implant talkers and normal-hearing talkers.

Cochlear implants (CIs) are commonly used to restore the ability to hear in those with severe or profound hearing loss. CIs provide the necessary auditory feedback for them to monitor and control speech production. However, the speech produced by CI users may not be fully restored to achieve similar perceived sound quality to that produced by normal-hearing talkers and this difference is easily noticeable in their daily conversation. In this study, we attempt to address this difference as perceived by normal-hearing listeners, when listening to continuous speech produced by CI talkers and normal-hearing talkers. We used a regenerative model to decode and reconstruct the speech envelope from the single-trial electroencephalogram (EEG) recorded on the scalp of the normal-hearing listeners. Bootstrap Spearman correlation between the actual speech envelope and the envelope reconstructed from the EEG was computed as a metric to quantify the difference in response to the speech produced by the two talker groups. The same listeners were asked to rate the perceived sound quality of the speech produced by the two talker groups as a behavioral sound quality assessment. The results show that both the perceived sound quality ratings and the computed metric, which can be seen as the degree of cortical entrainment to the actual speech envelope across the normal-hearing listeners, were higher in value for speech produced by normal hearing talkers than that for CI talkers. The first purpose of the study was to determine how well the envelope of speech is represented neurophysiologically via its similarity to the envelope reconstructed from EEG. The second purpose was to show how well this representation of speech for both CI and normal hearing talker groups differentiates in term of perceived sound quality.

Valid Acoustic Models of Cochlear Implants: One Size Does Not Fit All.

HYPOTHESIS: This study tests the hypothesis that it is possible to find tone or noise vocoders that sound similar and result in similar speech perception scores to a cochlear implant (CI). This would validate the use of such vocoders as acoustic models of CIs. We further hypothesize that those valid acoustic models will require a personalized amount of frequency mismatch between input filters and output tones or noise bands. BACKGROUND: Noise or tone vocoders have been used as acoustic models of CIs in hundreds of publications but have never been convincingly validated. METHODS: Acoustic models were evaluated by single-sided deaf CI users who compared what they heard with the CI in one ear to what they heard with the acoustic model in the other ear. We evaluated frequency-matched models (both all-channel and 6-channel models, both tone and noise vocoders) as well as self-selected models that included an individualized level of frequency mismatch. RESULTS: Self-selected acoustic models resulted in similar levels of speech perception and similar perceptual quality as the CI. These models also matched the CI in terms of perceived intelligibility, harshness, and pleasantness. CONCLUSION: Valid acoustic models of CIs exist, but they are different from the models most widely used in the literature. Individual amounts of frequency mismatch may be required to optimize the validity of the model. This may be related to the basalward frequency mismatch experienced by postlingually deaf patients after cochlear implantation.

The Spatial Selective Auditory Attention of Cochlear Implant Users in Different Conversational Sound Levels.

In multi-speaker environments, cochlear implant (CI) users may attend to a target sound source in a different manner from normal hearing (NH) individuals during a conversation. This study attempted to investigate the effect of conversational sound levels on the mechanisms adopted by CI and NH listeners in selective auditory attention and how it affects their daily conversation. Nine CI users (five bilateral, three unilateral, and one bimodal) and eight NH listeners participated in this study. The behavioral speech recognition scores were collected using a matrix sentences test, and neural tracking to speech envelope was recorded using electroencephalography (EEG). Speech stimuli were presented at three different levels (75, 65, and 55 dB SPL) in the presence of two maskers from three spatially separated speakers. Different combinations of assisted/impaired hearing modes were evaluated for CI users, and the outcomes were analyzed in three categories: electric hearing only, acoustic hearing only, and electric + acoustic hearing. Our results showed that increasing the conversational sound level degraded the selective auditory attention in electrical hearing. On the other hand, increasing the sound level improved the selective auditory attention for the acoustic hearing group. In the NH listeners, however, increasing the sound level did not cause a significant change in the auditory attention. Our result implies that the effect of the sound level on selective auditory attention varies depending on the hearing modes, and the loudness control is necessary for the ease of attending to the conversation by CI users.

Wireless And Portable Daily Soundscape Monitoring System For Auditory Profiling: Diagnostic Tool For Alzheimer's Disease.

The purpose of this paper is to develop an inexpensive, wearable, and portable monitoring system with wireless capabilities for signal acquisition of the user's surrounding soundscape and electroencephalography (EEG). The end-goal of this device is to monitor high-risk populations that are developing into earlier stages of Alzheimer's Disease (AD). Currently, the development of such device is still within preliminary phase and has only been tested in healthy individuals. Future applications of our monitoring system may be used as a non-invasive and inexpensive diagnostic tool for early detection of AD, potentially paving a new platform for therapeutic intervention. The system consists of low-weight bearing components, including an analog front-end and a single-board computer. The analog front-end contains three independent EEG, reference, bias, and auditory recording channels. The single-board computer timestamps and encrypts the incoming channels prior to local or "cloud" storage. Cloud storage provides ease-of-access and offline data analysis without the need to physically extract the data from the monitoring system. A portable/rechargeable battery provides power to the entire monitoring system for over 4 hours of operation. A graphical user-interface (GUI) was developed for secured remote access to data, parameter settings, and system configurations. The performance of the system was tested by measuring the frequency following response (FFR) in the captured EEG signals with respect to periodic auditory stimuli.

The effect of perceived sound quality of speech in noisy speech perception by normal hearing and hearing impaired listeners.

Most noise reduction algorithms in current assistive hearing devices were designed to reduce noise to achieve the best signal to noise ratio (SNR) possible, presuming that the perceived quality of speech can be compromised as long as the level of noise is relatively much lower than the level of speech. In this study, we investigated and compared various factors other than SNR that may possibly affect the quality rating pattern of normal hearing (NH), cochlear implant (CI) and hearing aid (HA) subjects. In one of the comparisons, we found that the perceived sound quality ratings of noisy sentences at the same SNR were dependent on the level of speech present in the sentences rather than the noise. Noisy sentences with a higher level of speech were rated higher in perceived sound quality by NH subjects. However, in a similar comparison, CI subjects rated higher in perceived sound quality when the level of speech present was lower, but HA subjects do not show any clear trend with their perceived sound quality ratings on the same sentences.

A potential neurophysiological correlate of electric-acoustic pitch matching in adult cochlear implant users: Pilot data.

The overall goal of this study was to identify an objective physiological correlate of electric-acoustic pitch matching in unilaterally implanted cochlear implant (CI) participants with residual hearing in the non-implanted ear. Electrical and acoustic stimuli were presented in a continuously alternating fashion across ears. The acoustic stimulus and the electrical stimulus were either matched or mismatched in pitch. Auditory evoked potentials were obtained from nine CI users. Results indicated that N1 latency was stimulus-dependent, decreasing when the acoustic frequency of the tone presented to the non-implanted ear was increased. More importantly, there was an additional decrease in N1 latency in the pitch-matched condition. These results indicate the potential utility of N1 latency as an index of pitch matching in CI users.

Self-Selection of Frequency Tables with Bilateral Mismatches in an Acoustic Simulation of a Cochlear Implant.

BACKGROUND: Many recipients of bilateral cochlear implants (CIs) may have differences in electrode insertion depth. Previous reports indicate that when a bilateral mismatch is imposed, performance on tests of speech understanding or sound localization becomes worse. If recipients of bilateral CIs cannot adjust to a difference in insertion depth, adjustments to the frequency table may be necessary to maximize bilateral performance. PURPOSE: The purpose of this study was to examine the feasibility of using real-time manipulations of the frequency table to offset any decrements in performance resulting from a bilateral mismatch. RESEARCH DESIGN: A simulation of a CI was used because it allows for explicit control of the size of a bilateral mismatch. Such control is not available with users of CIs. STUDY SAMPLE: A total of 31 normal-hearing young adults participated in this study. DATA COLLECTION AND ANALYSIS: Using a CI simulation, four bilateral mismatch conditions (0, 0.75, 1.5, and 3 mm) were created. In the left ear, the analysis filters and noise bands of the CI simulation were the same. In the right ear, the noise bands were shifted higher in frequency to simulate a bilateral mismatch. Then, listeners selected a frequency table in the right ear that was perceived as maximizing bilateral speech intelligibility. Word-recognition scores were then assessed for each bilateral mismatch condition. Listeners were tested with both a standard frequency table, which preserved a bilateral mismatch, or with their self-selected frequency table. RESULTS: Consistent with previous reports, bilateral mismatches of 1.5 and 3 mm yielded decrements in word recognition when the standard table was used in both ears. However, when listeners used the self-selected frequency table, performance was the same regardless of the size of the bilateral mismatch. CONCLUSIONS: Self-selection of a frequency table appears to be a feasible method for ameliorating the negative effects of a bilateral mismatch. These data may have implications for recipients of bilateral CIs who cannot adapt to a bilateral mismatch, because they suggest that (1) such individuals may benefit from modification of the frequency table in one ear and (2) self-selection of a "most intelligible" frequency table may be a useful tool for determining how the frequency table should be altered to optimize speech recognition.

Pitch Matching between Electrical Stimulation of a Cochlear Implant and Acoustic Stimuli Presented to a Contralateral Ear with Residual Hearing.

BACKGROUND: Cochlear implants (CIs) successfully restore hearing in postlingually deaf adults, but in doing so impose a frequency-position function in the cochlea that may differ from the physiological one. PURPOSE: The CI-imposed frequency-position function is determined by the frequency allocation table programmed into the listener's speech processor and by the location of the electrode array along the cochlea. To what extent can postlingually deaf CI users successfully adapt to the difference between physiological and CI-imposed frequency-position functions? RESEARCH DESIGN: We attempt to answer the question by combining behavioral measures of electroacoustic pitch matching (PM) and measures of electrode location within the cochlea. STUDY SAMPLE: The participants in this study were 16 adult CI users with residual hearing who could match the pitch of acoustic pure tones presented to their unimplanted ears to the pitch resulting from stimulation of different CI electrodes. DATA COLLECTION AND ANALYSIS: We obtained data for four to eight apical electrodes from 16 participants with CIs (most of whom were long-term users), and estimated electrode insertion angle for 12 of these participants. PM functions in this group were compared with the two frequency-position functions discussed above. RESULTS: Taken together, the findings were consistent with the possibility that adaptation to the frequency-position function imposed by CIs does happen, but it is not always complete. CONCLUSIONS: Some electrodes continue to be perceived as higher pitched than the acoustic frequencies with which they are associated despite years of listening experience after cochlear implantation.

Feasibility of real-time selection of frequency tables in an acoustic simulation of a cochlear implant.

OBJECTIVES: Perception of spectrally degraded speech is particularly difficult when the signal is also distorted along the frequency axis. This might be particularly important for post-lingually deafened recipients of cochlear implants (CIs), who must adapt to a signal where there may be a mismatch between the frequencies of an input signal and the characteristic frequencies of the neurons stimulated by the CI. However, there is a lack of tools that can be used to identify whether an individual has adapted fully to a mismatch in the frequency-to-place relationship and if so, to find a frequency table that ameliorates any negative effects of an unadapted mismatch. The goal of the proposed investigation is to test the feasibility of whether real-time selection of frequency tables can be used to identify cases in which listeners have not fully adapted to a frequency mismatch. The assumption underlying this approach is that listeners who have not adapted to a frequency mismatch will select a frequency table that minimizes any such mismatches, even at the expense of reducing the information provided by this frequency table. DESIGN: Thirty-four normal-hearing adults listened to a noise-vocoded acoustic simulation of a CI and adjusted the frequency table in real time until they obtained a frequency table that sounded "most intelligible" to them. The use of an acoustic simulation was essential to this study because it allowed the authors to explicitly control the degree of frequency mismatch present in the simulation. None of the listeners had any previous experience with vocoded speech, in order to test the hypothesis that the real-time selection procedure could be used to identify cases in which a listener has not adapted to a frequency mismatch. After obtaining a self-selected table, the authors measured consonant nucleus consonant word-recognition scores with that self-selected table and two other frequency tables: a "frequency-matched" table that matched the analysis filters with the noisebands of the noise-vocoder simulation, and a "right information" table that is similar to that used in most CI speech processors, but in this simulation results in a frequency shift equivalent to 6.5 mm of cochlear space. RESULTS: Listeners tended to select a table that was very close to, but shifted slightly lower in frequency from the frequency-matched table. The real-time selection process took on average 2 to 3 min for each trial, and the between-trial variability was comparable with that previously observed with closely related procedures. The word-recognition scores with the self-selected table were clearly higher than with the right-information table and slightly higher than with the frequency-matched table. CONCLUSIONS: Real-time self-selection of frequency tables may be a viable tool for identifying listeners who have not adapted to a mismatch in the frequency-to-place relationship, and to find a frequency table that is more appropriate for them. Moreover, the small but significant improvements in word-recognition ability observed with the self-selected table suggest that these listeners based their selections on intelligibility rather than some other factor. The within-subject variability in the real-time selection procedure was comparable with that of a genetic algorithm, and the speed of the real-time procedure appeared to be faster than either a genetic algorithm or a simplex procedure.

Real-time measurement of electrode impedance during intracochlear electrode insertion.

OBJECTIVES/HYPOTHESIS: This pilot study details the use of a software tool that uses continuous impedance measurement during electrode insertion, with the eventual potential to assess and optimize electrode position and reduce insertional trauma. STUDY DESIGN: Software development and experimental study with human cadaveric cochleae and two live surgeries. METHODS: A prototype program to measure intracochlear electrode impedance and display it graphically in real time has been developed. The software was evaluated in human cadaveric temporal bones while simultaneously making real-time fluoroscopic recordings and in two live surgeries during intracochlear electrode insertion. RESULTS: Impedance changes were observed with various scalar positions, and values were consistent with those obtained using clinically available software. Using Contour Advance electrodes, impedance values increased after stylet removal, particularly when using the monopolar mode. CONCLUSIONS: Impedance values seem systematically affected by electrode position, with higher values being associated with proximity to the cochlear wall. The new software is capable of acquiring impedance measurements during electrode insertion, and these data may be useful to guide surgeons to achieve optimal and atraumatic electrode insertion, to guide robotic electrode insertion, and to provide insights about electrode position in the cochlea.

VALIDATION OF ACOUSTIC MODELS OF AUDITORY NEURAL PROSTHESES.

Acoustic models have been used in numerous studies over the past thirty years to simulate the percepts elicited by auditory neural prostheses. In these acoustic models, incoming signals are processed the same way as in a cochlear implant speech processor. The percepts that would be caused by electrical stimulation in a real cochlear implant are simulated by modulating the amplitude of either noise bands or sinusoids. Despite their practical usefulness these acoustic models have never been convincingly validated. This study presents a tool to conduct such validation using subjects who have a cochlear implant in one ear and have near perfect hearing in the other ear, allowing for the first time a direct perceptual comparison of the output of acoustic models to the stimulation provided by a cochlear implant.

Current and planned cochlear implant research at New York University Laboratory for Translational Auditory Research.

The Laboratory of Translational Auditory Research (LTAR/NYUSM) is part of the Department of Otolaryngology at the New York University School of Medicine and has close ties to the New York University Cochlear Implant Center. LTAR investigators have expertise in multiple related disciplines including speech and hearing science, audiology, engineering, and physiology. The lines of research in the laboratory deal mostly with speech perception by hearing impaired listeners, and particularly those who use cochlear implants (CIs) or hearing aids (HAs). Although the laboratory's research interests are diverse, there are common threads that permeate and tie all of its work. In particular, a strong interest in translational research underlies even the most basic studies carried out in the laboratory. Another important element is the development of engineering and computational tools, which range from mathematical models of speech perception to software and hardware that bypass clinical speech processors and stimulate cochlear implants directly, to novel ways of analyzing clinical outcomes data. If the appropriate tool to conduct an important experiment does not exist, we may work to develop it, either in house or in collaboration with academic or industrial partners. Another notable characteristic of the laboratory is its interdisciplinary nature where, for example, an audiologist and an engineer might work closely to develop an approach that would not have been feasible if each had worked singly on the project. Similarly, investigators with expertise in hearing aids and cochlear implants might join forces to study how human listeners integrate information provided by a CI and a HA. The following pages provide a flavor of the diversity and the commonalities of our research interests.

A new software tool to optimize frequency table selection for cochlear implants.

HYPOTHESIS: When cochlear implant (CI) users are allowed to self-select the "most intelligible" frequency-to-electrode table, some of them choose one that differs from the default frequency table that is normally used in clinical practice. BACKGROUND: CIs reproduce the tonotopicity of normal cochleas using frequency-to-electrode tables that assign stimulation of more basal electrodes to higher frequencies and more apical electrodes to lower frequency sounds. Current audiologic practice uses a default frequency-to-electrode table for most patients. However, individual differences in cochlear size, neural survival, and electrode positioning may result in different tables sounding most intelligible to different patients. No clinical tools currently exist to facilitate this fitting. METHODS: A software tool was designed that enables CI users to self-select a most intelligible frequency table. Users explore a 2-dimensional space that represents a range of different frequency tables. Unlike existing tools, this software enables users to interactively audition speech processed by different frequency tables and quickly identify a preferred one. Pilot testing was performed in 11 long-term, postlingually deaf CI users. RESULTS: The software tool was designed, developed, tested, and debugged. Patients successfully used the tool to sample frequency tables and to self-select tables deemed most intelligible, which for approximately half of the users differed from the clinical default. CONCLUSION: A software tool allowing CI users to self-select frequency-to-electrode tables may help in fitting postlingually deaf users. This novel approach may transform current methods of CI fitting.

Perception of nonlinear distortion by hearing-impaired people.

All hearing aids and communication devices introduce nonlinear distortion. The perception of distortion by hearing-impaired subjects was studied using artificial controlled distortions of various amounts and types. Subjects were asked to rate the perceived quality of distorted speech and music. Stimuli were subjected to frequency-dependent amplification as prescribed by the 'Cambridge formula' before presentation via Sennheiser HD580 earphones. The pattern of the ratings was reasonably consistent across subjects, but two of the eight subjects showed inconsistent results for the speech stimuli. Center clipping and soft clipping had only small effects on the ratings, while hard clipping and 'full-range' distortion had large effects. The results indicate that most hearing-impaired subjects are able to make orderly and consistent ratings of degradations in sound quality introduced by nonlinear distortion. The pattern of results could be predicted reasonably well using a model developed to account for the perception of distortion by normally hearing subjects.

Perceived naturalness of spectrally distorted speech and music.

We determined how the perceived naturalness of music and speech (male and female talkers) signals was affected by various forms of linear filtering, some of which were intended to mimic the spectral "distortions" introduced by transducers such as microphones, loudspeakers, and earphones. The filters introduced spectral tilts and ripples of various types, variations in upper and lower cutoff frequency, and combinations of these. All of the differently filtered signals (168 conditions) were intermixed in random order within one block of trials. Levels were adjusted to give approximately equal loudness in all conditions. Listeners were required to judge the perceptual quality (naturalness) of the filtered signals on a scale from 1 to 10. For spectral ripples, perceived quality decreased with increasing ripple density up to 0.2 ripple/ERB(N) and with increasing ripple depth. Spectral tilts also degraded quality, and the effects were similar for positive and negative tilts. Ripples and/or tilts degraded quality more when they extended over a wide frequency range (87-6981 Hz) than when they extended over subranges. Low- and mid-frequency ranges were roughly equally important for music, but the mid-range was most important for speech. For music, the highest quality was obtained for the broadband signal (55-16,854 Hz). Increasing the lower cutoff frequency from 55 Hz resulted in a clear degradation of quality. There was also a distinct degradation as the upper cutoff frequency was decreased from 16,845 Hz. For speech, there was a marked degradation when the lower cutoff frequency was increased from 123 to 208 Hz and when the upper cutoff frequency was decreased from 10,869 Hz. Typical telephone bandwidth (313 to 3547 Hz) gave very poor quality.