From Microphone to Phoneme: An End-to-End Computational Neural Model for Predicting Speech Perception With Cochlear Implants.

GOAL: Advances in computational models of biological systems and artificial neural networks enable rapid virtual prototyping of neuroprostheses, accelerating innovation in the field. Here, we present an end-to-end computational model for predicting speech perception with cochlear implants (CI), the most widely-used neuroprosthesis. METHODS: The model integrates CI signal processing, a finite element model of the electrically-stimulated cochlea, and an auditory nerve model to predict neural responses to speech stimuli. An automatic speech recognition neural network is then used to extract phoneme-level speech perception from these neural response patterns. RESULTS: Compared to human CI listener data, the model predicts similar patterns of speech perception and misperception, captures between-phoneme differences in perceptibility, and replicates effects of stimulation parameters and noise on speech recognition. Information transmission analysis at different stages along the CI processing chain indicates that the bottleneck of information flow occurs at the electrode-neural interface, corroborating studies in CI listeners. CONCLUSION: An end-to-end model of CI speech perception replicated phoneme-level CI speech perception patterns, and was used to quantify information degradation through the CI processing chain. SIGNIFICANCE: This type of model shows great promise for developing and optimizing new and existing neuroprostheses.

The Panoramic ECAP Method: Estimating Patient-Specific Patterns of Current Spread and Neural Health in Cochlear Implant Users.

The knowledge of patient-specific neural excitation patterns from cochlear implants (CIs) can provide important information for optimizing efficacy and improving speech perception outcomes. The Panoramic ECAP ('PECAP') method (Cosentino et al. 2015) uses forward-masked electrically evoked compound action-potentials (ECAPs) to estimate neural activation patterns of CI stimulation. The algorithm requires ECAPs be measured for all combinations of probe and masker electrodes, exploiting the fact that ECAP amplitudes reflect the overlapping excitatory areas of both probes and maskers. Here we present an improved version of the PECAP algorithm that imposes biologically realistic constraints on the solution, that, unlike the previous version, produces detailed estimates of neural activation patterns by modelling current spread and neural health along the intracochlear electrode array and is capable of identifying multiple regions of poor neural health. The algorithm was evaluated for reliability and accuracy in three ways: (1) computer-simulated current-spread and neural-health scenarios, (2) comparisons to psychophysical correlates of neural health and electrode-modiolus distances in human CI users, and (3) detection of simulated neural 'dead' regions (using forward masking) in human CI users. The PECAP algorithm reliably estimated the computer-simulated scenarios. A moderate but significant negative correlation between focused thresholds and the algorithm's neural-health estimates was found, consistent with previous literature. It also correctly identified simulated 'dead' regions in all seven CI users evaluated. The revised PECAP algorithm provides an estimate of neural excitation patterns in CIs that could be used to inform and optimize CI stimulation strategies for individual patients in clinical settings.

An Instrumented Cochlea Model for the Evaluation of Cochlear Implant Electrical Stimulus Spread.

Cochlear implants use electrical stimulation of the auditory nerve to restore the sensation of hearing to deaf people. Unfortunately, the stimulation current spreads extensively within the cochlea, resulting in "blurring" of the signal, and hearing that is far from normal. Current spread can be indirectly measured using the implant electrodes for both stimulating and sensing, but this provides incomplete information near the stimulating electrode due to electrode-electrolyte interface effects. Here, we present a 3D-printed "unwrapped" physical cochlea model with integrated sensing wires. We integrate resistors into the walls of the model to simulate current spread through the cochlear bony wall, and "tune" these resistances by calibration with an in-vivo electrical measurement from a cochlear implant patient. We then use this model to compare electrical current spread under different stimulation modes including monopolar, bipolar and tripolar configurations. Importantly, a trade-off is observed between stimulation amplitude and current focusing among different stimulation modes. By combining different stimulation modes and changing intracochlear current sinking configurations in the model, we explore this trade-off between stimulation amplitude and focusing further. These results will inform clinical strategies for use in delivering speech signals to cochlear implant patients.

Evaluating and Comparing Behavioural and Electrophysiological Estimates of Neural Health in Cochlear Implant Users.

Variations in neural health along the cochlea can degrade the spectral and temporal representation of sounds conveyed by cochlear implants (CIs). We evaluated and compared one electrophysiological measure and two behavioural measures that have been proposed as estimates of neural health patterns, in order to explore the extent to which the different measures provide converging and consistent neural health estimates. All measures were obtained from the same 11 users of the Cochlear Corporation CI. The two behavioural measures were multipulse integration (MPI) and the polarity effect (PE), both measured on each of seven electrodes per subject. MPI was measured as the difference between thresholds at 80 pps and 1000 pps, and PE as the difference in thresholds between cathodic- and anodic-centred quadraphasic (QP) 80-pps pulse trains. It has been proposed that good neural health corresponds to a large MPI and to a large negative PE (lower thresholds for cathodic than anodic pulses). The electrophysiological measure was the effect of interphase gap (IPG) on the offset of the ECAP amplitude growth function (AGF), which has been correlated with spiral ganglion neuron density in guinea pigs. This 'IPG offset' was obtained on the same subset of electrodes used for the behavioural measures. Despite high test-retest reliability, there were no significant correlations between the neural health estimates for either within-subject comparisons across the electrode array, or between-subject comparisons of the means. A phenomenological model of a population of spiral ganglion neurons was then used to investigate physiological mechanisms that might underlie the different neural health estimates. The combined experimental and modelling results provide evidence that PE, MPI and IPG offset may reflect different characteristics of the electrode-neural interface.

Interpreting the Effect of Stimulus Parameters on the Electrically Evoked Compound Action Potential and on Neural Health Estimates.

Variations in the condition of the neural population along the length of the cochlea can degrade the spectral and temporal representation of sounds conveyed by CIs, thereby limiting speech perception. One measurement that has been proposed as an estimate of neural survival (the number of remaining functional neurons) or neural health (the health of those remaining neurons) is the effect of stimulation parameters, such as the interphase gap (IPG), on the amplitude growth function (AGF) of the electrically evoked compound action potential (ECAP). The extent to which such measures reflect neural factors, rather than non-neural factors (e.g. electrode orientation, electrode-modiolus distance, and impedance), depends crucially upon how the AGF data are analysed. However, there is currently no consensus in the literature for the correct method to interpret changes in the ECAP AGF due to changes in stimulation parameters. We present a simple theoretical model for the effect of IPG on ECAP AGFs, along with a re-analysis of both animal and human data that measured the IPG effect. Both the theoretical model and the re-analysis of the animal data suggest that the IPG effect on ECAP AGF slope (IPG slope effect), measured using either a linear or logarithmic input-output scale, does not successfully control for the effects of non-neural factors. Both the model and the data suggest that the appropriate method to estimate neural health is by measuring the IPG offset effect, defined as the dB offset between the linear portions of ECAP AGFs for two stimuli differing only in IPG.

Encoding speech in cochlear implants using simultaneous amplitude and rate modulation.

To improve speech perception for cochlear implant (CI) users, it is essential to improve the transmission of temporal envelopes. The most common speech processors deliver temporal envelopes via the CI using fixed-rate amplitude modulated (AM) pulse trains. Psychophysical studies suggest that rate modulation (RM) and AM are perceived by a shared temporal integration mechanism, but the potential for them to constructively combine to encode temporal envelopes has yet to be explored. In this experiment, a speech processing strategy called amplitude and rate temporal modulation was developed to encode speech temporal envelopes with simultaneous AM and RM. The strategy was tested for perception of clean speech at 60 and 40 dBA, and 60 dBA speech in noise (+10 dB SNR). The amount of RM was varied and the amount of AM was held constant to determine whether the addition of RM could enhance the perception of temporal envelopes and improve speech understanding. At the lowest RM amount, speech scores were poorest for all speech conditions. For 60 dBA clean speech and speech in noise, speech scores were significantly better at the highest RM amounts, suggesting that RM combined with AM can be used to enhance perception of temporal envelopes.

Rate modulation detection thresholds for cochlear implant users.

The perception of temporal amplitude modulations is critical for speech understanding by cochlear implant (CI) users. The present study compared the ability of CI users to detect sinusoidal modulations of the electrical stimulation rate and current level, at different presentation levels (80% and 40% of the dynamic range) and modulation frequencies (10 and 100 Hz). Rate modulation detection thresholds (RMDTs) and amplitude modulation detection thresholds (AMDTs) were measured and compared to assess whether there was a perceptual advantage to either modulation method. Both RMDTs and AMDTs improved with increasing presentation level and decreasing modulation frequency. RMDTs and AMDTs were correlated, indicating that a common processing mechanism may underlie the perception of rate modulation and amplitude modulation, or that some subject-dependent factors affect both types of modulation detection.

The effect of presentation level and stimulation rate on speech perception and modulation detection for cochlear implant users.

In order to improve speech understanding for cochlear implant users, it is important to maximize the transmission of temporal information. The combined effects of stimulation rate and presentation level on temporal information transfer and speech understanding remain unclear. The present study systematically varied presentation level (60, 50, and 40 dBA) and stimulation rate [500 and 2400 pulses per second per electrode (pps)] in order to observe how the effect of rate on speech understanding changes for different presentation levels. Speech recognition in quiet and noise, and acoustic amplitude modulation detection thresholds (AMDTs) were measured with acoustic stimuli presented to speech processors via direct audio input (DAI). With the 500 pps processor, results showed significantly better performance for consonant-vowel nucleus-consonant words in quiet, and a reduced effect of noise on sentence recognition. However, no rate or level effect was found for AMDTs, perhaps partly because of amplitude compression in the sound processor. AMDTs were found to be strongly correlated with the effect of noise on sentence perception at low levels. These results indicate that AMDTs, at least when measured with the CP910 Freedom speech processor via DAI, explain between-subject variance of speech understanding, but do not explain within-subject variance for different rates and levels.