Learning to see via epiretinal implant stimulationin silicowith model-based deep reinforcement learning.

OBJECTIVE: Diseases such as age-related macular degeneration and retinitis pigmentosa cause the degradation of the photoreceptor layer. One approach to restore vision is to electrically stimulate the surviving retinal ganglion cells with a microelectrode array such as epiretinal implants. Epiretinal implants are known to generate visible anisotropic shapes elongated along the axon fascicles of neighboring retinal ganglion cells. Recent work has demonstrated that to obtain isotropic pixel-like shapes, it is possible to map axon fascicles and avoid stimulating them by inactivating electrodes or lowering stimulation current levels. Avoiding axon fascicule stimulation aims to remove brushstroke-like shapes in favor of a more reduced set of pixel-like shapes. APPROACH: In this study, we propose the use of isotropic and anisotropic shapes to render intelligible images on the retina of a virtual patient in a reinforcement learning environment named rlretina. The environment formalizes the task as using brushstrokes in a stroke-based rendering task. MAIN RESULTS: We train a deep reinforcement learning agent that learns to assemble isotropic and anisotropic shapes to form an image. We investigate which error-based or perception-based metrics are adequate to reward the agent. The agent is trained in a model-based data generation fashion using the psychophysically validated axon map model to render images as perceived by different virtual patients. We show that the agent can generate more intelligible images compared to the naive method in different virtual patients. SIGNIFICANCE: This work shares a new way to address epiretinal stimulation that constitutes a first step towards improving visual acuity in artificially-restored vision using anisotropic phosphenes.

The effect of input noises on the activity of auditory neurons using GLM-based metrics.

Objective.The auditory system is extremely efficient in extracting auditory information in the presence of background noise. However, people with auditory implants have a hard time understanding speech in noisy conditions. The neural mechanisms related to the processing of background noise, especially in the inferior colliculus (IC) where the auditory midbrain implant is located, are still not well understood. Understanding the mechanisms of perception in noise could lead to better stimulation or preprocessing strategies for such implants. We thus wish to investigate if there is a difference in the activity of neurons in the IC when presenting noisy vocalizations with different types of noise (stationary vs. non-stationary), input signal-to-noise ratios (SNR) and signal levels.Approach.We developed novel metrics based on a generalized linear model (GLM) to investigate the effect of a given input noise on neural activity. We used these metrics to analyze neural data recorded from the IC in ketamine-anesthetized female Hartley guinea pigs while presenting noisy vocalizations.Main results.We found that non-stationary noise clearly contributes to the multi-unit neural activity in the IC by causing excitation, regardless of the SNR, input level or vocalization type. However, when presenting white or natural stationary noises, a great diversity of responses was observed for the different conditions, where the multi-unit activity of some sites was affected by the presence of noise and the activity of others was not.Significance.The GLM-based metrics allowed the identification of a clear distinction between the effect of white or natural stationary noises and that of non-stationary noise on the multi-unit activity in the IC. This had not been observed before and indicates that the so-called noise invariance in the IC is dependent on the input noisy conditions. This could suggest different preprocessing or stimulation approaches for auditory midbrain implants depending on the noisy conditions.

A frequency-band coupling model of EEG signals can capture features from an input audio stimulus.

Despite decades of research, the features of an input audio stimulus that are encoded in an electroencephalogram (EEG) are still not clearly identified. We wish to investigate whether a frequency-band coupling model that estimates the cortical neural activity from EEGs can capture the important features of an input audio stimulus. To do so, EEG recordings were acquired from 8 subjects during a listening task where the vowels a, i and u were randomly presented. The neural activity was estimated from the EEG using a frequency-band coupling model that combined the EEG's phase in the delta band (2 Hz-4 Hz) and its amplitude in the gamma band (30 Hz-100 Hz). To investigate if the estimated neural activity could capture relevant features of an input audio stimulus, we fitted a generalized linear model (GLM) to the estimated neural activity and applied a statistical relative deviance metric to evaluate how important is the input audio stimulus in the estimated neural activity. We demonstrate that the input audio stimulus is the main component explaining the estimated neural activity and that other aspects such as the contribution of the surrounding network dynamics do not contribute significantly to the estimated neural activity. These results confirm that the features of the EEG used in the coupling model, namely the phase of the delta band and the power of the gamma band, do encode relevant aspects of an input audio signal. This non-invasive approach could be used, for example, to study how the presence of spectro-temporal features in the estimated neural activity is modified depending on different listening conditions or types of input sounds.

Novel metrics to measure the effect of additive inputs on the activity of sensory system neurons.

Sensory systems, such as the visual or auditory system, are highly non linear. It is therefore not easy to predict the effect of additive inputs on the spiking activity of related brain structures. Here, we propose two metrics to study the effect of additive covariates on the spiking activity of neurons. These metrics are directly obtained from a generalized linear model. We apply these metrics to the study of the effect of additive input audio noise on the spiking activity of neurons in the auditory system. To do so, we combine clean vocalisations with natural stationary or non-stationary noises and record activity in the auditory system while presenting the noisy vocalisations. We found that non-stationary noise has a greater effect on the neural activity than stationary noise. We observe that the results, obtained using the proposed metrics, is more consistent with current knowledge in auditory neuroscience than the results obtained when using a common metric from the literature, the extraction index.

Improving neural decoding in the central auditory system using bio-inspired spectro-temporal representations and a generalized bilinear model.

We study the impact of different encoding models and spectro-temporal representations on the accuracy of Bayesian decoding of neural activity recorded from the central auditory system. Two encoding models, a generalized linear model (GLM) and a generalized bilinear model (GBM), are compared along with three different spectro-temporal representations of the input stimuli: a spectrogram and two bio-inspired representations, i.e. a gammatone filter bank (GFB) and a spikegram. Signal to noise ratios between the reconstructed and original representations are used to evaluate the decoding, or reconstruction accuracy. We experimentally show that the reconstruction accuracy is best with the spikegram representation and worst with the spectrogram representation and, furthermore, that using a GBM instead of a GLM significantly increases the reconstruction accuracy. In fact, our results show that the spikegram reconstruction accuracy with a GBM fitting yields an SNR that is 3.3 dB better than when using the standard decoding approach of reconstructing a spectrogram with GLM fitting.

A point process model for auditory neurons considering both their intrinsic dynamics and the spectrotemporal properties of an extrinsic signal.

We propose a point process model of spiking activity from auditory neurons. The model takes account of the neuron's intrinsic dynamics as well as the spectrotemporal properties of an input stimulus. A discrete Volterra expansion is used to derive the form of the conditional intensity function. The Volterra expansion models the neuron's baseline spike rate, its intrinsic dynamics-spiking history-and the stimulus effect which in this case is the analog of the spectrotemporal receptive field (STRF). We performed the model fitting efficiently in a generalized linear model framework using ridge regression to address properly this ill-posed maximum likelihood estimation problem. The model provides an excellent fit to spiking activity from 55 auditory nerve neurons. The STRF-like representation estimated jointly with the neuron's intrinsic dynamics may offer more accurate characterizations of neural activity in the auditory system than current ones based solely on the STRF.