Utilising reinforcement learning to develop strategies for driving auditory neural implants.

OBJECTIVE: In this paper we propose a novel application of reinforcement learning to the area of auditory neural stimulation. We aim to develop a simulation environment which is based off real neurological responses to auditory and electrical stimulation in the cochlear nucleus (CN) and inferior colliculus (IC) of an animal model. Using this simulator we implement closed loop reinforcement learning algorithms to determine which methods are most effective at learning effective acoustic neural stimulation strategies. APPROACH: By recording a comprehensive set of acoustic frequency presentations and neural responses from a set of animals we created a large database of neural responses to acoustic stimulation. Extensive electrical stimulation in the CN and the recording of neural responses in the IC provides a mapping of how the auditory system responds to electrical stimuli. The combined dataset is used as the foundation for the simulator, which is used to implement and test learning algorithms. MAIN RESULTS: Reinforcement learning, utilising a modified n-Armed Bandit solution, is implemented to demonstrate the model's function. We show the ability to effectively learn stimulation patterns which mimic the cochlea's ability to covert acoustic frequencies to neural activity. Time taken to learn effective replication using neural stimulation takes less than 20 min under continuous testing. SIGNIFICANCE: These results show the utility of reinforcement learning in the field of neural stimulation. These results can be coupled with existing sound processing technologies to develop new auditory prosthetics that are adaptable to the recipients current auditory pathway. The same process can theoretically be abstracted to other sensory and motor systems to develop similar electrical replication of neural signals.

Modeling of drug release from collagen matrices.

Drug release from collagen matrices is in most cases governed by diffusion from swollen matrices but also enzymatic matrix degradation or hydrophobic drug/collagen interactions may contribute. To reduce water uptake and to prolong the release, insoluble collagen matrices have been chemically or dehydrothermally crosslinked. Assuming Fickian diffusion a one-dimensional model was developed and tested that allows description of water penetration, swelling and drug release and that may be expanded considering a subsequent erosion process or interactions. Swelling is described by a volume balance. For dry collagen matrices crosslinked by thermal treatment the existence of a moving front separating the polymer from a gel phase was considered, and a convective term induced by the volume expansion was incorporated. The resulting moving boundary problem was solved using a method based on biquadratic finite elements in both space and time that is stable, shows high accuracy, and is suitable for solving problems with a singularity at the initial time point. The model was verified for insoluble collagen matrices at different crosslinking degrees for both chemical and thermal treatment. For constant diffusion coefficients a close form of the solution was derived yielding equivalent results to the numerical approach.