Seizure Control in a Computational Model Using a Reinforcement Learning Stimulation Paradigm.

Neuromodulation technologies such as vagus nerve stimulation and deep brain stimulation, have shown some efficacy in controlling seizures in medically intractable patients. However, inherent patient-to-patient variability of seizure disorders leads to a wide range of therapeutic efficacy. A patient specific approach to determining stimulation parameters may lead to increased therapeutic efficacy while minimizing stimulation energy and side effects. This paper presents a reinforcement learning algorithm that optimizes stimulation frequency for controlling seizures with minimum stimulation energy. We apply our method to a computational model called the epileptor. The epileptor model simulates inter-ictal and ictal local field potential data. In order to apply reinforcement learning to the Epileptor, we introduce a specialized reward function and state-space discretization. With the reward function and discretization fixed, we test the effectiveness of the temporal difference reinforcement learning algorithm (TD(0)). For periodic pulsatile stimulation, we derive a relation that describes, for any stimulation frequency, the minimal pulse amplitude required to suppress seizures. The TD(0) algorithm is able to identify parameters that control seizures quickly. Additionally, our results show that the TD(0) algorithm refines the stimulation frequency to minimize stimulation energy thereby converging to optimal parameters reliably. An advantage of the TD(0) algorithm is that it is adaptive so that the parameters necessary to control the seizures can change over time. We show that the algorithm can converge on the optimal solution in simulation with slow and fast inter-seizure intervals.

Future of seizure prediction and intervention: closing the loop.

The ultimate goal of epilepsy therapies is to provide seizure control for all patients while eliminating side effects. Improved specificity of intervention through on-demand approaches may overcome many of the limitations of current intervention strategies. This article reviews the progress in seizure prediction and detection, potential new therapies to provide improved specificity, and devices to achieve these ends. Specifically, we discuss (1) potential signal modalities and algorithms for seizure detection and prediction, (2) closed-loop intervention approaches, and (3) hardware for implementing these algorithms and interventions. Seizure prediction and therapies maximize efficacy, whereas minimizing side effects through improved specificity may represent the future of epilepsy treatments.