Nanophotonic reservoir computing with photonic crystal cavities to generate periodic patterns.

Reservoir computing (RC) is a technique in machine learning inspired by neural systems. RC has been used successfully to solve complex problems such as signal classification and signal generation. These systems are mainly implemented in software, and thereby they are limited in speed and power efficiency. Several optical and optoelectronic implementations have been demonstrated, in which the system has signals with an amplitude and phase. It is proven that these enrich the dynamics of the system, which is beneficial for the performance. In this paper, we introduce a novel optical architecture based on nanophotonic crystal cavities. This allows us to integrate many neurons on one chip, which, compared with other photonic solutions, closest resembles a classical neural network. Furthermore, the components are passive, which simplifies the design and reduces the power consumption. To assess the performance of this network, we train a photonic network to generate periodic patterns, using an alternative online learning rule called first-order reduced and corrected error. For this, we first train a classical hyperbolic tangent reservoir, but then we vary some of the properties to incorporate typical aspects of a photonics reservoir, such as the use of continuous-time versus discrete-time signals and the use of complex-valued versus real-valued signals. Then, the nanophotonic reservoir is simulated and we explore the role of relevant parameters such as the topology, the phases between the resonators, the number of nodes that are biased and the delay between the resonators. It is important that these parameters are chosen such that no strong self-oscillations occur. Finally, our results show that for a signal generation task a complex-valued, continuous-time nanophotonic reservoir outperforms a classical (i.e., discrete-time, real-valued) leaky hyperbolic tangent reservoir (normalized root-mean-square errors=0.030 versus NRMSE=0.127).

Experimental demonstration of reservoir computing on a silicon photonics chip.

In today's age, companies employ machine learning to extract information from large quantities of data. One of those techniques, reservoir computing (RC), is a decade old and has achieved state-of-the-art performance for processing sequential data. Dedicated hardware realizations of RC could enable speed gains and power savings. Here we propose the first integrated passive silicon photonics reservoir. We demonstrate experimentally and through simulations that, thanks to the RC paradigm, this generic chip can be used to perform arbitrary Boolean logic operations with memory as well as 5-bit header recognition up to 12.5 Gbit s(-1), without power consumption in the reservoir. It can also perform isolated spoken digit recognition. Our realization exploits optical phase for computing. It is scalable to larger networks and much higher bitrates, up to speeds >100 Gbit s(-1). These results pave the way for the application of integrated photonic RC for a wide range of applications.

Real-time detection of epileptic seizures in animal models using reservoir computing.

In recent years, an increasing number of studies have investigated the effects of closed-loop anti-epileptic treatments. Most of the current research still is very labour intensive: real-time treatment is manually triggered and conclusions can only be drawn after multiple days of manual review and annotation of the electroencephalogram (EEG). In this paper we propose a technique based on reservoir computing (RC) to automatically and in real-time detect epileptic seizures in the intra-cranial EEG (iEEG) of epileptic rats in order to immediately trigger seizure treatment. The performance of the system is evaluated in two different seizure types: absence seizures from genetic absence epilepsy rats from Strasbourg (GAERS) and limbic seizures from post status epilepticus (PSE) rats. The dataset consists of 452 hours iEEG from 23 GAERS and 2083 hours iEEG from 22 PSE rats. In the default set-up the system detects 0.09 and 0.13 false positives per seizure and misses 0.07 and 0.005 events per seizure for GAERS and PSE rats respectively. It achieves an average detection delay below 1s in GAERS and less than 10s in the PSE data. This detection delay and the number of missed seizures can be further decreased when a higher false positive rate is allowed. Our method outperforms state-of-the-art detection techniques and only a few parameters require optimization on a limited training set. It is therefore suited for automatic seizure detection based on iEEG and may serve as a useful tool for epilepsy researchers. The technique avoids the time-consuming manual review and annotation of EEG and can be incorporated in a closed-loop treatment strategy.

Information processing capacity of dynamical systems.

Many dynamical systems, both natural and artificial, are stimulated by time dependent external signals, somehow processing the information contained therein. We demonstrate how to quantify the different modes in which information can be processed by such systems and combine them to define the computational capacity of a dynamical system. This is bounded by the number of linearly independent state variables of the dynamical system, equaling it if the system obeys the fading memory condition. It can be interpreted as the total number of linearly independent functions of its stimuli the system can compute. Our theory combines concepts from machine learning (reservoir computing), system modeling, stochastic processes, and functional analysis. We illustrate our theory by numerical simulations for the logistic map, a recurrent neural network, and a two-dimensional reaction diffusion system, uncovering universal trade-offs between the non-linearity of the computation and the system's short-term memory.

A bayesian model for exploiting application constraints to enable unsupervised training of a P300-based BCI.

This work introduces a novel classifier for a P300-based speller, which, contrary to common methods, can be trained entirely unsupervisedly using an Expectation Maximization approach, eliminating the need for costly dataset collection or tedious calibration sessions. We use publicly available datasets for validation of our method and show that our unsupervised classifier performs competitively with supervised state-of-the-art spellers. Finally, we demonstrate the added value of our method in different experimental settings which reflect realistic usage situations of increasing difficulty and which would be difficult or impossible to tackle with existing supervised or adaptive methods.

Automatic detection of epileptic seizures on the intra-cranial electroencephalogram of rats using reservoir computing.

INTRODUCTION: In this paper we propose a technique based on reservoir computing (RC) to mark epileptic seizures on the intra-cranial electroencephalogram (EEG) of rats. RC is a recurrent neural networks training technique which has been shown to possess good generalization properties with limited training. MATERIALS: The system is evaluated on data containing two different seizure types: absence seizures from genetic absence epilepsy rats from Strasbourg (GAERS) and tonic-clonic seizures from kainate-induced temporal-lobe epilepsy rats. The dataset consists of 452hours from 23 GAERS and 982hours from 15 kainate-induced temporal-lobe epilepsy rats. METHODS: During the preprocessing stage, several features are extracted from the EEG. A feature selection algorithm selects the best features, which are then presented as input to the RC-based classification algorithm. To classify the output of this algorithm a two-threshold technique is used. This technique is compared with other state-of-the-art techniques. RESULTS: A balanced error rate (BER) of 3.7% and 3.5% was achieved on the data from GAERS and kainate rats, respectively. This resulted in a sensitivity of 96% and 94% and a specificity of 96% and 99% respectively. The state-of-the-art technique for GAERS achieved a BER of 4%, whereas the best technique to detect tonic-clonic seizures achieved a BER of 16%. CONCLUSION: Our method outperforms up-to-date techniques and only a few parameters need to be optimized on a limited training set. It is therefore suited as an automatic aid for epilepsy researchers and is able to eliminate the tedious manual review and annotation of EEG.

Parallel reservoir computing using optical amplifiers.

Reservoir computing (RC), a computational paradigm inspired on neural systems, has become increasingly popular in recent years for solving a variety of complex recognition and classification problems. Thus far, most implementations have been software-based, limiting their speed and power efficiency. Integrated photonics offers the potential for a fast, power efficient and massively parallel hardware implementation. We have previously proposed a network of coupled semiconductor optical amplifiers as an interesting test case for such a hardware implementation. In this paper, we investigate the important design parameters and the consequences of process variations through simulations. We use an isolated word recognition task with babble noise to evaluate the performance of the photonic reservoirs with respect to traditional software reservoir implementations, which are based on leaky hyperbolic tangent functions. Our results show that the use of coherent light in a well-tuned reservoir architecture offers significant performance benefits. The most important design parameters are the delay and the phase shift in the system's physical connections. With optimized values for these parameters, coherent semiconductor optical amplifier (SOA) reservoirs can achieve better results than traditional simulated reservoirs. We also show that process variations hardly degrade the performance, but amplifier noise can be detrimental. This effect must therefore be taken into account when designing SOA-based RC implementations.

Toward optical signal processing using photonic reservoir computing.

We propose photonic reservoir computing as a new approach to optical signal processing in the context of large scale pattern recognition problems. Photonic reservoir computing is a photonic implementation of the recently proposed reservoir computing concept, where the dynamics of a network of nonlinear elements are exploited to perform general signal processing tasks. In our proposed photonic implementation, we employ a network of coupled Semiconductor Optical Amplifiers (SOA) as the basic building blocks for the reservoir. Although they differ in many key respects from traditional software-based hyperbolic tangent reservoirs, we show using simulations that such a photonic reservoir can outperform traditional reservoirs on a benchmark classification task. Moreover, a photonic implementation offers the promise of massively parallel information processing with low power and high speed.

Compact hardware liquid state machines on FPGA for real-time speech recognition.

Hardware implementations of Spiking Neural Networks are numerous because they are well suited for implementation in digital and analog hardware, and outperform classic neural networks. This work presents an application driven digital hardware exploration where we implement real-time, isolated digit speech recognition using a Liquid State Machine. The Liquid State Machine is a recurrent neural network of spiking neurons where only the output layer is trained. First we test two existing hardware architectures which we improve and extend, but that appears to be too fast and thus area consuming for this application. Next, we present a scalable, serialized architecture that allows a very compact implementation of spiking neural networks that is still fast enough for real-time processing. All architectures support leaky integrate-and-fire membranes with exponential synaptic models. This work shows that there is actually a large hardware design space of Spiking Neural Network hardware that can be explored. Existing architectures have only spanned part of it.