Spiking Neural Networks and online learning: An overview and perspectives.

Applications that generate huge amounts of data in the form of fast streams are becoming increasingly prevalent, being therefore necessary to learn in an online manner. These conditions usually impose memory and processing time restrictions, and they often turn into evolving environments where a change may affect the input data distribution. Such a change causes that predictive models trained over these stream data become obsolete and do not adapt suitably to new distributions. Specially in these non-stationary scenarios, there is a pressing need for new algorithms that adapt to these changes as fast as possible, while maintaining good performance scores. Unfortunately, most off-the-shelf classification models need to be retrained if they are used in changing environments, and fail to scale properly. Spiking Neural Networks have revealed themselves as one of the most successful approaches to model the behavior and learning potential of the brain, and exploit them to undertake practical online learning tasks. Besides, some specific flavors of Spiking Neural Networks can overcome the necessity of retraining after a drift occurs. This work intends to merge both fields by serving as a comprehensive overview, motivating further developments that embrace Spiking Neural Networks for online learning scenarios, and being a friendly entry point for non-experts.

Exploiting the stimuli encoding scheme of evolving Spiking Neural Networks for stream learning.

Stream data processing has lately gained momentum with the arrival of new Big Data scenarios and applications dealing with continuously produced information flows. Unfortunately, traditional machine learning algorithms are not prepared to tackle the specific challenges imposed by data stream processing, such as the need for learning incrementally, limited memory and processing time requirements, and adaptation to non-stationary data, among others. To face these paradigms, Spiking Neural Networks have emerged as one of the most promising stream learning techniques, with variants such as Evolving Spiking Neural Networks capable of efficiently addressing many of these challenges. Interestingly, these networks resort to a particular population encoding scheme - Gaussian Receptive Fields - to transform the incoming stimuli into temporal spikes. The study presented in this manuscript sheds light on the predictive potential of this encoding scheme, focusing on how it can be applied as a computationally lightweight, model-agnostic preprocessing step for data stream learning. We provide informed intuition to unveil under which circumstances the aforementioned population encoding method yields effective prediction gains in data stream classification with respect to the case where no preprocessing is performed. Results obtained for a variety of stream learning models and both synthetic and real stream datasets are discussed to empirically buttress the capability of Gaussian Receptive Fields to boost the predictive performance of stream learning methods, spanning further research towards extrapolating our findings to other machine learning problems.

Active learning with drifting streaming data.

In learning to classify streaming data, obtaining true labels may require major effort and may incur excessive cost. Active learning focuses on carefully selecting as few labeled instances as possible for learning an accurate predictive model. Streaming data poses additional challenges for active learning, since the data distribution may change over time (concept drift) and models need to adapt. Conventional active learning strategies concentrate on querying the most uncertain instances, which are typically concentrated around the decision boundary. Changes occurring further from the boundary may be missed, and models may fail to adapt. This paper presents a theoretically supported framework for active learning from drifting data streams and develops three active learning strategies for streaming data that explicitly handle concept drift. They are based on uncertainty, dynamic allocation of labeling efforts over time, and randomization of the search space. We empirically demonstrate that these strategies react well to changes that can occur anywhere in the instance space and unexpectedly.