Combining STDP and binary networks for reinforcement learning from images and sparse rewards.

Spiking neural networks (SNNs) aim to replicate energy efficiency, learning speed and temporal processing of biological brains. However, accuracy and learning speed of such networks is still behind reinforcement learning (RL) models based on traditional neural models. This work combines a pre-trained binary convolutional neural network with an SNN trained online through reward-modulated STDP in order to leverage advantages of both models. The spiking network is an extension of its previous version, with improvements in architecture and dynamics to address a more challenging task. We focus on extensive experimental evaluation of the proposed model with optimized state-of-the-art baselines, namely proximal policy optimization (PPO) and deep Q network (DQN). The models are compared on a grid-world environment with high dimensional observations, consisting of RGB images with up to 256 × 256 pixels. The experimental results show that the proposed architecture can be a competitive alternative to deep reinforcement learning (DRL) in the evaluated environment and provide a foundation for more complex future applications of spiking networks.

Mechanism of KCl enhancement in detection of nonionic polymers by nanopore sensors.

The mechanisms of KCl-induced enhancement in identification of individual molecules of poly(ethylene glycol) using solitary alpha-hemolysin nanoscale pores are described. The interaction of single molecules with the nanopore causes changes in the ionic current flowing through the pore. We show that the on-rate constant of the process is several hundred times larger and that the off-rate is several hundred times smaller in 4 M KCl than in 1 M KCl. These shifts dramatically improve detection and make single molecule identification feasible. KCl also changes the solubility of poly(ethylene glycol) by the same order of magnitude as it changes the rate constants. In addition, the polymer-nanopore interaction is determined to be a strong non-monotonic function of voltage, indicating that the flexible, nonionic poly(ethylene glycol) acts as a charged molecule. Therefore, salting-out and Coulombic interactions are responsible for the KCl-induced enhancement. These results will advance the development of devices with sensor elements based on single nanopores.