Memristor-based spiking neural network with online reinforcement learning.

Neural networks implemented in memristor-based hardware can provide fast and efficient in-memory computation, but traditional learning methods such as error back-propagation are hardly feasible in it. Spiking neural networks (SNNs) are highly promising in this regard, as their weights can be changed locally in a self-organized manner without the demand for high-precision changes calculated with the use of information almost from the entire network. This problem is rather relevant for solving control tasks with neural-network reinforcement learning methods, as those are highly sensitive to any source of stochasticity in a model initialization, training, or decision-making procedure. This paper presents an online reinforcement learning algorithm in which the change of connection weights is carried out after processing each environment state during interaction-with-environment data generation. Another novel feature of the algorithm is that it is applied to SNNs with memristor-based STDP-like learning rules. The plasticity functions are obtained from real memristors based on poly-p-xylylene and CoFeB-LiNbO3 nanocomposite, which were experimentally assembled and analyzed. The SNN is comprised of leaky integrate-and-fire neurons. Environmental states are encoded by the timings of input spikes, and the control action is decoded by the first spike. The proposed learning algorithm solves the Cart-Pole benchmark task successfully. This result could be the first step towards implementing a real-time agent learning procedure in a continuous-time environment that can be run on neuromorphic systems with memristive synapses.

The Endocannabinoid System and Oligodendrocytes in Health and Disease.

Cannabinoid-based interventions are being explored for central nervous system (CNS) pathologies such as neurodegeneration, demyelination, epilepsy, stroke, and trauma. As these disease states involve dysregulation of myelin integrity and/or remyelination, it is important to consider effects of the endocannabinoid system on oligodendrocytes and their precursors. In this review, we examine research reports on the effects of the endocannabinoid system (ECS) components on oligodendrocytes and their precursors, with a focus on therapeutic implications. Cannabinoid ligands and modulators of the endocannabinoid system promote cell signaling in oligodendrocyte precursor survival, proliferation, migration and differentiation, and mature oligodendrocyte survival and myelination. Agonist stimulation of oligodendrocyte precursor cells (OPCs) at both CB1 and CB2 receptors counter apoptotic processes via Akt/PI3K, and promote proliferation via Akt/mTOR and ERK pathways. CB1 receptors in radial glia promote proliferation and conversion to progenitors fated to become oligodendroglia, whereas CB2 receptors promote OPC migration in neonatal development. OPCs produce 2-arachidonoylglycerol (2-AG), stimulating cannabinoid receptor-mediated ERK pathways responsible for differentiation to arborized, myelin basic protein (MBP)-producing oligodendrocytes. In cell culture models of excitotoxicity, increased reactive oxygen species, and depolarization-dependent calcium influx, CB1 agonists improved viability of oligodendrocytes. In transient and permanent middle cerebral artery occlusion models of anoxic stroke, WIN55212-2 increased OPC proliferation and maturation to oligodendroglia, thereby reducing cerebral tissue damage. In several models of rodent encephalomyelitis, chronic treatment with cannabinoid agonists ameliorated the damage by promoting OPC survival and oligodendrocyte function. Pharmacotherapeutic strategies based upon ECS and oligodendrocyte production and survival should be considered.

Cannabinoid Receptor Interacting Protein 1a Competition with ß-Arrestin for CB1 Receptor Binding Sites.

Cannabinoid receptor interacting protein 1a (CRIP1a) is a CB1 receptor (CB1R) distal C-terminal-associated protein that alters CB1R interactions with G-proteins. We tested the hypothesis that CRIP1a is capable of also altering CB1R interactions with ß-arrestin proteins that interact with the CB1R at the C-terminus. Coimmunoprecipitation studies indicated that CB1R associates in complexes with either CRIP1a or ß-arrestin, but CRIP1a and ß-arrestin fail to coimmunoprecipitate with each other. This suggests a competition for CRIP1a and ß-arrestin binding to the CB1R, which we hypothesized could attenuate the action of ß-arrestin to mediate CB1R internalization. We determined that agonist-mediated reduction of the density of cell surface endogenously expressed CB1Rs was clathrin and dynamin dependent and could be modeled as agonist-induced aggregation of transiently expressed GFP-CB1R. CRIP1a overexpression attenuated CP55940-mediated GFP-CB1R as well as endogenous ß-arrestin redistribution to punctae, and conversely, CRIP1a knockdown augmented ß-arrestin redistribution to punctae. Peptides mimicking the CB1R C-terminus could bind to both CRIP1a in cell extracts as well as purified recombinant CRIP1a. Affinity pull-down studies revealed that phosphorylation at threonine-468 of a CB1R distal C-terminus 14-mer peptide reduced CB1R-CRIP1a association. Coimmunoprecipitation of CB1R protein complexes demonstrated that central or distal C-terminal peptides competed for the CB1R association with CRIP1a, but that a phosphorylated central C-terminal peptide competed for association with ß-arrestin 1, and phosphorylated central or distal C-terminal peptides competed for association with ß-arrestin 2. Thus, CRIP1a can compete with ß-arrestins for interaction with C-terminal CB1R domains that could affect agonist-driven, ß-arrestin-mediated internalization of the CB1R.