Neuromorphic Systems Design by Matching Inductive Biases to Hardware Constraints.

Neuromorphic systems are designed with careful consideration of the physical properties of the computational substrate they use. Neuromorphic engineers often exploit physical phenomena to directly implement a desired functionality, enabled by "the isomorphism between physical processes in different media" (Douglas et al., 1995). This bottom-up design methodology could be described as matching computational primitives to physical phenomena. In this paper, we propose a top-down counterpart to the bottom-up approach to neuromorphic design. Our top-down approach, termed "bias matching," is to match the inductive biases required in a learning system to the hardware constraints of its implementation; a well-known example is enforcing translation equivariance in a neural network by tying weights (replacing vector-matrix multiplications with convolutions), which reduces memory requirements. We give numerous examples from the literature and explain how they can be understood from this perspective. Furthermore, we propose novel network designs based on this approach in the context of collaborative filtering. Our simulation results underline our central conclusions: additional hardware constraints can improve the predictions of a Machine Learning system, and understanding the inductive biases that underlie these performance gains can be useful in finding applications for a given constraint.

Neural Sensors: Learning Pixel Exposures for HDR Imaging and Video Compressive Sensing With Programmable Sensors.

Camera sensors rely on global or rolling shutter functions to expose an image. This fixed function approach severely limits the sensors' ability to capture high-dynamic-range (HDR) scenes and resolve high-speed dynamics. Spatially varying pixel exposures have been introduced as a powerful computational photography approach to optically encode irradiance on a sensor and computationally recover additional information of a scene, but existing approaches rely on heuristic coding schemes and bulky spatial light modulators to optically implement these exposure functions. Here, we introduce neural sensors as a methodology to optimize per-pixel shutter functions jointly with a differentiable image processing method, such as a neural network, in an end-to-end fashion. Moreover, we demonstrate how to leverage emerging programmable and re-configurable sensor-processors to implement the optimized exposure functions directly on the sensor. Our system takes specific limitations of the sensor into account to optimize physically feasible optical codes and we evaluate its performance for snapshot HDR and high-speed compressive imaging both in simulation and experimentally with real scenes.

Visualizing the Synaptic and Cellular Ultrastructure in Neurons Differentiated from Human Induced Neural Stem Cells-An Optimized Protocol.

The size of the synaptic subcomponents falls below the limits of visible light microscopy. Despite new developments in advanced microscopy techniques, the resolution of transmission electron microscopy (TEM) remains unsurpassed. The requirements of tissue preservation are very high, and human post mortem material often does not offer adequate quality. However, new reprogramming techniques that generate human neurons in vitro provide samples that can easily fulfill these requirements. The objective of this study was to identify the culture technique with the best ultrastructural preservation in combination with the best embedding and contrasting technique for visualizing neuronal elements. Two induced neural stem cell lines derived from healthy control subjects underwent differentiation either adherent on glass coverslips, embedded in a droplet of highly concentrated Matrigel, or as a compact neurosphere. Afterward, they were fixed using a combination of glutaraldehyde (GA) and paraformaldehyde (PFA) followed by three approaches (standard stain, Ruthenium red stain, high contrast en-bloc stain) using different combinations of membrane enhancing and contrasting steps before ultrathin sectioning and imaging by TEM. The compact free-floating neurospheres exhibited the best ultrastructural preservation. High-contrast en-bloc stain offered particularly sharp staining of membrane structures and the highest quality visualization of neuronal structures. In conclusion, compact neurospheres growing under free-floating conditions in combination with a high contrast en-bloc staining protocol, offer the optimal preservation and contrast with a particular focus on visualizing membrane structures as required for analyzing synaptic structures.

A neuromorphic systems approach to in-memory computing with non-ideal memristive devices: from mitigation to exploitation.

Memristive devices represent a promising technology for building neuromorphic electronic systems. In addition to their compactness and non-volatility, they are characterized by their computationally relevant physical properties, such as their state-dependence, non-linear conductance changes, and intrinsic variability in both their switching threshold and conductance values, that make them ideal devices for emulating the bio-physics of real synapses. In this paper we present a spiking neural network architecture that supports the use of memristive devices as synaptic elements and propose mixed-signal analog-digital interfacing circuits that mitigate the effect of variability in their conductance values and exploit their variability in the switching threshold for implementing stochastic learning. The effect of device variability is mitigated using pairs of memristive devices configured in a complementary push-pull mechanism and interfaced to a current-mode normalizer circuit. The stochastic learning mechanism is obtained by mapping the desired change in synaptic weight into a corresponding switching probability that is derived from the intrinsic stochastic behavior of memristive devices. We demonstrate the features of the CMOS circuits and apply the architecture proposed to a standard neural network hand-written digit classification benchmark based on the MNIST data-set. We evaluate the performance of the approach proposed in this benchmark using behavioral-level spiking neural network simulation, showing both the effect of the reduction in conductance variability produced by the current-mode normalizer circuit and the increase in performance as a function of the number of memristive devices used in each synapse.

Anti-GAD65 Containing Cerebrospinal Fluid Does not Alter GABAergic Transmission.

Glutamic acid decarboxylase of 65 kDa (GAD65) antibodies have been reported in a variety of neurological disorders such as stiff-person syndrome (SPS), sporadic ataxia and some cases of epilepsy. Since the target is believed to be the cytoplasmic enzyme GAD65, the key enzyme of γ-aminobutyric acid (GABA) synthesis, the pathophysiological role of these antibodies is poorly understood. Here, we stereotactically injected human cerebrospinal fluid (CSF) containing GAD65-antibodies into the hippocampus of rats in vivo and then prepared hippocampal slices 1-2 days after post-operative recovery. We characterized both evoked and spontaneous GABAergic transmission in vitro using sharp microelectrode and patch-clamp recordings in CA1 neurons. Intracellular recordings with sharp microelectrodes from CA1 neurons showed that evoked GABAAR- or GABABR-mediated inhibitory postsynaptic potentials (IPSP) remained unaltered in anti-GAD65 tissue. These results were confirmed with patch-clamp recordings showing no difference in evoked gabazine-sensitive inhibitory postsynaptic currents (IPSCs). In addition, spontaneous IPSCs also showed no difference between anti-GAD65 tissue and controls with respect to the mean frequency, the mean amplitude and the sIPSC distribution. In conclusion, stereotactic injection of GAD65-antibodies into the hippocampus leaves evoked and spontaneous GABAergic synaptic transmission intact. Hence, dysfunction of the inhibitory GABAergic system does not appear to be the major mechanism of epileptogenicity in this disease.

Directional deep brain stimulation of the subthalamic nucleus: A pilot study using a novel neurostimulation device.

INTRODUCTION: A novel neurostimulation system allows steering current in horizontal directions by combining segmented leads and multiple independent current control. The aim of this study was to evaluate directional DBS effects on parkinsonian motor features and adverse effects of subthalamic neurostimulation. METHODS: Seven PD patients implanted with the novel directional DBS system for bilateral subthalamic DBS underwent an extended monopolar review session during the first postoperative week, in which current thresholds were determined for rigidity control and stimulation-induced adverse effects using either directional or ring-mode settings. RESULTS: Effect or adverse effect thresholds were modified by directional settings for each of the 14 STN leads. Magnitude of change varied markedly between leads, as did orientation of optimal horizontal current steering. CONCLUSION: Directional current steering through chronically implanted segmented electrodes is feasible, alters adverse effect and efficacy thresholds in a highly individual manner, and expands the therapeutic window in a monopolar review as compared to ring-mode DBS. © 2016 The Authors. Movement Disorders published by Wiley Periodicals, Inc. on behalf of International Parkinson and Movement Disorder Society.

An event-based architecture for solving constraint satisfaction problems.

Constraint satisfaction problems are ubiquitous in many domains. They are typically solved using conventional digital computing architectures that do not reflect the distributed nature of many of these problems, and are thus ill-suited for solving them. Here we present a parallel analogue/digital hardware architecture specifically designed to solve such problems. We cast constraint satisfaction problems as networks of stereotyped nodes that communicate using digital pulses, or events. Each node contains an oscillator implemented using analogue circuits. The non-repeating phase relations among the oscillators drive the exploration of the solution space. We show that this hardware architecture can yield state-of-the-art performance on random SAT problems under reasonable assumptions on the implementation. We present measurements from a prototype electronic chip to demonstrate that a physical implementation of the proposed architecture is robust to practical non-idealities and to validate the theory proposed.

Rhythmic Inhibition Allows Neural Networks to Search for Maximally Consistent States.

Gamma-band rhythmic inhibition is a ubiquitous phenomenon in neural circuits, yet its computational role remains elusive. We show that a model of gamma-band rhythmic inhibition allows networks of coupled cortical circuit motifs to search for network configurations that best reconcile external inputs with an internal consistency model encoded in the network connectivity. We show that Hebbian plasticity allows the networks to learn the consistency model by example. The search dynamics driven by rhythmic inhibition enable the described networks to solve difficult constraint satisfaction problems without making assumptions about the form of stochastic fluctuations in the network. We show that the search dynamics are well approximated by a stochastic sampling process. We use the described networks to reproduce perceptual multistability phenomena with switching times that are a good match to experimental data and show that they provide a general neural framework that can be used to model other perceptual inference phenomena.

Effects of oxygen insufflation during pilocarpine-induced status epilepticus on mortality, tissue damage and seizures.

PURPOSE: This prospective, randomized study was performed to investigate the effects of oxygen (O2) treatment during sustained epileptic activity on mortality, subsequent seizure frequency, and neuronal damage. METHODS: Status epilepticus (SE) was induced by intraperitoneal injection of 340mg/kg pilocarpine, and terminated by diazepam after 40min. During SE, rats were randomized to O2 treatment (insufflation rate of 1.5l/min O2) during SE or normal air conditions. Outcome measures were SE-related mortality, seizure occurrence, mossy fiber sprouting, neuronal cell loss and expression of 27-kDa heat-shock protein (Hsp27). RESULTS: O2-treated and O2-untreated animals did not differ with respect to SE latency, diazepam dose required to stop SE. While 7/38 rats died during SE in the O2-untreated group, very little mortality (1/38) occurred in the O2-treated group (P<0.05). However, within 1h after SE termination, seven O2-treated rats died which was not observed in the O2-untreated group indicating no significant difference in overall mortality. There was a tendency towards lower seizure rate in the O2-treated group at one month after pilocarpine-induced SE. Three months after SE, however, seizure rates were no longer different between both groups. Moreover, mossy fiber sprouting, neuronal cell loss and Hsp27 expression did not differ between O2-treated and O2-untreated groups. CONCLUSION: Our findings indicate that O2 treatment might delay the relative risk of epileptic seizures following an initial brain injury, but it may also lead to a rather unfavorably increased heterogeneity of epileptogenesis in experimental studies.

Impaired D-serine-mediated cotransmission mediates cognitive dysfunction in epilepsy.

The modulation of synaptic plasticity by NMDA receptor (NMDAR)-mediated processes is essential for many forms of learning and memory. Activation of NMDARs by glutamate requires the binding of a coagonist to a regulatory site of the receptor. In many forebrain regions, this coagonist is d-serine. Here, we show that experimental epilepsy in rats is associated with a reduction in the CNS levels of d-serine, which leads to a desaturation of the coagonist binding site of synaptic and extrasynaptic NMDARs. In addition, the subunit composition of synaptic NMDARs changes in chronic epilepsy. The desaturation of NMDARs causes a deficit in hippocampal long-term potentiation, which can be rescued with exogenously supplied d-serine. Importantly, exogenous d-serine improves spatial learning in epileptic animals. These results strongly suggest that d-serine deficiency is important in the amnestic symptoms of temporal lobe epilepsy. Our results point to a possible clinical utility of d-serine to alleviate these disease manifestations.

Enhanced NMDA receptor-dependent LTP in the epileptic CA1 area via upregulation of NR2B.

Impairment of synaptic plasticity such as long-term potentiation (LTP) is a common finding in various animal models of a number of neurodegenerative disorders. While cognitive deficits associated with these models are plausibly attributed to impaired plasticity, it is an intriguing question whether learning impairment correlates in general with compromised synaptic plasticity. In the present study, we have addressed this issue and discovered an enhancement of theta-burst stimulation-induced LTP at Schaffer collateral-CA1 synapses from chronically epileptic animals. The LTP enhancement was abolished by the NMDA receptor 2B (NR2B) blocker Ro 25-6981 (1µM) while it was preserved following application of the NR2A blocker NVP-AAM077 (50nM). Moreover, pharmacological characterization of intracellularly recorded excitatory postsynaptic potentials (EPSP) from CA1 pyramidal neurons indicated an increased NR2B/NR2A ratio in epileptic tissue, and NMDA receptor mediated excitatory postsynaptic currents showed significantly longer decay times. Quantitative reverse-transcriptase PCR confirmed the transcriptional up-regulation of NR2B-mRNA in chronically epileptic animals. To test the significance for epileptiform activity, recurrent epileptiform discharges (REDs) in the CA1 area induced by bath application of either high K(+) (8mM) plus gabazine (5µM) or 4-aminopyridine (50µM), were also characterized pharmacologically. While in control slices the presence of Ro 25-6981 had no effect on the RED frequency, NR2B inhibition significantly increased epileptic activity in tissue from epileptic animals. Our results demonstrate that CA1 synapses in chronically epileptic tissue can undergo an LTP enhancement due to an NR2B up-regulation in CA1 pyramidal neurons. On the network level, this up-regulation appears to be a compensatory process, since blockade of these receptors leaves the tissue more susceptible to hyperexcitability.

Upregulation of presynaptic mGluR2, but not mGluR3 in the epileptic medial perforant path.

Presynaptic metabotropic glutamate receptors (mGluRs) at glutamatergic synapses play a major role in governing release probability. Previous reports indicated a downregulation of group III mGluRs at the lateral perforant path-granule cell synapse in the chronically epileptic hippocampus. Here, we investigated the mGluR-dependent presynaptic inhibition at the medial perforant path-granule cell synapse in the pilocarpine-treated chronically epileptic rat. The specific group II mGluR agonist (2S,2'R,3'R)-2-(2',3'-dicarboxycyclopropyl)glycine (DCG-IV, 10µM) significantly depressed medial perforant path-evoked responses in control slices, but significantly more so in epileptic tissue. This depression was accompanied by a significant increase of the paired-pulse ratio in both animal groups indicating a presynaptic mechanism. Moreover, we also found that this significantly enhanced DCG-IV effect in the medial perforant path recorded in slices from pilocarpine-treated rats was due to a significant increase of mGluR2, but not mGluR3 transcripts in the entorhinal cortex using quantitative real-time reverse transcriptase-PCR. Immunohistochemistry confirmed the increased expression of group II mGluRs in the epileptic medial molecular layer. These results demonstrate that chronic epilepsy not only causes downregulation of mGluRs in the hippocampus, but may also lead to enhanced expression of these receptors - at least in the medial perforant path.

Sensitization to wasp venom does not induce autoantibodies leading to infertility.

Antigenic cross-reactivity has been described between the venom allergen (antigen 5) and mammalian testis proteins. Based on an allergen database we have previously shown that allergens can be represented by allergen motifs. A motif group was found containing venom antigen 5 sequences from different vespids. Using an optimized amino acid profile based on antigen 5 sequences for searching cross-reactive proteins, three human semen proteins belonging to the family of cysteine-rich secretory proteins (hCRISP) were found in the Swiss Protein database. To analyze antigenic cross-reactivity between antigen 5 and hCRISPs, antigen 5 from yellow jacket venom (Ves v 5) and two hCRISPs (CRISP-2 and -3) were chosen and produced as recombinant proteins in E. coli. A correlation was found between antibodies reacting with rVes v 5 and rhCRISP-2, -3 in a small human sera population indicating the presence of cross-reactive antibodies in human serum. Using intravenous immunoglobulin (IVIg), a therapeutic multidonor IgG preparation, cross-reactive antibodies were isolated that recognize rVes v 5, hCRISP-2 and -3 suggesting the presence of common epitopes between Ves v 5 and hCRISPs. However this cross-reactivity seems not to be linked to allergy to wasp venom as we could show no correlation between increasing CAP-class IgE level to wasp venom and IgG to sperm extract and hCRISPs. These data suggest that higher sensitization to wasp venom does not induce more antibodies against autoantigens and might not represent a higher risk to develop autoantibodies leading to infertility.

Loss of metabotropic glutamate receptor-dependent long-term depression via downregulation of mGluR5 after status epilepticus.

Synaptic plasticity is thought to be a key mechanism of information storage in the CNS. Different forms of synaptic long-term potentiation have been shown to be impaired in neurological disorders. Here, we show that metabotropic glutamate receptor (mGluR)-dependent long-term depression (LTD), but not NMDA receptor-dependent LTD at Schaffer collateral-CA1 synapses, is profoundly impaired after status epilepticus. Brief application of the group I mGluR agonist (R,S)-3,5-dihydroxyphenylglycine (100 microM; 5 min) induced mGluR LTD in control, but not in pilocarpine-treated rats. Experiments in the presence of selective inhibitors of either mGluR5 [2-methyl-6-(phenylethynyl)-pyridine] or mGluR1 [7-(hydroxyimino)cyclopropachromen-carboxylate ethyl ester and (S)-(+)-alpha-amino-4-carboxy-2-methylbenzeneacetic acid] demonstrate that loss of mGluR LTD is most likely attributable to a loss of mGluR5 function. Quantitative real-time reverse transcription PCR revealed a specific downregulation of mGluR5 mRNA, but not of mGluR1 mRNA in the CA1 region. Furthermore, we detected a strong reduction in mGluR5 protein expression by immunofluorescence and quantitative immunoblotting. Additionally, the scaffolding protein Homer that mediates coupling of mGluR5 to downstream signaling cascades was downregulated. Thus, we conclude that the reduction of mGluR LTD after pilocarpine-induced status epilepticus is the result of the subtype-specific downregulation of mGluR5 and associated downstream signaling components.

Intranasal immunisation using recombinant Lactobacillus johnsonii as a new strategy to prevent allergic disease.

We have previously demonstrated the induction of a specific anti-IgE response in vivo by parenteral immunisation of rhesus monkeys using short IgE mimotopes or an anti-idiotypic antibody mimicking an IgE epitope. Such specific anti-IgE responses may be of clinical benefit for atopic patients. In this study, we examined the potential for a more convenient therapy via mucosal immunisation using live recombinant Lactobacillus johnsonii (Lj) as a vaccine delivery vehicle. Either an anti-idiotypic scFv or an IgE mimotope were expressed on the surface of Lj as fusion proteins using the cell wall anchored proteinase PrtB from Lactobacillus delbrueckii subsp. bulgaricus. The recombinant Lj were shown to express the heterologous fusion proteins and were specifically recognised by the corresponding anti-human IgE monoclonal antibody. Subcutaneous and intranasal immunisation of mice with recombinant Lj, expressing these fusion proteins induced a systemic IgG response against human IgE. Our data suggest that recombinant Lactobacilli expressing IgE epitopes may represent a novel means of vaccination to induce a beneficial anti-IgE response.