SHIP: a computational framework for simulating and validating novel technologies in hardware spiking neural networks.

Investigations in the field of spiking neural networks (SNNs) encompass diverse, yet overlapping, scientific disciplines. Examples range from purely neuroscientific investigations, researches on computational aspects of neuroscience, or applicative-oriented studies aiming to improve SNNs performance or to develop artificial hardware counterparts. However, the simulation of SNNs is a complex task that can not be adequately addressed with a single platform applicable to all scenarios. The optimization of a simulation environment to meet specific metrics often entails compromises in other aspects. This computational challenge has led to an apparent dichotomy of approaches, with model-driven algorithms dedicated to the detailed simulation of biological networks, and data-driven algorithms designed for efficient processing of large input datasets. Nevertheless, material scientists, device physicists, and neuromorphic engineers who develop new technologies for spiking neuromorphic hardware solutions would find benefit in a simulation environment that borrows aspects from both approaches, thus facilitating modeling, analysis, and training of prospective SNN systems. This manuscript explores the numerical challenges deriving from the simulation of spiking neural networks, and introduces SHIP, Spiking (neural network) Hardware In PyTorch, a numerical tool that supports the investigation and/or validation of materials, devices, small circuit blocks within SNN architectures. SHIP facilitates the algorithmic definition of the models for the components of a network, the monitoring of states and output of the modeled systems, and the training of the synaptic weights of the network, by way of user-defined unsupervised learning rules or supervised training techniques derived from conventional machine learning. SHIP offers a valuable tool for researchers and developers in the field of hardware-based spiking neural networks, enabling efficient simulation and validation of novel technologies.

Improving HfO2-Based Resistive Switching Devices by Inserting a TaOx Thin Film via Engineered In Situ Oxidation.

Resistive switching (RS) devices with binary and analogue operation are expected to play a key role in the hardware implementation of artificial neural networks. However, state of the art RS devices based on binary oxides (e.g., HfO2) still do not exhibit enough competitive performance. In particular, variability and yield still need to be improved to fit industrial requirements. In this study, we fabricate RS devices based on a TaOx/HfO2 bilayer stack, using a novel methodology that consists of the in situ oxidation of a Ta film inside the atomic layer deposition (ALD) chamber in which the HfO2 film is deposited. By means of X-ray reflectivity (XRR) and time-of-flight secondary ion mass spectrometry (ToF-SIMS), we realized that the TaOx film shows a substoichiometric structure, and that the TaOx/HfO2 bilayer stack holds a well-layered structure. An exhaustive electrical characterization of the TaOx/HfO2-based RS devices shows improved switching performance compared to the single-layer HfO2 counterparts. The main advantages are higher forming yield, self-compliant switching, lower switching variability, enhanced reliability, and better synaptic plasticity.

Non-linear Memristive Synaptic Dynamics for Efficient Unsupervised Learning in Spiking Neural Networks.

Spiking neural networks (SNNs) are a computational tool in which the information is coded into spikes, as in some parts of the brain, differently from conventional neural networks (NNs) that compute over real-numbers. Therefore, SNNs can implement intelligent information extraction in real-time at the edge of data acquisition and correspond to a complementary solution to conventional NNs working for cloud-computing. Both NN classes face hardware constraints due to limited computing parallelism and separation of logic and memory. Emerging memory devices, like resistive switching memories, phase change memories, or memristive devices in general are strong candidates to remove these hurdles for NN applications. The well-established training procedures of conventional NNs helped in defining the desiderata for memristive device dynamics implementing synaptic units. The generally agreed requirements are a linear evolution of memristive conductance upon stimulation with train of identical pulses and a symmetric conductance change for conductance increase and decrease. Conversely, little work has been done to understand the main properties of memristive devices supporting efficient SNN operation. The reason lies in the lack of a background theory for their training. As a consequence, requirements for NNs have been taken as a reference to develop memristive devices for SNNs. In the present work, we show that, for efficient CMOS/memristive SNNs, the requirements for synaptic memristive dynamics are very different from the needs of a conventional NN. System-level simulations of a SNN trained to classify hand-written digit images through a spike timing dependent plasticity protocol are performed considering various linear and non-linear plausible synaptic memristive dynamics. We consider memristive dynamics bounded by artificial hard conductance values and limited by the natural dynamics evolution toward asymptotic values (soft-boundaries). We quantitatively analyze the impact of resolution and non-linearity properties of the synapses on the network training and classification performance. Finally, we demonstrate that the non-linear synapses with hard boundary values enable higher classification performance and realize the best trade-off between classification accuracy and required training time. With reference to the obtained results, we discuss how memristive devices with non-linear dynamics constitute a technologically convenient solution for the development of on-line SNN training.

Stimulated Ionic Telegraph Noise in Filamentary Memristive Devices.

Random telegraph noise is a widely investigated phenomenon affecting the reliability of the reading operation of the class of memristive devices whose operation relies on formation and dissolution of conductive filaments. The trap and the release of electrons into and from defects surrounding the filament produce current fluctuations at low read voltages. In this work, telegraphic resistance variations are intentionally stimulated through pulse trains in HfO2-based memristive devices. The stimulated noise results from the re-arrangement of ionic defects constituting the filament responsible for the switching. Therefore, the stimulated noise has an ionic origin in contrast to the electronic nature of conventional telegraph noise. The stimulated noise is interpreted as raising from a dynamic equilibrium establishing from the tendencies of ionic drift and diffusion acting on the edges of conductive filament. We present a model that accounts for the observed increase of noise amplitude with the average device resistance. This work provides the demonstration and the physical foundation for the intentional stimulation of ionic telegraph noise which, on one hand, affects the programming operations performed with trains of identical pulses, as for neuromorphic computing, and on the other hand, it can open opportunities for applications relying on stochastic processes in nanoscaled devices.

Evidence of soft bound behaviour in analogue memristive devices for neuromorphic computing.

The development of devices that can modulate their conductance under the application of electrical stimuli constitutes a fundamental step towards the realization of synaptic connectivity in neural networks. Optimization of synaptic functionality requires the understanding of the analogue conductance update under different programming conditions. Moreover, properties of physical devices such as bounded conductance values and state-dependent modulation should be considered as they affect storage capacity and performance of the network. This work provides a study of the conductance dynamics produced by identical pulses as a function of the programming parameters in an HfO2 memristive device. The application of a phenomenological model that considers a soft approach to the conductance boundaries allows the identification of different operation regimes and to quantify conductance modulation in the analogue region. Device non-linear switching kinetics is recognized as the physical origin of the transition between different dynamics and motivates the crucial trade-off between degree of analog modulation and memory window. Different kinetics for the processes of conductance increase and decrease account for device programming asymmetry. The identification of programming trade-off together with an evaluation of device variations provide a guideline for the optimization of the analogue programming in view of hardware implementation of neural networks.

Role of Al doping in the filament disruption in HfO2 resistance switches.

Resistance switching devices, whose operation is driven by formation (SET) and dissolution (RESET) of conductive paths shorting and disconnecting the two metal electrodes, have recently received great attention and a deep general comprehension of their operation has been achieved. However, the link between switching characteristics and material properties is still quite weak. In particular, doping of the switching oxide layer has often been investigated only for looking at performance upgrade and rarely for a meticulous investigation of the switching mechanism. In this paper, the impact of Al doping of HfO2 devices on their switching operations, retention loss mechanisms and random telegraph noise traces is investigated. In addition, phenomenological modeling of the switching operation is performed for device employing both undoped and doped HfO2. We demonstrate that Al doping influences the filament disruption process during the RESET operation and, in particular, it contributes in preventing an efficient restoration of the oxide with respect to undoped devices.

Analog Memristive Synapse in Spiking Networks Implementing Unsupervised Learning.

Emerging brain-inspired architectures call for devices that can emulate the functionality of biological synapses in order to implement new efficient computational schemes able to solve ill-posed problems. Various devices and solutions are still under investigation and, in this respect, a challenge is opened to the researchers in the field. Indeed, the optimal candidate is a device able to reproduce the complete functionality of a synapse, i.e., the typical synaptic process underlying learning in biological systems (activity-dependent synaptic plasticity). This implies a device able to change its resistance (synaptic strength, or weight) upon proper electrical stimuli (synaptic activity) and showing several stable resistive states throughout its dynamic range (analog behavior). Moreover, it should be able to perform spike timing dependent plasticity (STDP), an associative homosynaptic plasticity learning rule based on the delay time between the two firing neurons the synapse is connected to. This rule is a fundamental learning protocol in state-of-art networks, because it allows unsupervised learning. Notwithstanding this fact, STDP-based unsupervised learning has been proposed several times mainly for binary synapses rather than multilevel synapses composed of many binary memristors. This paper proposes an HfO2-based analog memristor as a synaptic element which performs STDP within a small spiking neuromorphic network operating unsupervised learning for character recognition. The trained network is able to recognize five characters even in case incomplete or noisy images are displayed and it is robust to a device-to-device variability of up to ±30%.

Resistive switching in high-density nanodevices fabricated by block copolymer self-assembly.

Bipolar resistive switching memories based on metal oxides offer a great potential in terms of simple process integration, memory performance, and scalability. In view of ultrahigh density memory applications, a reduced device size is not the only requirement, as the distance between different devices is a key parameter. By exploiting a bottom-up fabrication approach based on block copolymer self-assembling, we obtained the parallel production of bilayer Pt/Ti top electrodes arranged in periodic arrays over the HfO2/TiN surface, building memory devices with a diameter of 28 nm and a density of 5 × 10(10) devices/cm(2). For an electrical characterization, the sharp conducting tip of an atomic force microscope was adopted for a selective addressing of the nanodevices. The presence of devices showing high conductance in the initial state was directly connected with scattered leakage current paths in the bare oxide film, while with bipolar voltage operations we obtained reversible set/reset transitions irrespective of the conductance variability in the initial state. Finally, we disclosed a scalability limit for ultrahigh density memory arrays based on continuous HfO2 thin films, in which a cross-talk between distinct nanodevices can occur during both set and reset transitions.

Engineered fabrication of ordered arrays of Au-NiO-Au nanowires.

In the present paper, a novel method to fabricate ordered arrays of Au/NiO/Au nanowires is described, with the aim of filling the gap between the fundamental study of the electrical properties of scattered single nanowires and the engineered fabrication of nanowire arrays. This approach mainly consists of the following steps: (a) electrodeposition of Au/Ni/Au nanowires into an ordered porous anodic aluminum oxide template; (b) mechanical polishing of the sample to expose the gold tips of Au/Ni/Au nanowires to the template surface; (c) in situ annealing of the Au/Ni/Au nanowires without removing the template. The resulting structure consists in an ordered array of Au/NiO/Au nanowires slightly protruding out of a flat aluminum oxide template. Unlike current approaches, with the described method it is not necessary to remove the template in order to oxidize the middle metal, thus allowing the availability of an entire set of metal/oxide/metal nanowires ordered in a two-dimensional matrix and where single heterojunctions can be accessed individually.