Hardware implementation of memristor-based artificial neural networks.

Artificial Intelligence (AI) is currently experiencing a bloom driven by deep learning (DL) techniques, which rely on networks of connected simple computing units operating in parallel. The low communication bandwidth between memory and processing units in conventional von Neumann machines does not support the requirements of emerging applications that rely extensively on large sets of data. More recent computing paradigms, such as high parallelization and near-memory computing, help alleviate the data communication bottleneck to some extent, but paradigm- shifting concepts are required. Memristors, a novel beyond-complementary metal-oxide-semiconductor (CMOS) technology, are a promising choice for memory devices due to their unique intrinsic device-level properties, enabling both storing and computing with a small, massively-parallel footprint at low power. Theoretically, this directly translates to a major boost in energy efficiency and computational throughput, but various practical challenges remain. In this work we review the latest efforts for achieving hardware-based memristive artificial neural networks (ANNs), describing with detail the working principia of each block and the different design alternatives with their own advantages and disadvantages, as well as the tools required for accurate estimation of performance metrics. Ultimately, we aim to provide a comprehensive protocol of the materials and methods involved in memristive neural networks to those aiming to start working in this field and the experts looking for a holistic approach.

Thin-film design of amorphous hafnium oxide nanocomposites enabling strong interfacial resistive switching uniformity.

A design concept of phase-separated amorphous nanocomposite thin films is presented that realizes interfacial resistive switching (RS) in hafnium oxide-based devices. The films are formed by incorporating an average of 7% Ba into hafnium oxide during pulsed laser deposition at temperatures ≤400°C. The added Ba prevents the films from crystallizing and leads to ∼20-nm-thin films consisting of an amorphous HfOx host matrix interspersed with ∼2-nm-wide, ∼5-to-10-nm-pitch Ba-rich amorphous nanocolumns penetrating approximately two-thirds through the films. This restricts the RS to an interfacial Schottky-like energy barrier whose magnitude is tuned by ionic migration under an applied electric field. Resulting devices achieve stable cycle-to-cycle, device-to-device, and sample-to-sample reproducibility with a measured switching endurance of ≥104 cycles for a memory window ≥10 at switching voltages of ±2 V. Each device can be set to multiple intermediate resistance states, which enables synaptic spike-timing-dependent plasticity. The presented concept unlocks additional design variables for RS devices.

Nonideality-Aware Training for Accurate and Robust Low-Power Memristive Neural Networks.

Recent years have seen a rapid rise of artificial neural networks being employed in a number of cognitive tasks. The ever-increasing computing requirements of these structures have contributed to a desire for novel technologies and paradigms, including memristor-based hardware accelerators. Solutions based on memristive crossbars and analog data processing promise to improve the overall energy efficiency. However, memristor nonidealities can lead to the degradation of neural network accuracy, while the attempts to mitigate these negative effects often introduce design trade-offs, such as those between power and reliability. In this work, authors design nonideality-aware training of memristor-based neural networks capable of dealing with the most common device nonidealities. The feasibility of using high-resistance devices that exhibit high I-V nonlinearity is demonstrated-by analyzing experimental data and employing nonideality-aware training, it is estimated that the energy efficiency of memristive vector-matrix multipliers is improved by almost three orders of magnitude (0.715 TOPs-1 W-1 to 381 TOPs-1 W-1 ) while maintaining similar accuracy. It is shown that associating the parameters of neural networks with individual memristors allows to bias these devices toward less conductive states through regularization of the corresponding optimization problem, while modifying the validation procedure leads to more reliable estimates of performance. The authors demonstrate the universality and robustness of this approach when dealing with a wide range of nonidealities.

Multimodal transistors as ReLU activation functions in physical neural network classifiers.

Artificial neural networks (ANNs) providing sophisticated, power-efficient classification are finding their way into thin-film electronics. Thin-film technologies require robust, layout-efficient devices with facile manufacturability. Here, we show how the multimodal transistor's (MMT's) transfer characteristic, with linear dependence in saturation, replicates the rectified linear unit (ReLU) activation function of convolutional ANNs (CNNs). Using MATLAB, we evaluate CNN performance using systematically distorted ReLU functions, then substitute measured and simulated MMT transfer characteristics as proxies for ReLU. High classification accuracy is maintained, despite large variations in geometrical and electrical parameters, as CNNs use the same activation functions for training and classification.

Standards for the Characterization of Endurance in Resistive Switching Devices.

Resistive switching (RS) devices are emerging electronic components that could have applications in multiple types of integrated circuits, including electronic memories, true random number generators, radiofrequency switches, neuromorphic vision sensors, and artificial neural networks. The main factor hindering the massive employment of RS devices in commercial circuits is related to variability and reliability issues, which are usually evaluated through switching endurance tests. However, we note that most studies that claimed high endurances >106 cycles were based on resistance versus cycle plots that contain very few data points (in many cases even <20), and which are collected in only one device. We recommend not to use such a characterization method because it is highly inaccurate and unreliable (i.e., it cannot reliably demonstrate that the device effectively switches in every cycle and it ignores cycle-to-cycle and device-to-device variability). This has created a blurry vision of the real performance of RS devices and in many cases has exaggerated their potential. This article proposes and describes a method for the correct characterization of switching endurance in RS devices; this method aims to construct endurance plots showing one data point per cycle and resistive state and combine data from multiple devices. Adopting this recommended method should result in more reliable literature in the field of RS technologies, which should accelerate their integration in commercial products.

Simulation of Inference Accuracy Using Realistic RRAM Devices.

Resistive Random Access Memory (RRAM) is a promising technology for power efficient hardware in applications of artificial intelligence (AI) and machine learning (ML) implemented in non-von Neumann architectures. However, there is an unanswered question if the device non-idealities preclude the use of RRAM devices in this potentially disruptive technology. Here we investigate the question for the case of inference. Using experimental results from silicon oxide (SiO x ) RRAM devices, that we use as proxies for physical weights, we demonstrate that acceptable accuracies in classification of handwritten digits (MNIST data set) can be achieved using non-ideal devices. We find that, for this test, the ratio of the high- and low-resistance device states is a crucial determinant of classification accuracy, with ~96.8% accuracy achievable for ratios >3, compared to ~97.3% accuracy achieved with ideal weights. Further, we investigate the effects of a finite number of discrete resistance states, sub-100% device yield, devices stuck at one of the resistance states, current/voltage non-linearities, programming non-linearities and device-to-device variability. Detailed analysis of the effects of the non-idealities will better inform the need for the optimization of particular device properties.

Memristor-Based Edge Detection for Spike Encoded Pixels.

Memristors have many uses in machine learning and neuromorphic hardware. From memory elements in dot product engines to replicating both synapse and neuron wall behaviors, the memristor has proved a versatile component. Here we demonstrate an analog mode of operation observed in our silicon oxide memristors and apply this to the problem of edge detection. We demonstrate how a potential divider exploiting this analog behavior can prove a scalable solution to edge detection. We confirm its behavior experimentally and simulate its performance on a standard testbench. We show good performance comparable to existing memristor based work with a benchmark score of 0.465 on the BSDS500 dataset, while simultaneously maintaining a lower component count.

The interplay between structure and function in redox-based resistance switching.

We report a study of the relationship between oxide microstructure at the scale of tens of nanometres and resistance switching behaviour in silicon oxide. In the case of sputtered amorphous oxides, the presence of columnar structure enables efficient resistance switching by providing an initial structured distribution of defects that can act as precursors for the formation of chains of conductive oxygen vacancies under the application of appropriate electrical bias. Increasing electrode interface roughness decreases electroforming voltages and reduces the distribution of switching voltages. Any contribution to these effects from field enhancement at rough interfaces is secondary to changes in oxide microstructure templated by interface structure.

Silicon Oxide (SiOx ): A Promising Material for Resistance Switching?

Interest in resistance switching is currently growing apace. The promise of novel high-density, low-power, high-speed nonvolatile memory devices is appealing enough, but beyond that there are exciting future possibilities for applications in hardware acceleration for machine learning and artificial intelligence, and for neuromorphic computing. A very wide range of material systems exhibit resistance switching, a number of which-primarily transition metal oxides-are currently being investigated as complementary metal-oxide-semiconductor (CMOS)-compatible technologies. Here, the case is made for silicon oxide, perhaps the most CMOS-compatible dielectric, yet one that has had comparatively little attention as a resistance-switching material. Herein, a taxonomy of switching mechanisms in silicon oxide is presented, and the current state of the art in modeling, understanding fundamental switching mechanisms, and exciting device applications is summarized. In conclusion, silicon oxide is an excellent choice for resistance-switching technologies, offering a number of compelling advantages over competing material systems.

Spike-Timing Dependent Plasticity in Unipolar Silicon Oxide RRAM Devices.

Resistance switching, or Resistive RAM (RRAM) devices show considerable potential for application in hardware spiking neural networks (neuro-inspired computing) by mimicking some of the behavior of biological synapses, and hence enabling non-von Neumann computer architectures. Spike-timing dependent plasticity (STDP) is one such behavior, and one example of several classes of plasticity that are being examined with the aim of finding suitable algorithms for application in many computing tasks such as coincidence detection, classification and image recognition. In previous work we have demonstrated that the neuromorphic capabilities of silicon-rich silicon oxide (SiOx) resistance switching devices extend beyond plasticity to include thresholding, spiking, and integration. We previously demonstrated such behaviors in devices operated in the unipolar mode, opening up the question of whether we could add plasticity to the list of features exhibited by our devices. Here we demonstrate clear STDP in unipolar devices. Significantly, we show that the response of our devices is broadly similar to that of biological synapses. This work further reinforces the potential of simple two-terminal RRAM devices to mimic neuronal functionality in hardware spiking neural networks.

Investigation of resistance switching in SiO x RRAM cells using a 3D multi-scale kinetic Monte Carlo simulator.

We employ an advanced three-dimensional (3D) electro-thermal simulator to explore the physics and potential of oxide-based resistive random-access memory (RRAM) cells. The physical simulation model has been developed recently, and couples a kinetic Monte Carlo study of electron and ionic transport to the self-heating phenomenon while accounting carefully for the physics of vacancy generation and recombination, and trapping mechanisms. The simulation framework successfully captures resistance switching, including the electroforming, set and reset processes, by modeling the dynamics of conductive filaments in the 3D space. This work focuses on the promising yet less studied RRAM structures based on silicon-rich silica (SiO x ) RRAMs. We explain the intrinsic nature of resistance switching of the SiO x layer, analyze the effect of self-heating on device performance, highlight the role of the initial vacancy distributions acting as precursors for switching, and also stress the importance of using 3D physics-based models to capture accurately the switching processes. The simulation work is backed by experimental studies. The simulator is useful for improving our understanding of the little-known physics of SiO x resistive memory devices, as well as other oxide-based RRAM systems (e.g. transition metal oxide RRAMs), offering design and optimization capabilities with regard to the reliability and variability of memory cells.

Nanoscale Transformations in Metastable, Amorphous, Silicon-Rich Silica.

Electrically biasing thin films of amorphous, substoichiometric silicon oxide drives surprisingly large structural changes, apparent as density variations, oxygen movement, and ultimately, emission of superoxide ions. Results from this fundamental study are directly relevant to materials that are increasingly used in a range of technologies, and demonstrate a surprising level of field-driven local reordering of a random oxide network.

Emulating the Electrical Activity of the Neuron Using a Silicon Oxide RRAM Cell.

In recent years, formidable effort has been devoted to exploring the potential of Resistive RAM (RRAM) devices to model key features of biological synapses. This is done to strengthen the link between neuro-computing architectures and neuroscience, bearing in mind the extremely low power consumption and immense parallelism of biological systems. Here we demonstrate the feasibility of using the RRAM cell to go further and to model aspects of the electrical activity of the neuron. We focus on the specific operational procedures required for the generation of controlled voltage transients, which resemble spike-like responses. Further, we demonstrate that RRAM devices are capable of integrating input current pulses over time to produce thresholded voltage transients. We show that the frequency of the output transients can be controlled by the input signal, and we relate recent models of the redox-based nanoionic resistive memory cell to two common neuronal models, the Hodgkin-Huxley (HH) conductance model and the leaky integrate-and-fire model. We employ a simplified circuit model to phenomenologically describe voltage transient generation.

Conductance tomography of conductive filaments in intrinsic silicon-rich silica RRAM.

We present results from an imaging study of filamentary conduction in silicon suboxide resistive RAM devices. We used a conductive atomic force microscope to etch through devices while measuring current, allowing us to produce tomograms of conductive filaments. To our knowledge this is the first report of such measurements in an intrinsic resistance switching material.

Electrically tailored resistance switching in silicon oxide.

Resistive switching in a metal-free silicon-based material offers a compelling alternative to existing metal oxide-based resistive RAM (ReRAM) devices, both in terms of ease of fabrication and of enhanced device performance. We report a study of resistive switching in devices consisting of non-stoichiometric silicon-rich silicon dioxide thin films. Our devices exhibit multi-level switching and analogue modulation of resistance as well as standard two-level switching. We demonstrate different operational modes that make it possible to dynamically adjust device properties, in particular two highly desirable properties: nonlinearity and self-rectification. This can potentially enable high levels of device integration in passive crossbar arrays without causing the problem of leakage currents in common line semi-selected devices. Aspects of conduction and switching mechanisms are discussed, and scanning tunnelling microscopy (STM) measurements provide a more detailed insight into both the location and the dimensions of the conductive filaments.