Roadmap on emerging hardware and technology for machine learning.

Recent progress in artificial intelligence is largely attributed to the rapid development of machine learning, especially in the algorithm and neural network models. However, it is the performance of the hardware, in particular the energy efficiency of a computing system that sets the fundamental limit of the capability of machine learning. Data-centric computing requires a revolution in hardware systems, since traditional digital computers based on transistors and the von Neumann architecture were not purposely designed for neuromorphic computing. A hardware platform based on emerging devices and new architecture is the hope for future computing with dramatically improved throughput and energy efficiency. Building such a system, nevertheless, faces a number of challenges, ranging from materials selection, device optimization, circuit fabrication and system integration, to name a few. The aim of this Roadmap is to present a snapshot of emerging hardware technologies that are potentially beneficial for machine learning, providing the Nanotechnology readers with a perspective of challenges and opportunities in this burgeoning field.

High-Performance Mixed-Signal Neurocomputing With Nanoscale Floating-Gate Memory Cell Arrays.

Potential advantages of analog- and mixed-signal nanoelectronic circuits, based on floating-gate devices with adjustable conductance, for neuromorphic computing had been realized long time ago. However, practical realizations of this approach suffered from using rudimentary floating-gate cells of relatively large area. Here, we report a prototype $28\times28$ binary-input, ten-output, three-layer neuromorphic network based on arrays of highly optimized embedded nonvolatile floating-gate cells, redesigned from a commercial 180-nm nor flash memory. All active blocks of the circuit, including 101 780 floating-gate cells, have a total area below 1 mm2. The network has shown a 94.7% classification fidelity on the common Modified National Institute of Standards and Technology benchmark, close to the 96.2% obtained in simulation. The classification of one pattern takes a sub-1- $\mu \text{s}$ time and a sub-20-nJ energy-both numbers much better than in the best reported digital implementations of the same task. Estimates show that a straightforward optimization of the hardware and its transfer to the already available 55-nm technology may increase this advantage to more than $10^{2}\times $ in speed and $10^{4}\times $ in energy efficiency.

Publisher Correction: Stateful characterization of resistive switching TiO2 with electron beam induced currents.

The original version of this Article contained an error in Eq. 1. The arrows between the symbols "T" and "B", and "B" and "T", were written "↔" but should have been "→", and incorrectly read: IEBIC=IEBAC+ISEE+I(e↔h)+IEBICT↔B+IESEEB↔T The correct from of the Eq. 1 is as follows:IEBIC=IEBAC+ISEE+I(e↔h)+IEBICT→B+IESEEB→T This has now been corrected in both the PDF and HTML versions of the article.

Stateful characterization of resistive switching TiO2 with electron beam induced currents.

Metal oxide resistive switches are increasingly important as possible artificial synapses in next-generation neuromorphic networks. Nevertheless, there is still no codified set of tools for studying properties of the devices. To this end, we demonstrate electron beam-induced current measurements as a powerful method to monitor the development of local resistive switching in TiO2-based devices. By comparing beam energy-dependent electron beam-induced currents with Monte Carlo simulations of the energy absorption in different device layers, it is possible to deconstruct the origins of filament image formation and relate this to both morphological changes and the state of the switch. By clarifying the contrast mechanisms in electron beam-induced current microscopy, it is possible to gain new insights into the scaling of the resistive switching phenomenon and observe the formation of a current leakage region around the switching filament. Additionally, analysis of symmetric device structures reveals propagating polarization domains.

Flexible three-dimensional artificial synapse networks with correlated learning and trainable memory capability.

If a three-dimensional physical electronic system emulating synapse networks could be built, that would be a significant step toward neuromorphic computing. However, the fabrication complexity of complementary metal-oxide-semiconductor architectures impedes the achievement of three-dimensional interconnectivity, high-device density, or flexibility. Here we report flexible three-dimensional artificial chemical synapse networks, in which two-terminal memristive devices, namely, electronic synapses (e-synapses), are connected by vertically stacking crossbar electrodes. The e-synapses resemble the key features of biological synapses: unilateral connection, long-term potentiation/depression, a spike-timing-dependent plasticity learning rule, paired-pulse facilitation, and ultralow-power consumption. The three-dimensional artificial synapse networks enable a direct emulation of correlated learning and trainable memory capability with strong tolerances to input faults and variations, which shows the feasibility of using them in futuristic electronic devices and can provide a physical platform for the realization of smart memories and machine learning and for operation of the complex algorithms involving hierarchical neural networks.High-density information storage calls for the development of modern electronics with multiple stacking architectures that increase the complexity of three-dimensional interconnectivity. Here, Wu et al. build a stacked yet flexible artificial synapse network using layer-by-layer solution processing.

Modeling and Experimental Demonstration of a Hopfield Network Analog-to-Digital Converter with Hybrid CMOS/Memristor Circuits.

The purpose of this work was to demonstrate the feasibility of building recurrent artificial neural networks with hybrid complementary metal oxide semiconductor (CMOS)/memristor circuits. To do so, we modeled a Hopfield network implementing an analog-to-digital converter (ADC) with up to 8 bits of precision. Major shortcomings affecting the ADC's precision, such as the non-ideal behavior of CMOS circuitry and the specific limitations of memristors, were investigated and an effective solution was proposed, capitalizing on the in-field programmability of memristors. The theoretical work was validated experimentally by demonstrating the successful operation of a 4-bit ADC circuit implemented with discrete Pt/TiO2- x /Pt memristors and CMOS integrated circuit components.

Resistive switching and its suppression in Pt/Nb:SrTiO3 junctions.

Oxide-based resistive switching devices are promising candidates for new memory and computing technologies. Poor understanding of the defect-based mechanisms that give rise to resistive switching is a major impediment for engineering reliable and reproducible devices. Here we identify an unintentional interface layer as the origin of resistive switching in Pt/Nb:SrTiO3 junctions. We clarify the microscopic mechanisms by which the interface layer controls the resistive switching. We show that appropriate interface processing can eliminate this contribution. These findings are an important step towards engineering more reliable resistive switching devices.

Pattern classification by memristive crossbar circuits using ex situ and in situ training.

Memristors are memory resistors that promise the efficient implementation of synaptic weights in artificial neural networks. Whereas demonstrations of the synaptic operation of memristors already exist, the implementation of even simple networks is more challenging and has yet to be reported. Here we demonstrate pattern classification using a single-layer perceptron network implemented with a memrisitive crossbar circuit and trained using the perceptron learning rule by ex situ and in situ methods. In the first case, synaptic weights, which are realized as conductances of titanium dioxide memristors, are calculated on a precursor software-based network and then imported sequentially into the crossbar circuit. In the second case, training is implemented in situ, so the weights are adjusted in parallel. Both methods work satisfactorily despite significant variations in the switching behaviour of the memristors. These results give hope for the anticipated efficient implementation of artificial neuromorphic networks and pave the way for dense, high-performance information processing systems.

Memristive devices for computing.

Memristive devices are electrical resistance switches that can retain a state of internal resistance based on the history of applied voltage and current. These devices can store and process information, and offer several key performance characteristics that exceed conventional integrated circuit technology. An important class of memristive devices are two-terminal resistance switches based on ionic motion, which are built from a simple conductor/insulator/conductor thin-film stack. These devices were originally conceived in the late 1960s and recent progress has led to fast, low-energy, high-endurance devices that can be scaled down to less than 10 nm and stacked in three dimensions. However, the underlying device mechanisms remain unclear, which is a significant barrier to their widespread application. Here, we review recent progress in the development and understanding of memristive devices. We also examine the performance requirements for computing with memristive devices and detail how the outstanding challenges could be met.

High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm.

Using memristive properties common for titanium dioxide thin film devices, we designed a simple write algorithm to tune device conductance at a specific bias point to 1% relative accuracy (which is roughly equivalent to seven-bit precision) within its dynamic range even in the presence of large variations in switching behavior. The high precision state is nonvolatile and the results are likely to be sustained for nanoscale memristive devices because of the inherent filamentary nature of the resistive switching. The proposed functionality of memristive devices is especially attractive for analog computing with low precision data. As one representative example we demonstrate hybrid circuitry consisting of an integrated circuit summing amplifier and two memristive devices to perform the analog multiply-and-add (dot-product) computation, which is a typical bottleneck operation in information processing.

The switching location of a bipolar memristor: chemical, thermal and structural mapping.

Memristors are memory resistors promising a rapid integration into future memory technologies. However, progress is still critically limited by a lack of understanding of the physical processes occurring at the nanoscale. Here we correlate device electrical characteristics with local atomic structure, chemistry and temperature. We resolved a single conducting channel that is made up of a reduced phase of the as-deposited titanium oxide. Moreover, we observed sufficient Joule heating to induce a crystallization of the oxide surrounding the channel, with a peculiar pattern that finite element simulations correlated with the existence of a hot spot close to the bottom electrode, thus identifying the switching location. This work reports direct observations in all three dimensions of the internal structure of titanium oxide memristors.

Four-dimensional address topology for circuits with stacked multilayer crossbar arrays.

We present a topological framework that provides a simple yet powerful electronic circuit architecture for constructing and using multilayer crossbar arrays, allowing a significantly increased integration density of memristive crosspoint devices beyond the scaling limits of lateral feature sizes. The truly remarkable feature of such circuits, which is an extension of the CMOL (Cmos + MOLecular-scale devices) concept for an area-like interface to a three-dimensional system, is that a large-feature-size complimentary metal-oxide-semiconductor (CMOS) substrate can provide high-density interconnects to multiple crossbar layers through a single set of vertical vias. The physical locations of the memristive devices are mapped to a four-dimensional logical address space such that unique access from the CMOS substrate is provided to every device in a stacked array of crossbars. This hybrid architecture is compatible with digital memories, field-programmable gate arrays, and biologically inspired adaptive networks and with state-of-the-art integrated circuit foundries.

Memristor-CMOS hybrid integrated circuits for reconfigurable logic.

Hybrid reconfigurable logic circuits were fabricated by integrating memristor-based crossbars onto a foundry-built CMOS (complementary metal-oxide-semiconductor) platform using nanoimprint lithography, as well as materials and processes that were compatible with the CMOS. Titanium dioxide thin-film memristors served as the configuration bits and switches in a data routing network and were connected to gate-level CMOS components that acted as logic elements, in a manner similar to a field programmable gate array. We analyzed the chips using a purpose-built testing system, and demonstrated the ability to configure individual devices, use them to wire up various logic gates and a flip-flop, and then reconfigure devices.

Coupled ionic and electronic transport model of thin-film semiconductor memristive behavior.

The memristor, the fourth passive circuit element, was predicted theoretically nearly 40 years ago, but we just recently demonstrated both an intentional material system and an analytical model that exhibited the properties of such a device. Here we provide a more physical model based on numerical solutions of coupled drift-diffusion equations for electrons and ions with appropriate boundary conditions. We simulate the dynamics of a two-terminal memristive device based on a semiconductor thin film with mobile dopants that are partially compensated by a small amount of immobile acceptors. We examine the mobile ion distributions, zero-bias potentials, and current-voltage characteristics of the model for both steady-state bias conditions and for dynamical switching to obtain physical insight into the transport processes responsible for memristive behavior in semiconductor films.

The missing memristor found.

Anyone who ever took an electronics laboratory class will be familiar with the fundamental passive circuit elements: the resistor, the capacitor and the inductor. However, in 1971 Leon Chua reasoned from symmetry arguments that there should be a fourth fundamental element, which he called a memristor (short for memory resistor). Although he showed that such an element has many interesting and valuable circuit properties, until now no one has presented either a useful physical model or an example of a memristor. Here we show, using a simple analytical example, that memristance arises naturally in nanoscale systems in which solid-state electronic and ionic transport are coupled under an external bias voltage. These results serve as the foundation for understanding a wide range of hysteretic current-voltage behaviour observed in many nanoscale electronic devices that involve the motion of charged atomic or molecular species, in particular certain titanium dioxide cross-point switches.

Defect-tolerant architectures for nanoelectronic crossbar memories.

We have calculated the maximum useful bit density that may be achieved by the synergy of bad bit exclusion and advanced (BCH) error correcting codes in prospective crossbar nanoelectronic memories, as a function of defective memory cell fraction. While our calculations are based on a particular ("CMOL") memory topology, with naturally segmented nanowires and an area-distributed nano/CMOS interface, for realistic parameters our results are also applicable to "global" crossbar memories with peripheral interfaces. The results indicate that the crossbar memories with a nano/CMOS pitch ratio close to 1/3 (which is typical for the current, initial stage of the nanoelectronics development) may overcome purely semiconductor memories in useful bit density if the fraction of nanodevice defects (stuck-on-faults) is below approximately 15%, even under rather tough, 30 ns upper bound on the total access time. Moreover, as the technology matures, and the pitch ratio approaches an order of magnitude, the crossbar memories may be far superior to the densest semiconductor memories by providing, e.g., a 1 Tbit/cm2 density even for a plausible defect fraction of 2%. These highly encouraging results are much better than those reported in literature earlier, including our own early work, mostly due to more advanced error correcting codes.