Training fully connected networks with resistive memories: impact of device failures.

Hardware accelerators based on two-terminal non-volatile memories (NVMs) can potentially provide competitive speed and accuracy for the training of fully connected deep neural networks (FC-DNNs), with respect to GPUs and other digital accelerators. We recently proposed [S. Ambrogio et al., Nature, 2018] novel neuromorphic crossbar arrays, consisting of a pair of phase-change memory (PCM) devices combined with a pair of 3-Transistor 1-Capacitor (3T1C) circuit elements, so that each weight was implemented using multiple conductances of varying significance, and then showed that this weight element can train FC-DNNs to software-equivalent accuracies. Unfortunately, however, real arrays of emerging NVMs such as PCM typically include some failed devices (e.g., <100% yield), either due to fabrication issues or early endurance failures, which can degrade DNN training accuracy. This paper explores the impact of device failures, NVM conductances that may contribute read current but which cannot be programmed, on DNN training and test accuracy. Results show that "stuck-on" and "dead" devices, exhibiting high and low read conductances, respectively, do in fact degrade accuracy performance to some degree. We find that the presence of the CMOS-based and thus highly-reliable 3T1C devices greatly increase system robustness. After studying the inherent mechanisms, we study the dependence of DNN accuracy on the number of functional weights, the number of neurons in the hidden layer, and the number and type of damaged devices. Finally, we describe conditions under which making the network larger or adjusting the network hyperparameters can still improve the network accuracy, even in the presence of failed devices.

Equivalent-accuracy accelerated neural-network training using analogue memory.

Neural-network training can be slow and energy intensive, owing to the need to transfer the weight data for the network between conventional digital memory chips and processor chips. Analogue non-volatile memory can accelerate the neural-network training algorithm known as backpropagation by performing parallelized multiply-accumulate operations in the analogue domain at the location of the weight data. However, the classification accuracies of such in situ training using non-volatile-memory hardware have generally been less than those of software-based training, owing to insufficient dynamic range and excessive weight-update asymmetry. Here we demonstrate mixed hardware-software neural-network implementations that involve up to 204,900 synapses and that combine long-term storage in phase-change memory, near-linear updates of volatile capacitors and weight-data transfer with 'polarity inversion' to cancel out inherent device-to-device variations. We achieve generalization accuracies (on previously unseen data) equivalent to those of software-based training on various commonly used machine-learning test datasets (MNIST, MNIST-backrand, CIFAR-10 and CIFAR-100). The computational energy efficiency of 28,065 billion operations per second per watt and throughput per area of 3.6 trillion operations per second per square millimetre that we calculate for our implementation exceed those of today's graphical processing units by two orders of magnitude. This work provides a path towards hardware accelerators that are both fast and energy efficient, particularly on fully connected neural-network layers.

Poly(vinylidene fluoride) (PVDF) Binder Degradation in Li-O2 Batteries: A Consideration for the Characterization of Lithium Superoxide.

We show that a common Li-O2 battery cathode binder, poly(vinylidene fluoride) (PVDF), degrades in the presence of reduced oxygen species during Li-O2 discharge when adventitious impurities are present. This degradation process forms products that exhibit Raman shifts (∼1133 and 1525 cm-1) nearly identical to those reported to belong to lithium superoxide (LiO2), complicating the identification of LiO2 in Li-O2 batteries. We show that these peaks are not observed when characterizing extracted discharged cathodes that employ poly(tetrafluoroethylene) (PTFE) as a binder, even when used to bind iridium-decorated reduced graphene oxide (Ir-rGO)-based cathodes similar to those that reportedly stabilize bulk LiO2 formation. We confirm that for all extracted discharged cathodes on which the 1133 and 1525 cm-1 Raman shifts are observed, only a 2.0 e-/O2 process is identified during the discharge, and lithium peroxide (Li2O2) is predominantly formed (along with typical parasitic side product formation). Our results strongly suggest that bulk, stable LiO2 formation via the 1 e-/O2 process is not an active discharge reaction in Li-O2 batteries.

On the efficacy of electrocatalysis in nonaqueous Li-O2 batteries.

Heterogeneous electrocatalysis has become a focal point in rechargeable Li-air battery research to reduce overpotentials in both the oxygen reduction (discharge) and especially oxygen evolution (charge) reactions. In this study, we show that past reports of traditional cathode electrocatalysis in nonaqueous Li-O(2) batteries were indeed true, but that gas evolution related to electrolyte solvent decomposition was the dominant process being catalyzed. In dimethoxyethane, where Li(2)O(2) formation is the dominant product of the electrochemistry, no catalytic activity (compared to pure carbon) is observed using the same (Au, Pt, MnO(2)) nanoparticles. Nevertheless, the onset potential of oxygen evolution is only slightly higher than the open circuit potential of the cell, indicating conventional oxygen evolution electrocatalysis may be unnecessary.

Observation of the role of subcritical nuclei in crystallization of a glassy solid.

Phase transformation generally begins with nucleation, in which a small aggregate of atoms organizes into a different structural symmetry. The thermodynamic driving forces and kinetic rates have been predicted by classical nucleation theory, but observation of nanometer-scale nuclei has not been possible, except on exposed surfaces. We used a statistical technique called fluctuation transmission electron microscopy to detect nuclei embedded in a glassy solid, and we used a laser pump-probe technique to determine the role of these nuclei in crystallization. This study provides a convincing proof of the time- and temperature-dependent development of nuclei, information that will play a critical role in the development of advanced materials for phase-change memories.

Solution-phase deposition and nanopatterning of GeSbSe phase-change materials.

Chalcogenide films with reversible amorphous-crystalline phase transitions have been commercialized as optically rewritable data-storage media, and intensive effort is now focused on integrating them into electrically addressed non-volatile memory devices (phase-change random-access memory or PCRAM). Although optical data storage is accomplished by laser-induced heating of continuous films, electronic memory requires integration of discrete nanoscale phase-change material features with read/write electronics. Currently, phase-change films are most commonly deposited by sputter deposition, and patterned by conventional lithography. Metal chalcogenide films for transistor applications have recently been deposited by a low-temperature, solution-phase route. Here, we extend this methodology to prepare thin films and nanostructures of GeSbSe phase-change materials. We report the ready tuneability of phase-change properties in GeSbSe films through composition variation achieved by combining novel precursors in solution. Rapid, submicrosecond phase switching is observed by laser-pulse annealing. We also demonstrate that prepatterned holes can be filled to fabricate phase-change nanostructures from hundreds down to tens of nanometres in size, offering enhanced flexibility in fabricating PCRAM devices with reduced current requirements.

Sequence-independent helical wrapping of single-walled carbon nanotubes by long genomic DNA.

Because of their nanometer sizes and molecular recognition capabilities, biological systems have garnered much attention as vehicles for the directed assembly of nanoscale materials.(1-6) One of the greatest challenges of this research has been to successfully interface biological systems with electronic materials, such as semiconductors and metals. As a means to address some of these issues, Sarikaya, Belcher, and others have used a combinatorial technique called phage display(7-9) to discover new families of peptides that showed binding affinities to various substrates. More recently, Zheng and co-workers used combinatorial DNA libraries to isolate short DNA oligomers (30-90 bases) that could disperse single-walled carbon nanotubes (SWCNT) in water.(10) Through a systematic analysis, they found that short oligonucleotides having repeating sequences of gunanines and thymines (dGdT)(n) could wrap in a helical manner around a CNT with periodic pitch.(11) Although helix formation around SWCNTs having regular pitches is an effective method for dispersing and separating CNTs, the need for specific repeating sequences limits use to non-natural DNA that must be synthesized with optimal lengths of less than 150 bases. In contrast, we demonstrate here that long genomic single-stranded DNA (>>100 bases) of a completely random sequence of bases can be used to disperse CNTs efficiently through the single-stranded DNA's (ssDNA) ability to form tight helices around the CNTs with distinct periodic pitches. Although this process occurs irrespective of the DNA sequence, we show that this process is highly dependent on the removal of complementary strands. We also demonstrate that although the helix pitch-to-pitch distances remain constant down the length of a single CNT, the distances are variable from one DNA-CNT to another. Finally, we report initial work that shows that methods developed to align long dsDNA can be applied in a similar fashion to produce highly dense arrays of aligned ssDNA-CNT hybrids.