Mixed-Precision Deep Learning Based on Computational Memory.

Deep neural networks (DNNs) have revolutionized the field of artificial intelligence and have achieved unprecedented success in cognitive tasks such as image and speech recognition. Training of large DNNs, however, is computationally intensive and this has motivated the search for novel computing architectures targeting this application. A computational memory unit with nanoscale resistive memory devices organized in crossbar arrays could store the synaptic weights in their conductance states and perform the expensive weighted summations in place in a non-von Neumann manner. However, updating the conductance states in a reliable manner during the weight update process is a fundamental challenge that limits the training accuracy of such an implementation. Here, we propose a mixed-precision architecture that combines a computational memory unit performing the weighted summations and imprecise conductance updates with a digital processing unit that accumulates the weight updates in high precision. A combined hardware/software training experiment of a multilayer perceptron based on the proposed architecture using a phase-change memory (PCM) array achieves 97.73% test accuracy on the task of classifying handwritten digits (based on the MNIST dataset), within 0.6% of the software baseline. The architecture is further evaluated using accurate behavioral models of PCM on a wide class of networks, namely convolutional neural networks, long-short-term-memory networks, and generative-adversarial networks. Accuracies comparable to those of floating-point implementations are achieved without being constrained by the non-idealities associated with the PCM devices. A system-level study demonstrates 172 × improvement in energy efficiency of the architecture when used for training a multilayer perceptron compared with a dedicated fully digital 32-bit implementation.

Experimental Demonstration of Supervised Learning in Spiking Neural Networks with Phase-Change Memory Synapses.

Spiking neural networks (SNN) are computational models inspired by the brain's ability to naturally encode and process information in the time domain. The added temporal dimension is believed to render them more computationally efficient than the conventional artificial neural networks, though their full computational capabilities are yet to be explored. Recently, in-memory computing architectures based on non-volatile memory crossbar arrays have shown great promise to implement parallel computations in artificial and spiking neural networks. In this work, we evaluate the feasibility to realize high-performance event-driven in-situ supervised learning systems using nanoscale and stochastic analog memory synapses. For the first time, the potential of analog memory synapses to generate precisely timed spikes in SNNs is experimentally demonstrated. The experiment targets applications which directly integrates spike encoded signals generated from bio-mimetic sensors with in-memory computing based learning systems to generate precisely timed control signal spikes for neuromorphic actuators. More than 170,000 phase-change memory (PCM) based synapses from our prototype chip were trained based on an event-driven learning rule, to generate spike patterns with more than 85% of the spikes within a 25 ms tolerance interval in a 1250 ms long spike pattern. We observe that the accuracy is mainly limited by the imprecision related to device programming and temporal drift of conductance values. We show that an array level scaling scheme can significantly improve the retention of the trained SNN states in the presence of conductance drift in the PCM. Combining the computational potential of supervised SNNs with the parallel compute power of in-memory computing, this work paves the way for next-generation of efficient brain-inspired systems.

Accurate deep neural network inference using computational phase-change memory.

In-memory computing using resistive memory devices is a promising non-von Neumann approach for making energy-efficient deep learning inference hardware. However, due to device variability and noise, the network needs to be trained in a specific way so that transferring the digitally trained weights to the analog resistive memory devices will not result in significant loss of accuracy. Here, we introduce a methodology to train ResNet-type convolutional neural networks that results in no appreciable accuracy loss when transferring weights to phase-change memory (PCM) devices. We also propose a compensation technique that exploits the batch normalization parameters to improve the accuracy retention over time. We achieve a classification accuracy of 93.7% on CIFAR-10 and a top-1 accuracy of 71.6% on ImageNet benchmarks after mapping the trained weights to PCM. Our hardware results on CIFAR-10 with ResNet-32 demonstrate an accuracy above 93.5% retained over a one-day period, where each of the 361,722 synaptic weights is programmed on just two PCM devices organized in a differential configuration.

Treatment of Primary Central Nervous System Posttransplant Lymphoproliferative Disorder in an Adult Kidney Transplant Recipient: A Case Report.

Posttransplant lymphoproliferative disorder is a serious complication of solid-organ transplant. Extranodal involvement is common; however, isolated involvement of the central nervous system is extremely rare and represents a particularly difficult therapeutic challenge with no current consensus on optimal treatment. Here, we describe a 70-year-old woman who developed Epstein-Barr virus-related primary central nervous system lymphoma 19 months after kidney transplant. Immunosuppression was reduced, and the patient was started on high-dose methotrexate, which was complicated by acute kidney injury and discontinued. She then received a rituximab and temozolomide chemotherapeutic regimen and achieved complete clinical response. Seventeen months after diagnosis, she is alive and has not developed any other posttransplant lymphoproliferative disorder. We review the current literature and discuss treatment options for patients with primary central nervous system posttransplant lymphoproliferative disorder following kidney transplant.

Neuromorphic computing with multi-memristive synapses.

Neuromorphic computing has emerged as a promising avenue towards building the next generation of intelligent computing systems. It has been proposed that memristive devices, which exhibit history-dependent conductivity modulation, could efficiently represent the synaptic weights in artificial neural networks. However, precise modulation of the device conductance over a wide dynamic range, necessary to maintain high network accuracy, is proving to be challenging. To address this, we present a multi-memristive synaptic architecture with an efficient global counter-based arbitration scheme. We focus on phase change memory devices, develop a comprehensive model and demonstrate via simulations the effectiveness of the concept for both spiking and non-spiking neural networks. Moreover, we present experimental results involving over a million phase change memory devices for unsupervised learning of temporal correlations using a spiking neural network. The work presents a significant step towards the realization of large-scale and energy-efficient neuromorphic computing systems.

Spiking neural networks for handwritten digit recognition-Supervised learning and network optimization.

We demonstrate supervised learning in Spiking Neural Networks (SNNs) for the problem of handwritten digit recognition using the spike triggered Normalized Approximate Descent (NormAD) algorithm. Our network that employs neurons operating at sparse biological spike rates below 300Hz achieves a classification accuracy of 98.17% on the MNIST test database with four times fewer parameters compared to the state-of-the-art. We present several insights from extensive numerical experiments regarding optimization of learning parameters and network configuration to improve its accuracy. We also describe a number of strategies to optimize the SNN for implementation in memory and energy constrained hardware, including approximations in computing the neuronal dynamics and reduced precision in storing the synaptic weights. Experiments reveal that even with 3-bit synaptic weights, the classification accuracy of the designed SNN does not degrade beyond 1% as compared to the floating-point baseline. Further, the proposed SNN, which is trained based on the precise spike timing information outperforms an equivalent non-spiking artificial neural network (ANN) trained using back propagation, especially at low bit precision. Thus, our study shows the potential for realizing efficient neuromorphic systems that use spike based information encoding and learning for real-world applications.

A 250 mV Cu/SiO2/W Memristor with Half-Integer Quantum Conductance States.

Memristive devices, whose conductance depends on previous programming history, are of significant interest for building nonvolatile memory and brain-inspired computing systems. Here, we report half-integer quantized conductance transitions G = (n/2) (2e(2)/h) for n = 1, 2, 3, etc., in Cu/SiO2/W memristive devices observed below 300 mV at room temperature. This is attributed to the nanoscale filamentary nature of Cu conductance pathways formed inside SiO2. Retention measurements also show spontaneous filament decay with quantized conductance levels. Numerical simulations shed light into the dynamics underlying the data retention loss mechanisms and provide new insights into the nanoscale physics of memristive devices and trade-offs involved in engineering them for computational applications.

Novel synaptic memory device for neuromorphic computing.

This report discusses the electrical characteristics of two-terminal synaptic memory devices capable of demonstrating an analog change in conductance in response to the varying amplitude and pulse-width of the applied signal. The devices are based on Mn doped HfO2 material. The mechanism behind reconfiguration was studied and a unified model is presented to explain the underlying device physics. The model was then utilized to show the application of these devices in speech recognition. A comparison between a 20 nm × 20 nm sized synaptic memory device with that of a state-of-the-art VLSI SRAM synapse showed ~10× reduction in area and >10(6) times reduction in the power consumption per learning cycle.

Sublingual tissue perfusion improves during emergency treatment of acute decompensated heart failure.

OBJECTIVES: The aim of this study was to measure sublingual perfused capillary density (PCD) to assess sublingual microvascular perfusion during emergency department (ED) treatment of acute decompensated heart failure (ADHF). METHODS: This prospective, observational study enrolled ED patients with ADHF, measuring pre- and post-ED treatment PCD. Sidestream dark-field imaging was analyzed by 3 investigators blinded to patient identifiers and time points. Patient demographics, ADHF etiology, serum brain natriuretic peptide, and hemoglobin were measured along with a visual analogue scale (VAS), which assessed patient baseline characteristics and response to ED treatment. A paired t test analyzed changes in PCD, mean arterial pressure (MAP), and patient assessment. Interrater variability was assessed with an intraclass correlation coefficient (ICC), with a P value <.05 considered significant for all testing. RESULTS: Thirty-six patients were enrolled with a mean time between pretreatment and posttreatment PCD (±SD) of 138 ± 59 minutes and a hospital length of stay of 4.0 ± 4.1 days. During this time, PCD increased (difference, 1.3 mm/mm(2); 95% confidence interval, 0.4-2.1; P = .004), as did the MAP (P = .002), patient VAS score (P < .001), and observer VAS score (P < .001). There was no correlation between the change in PCD and time (R(2) = .016, P = .47), MAP (R(2) = .013, P = .54), or VAS scores. The ICC was 0.954. CONCLUSIONS: Sublingual tissue perfusion is diminished in ADHF but increases with treatment. It may represent a quantitative way to evaluate ADHF in the ED setting.