HFNet: A CNN Architecture Co-designed for Neuromorphic Hardware With a Crossbar Array of Synapses.

The hardware-software co-optimization of neural network architectures is a field of research that emerged with the advent of commercial neuromorphic chips, such as the IBM TrueNorth and Intel Loihi. Development of simulation and automated mapping software tools in tandem with the design of neuromorphic hardware, whilst taking into consideration the hardware constraints, will play an increasingly significant role in deployment of system-level applications. This paper illustrates the importance and benefits of co-design of convolutional neural networks (CNN) that are to be mapped onto neuromorphic hardware with a crossbar array of synapses. Toward this end, we first study which convolution techniques are more hardware friendly and propose different mapping techniques for different convolutions. We show that, for a seven-layered CNN, our proposed mapping technique can reduce the number of cores used by 4.9-13.8 times for crossbar sizes ranging from 128 × 256 to 1,024 × 1,024, and this can be compared to the toeplitz method of mapping. We next develop an iterative co-design process for the systematic design of more hardware-friendly CNNs whilst considering hardware constraints, such as core sizes. A python wrapper, developed for the mapping process, is also useful for validating hardware design and studies on traffic volume and energy consumption. Finally, a new neural network dubbed HFNet is proposed using the above co-design process; it achieves a classification accuracy of 71.3% on the IMAGENET dataset (comparable to the VGG-16) but uses 11 times less cores for neuromorphic hardware with core size of 1,024 × 1,024. We also modified the HFNet to fit onto different core sizes and report on the corresponding classification accuracies. Various aspects of the paper are patent pending.

Triplet Spike Time-Dependent Plasticity in a Floating-Gate Synapse.

Synapse plays an important role in learning in a neural network; the learning rules that modify the synaptic strength based on the timing difference between the pre- and postsynaptic spike occurrence are termed spike time-dependent plasticity (STDP) rules. The most commonly used rule posits weight change based on time difference between one presynaptic spike and one postsynaptic spike and is hence termed doublet STDP (D-STDP). However, D-STDP could not reproduce results of many biological experiments; a triplet STDP (T-STDP) that considers triplets of spikes as the fundamental unit has been proposed recently to explain these observations. This paper describes the compact implementation of a synapse using a single floating-gate (FG) transistor that can store a weight in a nonvolatile manner and demonstrates the T-STDP learning rule by modifying drain voltages according to triplets of spikes. We describe a mathematical procedure to obtain control voltages for the FG device for T-STDP and also show measurement results from an FG synapse fabricated in TSMC 0.35-µm CMOS process to support the theory. Possible very large scale integration implementation of drain voltage waveform generator circuits is also presented with the simulation results.

On the non-STDP behavior and its remedy in a floating-gate synapse.

This brief describes the neuromorphic very large scale integration implementation of a synapse utilizing a single floating-gate (FG) transistor that can be used to store a weight in a nonvolatile manner and demonstrate biological learning rules such as spike-timing-dependent plasticity (STDP). The experimental STDP plot (change in weight against ∆t=tpost - tpre ) of a traditional FG synapse from previous studies shows a depression instead of potentiation at some range of positive values of ∆t -we call this non-STDP behavior. In this brief, we first analyze theoretically the reason for this anomaly and then present a simple solution based on changing control gate waveforms of the FG device to make the weight change conform closely to biological observations over a wide range of parameters. The experimental results from an FG synapse fabricated in AMS 0.35- µ m CMOS process design are also presented to justify the claim. Finally, we present the simulation results of a circuit designed to create the modified gate voltage waveform.