UC Santa Cruz

This dissertation is dedicated to using Memristive Spiking Neural Networks (MSNNs) for deep learning tasks such as image classification, visual associative memory tasks such as pattern recognition, and auditory cortex processing tasks such as sound localization (SL). The image classification model consists entirely of memristive neurons and memristive synapses utilizing deep learning plausible supervised learning rules. The pattern recognition fully MSNNs consists of memristive neurons and memristive synapses harnessing biologically plausible unsupervised learning rules. SL MSNNs emulate biological brain functionality with volatile memristive synapses.

By developing a minimal circuit element memristive neuron -- Memristive Integrate-and-Fire (MIF) neuron -- with commercially accessible memristors, we are able to demonstrate large-scale fully MSNNs and apply the supervised Backpropagation Through Time (BPTT) algorithm to train networks, achieving state-of-the-art accuracy for several datasets. In addition, using a memristive unsupervised learning rule based on a continuously evolving alpha function membrane potential, we are also able to train large-scale fully MSNNs with Spiking-Time-Dependent Plasticity (STDP) and achieve high accuracies. Moreover, a volatile memristor is used for mimicking Short-Term Depression (STD) synapses such that we can simulate SL networks and pinpoint the direction of the sound coming from.

The major contributions reported in this proposal include: • Development of the MIF neuron SPICE-level model; • Validation of the SPICE-level MIF neuron model to a Python-based simulation of large-scale MSNNs; • Simulation of a small-scale neuron network consisting of a presynaptic, a memristive synapse, and a postsynaptic MIF neuron to generate the STDP learning window; • Abstraction of memristive STDP with alpha functions; • Simulation of a large-scale, fully MSNN consisting of MIF neurons and memristive synapses for unsupervised learning in Python; • Development of the MIF neuron numerical integration model; • Validation of the numerical integration model in a behavioral simulation of large-scale MSNNs; • Verification of the forward Euler numerical integration method to simplify the training process when compared to more complicated numerical methods; • Training fully MSNNs by directly applying the gradient descent learning algorithm to the MIF neuron numerical integration model; • Modeling a volatile memristor that exhibits Short-Term Plasticity (STP) based on real synapses; • Mimicking a real brain auditory cortex Spiking Neural Network (SNN) with the proposed memristor model.