Artificial intelligence (AI) technology like deep learning is powering our daily life in many areas such as pattern recognition. The artificial neural network (ANN) is one of the deep learning models to achieve pattern recognition. A well-trained ANN can recognize images with precision over 98%. Although traditional two terminals memristors like phase changing memory (PCM) are already used to build ANNs. But those devices typically suffer from nonlinear, asymmetric conductance tuning problems. The novel memristive device Ionic Floating Gate memory (IFG) could potentially solve those problems. In this paper, a compact Cadence model of fully connected ANN using IFG is presented. The devices are tuned to an optimized state and formed a well-trained network. The recognition accuracy reaches 93.8%. This work demonstrates the IFG device also has the potential to be further utilized into other deep neural networks as synaptic memory.

Modern Artificial Neural Network(ANN) is a kind of nonlinear statistical data modeling tool, which can be optimized by a learning method based on mathematical statistics. Therefore, it is a practical application of mathematical statics. The ANN can get abilities to make simple decisions and judgments, just like a human brain. It is superior to formal logical implication.

Since the advancement of the ANN study significantly depends on the expansion of networks in-depth, a massive amount of vector-matrix multiplications is required [3]. Thus, energy efficiency is a key factor in evaluating the performance of ANNs. Since researches on vector-matrix multiplications have made great achievements, large and deep ANNs have been used to handle complex tasks and process massive data. The memories utilized by conventional ANNs, such as Static Random Access Memory (SRAM) and Flash memory are charge-based, which is not efficient in view of energy consumption during ANN computation since it cannot directly implement vector-matrix multiplication in a crossbar array structure [3]. Over the past decade, the Nonvolatile Memory (NVM) crossbar array has shown its superiority on improving the energy efficiency. Unlike the conventional memory, NVM is current based. NVM crossbar array can calculate matrix multiplication in a single step by sampling the current flowing, therefore, it was utilized as the analog vector-matrix multiplier for on ANN [3]. However, the nonlinear I-V characteristics of NVM put hard constrains on critical design parameters, such as the read voltage and the range of weight, which causes substantially reduced accuracy.

In this work, we built an ionic floating-gate (IFG) memory unit device model on Cadence based on two presented models, ENODe and ionic floating-gate memory (IFG) memory [2]. The IFG model consists of a polymer redox transistor connected to a conductive-bridge memory (CBM) [2]. We would like to apply this IFG memory unit device to build Memristor-based Neural Networks (MNN) and explore the performance of the networks. In the MNNs, the selective and linear programming of a redox transistor array is executed in parallel by overcoming the bridging threshold voltage of the CBMs [2], which can improve the performance (accuracy). This thesis is mainly about the design and training of the MNNs. Since we would like to apply the MNNs to perform digital number recognition, we designed the architecture that is available to recognize Modified National Institute of Standards and Technology (MNIST) handwriting digits and trained the MNN with MNIST train dataset, then tested the performance (accuracy) by MNIST test dataset. In addition, we tried to make some handwriting images by ourselves and used them to test the trained network to consolidate our conclusion. We also attempted to resize the original MNIST dataset by an image data pixel converting algorithm and make the newly created train dataset applicable to train an MNN with smaller scale. We used both the MNIST test dataset and the handwritten image data newly created to test the network and compared the performance with that of the original one. We hope that the IFG memory unit device can impose a significant impact on the MNN design.

A set of new neuron model and neural network architectures are introduced for the exploration of spiking neural networks to classify handwritten digits and images. Brain-like neural network powered artificial intelligence has become a driving force behind everyday applications from autonomous vehicles, facial recognition applications, to business analytics and energy market prediction. Biological brain-inspired Deep Neural Networks (DNN) have been around for a while. However, spiking Neural Networks (SNNs) have come to the spotlight due to higher energy efficiency and brain-like computing power. In this thesis, various models of spiking neurons such as Leaky Integrate-and-Fire (LIF), and Memristive Integrate-and-Fire (MIF) are examined and explored for applications of spiking neural networks. It has become apparent that large-scale neural network models will not only be in data-centers but also in edge computing devices and embedded systems, thus more energy-efficient and faster neural network types are needed.

The NAND technology has become a popular research area and implementation choice due to its non-volatile flash memory characteristics. There are many engineering challenges when it comes to NAND technology. Some of the limiting factors are reducing the transistor width and increasing read and write performance. The device physics for NAND floating gate cell technology also introduces challenges. Using a floating gate transistor that can address up to 4 bits per cell allows for higher density. However, this introduces a higher internal current leakage that can cause read and program inaccuracies. With floating-gate 3D NAND technology, we are able to better the data retention and have better QLC (Quad-Level Cell) capability. 3D NAND technology has a series of memory cells interconnected that can achieve higher data density and increased storage capacity. In this research, the Gen-4 NAND Flash device functionality is explored by design validation to achieve higher solid-state drive performance. This technology is 144-Layer QLC NAND Flash which is 1024Gb in density. My contributions are implemented showing improved read and write performance for customer operations and how inaccuracies are dealt with from a software and hardware perspective. Silicon wafer testing is also explored to fully characterize and analyze the problem.

This thesis focuses on integrated and miniaturized wireless neural recording systems for bio-instruments for the in-depth understanding on animal behavior, human brain activities, and complex neuroprosthetic devices to treat various neurological diseases. The interdisciplinary nature of the system requires a wide range of knowledge in biology and electronics to build such systems. A unique environment where the system should operate imposes challenging design constraints and system-level issues, which can be solved only by considering both biology and electronics. Fundamental building circuits including amplifiers, filters, analog-to- digital converters (ADCs) are addressed first. Then sub-systems, which consist of those basic circuits, are analyzed with an emphasis on trade-offs, which should be carefully considered to achieve optimal design. Specifically, an ultra wideband communication system for biomedical applications is proposed to overcome the limitation of the data bandwidth and power consumption existing in current conventional systems. Several ICs are designed and fabricated in 0.35um CMOS process to verify the proposed concepts and ideas. Prototype systems are implemented using those fabricated chips, and their test results from the bench top and in animal implantation are presented.

This dissertation is dedicated to using Memristive Spiking Neural Networks (MSNNs) for deep learning tasks such as image classification, visual associative memory tasks such as pattern recognition, and auditory cortex processing tasks such as sound localization (SL). The image classification model consists entirely of memristive neurons and memristive synapses utilizing deep learning plausible supervised learning rules. The pattern recognition fully MSNNs consists of memristive neurons and memristive synapses harnessing biologically plausible unsupervised learning rules. SL MSNNs emulate biological brain functionality with volatile memristive synapses.

By developing a minimal circuit element memristive neuron -- Memristive Integrate-and-Fire (MIF) neuron -- with commercially accessible memristors, we are able to demonstrate large-scale fully MSNNs and apply the supervised Backpropagation Through Time (BPTT) algorithm to train networks, achieving state-of-the-art accuracy for several datasets. In addition, using a memristive unsupervised learning rule based on a continuously evolving alpha function membrane potential, we are also able to train large-scale fully MSNNs with Spiking-Time-Dependent Plasticity (STDP) and achieve high accuracies. Moreover, a volatile memristor is used for mimicking Short-Term Depression (STD) synapses such that we can simulate SL networks and pinpoint the direction of the sound coming from.

The major contributions reported in this proposal include: • Development of the MIF neuron SPICE-level model; • Validation of the SPICE-level MIF neuron model to a Python-based simulation of large-scale MSNNs; • Simulation of a small-scale neuron network consisting of a presynaptic, a memristive synapse, and a postsynaptic MIF neuron to generate the STDP learning window; • Abstraction of memristive STDP with alpha functions; • Simulation of a large-scale, fully MSNN consisting of MIF neurons and memristive synapses for unsupervised learning in Python; • Development of the MIF neuron numerical integration model; • Validation of the numerical integration model in a behavioral simulation of large-scale MSNNs; • Verification of the forward Euler numerical integration method to simplify the training process when compared to more complicated numerical methods; • Training fully MSNNs by directly applying the gradient descent learning algorithm to the MIF neuron numerical integration model; • Modeling a volatile memristor that exhibits Short-Term Plasticity (STP) based on real synapses; • Mimicking a real brain auditory cortex Spiking Neural Network (SNN) with the proposed memristor model.

Data-intensive storage and computing systems call for continuing advancements both in data latency and energy efficiency. However, the major benefits from CMOS technologies, such as high packing density and processing speed, are getting desensitized primarily due to the prohibitively increasing power density. In order for the next-generation storage and computing systems to be capable of high performance data-intensive applications, it is necessary to continue innovations in creating new circuits and system architectures, together with searching for new materials and devices.

The memristor, short for memory resistor, is a two-terminal passive device whose resistance is controlled by external electrical signals and exhibits non-volatile memory function. Due to its capabilities of non-volatile resistive memories, nanoscale miniaturization in an ultra-high packing density, and intriguing nonlinear dynamics, the memristor is being widely investigated to create new advanced circuit functions and to complement CMOS systems. The memristor technologies will lead current CMOS-based storage and computing systems to the data-intensive electronic systems, with significantly reduced stand-by power, form factor and manufacturing cost.

Developing compact models for the memristor is essential to facilitate circuit analyses and designs with memristors. While the previously reported memristor models exhibit limitations in the model stability, versatility and adaptability, we propose a new module-based memristor model that covers a wide range of device behaviors. Coincident to the theoretic memristor behaviors, the proposed model uniquely reveals that an effective charge-flux constitutive relationship can always be obtained from various types of memristors. The stability of the proposed model is also significantly enhanced by adapting the new charge (or flux)-based window function.

Associative lookup functions with high throughputs are widely implemented in Ternary Content Addressable Memories (TCAMs). The TCAM holds the potential to curb the latency and power requirements of data-intensive systems. However existing TCAMs that commonly utilize Static Random Access Memories (SRAMs) as the storage units exhibit low storage capacity/density and high cost-per-bit due to bit cells with large areas. We propose a memristor-based ternary content addressable memory (mTCAM) for data-intensive applications. A novel bit cell structure is presented that not only minimizes the bit cell area but also is capable of performance optimizations on the latency and energy consumption. Detailed design issues such as voltage compliance to ensure correct write/search operations, parameter-dependent sensing margins and device variations are also discussed. Circuit level simulations have demonstrated functionalities of the mTCAM. Performance evaluation has shown that mTCAM achieves impressive storage density, search latency and energy consumption. The proposed mTCAM is an attractive candidate in building future computing systems for data-intensive applications.