UC Santa Barbara

In the field of neuromorphic VLSI connectivity is a huge bottleneck in implementing brain-inspired circuits due to the large number of synapses needed for performing brain-like functions. (E.g. pattern recognition, classification, etc.). In this thesis I have addressed this problem using a two pronged approach namely spatial and temporal.

Spatial: The real-estate occupied by silicon synapses have been an impediment to implementing neuromorphic circuits. In recent years, memristors have emerged as a nano-scale analog synapse. Furthermore, these nano-devices can be integrated on top of CMOS chips enabling the realization of dense neural networks. As a first step in realizing this vision, a programmable CMOS chip enabling direct integration of memristors was realized. In a collaborative MURI project, a CMOS memory platform was designed for the memristive memory array in a hybrid/3D architecture (CMOL architecture) and memristors were successfully integrated on top of it. After demonstrating feasibility of post-CMOS integration of memristors, a second design containing an array of spiking CMOS neurons was designed in a 5mm x 5mm chip in a 180nm CMOS process to explore the role of memristors as synapses in neuromorphic chips.

8

Temporal: While physical miniaturization by integrating memristors is one facet of realizing area-efficient neural networks, on-chip routing between silicon neurons prevents the complete realization of complex networks containing large number of neurons. A promising solution for the connectivity problem is to employ spatio-temporal coding to encode neuronal information in the time of arrival of the spikes. Temporal codes open up a whole new range of coding schemes which not only are energy efficient (computation with one spike) but also have much larger information capacity than their conventional counterparts. This can result in reducing the number of connections to do similar tasks with traditional rate-based methods.

By choosing an efficient temporal coding scheme we developed a system architecture by which pattern classification can be done using a “Winners-share-all” instead of a “Winner-takes-all” mechanism. Winner-takes-all limits the code space to the number of output neurons, meaning n output neurons can only classify n pattern. In winners-share-all we exploit the code space provided by the temporal code by training different combination of k out of n neurons to fire together in response to different patterns. Optimal values of k in order to maximize information capacity using n output neurons were theoretically determined and utilized. An unsupervised network of 3 layers was trained to classify 14 patterns of 15 x 15 pixels while using only 6 output neurons to demonstrate the power of the technique. The reduction in the number of output neurons results in the reduction of number of training parameters and results in lower power, area and memory required for the same functionality.