The Internet of Things (IoT) integrates heterogeneous devices, ranging from sensors to smartphones, tablets and edge servers, and can provide a variety of services, beyond the traditional Internet. Unfortunately, due to its unprecedented scale and ubiquity, IoT faces a maintainability challenge and a set of interrelated problems. With the emergence of edge computing, IoT devices execute various tasks that consume a significant amount of power to deliver high quality of service, which can drain their battery in short time. High peak power increases the device temperature stress, which worsens the impact of transistors and interconnects reliability degradation mechanisms. Such mechanisms lead to early device failures and are costly to fix. In this dissertation, we focus on novel solutions for energy-efficient and reliability-driven management of IoT systems. We introduce a simulation framework called RelIoT to enable reliability evaluation and analysis in IoT networks, which paves the way for the development of new network management solutions. We develop a dynamic reliability management technique based on computation offloading for IoT edge computing architectures. Our approach achieves 20.5% longer mean time to failure than the next best network management solution. We also present an adaptive and distributed reliability-aware routing protocol using reinforcement learning. We show that our routing protocol can improve reliability of a network up to 73.2% compared to state-of-the-art routing approaches. The main focus inall our solutions is to use device batteries efficiently, satisfy QoS requirements, and improve overall network lifetime by mitigating reliability degradation. Lastly, we complement this to specifically study battery health and associated degradation mechanisms, as the traditional techniques developed for optimizing the energy consumption of networks do not yield optimal battery life. An improvement in network lifetime up to 68.5% can be achieved with our approach compared to energy consumption optimization approaches.

The rapid proliferation of data has expedited the development of cutting-edge machine learning (ML) methods, including deep neural networks, to find underlying insights from data. However, the large amount of data movement has driven up energy requirements, posing a new threat of surpassing global energy production. In this dissertation, we explore a novel computing paradigm, hyperdimensional computing (HDC), which mimics attributes of the human brain's neuronal circuits with lightweight arithmetic on low-precision high-dimensional vectors and significantly enhances speed and energy efficiency. We present OpenHD, a GPU-based software infrastructure for HDC, which automatically generates optimized GPU code of HDC applications with the Just-in-Time compilation to ease the development and optimization efforts for the deployment. The proposed framework achieves 4.5x and 146x speedup on average for GPU-based HDC classification and clustering, respectively.

Using OpenHD, we first expand the variety of data structures that can be supported in HDC applications. We design an HDC-based ML solution that supports graph data, and implement it on GPU. Our solution, RelHD, enables graph-based ML by aggregating the relationship between data and features of data into a single high-dimensional vector. We further accelerate RelHD using processing in-memory (PIM) based on FeFET. The PIM accelerator offers 10x speedup and 986x energy efficiency improvement over the state-of-the-art crossbar memory-based graph neural network accelerator.

While most existing HDC-based applications have used small-scale datasets, we show that HDC can scale to tackle the large-scale problem. We present an HDC-based approach called HyperOMS for large-scale open modification spectral library searching (OMS) in mass spectrometry-based proteomics analysis. We develop a novel HDC-based OMS algorithm and accelerate it on GPU using the OpenHD framework. To run HyperOMS efficiently, we devise a DRAM-based PIM accelerator with optimization strategies to maximize parallelism. Evaluation results show that the accelerator yields up to 100x speedup and has 1,337x higher energy efficiency over the state-of-the-art OMS tool running on GPU while offering comparable search quality to competing solutions.

With the rapid development of the Internet of things (IoT), networks, software, and computing platforms, the size of the generated data is dramatically increasing, bringing the dawn of the big data era. These ever-increasing data volumes and complexity require new algorithms and hardware platforms to deliver sufficient performance. Data from sensors, such as images, video, and text, contributed to 2.5 quintillions bytes generated every day in 2020. The rate of generating data is outpacing the computational capabilities of conventional computing platforms and algorithms. CPU performance improvement has been stagnating in recent years, which is one of the causes of the rise of application-specific accelerators that process big data applications. FPGAs are also more commonly used for accelerating big data algorithms, such as machine learning.In this work, we develop and optimize both the hardware implementation and also algorithms for FPGA-based accelerators to increase the performance of machine learning applications. We leverage Residue Number System (RNS) to optimize the deep neural networks (DNNs) execution and develop an FPGA-based accelerator, called Residue-Net, to entirely execute DNNs using RNS on FPGAs. Residue-Net improves the DNNs throughput by 2.8x compared to running the FPGA-based baseline. Even though running DNNs on FPGAs provides higher performance compared to running on general-purpose processors, due to their intrinsic computation complexity, it is challenging to deliver high performance and low energy consumption using FPGAs, especially for the edge devices. Less complex and more hardware-friendly machine learning algorithms are needed in order to revolutionize the performance at and beyond the edge. Hyperdimensional computing (HD) is a great example of a very efficient paradigm for machine learning. HD is intrinsically parallelizable with significantly fewer operations than DNNs, and thus can easily be accelerated in hardware. We develop an automated tool to generate an FPGA-based accelerator, called HD2FPGA, for classification and clustering applications, with accuracy that is comparable to the state-of-the-art machine learning algorithms, but orders of magnitude higher efficiency. HD2FPGA achieves 578x speedup and 1500x energy reduction in end-to-end execution of HD classification compared to the CPU baseline. HD2FPGA, compared to state-of-the-art DNN running on an FPGA, delivers 277x speedup and 172x energy reduction. As the volume of data increases, a single FPGA is not enough to get the desired performance. Thus, many cloud service providers offer multi-FPGA platforms. The size of the data centers workloads varies dramatically over time, leading to significant underutilization of computing resources such as FPGAs while consuming a large amount of power, which is a critical contributor to data center inefficiency. We propose an efficient framework to throttle the power consumption of multi-FPGA platforms by dynamically scaling the voltage and hereby frequency at run time according to the prediction of, and adjustment to the workload level while maintaining the desired Quality of Service (QoS). Our evaluations by implementing state-of-the-art deep neural network accelerators revealed that providing an average power reduction of 4.0x, the proposed framework surpasses the previous works by 33.6% (up to 83%).

The rapidly growing number of edge devices continuously generating data with real-time response constraints coupled with the bandwidth, latency, and reliability issues of centralized cloud computing have made computing near the edge indispensable. As a result, using Field Programmable Gate Arrays (FPGAs) at the edge, due to their unique capabilities that meet the requirements of both high-performance applications and the Internet of Things (IoT) domain, is becoming prevalent. However, designs deployed on these devices suffer from efficiency gap versus custom implementations mainly due to the overhead associated with the FPGAs reconfigurability. This problem is more pronounced in the edge domain, where most devices are battery-powered. In the first part of this dissertation, we identify and overcome the challenges xvi behind the power reduction of FPGA-based applications and propose techniques to lower their energy consumption. Our approach exploits the pessimistic timing margin of the designs to tune the voltage and improves the energy consumption by 66%. An increasing number of edge applications rely on machine learning (ML) algorithms to generate useful insights from data. While modern machine learning techniques – in particular deep neural networks (DNNs) – can produce state-of-the-art results, they often entail substantial memory and compute requirements that may exceed the power and resources available on lightweight error-prone edge devices. Hyperdimensional Computing (HDC) is an emerging lightweight and robust learning paradigm suited for the edge domain that copes with the memory and compute overhead of conventional ML algorithms. The next part of the dissertation proposes efficient FPGA-based and custom hardware implementations of HDC to enable intelligence on devices with limited resources, strict energy constraints, and in noisy environments. The proposed HDC algorithms and accelerators reduce the energy consumption by more than three orders of magnitude compared to other ML solutions, with a comparable or better accuracy. The last part of the dissertation seeks to resolve the privacy concerns of HDC that stem from its reversible algorithm and pose challenges for HDC-based learning and inference. We propose hardware- and communication-efficient techniques that improve the ‘inference’ privacy of HDC by reducing the information of the transferred data while consuming less energy than the non-private baseline. We then show that HDC ‘learning’ can meet tight privacy budgets with negligible accuracy degradation. We also propose a hybrid CNN and HDC model for differentially-private training over image data, which achieves comparable or better accuracy than the state-of-the-art CNN-only methods with more than three orders of magnitude faster training.

The explosive increase in data volume in emerging applications poses grand challenges to computing systems because the bandwidth between compute and memory cannot keep up with exploding data volumes. Processing in memory (PIM) is a promising technology to solve this memory problem by performing some key operations directly in and near memory. There remain several challenges in fully unleashing the power of PIM. Such challenges come from both the software and the hardware sides. On the software side, PIM requires that each operation happens where the data is. As a result, we need to first find the optimal data layout for each application, prior to running it in PIM. On the hardware side, due to the limited functionality of PIM operations, PIM acceleration may require customized logic to achieve high performance. Software-hardware co-design plays a critical role in order to fully exploit PIM acceleration. There are a number of challenges to PIM-based software-hardware co-design. First, software mapping (data layout) in PIM architecture has an extremely large design space. Second, the hardware customization should have minimum overhead to maximize memory capacity. This thesis presents several novel techniques for PIM software-hardware co-design. To tackle the challenges of mapping applications to PIM architecture, this thesis proposes a PIM data layout framework that efficiently optimizes data layout of widely-used machine learning (ML) operators onto general PIM architectures. This thesis also presents the software-hardware co-design for both conventional and non-convolution ML models using PIM architecture. The presented optimizations provide at least 3.7× speedup over conventional PIM acceleration methods. Finally, this thesis proposes a software-hardware co-design for fully homomorphic encryption, which is a challenging and critical application in cryptography, resulting in 30× speedup and energy efficiency over the existing PIM solutions. The target applications in this thesis cover a wide range of challenging operations and data transfer patterns needed for various emerging computing tasks, shedding light on the software-hardware co-design for future systems with PIM acceleration.

The recent advances in Deep Learning (DL) have changed the landscape of Machine Learning significantly. However, over the last few years, we observe that to enable complex applications with DL, an increasing amount of compute resources are required for deployment. This has several undesirable effects ranging from increased deployment costs and energy consumption to harmful environmental effects. Recently, as the community continues to explore alternate paradigms for machine learning, Hyperdimensional Computing (HDC) has gained popularity, due to its low-footprint, low energy consumption, and ease of acceleration on parallel hardware. However, prior work has shown that HDC is not sufficiently accurate on a few complex applications. This dissertation explores the synergy between DL and HDC for improving the compute efficiency of learning while keeping state-of-the-art accuracy. We propose novel hybrid models that leverage the salient properties of HDC and DL to address each methods drawbacks.

We first introduce FHDnn, a hybrid architecture for federated learning that leverages HDC to improve compute efficiency and robustness to unreliable network conditions in IoT Networks. Our architecture uses a frozen pre-trained CNN for feature extraction with a HDC classifier that is trained in a federated manner. By training and transmitting only the HDC classifier, we avoid having to transmit large DNN models and consequently mitigate the robustness issues. Our experiments show that our proposed methods improve communication efficiency by 66$\times$ and are 6$\times$ faster than federated learning with DNNs. We then extend this work to analyze the convergence properties of FHDnn and show that FHDnn provides convergence guarantees for federated learning.

Next, we propose systematic approaches for designing hybrid architectures for different modalities of data. We leverage the binding operator in HDC to capture relationships between data and generate HDC representations that capture these relationships in a single vector. We then use these informative data representations as input to DNN models. This facilitates ease of learning as the model no longer needs to learn relationships in data from scratch. We first consider text data for multi-label classification and demonstrate on large-scale real-world datasets that our proposed architecture is 231$\times$ smaller and 16$\times$ faster compared to SoA models.

Finally, we propose a novel architecture for image classification that leverages the symbolic properties of HDC to structure the problem hierarchically to reduce learning complexity. We construct a label hierarchy by grouping together similar images into groups and generate label representations using HDC, that captures these group relationships. This allows us to break down the classification into 2 stages: detecting the group the image belongs to and identifying the specific label within the group. We show that this improves efficiency by up to 200$\times$ compared to SoA models with minimal loss in accuracy.

Wireless sensor networks (WSNs) form a critical interface between physical and digital worlds by converting physical qualities into measurements which can be used for a wide-ranging spectrum of applications. In the past, these WSNs have been application-specific and exposed only to a limited set of users. Going forward, WSNs will no longer be specialized networks. The new emerging IoT will run multiple applications that have diverse delay requirements for generated and received measurements. In this thesis, we propose a power management framework that operates on the sensing platforms which have multiple power managers. For example, sensors are controlled by sensor controller and battery is managed by battery manager in order to minimize the unnecessary energy consumption. The proposed framework integrates these different power management approaches and optimizes their interactions to achieve optimality in terms of energy efficiency. Proposed approach saves 20\% to 60\% of energy consumption compared to the state of art approaches.

In addition, we propose an optimal transmission manager that supports multiple applications in single-hop wireless sensor networks. We formulate the problem with Markov decision process model and dynamically adjust transmission instances based on random delay requirements of buffered measurements. Then, we propose a distributed transmission manager that leverages the optimal transmission manager to operate in multi-hop WSNs. We implement both transmission managers in ns3 simulator and compare with other state of the art designs. The results show that the proposed transmission manager consumes on average 148.3% less energy than the state of the art approaches while on average having 14.1% fewer measurements that expire.

Lastly, we propose adaptive information dissemination protocol in order to provide information to users in vicinity. Sensors estimate users moving speed and adjust information provision interval in order to save energy. The results show that our proposed approach decrease in power consumption by a factor of 2x to 8x in a single sensor, and 2x to 16x in 10 node sensor network, when compared to the state of art approaches.

With the emergence of the Internet of Things (IoT), devices are generating massive amounts of data. Running machine learning algorithms on IoT devices poses substantial technical challenges due to their limited resources. The focus of this dissertation is to dramatically increase computing efficiency as well as the learning capability of today’s IoT systems by accelerating existing algorithms in hardware and designing new classes of light-weight machine learning algorithms. Our design makes a modification to storage-class memory to support search-based and vector-based computation in memory. We show how this architecture can be used to accelerate deep neural networks in both training and inference phases, resulting in 303× faster and 48× more energy efficient training as compared to the state-of-the-art GPU.

Hardware acceleration alone does not provide all the efficiency and robustness that we need. Therefore, we present Hyperdimensional (HD) computing, an alternative method of learning that implements principles of the functionality in the brain: (i) fast learning, (ii) robustness to noise/error, and (iii) intertwined memory and logic. These features make HD computing a promising solution for today’s embedded devices with limited resources as well as future computing systems in deep nanoscaled technology that have issues of high noise and variability. We exploit emerging technologies to enable processing in-memory which is capable of highly-parallel computation and data movement reduction. Our evaluations show that HD computing provides 39X faster and 56X more energy efficiency as compared to state-of-the-art deep learning accelerator.

Recent years have witnessed a rapid growth in the amount of generated data, owing to the emergence of Internet of Things (IoT). Processing such huge data on traditional computing systems is highly inefficient, mainly due to the limited cache capacity and memory bandwidth. Processing in-memory (PIM) is an emerging paradigm which tries to address this issue. It uses memories as computing units, hence reducing the data transfers between memory and processing cores. However, the application of present PIM techniques is restricted by their limited functionality and inability to process large amounts of data efficiently. In this thesis, we propose novel techniques which exploit the analog properties of emerging memory technologies. Not only do these support more complex functions such as addition, multiplication, and search but also manage and process large data more efficiently. We present a new blocked PIM architecture which uses inter-block interconnects to accelerate data intensive processing. We also introduce a heterogeneous architecture having general purpose cores and PIM-enable memory and a data-dependent task allocation scheme for it. We also apply application specific optimizations and approximation techniques to further design accelerators for neural networks and database query systems. While we design a multiplication-by-constant hardware for neural networks, query processing is accelerated by a novel in-memory nearest search technique. Our neural network accelerator achieves 113.9x higher energy efficiency and 56.3x speedup as compared to AMD GPU. Also, the query accelerator provides 49.3x performance speedup and 32.9x energy savings as compared to recent Intel CPU.

Learning with data from multiple domains is a longstanding topic in machine learning research. In recent years, deep neural networks (DNN) have shown remarkable performance on different machine learning tasks. However, how to efficiently utilize deep neural networks for learning with multiple domains is largely unexploited. A model aware of the relationships between different domains can be trained to work on new domains with fewer resources and achieve better performance. However, to identify and leverage the transferable structure is challenging.

In this dissertation, we propose novel methods which allow efficient learning across multiple domains in several different scenarios. First, we address the problem of learning across two image domains with deep neural networks. We propose two adaptive methods which allow different images to fine-tune and reuse different residual blocks and convolutional filters of the pre-trained model. Experimental results show that the proposed SpotTune outperforms the standard fine-tuning on 12 out of 14 datasets. Second, we consider the case that the target domain only has few examples per category which is referred to as the cross-domain few-shot problem. We establish a new benchmark for cross-domain few-shot learning and propose a multi-model selection algorithm which achieves an average improvement of 2\% compared with the state-of-the-art approach on the proposed benchmark. Third, we consider learning with multiple domains simultaneously. We propose a multi-domain learning method based on depthwise separable convolution which achieves the highest score on the Visual Decathlon Challenge and reduces the number of parameters by 50% compared with the state-of-the-art approach. We further propose an efficient multi-domain learning method for distributed training in sensor networks. The proposed method can reduce the communication cost by up to 53% and energy consumption by up to 67% without accuracy degradation compared with conventional approaches. Finally, we address learning with multiple domains sequentially. We propose an algorithm called mixed stochastic gradient descent (MEGA) which allows the model to maintain the performance on old domains while being trained on a new domain. MEGA achieves an average accuracy of 91.21±0.10% on Permuted MNIST, which is 2% better than the previous state-of-the-art model. On Split CIFAR, the proposed MEGA achieves an average accuracy of 66.12±1.93%, which is about 5% better than the state-of-the-art method.