With the rapid advancements in quantum hardware, we stand on the brink of the large-scale quantum computing (LSQC) era, poised to harness the power of thousands of even millions of noisy qubits. This heralds a transformative period where the computational prowess of LSQC holds promises for addressing practical scientific challenges, notably in the realms of molecule simulation and drug discovery. However, despite this exciting progress, a noticeable gap persists between the current Noisy Intermediate-Scale Quantum (NISQ)-era ecosystem and the full potential of LSQC. On the one hand, the scale of LSQC makes manual hardware/software optimizations untenable. On the other hand, the noisy nature of quantum hardware, accompanying the instruction scale of large quantum applications, will surely destroy the quantum computing outcome. These aspects of LSQC induces a series of challenges and requirements not well optimized and supported by NISQ computing stack: automation, scalability, and robustness.

The dissertation aims to fill this gap and synthesize a full-stack design for the LSQC computing stack. The proposed framework presents systematic optimizations for quantum software and quantum architecture. The vision is that significant advances in LSQC require full-stack infrastructure that could not only vertically integrate the advances from both the higher problem level and the lower hardware level. Concerning rapidly evolving problem size and hardware magnitude, the value of the proposed design will only increase. It provides a clearer road-map for building the LSQC in the near future. The evaluation demonstrates the superiority of the proposed framework.

Big data analytics enjoy increasingly wide applications in the real world enabled by the development of model, data, and hardware. However, the development of these three components usually shows a significant imbalance. While there are many new models and data, commercialized hardware usually provides only limited support. My research mitigates this gap by building systems for efficient big data analytics that stitch model, data, and hardware together. In particular, we find that given specialized hardware with limited compute primitives, system optimizations are the keys to generalizing such specialized hardware and efficiently supporting diverse model and data workload.

This thesis discusses systems for efficient big data analytics under three aspects. The first aspect introduces hardware-aware kernel tuning. Based on limited hardware compute primitives, my research builds more middle-level libraries to efficiently support diverse big data analytic workloads. The second aspect proposes runtime systems for efficient neural network inference. While larger neural networks are used for general workload, a specialized workload is usually observed in a specific scenario. My research builds runtime systems to automatically detect the scenario information during runtime and exploit such information for efficient big data analytics. The third aspect is to build secure deep learning frameworks to efficiently support diverse workloads such as zero-knowledge neural networks and neural network verification. My research abstracts the key computing patterns in secure deep learning and automatically optimizes diverse NN operators with framework supports.

With the end of Moore’s Law and the rise of compute- and data-intensive deep learning (DL) applications, the focus on arduous new processor design has shifted towards a more effective and agile approach: Intelligent Software to maximize the performance gains of DL hardware like GPUs. There are several highlights of such intelligent software design. First, it would maximize the execution efficiency of existing and emerging DL algorithms on powerful platforms like GPUs. Second, it would promote the adaptiveness of systems to handle a diverse range of inputs. Third, it would maintain sufficient portability and scalability across a diverse range of platforms, such as mobile devices and high-performance clusters.

In this thesis, I will first highlight the importance of software innovation to bridge the gap between the increasingly diverse DL applications and the existing powerful DL hardware platforms. The second part of my thesis will recap my research work on DL system software innovation, focusing on 1) Precision Mismatch between DL applications and high-performance GPU units like Tensor Cores (e.g., QGTC [PPoPP ’22] and APNN-TC [SC ’21]), to improve the efficiency of quantized deep learning on powerful GPU platforms, and 2) Computing Pattern Mismatch between the sparse and irregular DL applications, such as Graph Neural Networks, and the dense and regular tailored GPU computing paradigm (e.g., GNNAdvisor [OSDI ’21] and MGG [OSDI ’23]), to highlight system adaptability and scalability. Finally, I will conclude this thesis with my vision and future work for building efficient, scalable, and secure DL systems.

Both static and streaming graph processing are central in data analytics scenarios such as recommendation systems, financial fraud detection, and social network analysis. The rich space of graph applications poses several challenges for the computer architecture community. First, standard static graph algorithm performance is sub-optimal on today's general-purpose architectures such as CPUs due to inefficiencies in the memory subsystem. It is currently increasingly difficult to rely on relative compute/memory technology scaling for continued performance improvement for a given optimized static graph algorithm on a general-purpose CPU. Second, while a large body of research in the computer architecture community focuses on static graph workloads, streaming graphs remain completely unexplored. The primary practical barriers for computer architecture researchers toward studying streaming graphs are immature software, a lack of systematic software analysis, and an absence of open-source benchmarks. This dissertation seeks to solve these challenges for both static and streaming graph workloads through benchmarking, performance analysis, and CPU-centric domain-specific architectures using software/hardware co-design.

For static graph workloads, this thesis highlights novel performance bottleneck insights such as 1) the factors limiting memory-level parallelism, 2) the heterogeneous reuse distances of different application data types, and 3) the difference in the performance sensitivities of the different levels of the cache hierarchy. Guided by the workload characterization, a domain-specific prefetcher called DROPLET is proposed to solve the memory access bottleneck. DROPLET is a physically decoupled but functionally cooperative prefetcher co-located at the L2 cache and at the memory controller. Moreover, DROPLET is data-aware because it prefetches different graph data types differently according to their intrinsic reuse distances. DROPLET achieves 19%-102% performance improvement over a no-prefetch baseline and 14%-74% performance improvement over a Variable Length Delta Prefetcher (VLDP). DROPLET also performs 4%-12.5% better than a monolithic L1 prefetcher similar to the state-of-the-art prefetcher for graphs.

For streaming graph workloads, this thesis develops a performance analysis framework called SAGA-Bench and performs workload characterization at both the software and the architecture levels. The findings include 1) the performance limitation of the graph update phase, 2) the input-dependent software performance trade-offs in graph updates, and 3) the difference in architecture resource utilization (core counts, memory bandwidth, and cache hierarchy) between the graph update and the graph compute phases. In addition, the thesis proposes the SPRING approach to demonstrate that input knowledge-driven software and hardware co-design is critical to optimize the performance of streaming graph processing. Evaluated across 260 workloads, our input-aware techniques provide on average 4.55x and 2.6x improvement in graph update performance for different input types. The graph compute performance is improved by 1.26x (up to 2.7x).

Big Data refers to the massive and rapidly growing data in our daily life, which can be very helpful to dig valuable information and make better decisions. However, handling big data workloads poses new challenges in traditional memory subsystems due to the memory wall and power wall issues. Moving a large amount of data from memory to the processor is expensive, which can cause severe performance bottleneck and high energy consumption. Moreover, while the computation capability of modern processors grows fast with Moore’s Law, the bandwidth and latency of memory improve much slower due to the I/O’s physical constraints.

To address the memory wall challenge for big data applications, this dissertation aims to answer the following two questions: How can we improve the existing memory tech- nology, and what should future memory look like? For the first question, we investigate the approach of near-data processing (NDP). NDP technique adopts custom compute logic near the data storage, which leverages the much higher internal bandwidth and reduces the energy consumption by data movements. Second, we envision the in-memory processing (IMP) technique for next-generation memory. In particular, we study the Resistive Random Access Memory (RRAM), an analog device with tunable resistance. We use RRAM as both storage and computation hardware to process data in-situ and analyze its mapping and reliability issues.

With the continuous booming development of deep learning, many kinds of model variants are being proposed to tackle more difficult machine learning tasks, such as Transformers, Deep Learning Recommendation Models, and Graph Neural Networks. These emerging deep learning models, while being sufficiently better than prior methods, also require much more hardware resources to train and deploy. To systematically tackle this issue, we approach it from a data-centric perspective and argue that the root cause of the software-hardware imbalance is the data explosion in emerging deep learning models.

To continue the scaling of deep learning applications and bridge the gap between hardware performance and application requirements, this dissertation proposes to leverage data redundancy to effectively reduce model cost and benefit hardware design. We first categorize the data in a deep learning model into three types, namely input dataset, model parameters, and computational results. While parameter redundancy has been extensively studied in prior work, data representation and computational redundancy are rarely discussed. On the base of this observation, we introduce four software-hardware co-designs to explore the other two types of data redundancy and thus improve deep learning efficiency. Specifically, in order to reduce the cost of intermediate computational results in Transformer models, the first two designs leverage dynamic runtime approximation with customized GPU kernel and ASIC design. To release the memory and computation burden caused by massive input training data, the third and fourth design focuses on using high-order tensor decomposition with domain-specific knowledge to achieve high-quality and aggressive data compression. Training on the compressed dataset leads to comparable model accuracy with much less hardware consumption.

Overall, this dissertation demonstrates opportunities and approaches to tackle data explosion in emerging deep learning models. Our methods cover both dynamically generated data and offline trainable data, both deep learning training and inference, and both general computing platforms (e.g., GPGPU) and customized accelerators.

As the new ``race to the moon'', quantum computing can possibly trigger a computation revolution due to its strong potential in several important domains, e.g., cryptography, chemistry simulation, optimization, and machine learning. However, as an emerging research area, grand challenges remain ahead since state-of-the-art quantum computing, from software to hardware, is still highly immature. This dissertation explores high-performance, efficient, and reliable quantum computing systems, and strikes a synergy among different technology stacks, including application, programming language, compiler optimization, hardware architecture design, and simulation. In particular, this dissertation focuses on two directions: 1) cross-layer co-design for quantum computing system; and 2) enabling deep quantum software/compiler optimizations at the high level. In the first direction, this dissertation studies how to efficiently map quantum software to hardware via carefully designed compiler optimization, and then investigates the application-specific architecture design with substantial hardware efficiency improvement. Following the application-specific principle and putting the algorithm optimization and hardware design together, this dissertation proposed a software-hardware co-optimization for chemistry simulation and achieved a wide range of benefits across multiple system stacks. In the second direction, this dissertation explores leveraging the algorithmic information, which is usually carried by new high-level programming languages, to design quantum software optimizations that are hard to implement in conventional quantum software infrastructures. These optimizations include a Pauli-string-based intermediate representation for large-scope compiler optimization on quantum simulation programs, a projection-operator-based runtime assertion language for efficient quantum program testing and debugging, and a trial scheduling technique to identify and eliminate redundant computation in noisy quantum computing simulation.

Machine Learning (ML) techniques, especially Deep Neural Networks (DNNs), have been driving innovations in many application domains. These breakthroughs are powered by the computational improvements in processor technology driven by Moore’s Law. However, the need for computational resources is insatiable when applying ML to large- scale real-world problems. Energy efficiency is another major concern of large-scale ML. The enormous energy consumption of ML models not only increases costs in data-centers and decreases battery life of mobile devices but also has a severe environmental impact. Entering the post-Moore’s Law era, how to keep up performance and energy-efficiency with the scaling of ML remains challenging.This dissertation addresses the performance and energy-efficiency challenges of ML. The thesis can be encapsulated in a few questions. Do we need all the computations and data movements involved in conventional ML processing? Does redundancy exist at the hardware level? How can we better approach large-scale ML problems with new computing paradigms? This dissertation presents how to explore the elasticity in ML pro- cessing and hardware architectures: from the algorithm perspective, redundancy-aware processing methods are proposed for DNN training and inference, as well as large-scale classification problems and long-range Transformers; from the architecture perspective, balanced, specialized, and flexible designs are presented to improve efficiency.