UC San Diego

Artificial Intelligence (AI) has been extensively applied across various fields due to its exceptional performance. Deep Neural Networks (DNNs) are a subset of machine learning models inspired by the structure and function of the human brain. They consist of multiple layers of interconnected nodes (neurons) that can learn complex data representations through training on large datasets. However, DNNs are vulnerable to Trojan attacks, especially for constrained, real-time, security-sensitive applications.

This thesis focuses on designing DNN algorithms and architectures to enhance their robustness, enabling safer applications. Using different approaches, we effectively advance DNNs' trustworthiness, Trojan detection, and performance acceleration.

As AI technology evolves, security concerns are receiving increasing attention. This thesis advances the field by addressing the limitations and concerns of the latest AI technologies through the design of domain-specific DNN algorithms and systems.

This dissertation integrates theoretical foundations, domain-specific architecture design, and automated tools to facilitate the co-optimization of deep learning algorithms with the underlying platform while meeting various constraints. The key contributions of this dissertation are as follows:

• Proposing DeepTD, the first FPGA-based accelerator architecture for efficient DNN Trojan Detection. DeepTD significantly improves state-of-the-art works regarding both latency and memory efficiency for the same detection threshold. Proof of concept realization demonstrates up to 60x faster detection time than state-of-the-art CPU and GPU realizations.

• Devising AdaTest with a Software/Hardware co-design principle and providing an optimized on-chip architecture solution. AdaTest’s architecture minimizes the hardware overhead in two ways: (i) Deploying circuit emulation on programmable hardware to accelerate reward evaluation of the test input; (ii) Pipelining each computation stage in AdaTest by automatically constructing auxiliary circuit for test input generation, reward evaluation, and adaptive sampling. We evaluate AdaTest’s performance on various HT benchmarks and compare it with two prior works that use logic testing for HT detection. Experimental results show that AdaTest engenders up to two orders of test generation speedup and two orders of test set size reduction compared to the prior works while achieving the same or higher Trojan detection rate.

• Developing a lightweight cryptographic protocol explicitly designed to exploit the unique characteristics of Binary Neural Networks(BNNs) and presenting an advanced dynamic exploration of the runtime-accuracy tradeoff of scalable BNNs in a single-shot training process. While previous works trained multiple BNNs with different computational complexities (which is cumbersome due to the slow convergence of BNNs), we trained a single BNN that can perform inference under various computational budgets. Compared to CryptFlow2, the state-of-the-art technique in the oblivious inference of non-binary DNNs, our approach reaches 3× faster inference while keeping the same accuracy. Compared to XONN, the state-of-the-art technique in the oblivious inference of binary networks, we achieve 2× to 12× faster inference while obtaining higher accuracy.

• Establishing the first private robustness check that uses high break point rank-based statistics on aggregated model updates. By exploiting randomized clustering, we significantly improve the scalability of our defense without compromising privacy. We leverage the derived statistical bounds in zero-knowledge proofs to detect and remove malicious updateswithout revealing private user updates. Our novel framework, zPROBE, enables Byzantine resilient and secure federated learning. We show the effectiveness of zPROBE on several computer vision benchmarks. Empirical evaluations demonstrate that zPROBE provides a low-overhead solution to defend against state-of-the-art Byzantine attacks while preserving privacy.