UC Davis

In the last decade, thanks to the notable progress of Deep Learning (DL), Artificial Intelligence (AI) has achieved tremendous advancement. However, the lack of explainability has limited the wide use of deep models in real-world applications due to its black-box nature. This inherent defect of black-box AI systems confronts people with but is not limited to issues such as accessibility, improveability, and accountability. The paradigm underlying these problems falls into the so-called explainable AI (XAI) field, which is widely acknowledged as a crucial feature for the practical deployment of AI models. This dissertation presents a comprehensive study of the advanced progress of XAI in multiple aspects from parameter space to input feature space and provides practical algorithms. The research combines concept taxonomy, algorithm designs, and empirical studies in various deep-learning systems.

The contents of the dissertation are structured as follows. Chapter 1 motivates the necessity of the XAI module in a deep learning system. Based on the space they achieve explainability, we taxonomize the general XAI approaches into two genres: explanation of the input space and explanation of the parameter space. We sketch the state-of-the-art progress in this field and introduce the approaches in detail that are closely related to the works we present in the following chapters. Chapter 2 focuses on one of the most popular parameter space explanations -- loss landscapes. Given the efficacy of loss landscapes in model diagnosis, we devise novel visualization tools to provide synthesized insights for researchers. Chapter 3 comprises two post-hoc explanation works. One local explanation method, Shapley Value Integrated Gradients (SIG) is an extension of the Integrated Gradient algorithm. Another work focuses on benchmarking the existing post-hoc XAI methods in low signal-to-noise ratio environments. Chapter 4 applies post-hoc explanations on the machine learning domain to increase the trustworthiness and guide machine learning system development. We introduce how to use feature ablation to justify the deep learning system for clinical prognosis problems using task-fMRI data. Besides, we combine the large language models with exemplar selection algorithm to explain unsupervised clustering in topic modeling.

This dissertation provides an overview of the research completed at the moment of writing, which captures the essence of the research and its significance.