Deep unsupervised learning has emerged as a promising alternative to supervised approaches. However, supervised learning needs a tremendous amount of information in the form of annotations on specific pre-defined tasks. In contrast, human learning requires much fewer annotations and is flexible. Recent research efforts have been motivated to explore different deep unsupervised learning algorithms to leverage the massive unlabeled data for various applications that move beyond the supervised learning setting. While recent deep unsupervised learning works have shown their success in representation learning, clustering, and anomaly detection, many challenges remain unsolved. For example, how to improve the quality of learned representations used for downstream applications (the quality of learned representations challenge)? How to interpret and understand the deep unsupervised learning model predictions (the explainability challenge)? Is there any risk of bias for deep unsupervised learning applications (the bias and fairness challenge)?

To gain insights into the aforementioned challenges, we propose a broad range of novel techniques to address them. Each injects human-level knowledge into deep unsupervised learning. To be specific, this dissertation presents five approaches. The first two address the quality of representation challenge, the third the explainability challenge, and the last two the bias and fairness challenges. Our first formulation introduces a deep constrained clustering framework that enhances clustering performance via various constraints. Our second formulation is a self-supervised representation learning framework that automatically discovers and differentiates different categories. The third formulation simultaneously performs representation learning for clustering and describing the generated clusters with semantic tags associated with the clustered instances. Our fourth formulation proposes a novel deep fair anomaly detection architecture that uses adversarial learning to inject human fairness rules. Finally, our fifth formulation enforces disparate impact rules into deep clustering models via minimal modification learning. These methods are unified in modernizing deep unsupervised learning with different types of human guidance.

In the last decade, thanks to the notable progress of Deep Learning (DL), Artificial Intelligence (AI) has achieved tremendous advancement. However, the lack of explainability has limited the wide use of deep models in real-world applications due to its black-box nature. This inherent defect of black-box AI systems confronts people with but is not limited to issues such as accessibility, improveability, and accountability. The paradigm underlying these problems falls into the so-called explainable AI (XAI) field, which is widely acknowledged as a crucial feature for the practical deployment of AI models. This dissertation presents a comprehensive study of the advanced progress of XAI in multiple aspects from parameter space to input feature space and provides practical algorithms. The research combines concept taxonomy, algorithm designs, and empirical studies in various deep-learning systems.

The contents of the dissertation are structured as follows. Chapter 1 motivates the necessity of the XAI module in a deep learning system. Based on the space they achieve explainability, we taxonomize the general XAI approaches into two genres: explanation of the input space and explanation of the parameter space. We sketch the state-of-the-art progress in this field and introduce the approaches in detail that are closely related to the works we present in the following chapters. Chapter 2 focuses on one of the most popular parameter space explanations -- loss landscapes. Given the efficacy of loss landscapes in model diagnosis, we devise novel visualization tools to provide synthesized insights for researchers. Chapter 3 comprises two post-hoc explanation works. One local explanation method, Shapley Value Integrated Gradients (SIG) is an extension of the Integrated Gradient algorithm. Another work focuses on benchmarking the existing post-hoc XAI methods in low signal-to-noise ratio environments. Chapter 4 applies post-hoc explanations on the machine learning domain to increase the trustworthiness and guide machine learning system development. We introduce how to use feature ablation to justify the deep learning system for clinical prognosis problems using task-fMRI data. Besides, we combine the large language models with exemplar selection algorithm to explain unsupervised clustering in topic modeling.

This dissertation provides an overview of the research completed at the moment of writing, which captures the essence of the research and its significance.

Artificial intelligence (AI), if used correctly, has the capacity to improve human life by automating procedures that previously required human expertise and precision, particularly those that may have a great impact on people’s lives and where the cost of a mistake is high. Unfortunately, the use of machine learning (ML) algorithms carries with it certain risks that may limit their applicability in such sensitive domains. Particularly, ML algorithms solve tasks by optimizing a complex non-linear mapping between an input and output space. While the automated process of tuning this function is powerful, it ultimately renders these learners uninterpretable and subject to error, misuse, or harmful bias.The fields of explainable artificial intelligence (XAI) and fair machine learning exist to combat these issues. XAI seeks to explain how ML agents operate in human-interpretable terms, while fairness aims to correct or avoid potential unfair outcomes. While existing work has laid promising groundwork toward these ends, there are several limitations in both domains that should be rectified before AI can be trusted for particularly sensitive tasks. This dissertation aims to extend XAI and fair machine learning by making headway on these limitations. For XAI, we create approaches that explain the entire model, not just individual actions, we develop techniques tailored towards ML tasks beyond supervised learning, and we examine alternatives to input space as the means of providing that explanation. For fairness, we look to the literature in social sciences to create fair ML algorithms that match the models of how unfairness and discrimination occur, which we argue are superior to existing techniques that do not leverage this theory. Finally, we introduce the novel concept of machine-to-machine explanation: the idea that explanation technology can be used for additional computational tasks, enabling collaboration among ML models to improve their performance.