UC San Diego

Deep Generative Models are a kind of unsupervised deep learning methods that learn the data distribution from samples and then generate unseen, high-quality samples from the learned distributions. These models have achieved tremendous success in different domains and tasks. However, many questions are not well-understood for these models. In order to better understand these models, in this thesis, we investigate the following questions: ($i$) what is the representation power of deep generative models, and ($ii$) how to identify and mitigate trustworthy concerns in deep generative models.

We study the representation power of deep generative models by looking at which distributions they can approximate arbitrarily well. we study normalizing flows and rigorously establish bounds on their expressive power. Our results indicate that some basic flows are highly expressive in one dimension, but in higher dimensions their representation power may be limited, especially when the flows have moderate depth. We then prove residual flows are universal approximators in maximum mean discrepancy and provide upper bounds on the depths under different assumptions.

We next investigate three different trustworthy concerns. The first is how to explain the black box neural networks in these models. We introduce VAE-TracIn, a computationally efficient and theoretically sound interpretability solution, for VAEs. We evaluate VAE-TracIn on real world datasets with extensive quantitative and qualitative analysis.

The second is how to mitigate privacy issues in learned generative models. We propose a density-ratio-based framework for efficient approximate data deletion in generative models, which avoids expensive re-training. We provide theoretical guarantees under various learner assumptions and empirically demonstrate our methods across a variety of generative methods.

The third is how to prevent undesirable outputs from deep generative models. We take a compute-friendly approach and investigate how to post-edit a pre-trained model to \textit{redact} certain samples. We consider several unconditional and conditional generative models and various types of descriptions of redacted samples. Extensive evaluations on real-world datasets show our algorithms outperform baseline methods in redaction quality as well as robustness while retaining high generation quality.