UC Irvine

Probabilistic generative models offer a principled way to represent and learn complex data distributions. By defining a joint probability distribution over the data, these models enable a wide array of inference queries, such as imputation of missing data given observations of other data dimensions. State-of-the-art deep generative models, including Variational Autoencoders (VAEs) and Denoising Diffusion Models, integrate deep learning methods to better model complex data like images. Despite rapid advances in learning deep generative models, accurate estimation of inference queries remains a significant challenge, and standard methods perform poorly in many scenarios. VAEs typically employ learning-based amortized inference, which results in an inflexible model that struggles to handle queries it has not been trained on, and requires expensive retraining of the inference network for each new query task. Diffusion models often require heuristics to apply sampling-based inference, leading to slow convergence and highly inaccurate predictions for hard inference queries like large-mask image inpainting.

In cases where data sensitivity is crucial, we often seek not only accurate predictions, but estimates of posterior uncertainty. To address these challenges, we propose leveraging classical optimization-based variational inference that adapts to versatile queries, showcasing its efficacy for two example inference queries: 1) estimating partition functions in discrete graphical models, and 2) imputing missing features with VAEs and diffusion models. Through both theoretical analysis and empirical validation, we demonstrate that our approach significantly enhances the accuracy of inference while capturing posterior uncertainty, making it suitable for real-world user interactions. This thesis thus lays the foundation for further research into scalable and accurate inference techniques, and facilitates the practical deployment of probabilistic generative models in increasingly complex and high-dimensional domains.