UC Irvine

Markov chain Monte Carlo (MCMC) methods have been widely used in Bayesian inference involving intractable probabilistic models. However, simple MCMC algorithms (e.g., random walk Metropolis and Gibbs sampling) are notorious for their lack of computational efficiency in complex, high-dimensional models and poor scaling to large data sets. In recent years, many advanced MCMC methods (e.g., Hamiltonian Monte Carlo and Riemannian Manifold Hamiltonian Monte Carlo) have been proposed that utilize geometrical and statistical quantities from the model in order to explore the target distribution more effectively. The gain in the efficacy of exploration, however, often comes at a significant computational cost which hinders their application to problems with large data sets or complex likelihoods.

In practice, it remains challenging to design scalable MCMC algorithms that can balance computational complexity and exploration efficacy well. To address this issue, some recent algorithms rely on stochastic gradient methods by approximating full data gradients using mini-batches of data. In contrast, this thesis focuses on accelerating the computation of MCMC samplers based on various surrogate methods via exploring the regularity of the target distribution.

We start with a precomputing strategy that can be used to build efficient surrogates in relatively low-dimension parameter spaces. We then propose a random network surrogate architecture which can effectively capture the collective properties of large data sets or complex models with scalability, flexibility and efficiency. Finally, we provide a variational perspective for our random network surrogate methods and propose an approximate inference framework that combines the advantages of both variational Bayes and Markov chain Monte Carlo methods. The properties and efficiency of our proposed methods are demonstrated on a variety of synthetic and real-world data problems.