UC Davis

This dissertation explores various aspects of sampling algorithms and stochastic optimization algorithms. We investigate the efficiency and behavior of various sampling methods for target distributions, particularly those with heavy-tails, through the analysis of different discretization techniques, functional inequalities, and asymptotic limits. In Chapter 2, we study specific diffusion-based sampling algorithms, the randomized midpoint method, for simulating continuous-time Langevin diffusions, establishing its asymptotic normality and providing insights into its behavior. In Chapter 3, we introduce two algorithms to sample heavy-tail targets. In Section 3.2, we study the oracle complexity of sampling from polynomially decaying heavy-tailed target densities using the Transformed Unadjusted Langevin Algorithm (TULA), highlighting connections to functional inequalities. In Section 3.3, by discretizing a class of Itô diffusions associated with weighted Poincaré inequalities, we examine the complexity of sampling from heavy-tailed distributions and provide iteration complexity estimates in terms of the Wasserstein-2 distance. In Chapter 4, we propose the Regularized Stein Variational Gradient Flow, which interpolates between the Stein Variational Gradient Flow and the Wasserstein Gradient Flow, and establish its theoretical properties. We also introduce a particle-based algorithm based the Regularized Stein Variational Gradient Flow and provide preliminary numerical evidence on its improved performance. In Chapter 5, we derive high-dimensional scaling limits and fluctuations for online least-squares Stochastic Gradient Descent (SGD) algorithm by treating the iterates as an interacting particle system, characterizing the limiting mean-square estimation or prediction errors and their fluctuations.