UC Santa Barbara

Gaussian processes (GPs) provide a fundamental tool for statistical inference and experimental design in scientific computing, which provides an important framework for quantifying uncertainty. However, two challenges remain for Gaussian process models of complex real-world data. First, computational and storage costs are prohibitively large when dealing with problems requiring massive amounts of data. Second, reliable predictions, inverse estimation, and feature selection for data with high-dimensional inputs or large noise. This thesis addresses these two challenges by developing novel, efficient statistical approaches and fast algorithms that address these problems. Chapter 1 systematically reviews GPs and their theoretical connections to dynamic linear models and the Kalman filter, which form the foundation of the subsequent scalable inference methods, and active learning strategies for prediction and optimization, particularly for problems with a high-dimensional input space. Chapter 2 introduces the inverse Kalman filter, a novel algorithm that reduces the computational complexity of covariance matrix-vector multiplication from quadratic to linear order, for any covariance matrix induced by dynamic linear models, a broad class of models that include GPs having Matérn covariance with one-dimensional input and half-integer roughness parameter. By integrating the inverse Kalman filter with the conjugate gradient algorithm, we are able to compute the predictive distribution for GPs with a class of additive covariance matrix scalably, with applications in accurately estimating kernels between a large number of interacting particles over a long period of time and interpolating incomplete lattices. Chapter 3 develops data-driven statistical methods for modeling collective cell motion, guiding feature selection in adaptive model construction through composite hypothesis testing to achieve effective parameterized modeling. Chapter 4 proposes a physics-informed machine learning method for automatically identifying the phase structure of block copolymers. The selected feature set is independent of the chemical composition of the materials, allowing the model to generalize to block copolymers with new monomers. Chapter 5 proposes an active learning framework with probabilistic error control, applicable to high-dimensional input space, providing a reliable solution for surrogate modeling of expensive simulations. Finally, Chapter 6 provides an outlook on future research directions.