UCLA

Modern machine learning techniques rely heavily on iterative optimization algorithms to solve high dimensional estimation problems involving non-convex landscapes. However, in the absence of knowing the closed-form expression of the solution, analyzing statistical properties of the estimators remains challenging in most cases. This dissertation provides a framework, called Multi-layer Vector Approximate Message Passing (ML-VAMP), for analyzing optimization-based estimators for a broad class of inverse problems. This framework is based on new developments in random matrix theory. Importantly, it does not rely on convex analysis and applies more broadly to non-convex optimization problems.

The ML-VAMP framework enables exact analysis in a certain high dimensional asymptotic regime for several problems of interest in machine learning and signal processing. In particular, the following problems have been explored in some detail,- Reconstruction of signals from noisy measurements using deep generative models, - Generalization error of learned one-layer and two-layer neural networks, \label{prob:nn} to demonstrate the analytical capabilities of the framework.

Using this framework we can analyze the effect of important design choices such asdegree of overparameterization, loss function, regularization, initialization, feature correlation, and a mismatch between train and test data in several problems of interest in machine learning.