UC Santa Barbara

Modern data sets are large and complicated. The demand for understanding the nature of such big data has motivated the development of high-dimensional statistics. Understanding covariance matrices and their eigenstructure plays a central role in high-dimensional statistics. In this thesis, we examine the critical role of the eigenstructure of data covariance matrices in two popular high-dimensional problems. Understanding the covariance matrices and their eigenstructure is essential for both problems. First, we focus on overparameterized machine learning. This line of research is motivated by the empirical observation that deep neural networks can generalize well despite learning a number of parameters that far exceeds the size of the training set. Our work contributes to this field by proving that analogous behaviors can be observed in simpler, binary and multiclass linear classification models. We show the equivalence between support-vector machines (SVM) and the interpolating classifiers. We derive generalization bounds and characterize the role of regularization in the overparameterized regime. Our bounds reveal that the feature covariance matrix plays a central role in guaranteeing good generalization under overparameterization.Specifically, our analysis is the first to demonstrate this for multiclass rather than binary classification. The second topic is Gaussian graphical models (GGM). GGM are widely used to estimate network structures in several applications. Specifically, estimating graph structures accurately is challenging when latent confounders exist. In this work, we theoretically compare two commonly used methods that can remove latent confounders when estimating GGM. The theory depends heavily on the analysis on feature covariance matrices and their inverse. Our results reveal that the eigenstructure of feature covariance matrices is crucial to determine the performance of different methods. Based on the theory, we propose a new method that combines the strengths of previous approaches. We demonstrate the effectiveness of our methodology with simulations in two real-world applications.