In recent years, the L-1 regularization has been extensively used to estimate a sparse precision matrix and encode an undirected graphical model. Because the regularized estimates are biased, the application of the graphical models has largely been restricted and has been ignored in many areas. In this work, we show that graphical models and their regularized estimates can be useful in the area of finance. We present our discussion in three parts. First, we propose a graphical representation model for the asset returns. The model captures observed variance in the equity market endogenously and offers a new perspective on the covariance estimation for asset returns. We show that such a model may provide a straightforward interpretation for investors regarding investment decision making. Second, we show that regularized estimates of graphical models, though biased, are useful to estimate the minimum variance portfolio and determine the portfolio rebalancing strategy. Third, we discuss the algorithms of solving one of the graphical models -- the graphical Concord. We present the software development process for the graphical Concord and illustrate the usage of the packages with some examples.

Graphical models, which utilize graphs to encode conditional independencies between random variables, have proven to be versatile tools for encoding multivariate distributions. In addressing some practical considerations for using graphical models in applications, we developed two graphical model methods. Firstly, we develop the FROSTY method for directed graphical models focusing on scalability that performs well in simulation. FROSTY only requires observational data and user-specified confidence level as inputs and can estimate networks with thousands of variables. Notably, FROSTY does not require computationally expensive cross-validation for model selection. We learned that initial input and parameter tuning are important in estimation, and empirical evidence illustrates that other methods can be improved by adopting FROSTY's initial step. Secondly, we develop the ACCORD method for undirected graphical models based on pseudo-likelihood. For optimization, we propose a novel objective splitting that ensures linear convergence with a wide range of step size in both the objective values and the iterates despite the lack of strong convexity in ACCORD's objective function. Additionally, we implement a high-performance variant of ACCORD, named HP-ACCORD, that can scale up to 1 million variables utilizing communication-avoiding linear algebra on distributed computing environments.

Building on a recent framework for distributionally robust optimization, we consider

estimation of the inverse covariance matrix for multivariate data. We provide a novel

notion of a Wasserstein ambiguity set specifically tailored to this estimation problem,

leading to a tractable class of regularized estimators. Special cases include penalized

likelihood estimators for Gaussian data, specifically the graphical lasso estimator. As a

consequence of this formulation, the radius of the Wasserstein ambiguity set is directly

related to the regularization parameter in the estimation problem. Using this relationship, the level of robustness of the estimation procedure can be shown to correspond to

the level of confidence with which the ambiguity set contains a distribution with the population covariance. Furthermore, a unique feature of our formulation is that the radius

can be expressed in closed-form as a function of the ordinary sample covariance matrix.

Taking advantage of this finding, we develop a simple algorithm to determine a regularization parameter for graphical lasso, using only the bootstrapped sample covariance

matrices, meaning that computationally expensive repeated evaluation of the graphical

lasso algorithm is not necessary. Alternatively, the distributionally robust formulation

can also quantify the robustness of the corresponding estimator if one uses an off-the-shelf

method such as cross-validation. Finally, we numerically study the obtained regularization criterion and analyze the robustness of other automated tuning procedures used in practice.

The majority of methods for sparse precision matrix estimation rely on computationally expensive procedures, such as cross-validation, to determine the proper level of regularization.Recently, a special case of precision matrix estimation based on a distributionally robust optimization (DRO) framework has been shown to be equivalent to the graphical lasso. From this formulation, a method for choosing the regularization term, i.e., for graphical model selection, without tuning was proposed. In Chapter 2 of this thesis, we establish a theoretical connection between the confidence level of graphical model selection via the DRO formulation and the asymptotic family-wise error rate of estimating false edges. Simulation experiments and real data analyses illustrate the utility of the asymptotic family-wise error rate control behavior even in finite samples.\par Next, we propose a completely tuning-free approach to estimating sparse precision matrix based on linear regression in Chapter 3. Theoretically, the proposed estimator is minimax optimal under various norms. In addition, we propose a second-stage enhancement with non-convex penalties, which possesses strong oracle properties. We assessed our proposed methods through comprehensive simulations and real data application on human gene network analysis.

Modern data sets are large and complicated. The demand for understanding the nature of such big data has motivated the development of high-dimensional statistics. Understanding covariance matrices and their eigenstructure plays a central role in high-dimensional statistics. In this thesis, we examine the critical role of the eigenstructure of data covariance matrices in two popular high-dimensional problems. Understanding the covariance matrices and their eigenstructure is essential for both problems. First, we focus on overparameterized machine learning. This line of research is motivated by the empirical observation that deep neural networks can generalize well despite learning a number of parameters that far exceeds the size of the training set. Our work contributes to this field by proving that analogous behaviors can be observed in simpler, binary and multiclass linear classification models. We show the equivalence between support-vector machines (SVM) and the interpolating classifiers. We derive generalization bounds and characterize the role of regularization in the overparameterized regime. Our bounds reveal that the feature covariance matrix plays a central role in guaranteeing good generalization under overparameterization.Specifically, our analysis is the first to demonstrate this for multiclass rather than binary classification. The second topic is Gaussian graphical models (GGM). GGM are widely used to estimate network structures in several applications. Specifically, estimating graph structures accurately is challenging when latent confounders exist. In this work, we theoretically compare two commonly used methods that can remove latent confounders when estimating GGM. The theory depends heavily on the analysis on feature covariance matrices and their inverse. Our results reveal that the eigenstructure of feature covariance matrices is crucial to determine the performance of different methods. Based on the theory, we propose a new method that combines the strengths of previous approaches. We demonstrate the effectiveness of our methodology with simulations in two real-world applications.

Recent years have seen great advances in using Gaussian graphical models to characterize the conditional relationship among variables in many domains of study. In particular, many methods have been proposed for estimating the inverse covariance matrix. Along this line of research, glasso (graphical lasso, proposed by Friedman et al. (2008)) provides an $l_1$-regularized maximum likelihood estimator. One challenge in such regularization-based methods is determining the scalar tuning parameter that balances the model complexity and fit to the data, the latter frequently based on the likelihood. When working in high dimensions, traditional model selection methods such as $k$-fold cross-validation, Bayesian information criterion, and Akaike's information criterion can be challenging to apply for several reasons. First, the computation can be prohibitively expensive when estimating high-dimensional inverse covariances multiple times. In addition, reasonable search grids for candidate penalty parameter values can vary considerably across applications. Substantial effort is required to find reasonable search ranges for different applications. Furthermore, using homogeneous regularization for all entries in the inverse covariance matrices can be limiting.

To address these challenges, we first propose block-wise robust selection (BRS), a tuning method based on distributionally robust optimization for selecting block-wise regularization parameters in the glasso estimator. This method finds adaptive penalty parameters for different blocks in the inverse covariance matrix, where the blocks are determined based on data dispersion. In this formulation, the previous penalty parameter search in an arbitrary range now becomes a search of significance level within the fixed interval of $[0,1]$, regardless of the application of interest. Our method is computationally efficient and does not require data normalization prior to estimating the inverse covariance matrix.

Next, we demonstrate the application of BRS to the problem of climate field reconstruction, which aims to reconstruct the past temperature evolution by making use of the measurements in the post-instrumental period and partial records in the pre-instrumental times. The reconstruction can be viewed as a missing value imputation task. In these applications, we first use a Gaussian graphical model tuned via BRS to characterize the spatial field over the globe and then perform the imputation. In addition, we explore different clustering methods for grouping the variables in BRS. The reconstruction results confirm that our method is computationally attractive and provides similar imputed values when compared to using a graph tuned by environmental scientists. Furthermore, BRS can be used flexibly with different variable grouping methods.

Finally, we consider the emulation of physics-based simulators for environmental processes, leveraging Gaussian Processes. Our goal is to develop a computationally efficient surrogate model to closely approximate the outputs of the physics-based environmental simulator, which is expensive to run. Specifically, we develop an emulator for the Regional Hydro-Ecologic Simulation System (RHESSys) simulator. Our emulator leverages Gaussian Processes with embedded seasonality within the mean and a separable covariance. The emulator provides an efficient way to approximate the output that would be obtained by running the physics-based simulator at a substantially lower computational cost than running the simulator.Our emulator approximates environmental time series that would be generated by the physics-based model, e.g., streamflow, under different hydrological and ecological scenarios, e.g., different soil properties. In addition, the degree of approximation and computation efficiency of our built emulator enables us to conduct a global sensitivity analysis on the input-output relationship of the environmental process of interest and identify the key influential environmental factors. Without our emulator, such an analysis would be intractable (or very expensive), as one would need to run the physics-based simulator multiple times for various input settings, which is very costly.

This dissertation, “From Colonial to International: American knowledge construction of Korean history, 1880s-1960s” studies how knowledge on Korean history was constructed in the United States while being influenced by Japanese colonial scholarship from the late nineteenth century throughout the Japanese colonization of Korea (1910-1945), and how this knowledge influenced postwar Korean Studies in the U.S., established in the 1960s. Taking a transnational approach, the dissertation looks at how the knowledge on colonized Korea was constructed by multiple national agents—namely Japanese colonial scholars, American missionaries and their children, and Korean nationalist intellectuals—and how their knowledge on Korea, despite their different political purposes, was compatible with and influenced by each other. It also takes a fresh perspective in looking at Korean Studies in the U.S., which has been regarded as the product of Cold War politics during the postwar period, by tracing the earlier influence of prewar knowledge which reflected colonial scholarship. This dissertation argues that the history of colonized Korea was produced as a “discourse of failure” in which its contents were organized in a way to explain Korea’s being colonized and losing national sovereignty. From the late nineteenth century in the U.S., this knowledge construction was developed to emphasize Korea’s isolationism during the colonial period while partially integrating themes—such as stagnancy and heteronomy—from the Japanese colonial scholarship. This dissertation argues that the transnational co-authorship of Korean history confirmed it as the objective knowledge of Korea. Then, it argues that despite the discontinuity caused by changes in power dynamics, including the Pacific War and the emergence of Cold War politics, many themes from the colonial past were reconfigured to shape the basis of postwar Korean Studies in the U.S. in the 1960s. This dissertation looks at how these reshaped themes came to serve new functions, such as supporting modernization theory within Cold War politics.