In this dissertation, we study Bayesian inference methods for the Gaussian sequence model, where the problem is to estimate an unknown sequence from noisy observations. This model is fundamental to nonparametric function estimation and time series analysis, and finds application across diverse scientific disciplines.

Our goal is to develop flexible methods that do not impose restrictive assumptions on the structure of the unknown sequence, and that are able to adapt to different degrees of smoothness. Moreover, we aim to construct efficient algorithms that do not require any hyperparameter tuning and that provide automatic uncertainty quantification. To achieve these goals, we leverage properties of the Cauchy distribution, and of the Bayesian state-space model framework.

In Chapter 1, we focus on "weak smoothness" assumptions formulated in terms of the kth-order differences of the unknown sequence for k ∈ N. We motivate the use of Cauchy priors through the principle of Maximum Entropy, and propose two algorithms for posterior inference: an efficient Gibbs sampler and an approximate filtering-smoothing method. Through simulations, we compare the performance of our method to other approaches from the statistical literature. The Gibbs sampler implementation of our model demonstrates favorable empirical convergence rates across diverse test functions while maintaining linear computational complexity. Applications to real-world datasets, including financial returns, temperature anomalies, and economic indicators, underscore the method’s versatility and effectiveness compared to other Bayesian and frequentist alternatives.

Chapter 2 introduces several extensions of the method discussed in Chapter 1. We first present the Cauchy local linear trend model, and generalize it to the Cauchy decomposition model. Both these approaches rely on decomposing the unknown sequence into multiple components, each characterized by Cauchy priors on differences of distinct order. Next, we extend the method from Chapter 1 to allow for irregularly spaced observations by casting the problem in an infinite-dimensional space and leveraging Cauchy process priors. We also introduce the latent Cauchy AR(k) model, which further relaxes the weak smoothness assumptions of 1, leading to a method that is particularly suitable for sequences described by linear differential equations with shocks. We conclude by extending the first-order model from Chapter 1 to handle bivariate observations, broadening its applicability to multivariate settings. For each extension, we develop Gibbs samplers for efficient posterior inference. Moreover, we evaluate the performance of our methods through simulation studies.

Chapter 3 moves into a theoretical exploration of the frequentist properties of Cauchy priors in the context of the Normal means model characterized by nearly black signals. In particular, we derive bounds on the mean squared error of the posterior mean, and demonstrate that it attains minimax risk when the sparsity level is known. This analysis lays the groundwork for future research on theoretical guarantees for the more complex models developed in Chapters 1 and 2.

This dissertation contributes to Bayesian methodology by offering practical, flexible, and computationally efficient tools for sequence estimation. The methods presented herein are broadly applicable in contexts where signals must be estimated from noisy data without imposing strong smoothness constraints.

Modern science and engineering often generate data sets with a large sample size and a comparably large dimension which puts classic asymptotic theory into question in many ways. Therefore, the main focus of this thesis is to develop a fundamental understanding

of statistical procedures for estimation and hypothesis testing from a non-asymptotic point of view, where both the sample size and problem dimension grow hand in hand. A range of different problems are explored in this thesis, including work on the geometry of hypothesis testing, adaptivity to local structure in estimation, effective methods for shape-constrained problems, and early stopping with boosting algorithms.

Our treatment of these different problems shares the common theme of emphasizing the underlying geometric structure.

To be more specific, in our hypothesis testing problem, the null and alternative are specified by a pair of convex cones. This cone structure makes it possible for a sharp characterization of the behavior of Generalized Likelihood Ratio Test (GLRT) and its optimality property. The problem of planar set estimation based on noisy measurements of its support function, is a non-parametric problem in nature. It is interesting to see that estimators can be constructed such that they are more efficient in the case when the underlying set has a simpler structure, even without knowing the set beforehand. Moreover, when we consider applying boosting algorithms to estimate a function in reproducing kernel Hibert space (RKHS), the optimal stopping rule and the resulting estimator turn out to be determined by the localized complexity of the space.

These results demonstrate that, on one hand, one can benefit from respecting and making use of the underlying structure (optimal early stopping rule for different RKHS); on the other hand, some procedures (such as GLRT or local smoothing estimators) can achieve better performance when the underlying structure is simpler, without prior knowledge of the structure itself.

To evaluate the behavior of any statistical procedure, we follow the classic minimax framework and also discuss about more refined notion of local minimaxity.

Shape constraints encode a relatively weak form of prior information specifying the direction of certain relationships in an unknown signal. Classical examples include estimation of a convex function or a monotone density. Shape constraints are often strong enough to dramatically reduce statistical complexity while still yielding flexible, nonparametric estimators. This thesis brings shape constraints to bear on several recent research areas in statistics—distribution-free inference, high-dimensional covariance estimation, empirical Bayes, and multiple hypothesis testing.

Chapter 2 discusses my joint work with Professor Aditya Guntuboyina and Professor Jim Pitman on distribution-free properties of isotonic regression. In this work, we establish a distributional result for the components of the isotonic least squares estimator using its characterization as the derivative of the greatest convex minorant of a random walk. Provided the walk has exchangeable increments, we prove that the slopes of the greatest convex minorant are distributed as order statistics of the running averages. This result implies an exact formula for the squared error risk of least squares in homoscedastic isotonic regression when the true sequence is constant that holds for every exchangeable error distribution.

Chapter 3 discusses my joint work with Professor Aditya Guntuboyina and Professor Michael I. Jordan on sign-constrained precision matrix estimation. We investigate the problem of high-dimensional covariance estimation under the constraint that the partial correlations are nonnegative. The sign constraints dramatically simplify estimation: the Gaussian maximum likelihood estimator is well defined with only two observations regardless of the number of variables. We analyze its performance in the setting where the dimension may be much larger than the sample size. We establish that the estimator is both high-dimensionally consistent and minimax optimal in the symmetrized Stein loss. We also prove a negative result which shows that the sign-constraints can introduce substantial bias for estimating the top eigenvalue of the covariance matrix.

Chapter 4 discusses my joint work with Professor Aditya Guntuboyina and Professor Bodhisattva Sen on nonparametric empirical Bayes with multivariate, heteroscedastic Gaussian errors. Multivariate, heteroscedastic errors complicate statistical inference in many large- scale denoising problems. Empirical Bayes is attractive in such settings, but standard para- metric approaches rest on assumptions about the form of the prior distribution which can be hard to justify and which introduce unnecessary tuning parameters. We extend the nonparametric maximum likelihood estimator (NPMLE) for Gaussian location mixture densities to allow for multivariate, heteroscedastic errors. NPMLEs estimate an arbitrary prior by solving an infinite-dimensional, convex optimization problem; we show that this convex optimization problem can be tractably approximated by a finite-dimensional version. We introduce a dual mixture density whose modes contain the atoms of every NPMLE, and we leverage the dual both to establish non-uniqueness in multivariate settings as well as to construct explicit bounds on the support of the NPMLE.

The empirical Bayes posterior means based on an NPMLE have low regret, meaning they closely target the oracle posterior means one would compute with the true prior in hand. We prove an oracle inequality implying that the empirical Bayes estimator performs at nearly the optimal level (up to logarithmic factors) for denoising without prior knowledge. We provide finite-sample bounds on the average Hellinger accuracy of an NPMLE for estimating the marginal densities of the observations. We also demonstrate the adaptive and nearly- optimal properties of NPMLEs for deconvolution. We apply the method to two astronomy datasets, constructing a fully data-driven color-magnitude diagram of 1.4 million stars in the Milky Way and investigating the distribution of chemical abundance ratios for 27 thousand stars in the red clump.

Chapter 5 discusses my joint work with Daniel Xiang and Professor William Fithian on finite-sample control of the maximum local false discovery rate in multiple hypothesis testing. Despite the popularity of the false discovery rate (FDR) as an error control metric for large- scale multiple testing, its close Bayesian counterpart the local false discovery rate (lfdr), defined as the posterior probability that a particular null hypothesis is false, is a more directly relevant standard for justifying and interpreting individual rejections. However, the lfdr is difficult to work with in small samples, as the prior distribution is typically unknown. We propose a simple multiple testing procedure and prove that it controls the expectation of the maximum lfdr across all rejections; equivalently, it controls the probability that the rejection with the largest p-value is a false discovery. Our method operates without knowledge of the prior, assuming only that the p-value density is uniform under the null and decreasing under the alternative. We also show that our method asymptotically implements the oracle Bayes procedure for a weighted classification risk, optimally trading off between false positives and false negatives. We derive the limiting distribution of the attained maximum lfdr over the rejections, and the limiting empirical Bayes regret relative to the oracle procedure.

In the context of nonparametric regression, shape-constrained estimators such as isotonic regression have a number of attractive properties. The shape constraints are typically mild and are often justified by the context of the estimation problem, allowing for more flexible fits than a more restrictive parametric model; yet at the same time such estimators can be computed efficiently and have reasonable risk properties. Additionally, these estimators are free of tuning parameters and often exhibit adaptation to certain types of hidden structure in the data (e.g., isotonic regression and piecewise constant functions). Properties of such estimators in the setting of univariate function estimation are well-studied. This thesis provides some new insights for shape-constrained regression on two fronts: misspecification and the multivariate setting.

In Chapter 2, we study least squares estimators under polyhedral convex constraints. Many estimators fall in this category, including shape constrained estimators like isotonic regression and convex regression, as well as other estimators like LASSO. We give an explicit geometric characterization of how the risk of such an estimator behaves when the truth it is trying to estimate lies outside of the constraint set, and show how this result generalizes what is known in the well-specified setting. This result leads to a better understanding of how isotonic regression behaves when applied in settings where the true function is not isotonic. This chapter is joint work with Adityanand Guntuboyina.

There has been recent interest in understanding shape-constrained estimation in the multivariate setting. It is known that multivariate isotonic regression suffers from the curse of dimensionality, making it unsuitable for most high-dimensional applications. In Chapter 3, we propose and analyze an alternate multivariate generalization of isotonic regression that uses a notion of monotonicity called entire monotonicity. It is restrictive enough to avoid the curse of dimensionality (the dependence on the dimension is only in the logarithmic terms), yet rich enough to include non-smooth functions like rectangular piecewise constant functions. In parallel, we also propose and analyze a generalization of total variation denoising using a notion called Hardy-Krause variation, and show it has similar computational and statistical properties as the entirely monotonic estimator.

This chapter is joint work with Adityanand Guntuboyina and Bodhisattva Sen.

Finally in Chapter 4, we show how entire monotonicity can be viewed as the introduction of "positive interactions" to the interaction-less additive monotonic model. In making this comparison between entire monotonicity and additive monotonicity, we introduce various intermediate models that have different combinations of interactions. We prove a risk rate for some of these intermediate models that generalizes the analogous risk rate for entire monotonicity established in the previous chapter, and also discuss hypothesis testing for interaction terms in these models. This chapter is joint work with Adityanand Guntuboyina and Hansheng Jiang.

Modern marketplaces and societal systems are undergoing substantial transformations due to automated data collection and algorithmic decision-making. In response, my research focuses on developing methodologies and algorithms for sequential and data-driven decision-making, with a particular emphasis on modeling and analyzing human behavior within systems. To achieve this goal, I employ an eclectic set of techniques from statistics, optimization, and machine learning, while drawing on insights from economic theories and marketing literature. My dissertation contributes to the field of sequential and data-driven decision-making by addressing important real-world problems and developing novel methodologies and algorithms that bridge theory and practice.

Reference effects are fundamental consumer behavior models with roots from the renowned prospect theory in economics and have been widely observed in reality. Due to the existence of reference price effects by consumers, retailers need to be aware of the sequential effects of current prices on future demand when designing their dynamic pricing policies. In the first part of the dissertation, we build a random coefficient logit demand model that allows arbitrary joint distributions of valuations, responsiveness to prices, and responsiveness to reference prices among consumers to fully express consumer heterogeneity. We develop a nonparametric estimation method to learn heterogeneous consumer reference effects from transaction data, and we apply a modified policy iteration algorithm to find the optimal pricing policies. We verify the effectiveness of our approach through numerical experiments on large-scale individual-level transaction data from the online retailer JD.com.

Mixture of regression models are useful for regression analysis in heterogeneous populations, and have been applied in diverse fields including biology, economics, engineering, epidemiology, marketing, and transportation. In the second part of the dissertation, we study the nonparametric maximum likelihood estimator (NPMLE) for fitting these models, and notably our approach does not require prior specification of the number of mixture components. We establish existence of the NPMLE and prove finite-sample parametric Hellinger error bounds for the predicted density functions. We also provide an effective procedure for computing the NPMLE without ad-hoc discretization and prove a theoretical convergence rate under certain assumptions. This work provides the theoretical foundation for the nonparametric estimation method used in the first part of the dissertation for estimating consumer heterogeneity.

Aside from online platforms like retailing, another type of platform that is receiving increasing attention comes from sharing services such as ride-hailing and vehicle sharing. On-demand and one-way vehicle sharing (e.g., car sharing, bike sharing, scooter sharing) services are emerging and environment-friendly transportation options that promise consumers more flexibility and convenience. Despite the benefits of vehicle sharing, such services still suffer from many operational difficulties and struggle to thrive. One major challenge in operating such vehicle sharing systems is matching the supply and the demand in real time across multiple locations. In the third part of the dissertation, we consider how to dynamically reposition vehicles in order to minimize the total costs of repositioning and lost sales in the long run. The repositioning problem is critical in successful management of on-demand one-way vehicle sharing services, and it is challenging both analytically and computationally. The optimal repositioning policy under a general $n$-location network is inaccessible without knowing the optimal value function. Inspired by the base-stock reorder policy in inventory control, we propose a class of base-stock repositioning policies and prove that they are asymptotically optimal in certain limiting regimes.