UC Berkeley

In the context of nonparametric regression, shape-constrained estimators such as isotonic regression have a number of attractive properties. The shape constraints are typically mild and are often justified by the context of the estimation problem, allowing for more flexible fits than a more restrictive parametric model; yet at the same time such estimators can be computed efficiently and have reasonable risk properties. Additionally, these estimators are free of tuning parameters and often exhibit adaptation to certain types of hidden structure in the data (e.g., isotonic regression and piecewise constant functions). Properties of such estimators in the setting of univariate function estimation are well-studied. This thesis provides some new insights for shape-constrained regression on two fronts: misspecification and the multivariate setting.

In Chapter 2, we study least squares estimators under polyhedral convex constraints. Many estimators fall in this category, including shape constrained estimators like isotonic regression and convex regression, as well as other estimators like LASSO. We give an explicit geometric characterization of how the risk of such an estimator behaves when the truth it is trying to estimate lies outside of the constraint set, and show how this result generalizes what is known in the well-specified setting. This result leads to a better understanding of how isotonic regression behaves when applied in settings where the true function is not isotonic. This chapter is joint work with Adityanand Guntuboyina.

There has been recent interest in understanding shape-constrained estimation in the multivariate setting. It is known that multivariate isotonic regression suffers from the curse of dimensionality, making it unsuitable for most high-dimensional applications. In Chapter 3, we propose and analyze an alternate multivariate generalization of isotonic regression that uses a notion of monotonicity called entire monotonicity. It is restrictive enough to avoid the curse of dimensionality (the dependence on the dimension is only in the logarithmic terms), yet rich enough to include non-smooth functions like rectangular piecewise constant functions. In parallel, we also propose and analyze a generalization of total variation denoising using a notion called Hardy-Krause variation, and show it has similar computational and statistical properties as the entirely monotonic estimator.

This chapter is joint work with Adityanand Guntuboyina and Bodhisattva Sen.

Finally in Chapter 4, we show how entire monotonicity can be viewed as the introduction of "positive interactions" to the interaction-less additive monotonic model. In making this comparison between entire monotonicity and additive monotonicity, we introduce various intermediate models that have different combinations of interactions. We prove a risk rate for some of these intermediate models that generalizes the analogous risk rate for entire monotonicity established in the previous chapter, and also discuss hypothesis testing for interaction terms in these models. This chapter is joint work with Adityanand Guntuboyina and Hansheng Jiang.