Constructive Statistics: Estimators, Algorithms, and Asymptotics
Constructive mathematics is a philosophical doctrine that asserts that objects cannot be shown to exist unless a method is provided for producing them. The related usage Constructive Statistics is intended mean that statistical constructs, such as estimators, do not (usefully) exist unless they can be employed. A function defined from the data space to the parameter space is not an estimator by virtue of an existence proof, but only by virtue of a method for finding the answer. Furthermore, one cannot have a generally useful estimator unless it can be computed in a reasonable time (polynomial at a minimum). Asymptotics must be constrained by computational complexity. Another important point is that with many modern estimators that contain a stochastic component, the choice of algorithm determines the properties of the estimator. One could say that The Algorithm is the Estimator. Many of these modern stochastic estimators can be formulated as global optimization problems, with all that implies. In this climate, computational statistics is at the center of modern statistical science, not at the periphery.