Methods, like Maximum Empirical Likelihood (MEL), that operate within the Empirical Estimating Equations (E3) approach to estimation and inference are challenged by the Empty Set Problem (ESP). We propose to return from E3 back to the Estimating Equations, and to use the Maximum Likelihood method. In the discrete case the Maximum Likelihood with Estimating Equations (MLEE) method avoids ESP. In the continuous case, how to make ML-EE operational is an open question. Instead of it, we propose a Patched Empirical Likelihood, and demonstrate that it avoids ESP. The methods enjoy, in general, the same asymptotic properties as MEL.

Empirical Likelihood (EL) and other methods that operate within the Empirical Estimating Equations (E3) approach to estimation and inference are challenged by the Empty Set Problem (ESP). ESP concerns the possibility that a model set, which is data-dependent, may be empty for some data sets. To avoid ESP we return from E3 back to the Estimating Equations, and explore the Bayesian infinite-dimensional Maximum A-posteriori Probability (MAP) method. The Bayesian MAP with Dirichlet prior motivates a Revised EL (ReEL) method. ReEL i) avoids ESP as well as the convex hull restriction, ii) attains the same basic asymptotic properties as EL, and iii) its computation complexity is comparable to that of EL.

Using a large deviations approach, Maximum A-Posteriori Probability (MAP) and Empirical Likelihood (EL) are shown to possess, under misspecification, an exclusive property of Bayesian consistency. Under conditions of consistency, regardless of prior the MAP estimator asymptotically coincides with EL. The consistency property is also studied for sampling processes other than iid.

In this paper we demonstrate, in a parametric Estimating Equations setting, that the Empirical Likelihood (EL) method is an asymptotic instance of the Bayesian non-parametric Maximum-A-Posteriori approach. The resulting probabilistic interpretation and justifcation of EL rests on Bayesian non-parametric consistency in L-divergence.

In the Empirical Estimating Equations (E^3) approach to estimation and inference estimating equations are replaced by their data-dependent empirical counterparts. It is odd but with E^3 there are models where the E^3-based estimator does not exist for some data set, and does exist for others. This depends on whether or not a set of data-supported probability mass functions that satisfy the empirical estimating equations is empty for the data set. In a finite sample context, this unnoted feature invalidates methods of estimation and inference, such as the Maximum Empirical Likelihood, that operate within E^3. The empty set problem of E^3 is illustrated by several examples and possible remedies are discussed.

Criterion choice is such a hard problem in information recovery and in estimation and inference. In the case of inverse problems with noise, can probabilistic laws provide a basis for empirical estimator choice? That is the problem we investigate in this paper. Large Deviations Theory is used to evaluate the choice of estimator in the case of two fundamental situations-problems in modelling data. The probabilistic laws developed demonstrate that each problem has a unique solution-empirical estimator. Whether other members of the empirical estimator family can be associated a particular problem and conditional limit theorem, is an open question.

The Bayesian Sanov Theorem (BST) identifies, under both correct and incorrect specification of infinite dimensional model, the points of concentration of the posterior measure. Utilizing this insight in the context of Polya urn sampling, Bayesian nonparametric consistency is established. Polya BST is also used to provide an extension of Maximum Non-parametric Likelihood and Empirical Likelihood methods to the Polya case.