UC Santa Cruz

Nonlinearity and high-order auto-dependence are common traits of univariate time series tracking successive states from multidimensional systems. Standard statistical models based on linear stochastic processes are often inadequate to capture these complex dynamics. This work contributes Bayesian statistical methodology and modeling strategies to estimate Markovian transition distributions, particularly when these distributions exhibit non-Gaussianity and/or nonlinear dependence on multiple lags. Given the challenge of modeling high-order nonlinear dynamics, we place emphasis on detecting and exploiting low-order dependence.

We propose models for both discrete and continuous state spaces with a common theme of mixture modeling. We first utilize mixtures for soft model selection. To this end, we develop two prior distributions for probability vectors which, in contrast to the popular Dirichlet distribution, retain sparsity properties in the presence of data. Both priors are tractable, allowing for efficient posterior sampling and marginalization. We derive the priors, demonstrate their properties, and employ them for lag selection in the mixture transition distribution model.

We then extend the model for estimation and selection in higher-order, discrete-state Markov chains with two primary objectives: parsimonious approximation of high-order dynamics by mixing transition models of lower order, and model selection through over-specification and shrinkage with the new priors to an identifiable and interpretable parameterization. We also extend a continuous-state version of the mixture transition distribution model by admitting nonlinear dependence in the component distributions using Gaussian process priors. We discuss properties of the models and demonstrate their utility with simulation studies and applications to medical, geological, and ecological time series.

Finally, we propose and illustrate a Bayesian nonparametric autoregressive mixture model applied to flexibly estimate general transition densities exhibiting nonlinear lag dependence. Our approach is related to Bayesian curve fitting via joint density estimation using Dirichlet process mixtures, with the Markovian likelihood defined as the conditional distribution obtained from the mixture. We extended the model to include automatic relevance detection among a pre-specified set of lags. We illustrate the model by repeating earlier analyses.