UC San Diego

Gaussian mixture models are widely used in statistics and machine learning because of their simple formulation and superior fitting ability. High order moments of the Gaussian mixture model form incomplete symmetric tensors generated by hidden parameters in the model. This thesis studies how to recover unknown parameters in diagonal Gaussian mixture models using high order moment tensors. The problem can be formulated as computing incomplete symmetric tensor decompositions. We propose to use generating polynomials to compute incomplete symmetric tensor approximations and approximations. The obtained decomposition is utilized to recover parameters in the model.

In the first part of thesis, we propose a learning algorithm using the first and third order moment tensors and require that the number of components $r\leq\frac{d}{2}-1$ for mixture of $d$-dimensional Gaussians. In the second part of thesis, we generalize the previous algorithm using higher order moment tensors and therefore we can recover the unknown parameters of the model when the number of components $r>\frac{d}{2}-1$. We provide an upper bound of the number of components in the Gaussian mixture model that the generalized algorithm can compute. For both algorithms, we prove that our recovered parameters are accurate when the estimated moments are accurate. Numerical simulations and comparisons with EM algorithm are presented to show the performance of our algorithms.