Department of Statistics, UCLA

Independent component analysis (ICA) is an important method for blind source separation and unsupervised learning. Recently, the method has been extended to the overcomplete situation where the number of sources is greater than the number of receivers. Comparing complete ICA and overcomplete ICA in existing literature, one can notice that complete ICA does not assume noise in observations, and the sources can be ex- plicitly solved from the receivers, whereas the overcomplete ICA in gen- eral assumes noise in observations and the sources are implicitly solved by gradient type algorithms. In this paper, we present an explicit null- space representation for overcomplete ICA in the noiseless situation based on singular value decomposition (SVD), and develop an algorithm for estimating mixing matrix and recovering the sources. The null-space representation makes the connection between complete ICA and over- complete ICA more apparent, and leads to a mathematical explanation of lateral inhibition in the context of overcomplete linear model. It also appears to work well on the experimental examples. Moreover, it can be extended to the situation where there is noise in observations, and may lead to more efficient algorithms in this situation.