Accession Number:

ADA604494

Title:

Tensor Decompositions for Learning Latent Variable Models

Descriptive Note:

Technical rept.

Corporate Author:

CALIFORNIA UNIV IRVINE

Personal Author(s):

• Anandkumar, Anima
• Ge, Rong
• Hsu, Daniel
• Kakade, Sham M.
• Telgarsky, Matus

Report Date:

2012-12-08

Pagination or Media Count:

57.0

Abstract:

This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models--including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation--which exploits a certain tensor structure in their low-order observable moments typically, of second- and third-order. Specifically, parameter estimation is reduced to the problem of extracting a certain orthogonal decomposition of a symmetric tensor derived from the moments this decomposition can be viewed as a natural generalization of the singular value decomposition for matrices. Although tensor decompositions are generally intractable to compute, the decomposition of these specially structured tensors can be efficiently obtained by a variety of approaches, including power iterations and maximization approaches similar to the case of matrices. A detailed analysis of a robust tensor power method is provided, establishing an analogue of Wedins perturbation theorem for the singular vectors of matrices. This implies a robust and computationally tractable estimation approach for several popular latent variable models.

Descriptors:

• *MATHEMATICAL MODELS
• MARKOV PROCESSES
• STATISTICAL PROCESSES
• TENSOR ANALYSIS

Subject Categories:

• Theoretical Mathematics
• Statistics and Probability

Distribution Statement:

APPROVED FOR PUBLIC RELEASE