Stochastic complexity for latent variable models

In the area of machine learning and data mining, latent variable models are often used as knowledge representations. It is one of most important issues how one can select the best structures for such latent variable models from given data. The problem is that conventional information criteria cannot straightforwardly be applied to this issue. This is because latent variable models are irregular in general, in the sense that there is no one-to-one correspondence between parameters and distributions. In order to overcome this problem, I introduce the latent stochastic complexity to propose how to select the best latent structures. Through a number of concrete models, such as non-negative matrix factorization, mixture model, relational models, canonical correlation analysis, I show how to calculate latent stochastic complexity and how well it performs. Further we consider how we can truck changes of latent structures when they change over time.