Abstract

July 1 - 9.30-10.30
S.Amari - Information Geometry of Singular Statistical Models

Information geometry studies manifolds of probability distributions or statistical models. The intrinsic structure of regular finite-dimensional statistical models has been investigated well, and lots of applications emerge in a wide range of fields such as information theory, control systems theory, optimization, neural netwroks, and belief propagation. Many of these models are hierarchical, in the sense that smaller models are included in larger models as submanifolds. Typical such examples are multilayer perceptrons, ARMA time series models and Gaussian mixtures. In such a model, there exist critical areas corresponding to smaller models, on which the parameters become unidentifiable and the Fisher metric degenerates. Geometrically, such models include algebraic singularities. The present talk analyzes the structure of singularities by using a simple model of Gaussian mixtures. The cusp type singularities are found in this case, and accuracy of parameter estimation is analyzed when the true distribution is close to singularity. We also use a simple toy model (cone model) to show some other properties of singularities and the effects of singularities on learning. [an error occurred while processing this directive]