2015/08/26 by Galin Georgiev, Georgiev, Galin · 1 voice
Computer Science · Physics and Astronomy · #Generative Adversarial Networks and Image Synthesis #Neural Networks and Applications #Model Reduction and Neural Networks
paper · pdf · doi:10.48550/arxiv.1508.06585
We study from a physics viewpoint a class of generative neural nets, Gibbs machines, designed for gradual learning. While including variational auto-encoders, they offer a broader universal platform for incrementally adding newly learned features, including physical symmetries. Their direct connection to statistical physics and information geometry is established. A variational Pythagorean theorem justifies invoking the exponential/Gibbs class of probabilities for creating brand new objects. Combining these nets with classifiers, gives rise to a brand of universal generative neural nets - stochastic auto-classifier-encoders (ACE). ACE have state-of-the-art performance in their class, both for classification and density estimation for the MNIST data set.