2015/08/26 by Galin Georgiev, Georgiev, Galin · 1 voice · 2 citations
Computer Science · Mathematics · Physics and Astronomy · #Algorithm #Artificial intelligence #Artificial neural network #Class (philosophy) #Classifier (UML) #Computer science #Exponential function #Generative Adversarial Networks and Image Synthesis #Generative grammar #Generative model #MNIST database #Machine learning #Mathematical analysis #Mathematics #Model Reduction and Neural Networks #Neural Networks and Applications #Theoretical computer science #cs.CV #cs.LG #cs.NE
paper · pdf · doi:10.48550/arxiv.1508.06585
published in arXiv (Cornell University) (Cornell University) · v5: added thermodynamic identities and variational error estimation; expanded references
openalex publication_date 2015/08/26 · arxiv created 2016/06/30 · arxiv updated 2016/07/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We study from a physics viewpoint a class of generative neural nets, Gibbs machines, designed for gradual learning. While including variational auto-encoders, they offer a broader universal platform for incrementally adding newly learned features, including physical symmetries. Their direct connection to statistical physics and information geometry is established. A variational Pythagorean theorem justifies invoking the exponential/Gibbs class of probabilities for creating brand new objects. Combining these nets with classifiers, gives rise to a brand of universal generative neural nets - stochastic auto-classifier-encoders (ACE). ACE have state-of-the-art performance in their class, both for classification and density estimation for the MNIST data set.