2020/12/24 by Daniel O’Connor, Daniel O'Connor, O'Connor, Daniel +2
Computer Science · Mathematics · Physics and Astronomy · #Algorithm #Applied mathematics #Base (topology) #Bijection #Computer science #Discrete mathematics #Distribution (mathematics) #FOS: Computer and information sciences #Flow (mathematics) #Gaussian #Generative Adversarial Networks and Image Synthesis #Geometry #Invertible matrix #Limit (mathematics) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Mathematical analysis #Mathematical optimization #Mathematics #Model Reduction and Neural Networks #Music and Audio Processing #Pure mathematics #Smoothing #Space (punctuation) #Statistics #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.2012.13196
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2020/12/24 · openalex created_date 2021/01/05 · arxiv created 2021/07/12 · arxiv updated 2021/07/13 · openalex updated_date 2026/07/28
Efficient sampling of complex data distributions can be achieved using trained invertible flows (IF), where the model distribution is generated by pushing a simple base distribution through multiple non-linear bijective transformations. However, the iterative nature of the transformations in IFs can limit the approximation to the target distribution. In this paper we seek to mitigate this by implementing RBM-Flow, an IF model whose base distribution is a Restricted Boltzmann Machine (RBM) with a continuous smoothing applied. We show that by using RBM-Flow we are able to improve the quality of samples generated, quantified by the Inception Scores (IS) and Frechet Inception Distance (FID), over baseline models with the same IF transformations, but with less expressive base distributions. Furthermore, we also obtain D-Flow, an IF model with uncorrelated discrete latent variables. We show that D-Flow achieves similar likelihoods and FID/IS scores to those of a typical IF with Gaussian base variables, but with the additional benefit that global features are meaningfully encoded as discrete labels in the latent space.