2019/02/06 by Max F. Frenzel, Frenzel, Max F., Bogdan Teleaga +3
Computer Science · Mathematics · #Artificial Intelligence (cs.AI) #Artificial intelligence #Autoregressive model #Computation and Language (cs.CL) #Computational Physics and Python Applications #Computer science #Data mining #FOS: Computer and information sciences #Flexibility (engineering) #Generative Adversarial Networks and Image Synthesis #Generative grammar #Generative model #Generator (circuit theory) #Heuristic #Latent variable #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Mathematics #Measure (data warehouse) #Metric (unit) #Sequence (biology) #Space (punctuation) #Statistics #Topic Modeling #Transformation (genetics) #cs.AI #cs.CL #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.1902.02113
arxiv created 2019/02/06 · openalex publication_date 2019/02/06 · arxiv updated 2019/02/07 · openalex created_date 2019/02/21 · openalex updated_date 2026/07/28
Deep generative models are universal tools for learning data distributions on high dimensional data spaces via a mapping to lower dimensional latent spaces. We provide a study of latent space geometries and extend and build upon previous results on Riemannian metrics. We show how a class of heuristic measures gives more flexibility in finding meaningful, problem-specific distances, and how it can be applied to diverse generator types such as autoregressive generators commonly used in e.g. language and other sequence modeling. We further demonstrate how a diffusion-inspired transformation previously studied in cartography can be used to smooth out latent spaces, stretching them according to a chosen measure. In addition to providing more meaningful distances directly in latent space, this also provides a unique tool for novel kinds of data visualizations. We believe that the proposed methods can be a valuable tool for studying the structure of latent spaces and learned data distributions of generative models.