Flows for simultaneous manifold learning and density estimation

2020/03/31 by Johann Brehmer, K. Cranmer, Kyle Cranmer +2 · 1 voice · 33 citations
Computer Science · Mathematics · #Applied mathematics #Artificial intelligence #Class (philosophy) #Computational Physics and Python Applications #Computer science #Curse of dimensionality #Density estimation #Dimensionality reduction #Generative Adversarial Networks and Image Synthesis #Geometry #Inference #Information geometry #Manifold (fluid mechanics) #Manifold alignment #Mathematics #Nonlinear dimensionality reduction #Pattern recognition (psychology) #Range (aeronautics) #Space (punctuation) #Statistical manifold #Statistics #Time Series Analysis and Forecasting #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2003.13913

published in arXiv (Cornell University) 33, 442-453 (Cornell University) · Code at https://github.com/johannbrehmer/manifold-flow , v2: multiple new experiments, v3: added comparison with probabilistic auto-encoder

openalex publication_date 2020/03/31 · arxiv created 2020/11/13 · arxiv updated 2020/11/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce manifold-learning flows (M-flows), a new class of generative models that simultaneously learn the data manifold as well as a tractable probability density on that manifold. Combining aspects of normalizing flows, GANs, autoencoders, and energy-based models, they have the potential to represent datasets with a manifold structure more faithfully and provide handles on dimensionality reduction, denoising, and out-of-distribution detection. We argue why such models should not be trained by maximum likelihood alone and present a new training algorithm that separates manifold and density updates. In a range of experiments we demonstrate how M-flows learn the data manifold and allow for better inference than standard flows in the ambient data space.

Citations

• Approximating Likelihood Ratios with Calibrated Discriminative Classifiers
• Matplotlib: A 2D Graphics Environment
• Deterministic Nonperiodic Flow
• Are GANs Created Equal? A Large-Scale Study
• Invertible Autoencoder for domain adaptation
• Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows
• Mining gold from implicit models to improve likelihood-free inference

Cited by

• Understanding Self-Supervised Learning via Latent Distribution Matching
• Characterizing Neural Manifolds' Properties and Curvatures using Normalizing Flows
• Latent Diffusion Inversion Requires Understanding the Latent Space
• Split-Flows: Measure Transport and Information Loss Across Molecular Resolutions
• FloDR: An invertible dimensionality reduction method based on a normalising flow
• Marginal Flow: a flexible and efficient framework for density estimation
• Misspecification-robust amortised simulation-based inference using variational methods
• Measure Transport with Kernel Stein Discrepancy
• Latent Variable Modelling with Hyperbolic Normalizing Flows
• Offline RL with Smooth OOD Generalization in Convex Hull and its Neighborhood
• On Convolutions, Intrinsic Dimension, and Diffusion Models
• Neural Manifold Ordinary Differential Equations
• Generalized Energy Based Models
• Flow-based Generative Models for Learning Manifold to Manifold Mappings
• Sparse Autoencoders, Again?
• Autoencoding Under Normalization Constraints
• Beyond Diagonal Covariance: Flexible Posterior VAEs via Free-Form Injective Flows
• Learning Geometry and Topology via Multi-Chart Flows
• Probabilistic Autoencoder
• Follow the Energy, Find the Path: Riemannian Metrics from Energy-Based Models
• Getting High: High Fidelity Simulation of High Granularity Calorimeters with High Speed

Discussions

• Very similar to the figure that @johannbrehmer.bsky.social and I made in “Flows for simultaneous manifold learning and density estimation” arxiv.org/abs/2003.13913 [bsky, 32 points, 1 comments]

Related

• Density estimation on smooth manifolds with normalizing flows
• Joint Manifold Learning and Density Estimation Using Normalizing Flows
• Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows
• Learning Distributions on Manifolds with Free-Form Flows
• Nonlinear Isometric Manifold Learning for Injective Normalizing Flows