Topological Autoencoders

2019/06/03 by Michael Moor, Moor, Michael, Max Horn +5 · 1 voice · 11 citations
Computer Science · Mathematics · #Algebraic Topology (math.AT) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #cs.LG #math.AT #stat.ML

paper · pdf · doi:10.48550/arxiv.1906.00722

arxiv published 2019/06/03 · arxiv updated 2021/05/31

Abstract

We propose a novel approach for preserving topological structures of the input space in latent representations of autoencoders. Using persistent homology, a technique from topological data analysis, we calculate topological signatures of both the input and latent space to derive a topological loss term. Under weak theoretical assumptions, we construct this loss in a differentiable manner, such that the encoding learns to retain multi-scale connectivity information. We show that our approach is theoretically well-founded and that it exhibits favourable latent representations on a synthetic manifold as well as on real-world image data sets, while preserving low reconstruction errors.

Cited by

• CHyLL: Learning Continuous Neural Representations of Hybrid Systems
• HOLE: Homological Observation of Latent Embeddings for Neural Network Interpretability
• TopoReformer: Mitigating Adversarial Attacks Using Topological Purification in OCR Models
• Topological Alignment of Shared Vision-Language Embedding Space
• How Scale Breaks "Normalized Stress" and KL Divergence: Rethinking Quality Metrics
• Multi-Scale Geometric Autoencoder
• The Flood Complex: Large-Scale Persistent Homology on Millions of Points
• FADEx: Feature Attribution and Distortion-based Explanation of Dimensionality Reduction
• Learning Coherent Representations: A Topological Approach to Interpretability
• Variational Rank Reduction Autoencoders for Generative Thermal Design
• What are you sinking? A geometric approach on attention sink
• Path representations in multiparameter persistent homology

Discussions

• Topological Autoencoders [hn, 3 points, 0 comments]

Related

• Ensuring Topological Data-Structure Preservation under Autoencoder Compression due to Latent Space Regularization in Gauss--Legendre nodes
• Topological Autoencoders++: Fast and Accurate Cycle-Aware Dimensionality Reduction
• Autoencoding topology
• Topological De-Noising: Strengthening the Topological Signal
• Connectivity-Optimized Representation Learning via Persistent Homology