Parametric UMAP embeddings for representation and semi-supervised learning

2020/09/27 by Tim Sainburg, Sainburg, Tim, Leland McInnes +4 · 34 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · Mathematics · #Advanced Graph Neural Networks #Computational Geometry (cs.CG) #Domain Adaptation and Few-Shot Learning #FOS: Biological sciences #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Quantitative Methods (q-bio.QM) #Topological and Geometric Data Analysis #cs.CG #cs.LG #q-bio.QM #stat.ML

paper · pdf · doi:10.48550/arxiv.2009.12981

openalex publication_date 2020/09/27 · arxiv created 2021/08/29 · arxiv updated 2021/08/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/02

Abstract

UMAP is a non-parametric graph-based dimensionality reduction algorithm using applied Riemannian geometry and algebraic topology to find low-dimensional embeddings of structured data. The UMAP algorithm consists of two steps: (1) Compute a graphical representation of a dataset (fuzzy simplicial complex), and (2) Through stochastic gradient descent, optimize a low-dimensional embedding of the graph. Here, we extend the second step of UMAP to a parametric optimization over neural network weights, learning a parametric relationship between data and embedding. We first demonstrate that Parametric UMAP performs comparably to its non-parametric counterpart while conferring the benefit of a learned parametric mapping (e.g. fast online embeddings for new data). We then explore UMAP as a regularization, constraining the latent distribution of autoencoders, parametrically varying global structure preservation, and improving classifier accuracy for semi-supervised learning by capturing structure in unlabeled data. Google Colab walkthrough: https://colab.research.google.com/drive/1WkXVZ5pnMrm17m0YgmtoNjM_XHdnE5Vp?usp=sharing

Citations

• Learning Multiple Layers of Features from Tiny Images
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• Connectivity-Optimized Representation Learning via Persistent Homology
• Auto-Encoding Variational Bayes
• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• Generative adversarial interpolative autoencoding: adversarial training on latent space interpolations encourage convex latent distributions

Cited by

• Cross-modal Counterfactual Explanations: Uncovering Decision Factors and Dataset Biases in Subjective Classification
• From Minutes to Days: Scaling Intracranial Speech Decoding with Supervised Pretraining
• Linearly-scalable and entropy-optimal learning of nonstationary and nonlinear manifolds
• Generalization of Long-Range Machine Learning Potentials in Complex Chemical Spaces
• Probabilistic Foundations of Fuzzy Simplicial Sets for Nonlinear Dimensionality Reduction
• Associative Syntax and Maximal Repetitions reveal context-dependent complexity in fruit bat communication
• TopoReformer: Mitigating Adversarial Attacks Using Topological Purification in OCR Models
• MMCM: Multimodality-aware Metric using Clustering-based Modes for Probabilistic Human Motion Prediction
• Chemistry-Enhanced Diffusion-Based Framework for Small-to-Large Molecular Conformation Generation
• FloDR: An invertible dimensionality reduction method based on a normalising flow
• Sparsely gated tiny linear experts
• Efficient Manifold-Constrained Neural ODE for High-Dimensional Datasets
• Best practices and tools in R and Python for statistical processing and visualization of lipidomics and metabolomics data
• Guided Manifold Alignment with Geometry-Regularized Twin Autoencoders
• The GDN-CC Dataset: Automatic Corpus Clarification for AI-enhanced Democratic Citizen Consultations
• Latent Activation Editing: Inference-Time Refinement of Learned Policies for Safer Multirobot Navigation
• FADEx: Feature Attribution and Distortion-based Explanation of Dimensionality Reduction
• Conditional Multidimensional Scaling with Incomplete Conditioning Data
• Variational Gaussian Mixture Manifold Models for Client-Specific Federated Personalization
• TopoMap: A Feature-based Semantic Discriminator of the Topographical Regions in the Test Input Space
• UMATO: Bridging Local and Global Structures for Reliable Visual Analytics with Dimensionality Reduction
• Chronotome: Real-Time Topic Modeling for Streaming Embedding Spaces
• DE-VAE: Revealing Uncertainty in Parametric and Inverse Projections with Variational Autoencoders using Differential Entropy
• Leveraging Convolutional and Graph Networks for an Unsupervised Remote Sensing Labelling Tool
• Massive Atomic Diversity: a compact universal dataset for atomistic machine learning
• Manifold Learning for Personalized and Label-Free Detection of Cardiac Arrhythmias
• Unsupervised Machine Learning for Scientific Discovery: Workflow and Best Practices
• Latent Manifold Reconstruction and Representation with Topological and Geometrical Regularization
• Lost in Permissions: Exploring the Microsoft 365 App Ecosystem
• AKRMap: Adaptive Kernel Regression for Trustworthy Visualization of Cross-Modal Embeddings
• Mixed Signals: Understanding Model Disagreement in Multimodal Empathy Detection
• VIGIL: Vision-Language Guided Multiple Instance Learning Framework for Ulcerative Colitis Histological Healing Prediction
• Explanations Go Linear: Post-hoc Explainability for Tabular Data with Interpretable Meta-Encoding
• Evaluating Autoencoders for Parametric and Invertible Multidimensional Projections

Related

• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• Uniform Manifold Approximation and Projection (UMAP) and its Variants: Tutorial and Survey
• Relating tSNE and UMAP to Classical Dimensionality Reduction
• Facilitate the Parametric Dimension Reduction by Gradient Clipping
• GNUMAP: A Parameter-Free Approach to Unsupervised Dimensionality Reduction via Graph Neural Networks