Generalizable Spectral Embedding with an Application to UMAP

2025/01/20 by Ben-Ari, Nir, Yacobi, Amitai, Shaham, Uri · 1 citation
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.2501.11305

Abstract

Spectral Embedding (SE) is a popular method for dimensionality reduction, applicable across diverse domains. Nevertheless, its current implementations face three prominent drawbacks which curtail its broader applicability: generalizability (i.e., out-of-sample extension), scalability, and eigenvectors separation. Existing SE implementations often address two of these drawbacks; however, they fall short in addressing the remaining one. In this paper, we introduce Sep-SpectralNet (eigenvector-separated SpectralNet), a SE implementation designed to address all three limitations. Sep-SpectralNet extends SpectralNet with an efficient post-processing step to achieve eigenvectors separation, while ensuring both generalizability and scalability. This method expands the applicability of SE to a wider range of tasks and can enhance its performance in existing applications. We empirically demonstrate Sep-SpectralNet's ability to consistently approximate and generalize SE, while maintaining SpectralNet's scalability. Additionally, we show how Sep-SpectralNet can be leveraged to enable generalizable UMAP visualization. Our codes are publicly available.

Cited by

• Learning Shared Representations from Unpaired Data

Related

• Compressive spectral embedding: sidestepping the SVD
• SpecNet2: Orthogonalization-free spectral embedding by neural networks
• Incrementally Updated Spectral Embeddings
• Learning Structure Aware Deep Spectral Embedding
• BASiS: Batch Aligned Spectral Embedding Space