Convergence analysis of t-SNE as a gradient flow for point cloud on a manifold

2024/01/31 by Seonghyeon Jeong, Hau‐Tieng Wu, Jeong, Seonghyeon +1 · 1 citation
Computer Science · Engineering · Medicine · #3D Shape Modeling and Analysis #90C26 #90C30 #Computer Graphics and Visualization Techniques #Data Structures and Algorithms (cs.DS) #F.2.0 #F.2.2 #FOS: Computer and information sciences #G.4 #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Winter Sports Injuries and Performance

paper · pdf · doi:10.48550/arxiv.2401.17675

openalex publication_date 2024/01/31 · openalex created_date 2024/02/02 · openalex updated_date 2026/07/28

Abstract

We present a theoretical foundation regarding the boundedness of the t-SNE algorithm. t-SNE employs gradient descent iteration with Kullback-Leibler (KL) divergence as the objective function, aiming to identify a set of points that closely resemble the original data points in a high-dimensional space, minimizing KL divergence. Investigating t-SNE properties such as perplexity and affinity under a weak convergence assumption on the sampled dataset, we examine the behavior of points generated by t-SNE under continuous gradient flow. Demonstrating that points generated by t-SNE remain bounded, we leverage this insight to establish the existence of a minimizer for KL divergence.

Cited by

• t-SNE Exaggerates Clusters, Provably

Related

• Convergence properties of natural gradient descent for minimizing KL divergence
• Convergence Properties of Natural Gradient Descent for Minimizing KL Divergence
• KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint\n Support
• Gradients should stay on Path: Better Estimators of the Reverse- and Forward KL Divergence for Normalizing Flows
• Linear Convergence of Diffusion Models Under the Manifold Hypothesis