Clipped Hyperbolic Classifiers Are Super-Hyperbolic Classifiers

2021/07/23 by Yunhui Guo, Xudong Wang, Guo, Yunhui +5 · 8 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #Advanced Neural Network Applications #Cell Image Analysis Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2107.11472

openalex publication_date 2021/07/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Hyperbolic space can naturally embed hierarchies, unlike Euclidean space. Hyperbolic Neural Networks (HNNs) exploit such representational power by lifting Euclidean features into hyperbolic space for classification, outperforming Euclidean neural networks (ENNs) on datasets with known semantic hierarchies. However, HNNs underperform ENNs on standard benchmarks without clear hierarchies, greatly restricting HNNs' applicability in practice. Our key insight is that HNNs' poorer general classification performance results from vanishing gradients during backpropagation, caused by their hybrid architecture connecting Euclidean features to a hyperbolic classifier. We propose an effective solution by simply clipping the Euclidean feature magnitude while training HNNs. Our experiments demonstrate that clipped HNNs become super-hyperbolic classifiers: They are not only consistently better than HNNs which already outperform ENNs on hierarchical data, but also on-par with ENNs on MNIST, CIFAR10, CIFAR100 and ImageNet benchmarks, with better adversarial robustness and out-of-distribution detection.

Cited by

• PointCHR: Point Cloud Analysis via Curvature-Aware Hyperbolic Rectification
• Understanding and Improving Hyperbolic Deep Reinforcement Learning
• Adaptive Hyperbolic Kernels: Modulated Embedding in de Branges-Rovnyak Spaces
• Modality Alignment across Trees on Heterogeneous Hyperbolic Manifolds
• HIDISC: A Hyperbolic Framework for Domain Generalization with Generalized Category Discovery
• HyperHELM: Hyperbolic Hierarchy Encoding for mRNA Language Modeling
• Curvature Learning for Generalization of Hyperbolic Neural Networks
• Sharpness-Aware Geometric Defense for Robust Out-Of-Distribution Detection

Related

• Hyperbolic Convolutional Neural Networks
• Hyperbolic Deep Neural Networks: A Survey
• Why are hyperbolic neural networks effective? A study on hierarchical representation capability
• Hyperbolic Neural Networks
• Hyperbolic Graph Convolutional Neural Networks