PI-H2T: Enhancing Long-Tailed Visual Recognition with Permutation-Invariant and Head-to-Tail Feature Fusion

2025/05/31 by Mengke Li, Zhikai Hu, Li, Mengke +9
#cs.CV

paper · pdf · doi:10.48550/arxiv.2506.00625

Abstract

The imbalanced distribution of long-tailed data presents a significant challenge for deep learning models, causing them to prioritize head classes while neglecting tail classes. Two key factors contributing to low recognition accuracy are the deformed representation space and a biased classifier, stemming from insufficient semantic information in tail classes. To address these issues, we propose permutation-invariant and head-to-tail feature fusion (PI-H2T), a highly adaptable method. PI-H2T enhances the representation space through permutation-invariant representation fusion (PIF), yielding more clustered features and automatic class margins. Additionally, it adjusts the biased classifier by transferring semantic information from head to tail classes via head-to-tail fusion (H2TF), improving tail class diversity. Theoretical analysis and experiments show that PI-H2T optimizes both the representation space and decision boundaries. Its plug-and-play design ensures seamless integration into existing methods, providing a straightforward path to further performance improvements. Extensive experiments on long-tailed benchmarks confirm the effectiveness of PI-H2T.

Citations

• LoRA: Low-Rank Adaptation of Large Language Models
• Attention Is All You Need
• Deep Residual Learning for Image Recognition

Related

• Feature Fusion from Head to Tail for Long-Tailed Visual Recognition
• Deep Long-Tailed Learning: A Survey
• Constructing Balance from Imbalance for Long-tailed Image Recognition
• Distribution Alignment: A Unified Framework for Long-tail Visual Recognition
• Long-tailed Recognition by Learning from Latent Categories