Long-Tailed Recognition via Information-Preservable Two-Stage Learning

2025/10/09 by Fudong Lin, Yuan Xu, Lin, Fudong +1
Computer Science · #Imbalanced Data Classification Techniques #Machine Learning and Data Classification #Data Stream Mining Techniques

paper · pdf · doi:10.48550/arxiv.2510.08836

Abstract

The imbalance (or long-tail) is the nature of many real-world data distributions, which often induces the undesirable bias of deep classification models toward frequent classes, resulting in poor performance for tail classes. In this paper, we propose a novel two-stage learning approach to mitigate such a majority-biased tendency while preserving valuable information within datasets. Specifically, the first stage proposes a new representation learning technique from the information theory perspective. This approach is theoretically equivalent to minimizing intra-class distance, yielding an effective and well-separated feature space. The second stage develops a novel sampling strategy that selects mathematically informative instances, able to rectify majority-biased decision boundaries without compromising a model's overall performance. As a result, our approach achieves the state-of-the-art performance across various long-tailed benchmark datasets, validated via extensive experiments. Our code is available at https://github.com/fudong03/BNS_IPDPP.

Citations

• ECGN: A Cluster-Aware Approach to Graph Neural Networks for Imbalanced Classification
• KAN: Kolmogorov-Arnold Networks
• MIM4DD: Mutual Information Maximization for Dataset Distillation
• Mamba: Linear-Time Sequence Modeling with Selective State Spaces
• Subclass-balancing Contrastive Learning for Long-tailed Recognition
• Scaling Language-Image Pre-training via Masking
• Targeted Supervised Contrastive Learning for Long-Tailed Recognition
• Masked Autoencoders Are Scalable Vision Learners
• Self-supervised Learning is More Robust to Dataset Imbalance
• Parametric Contrastive Learning
• BEiT: BERT Pre-Training of Image Transformers
• An Empirical Study of Training Self-Supervised Vision Transformers
• Training data-efficient image transformers & distillation through attention
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Long-tailed Recognition by Routing Diverse Distribution-Aware Experts
• Long-tail learning via logit adjustment
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• Rethinking the Value of Labels for Improving Class-Imbalanced Learning
• A Simple Framework for Contrastive Learning of Visual Representations
• BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition
• Momentum Contrast for Unsupervised Visual Representation Learning
• Momentum Contrast for Unsupervised Visual Representation Learning
• Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss
• Large-Scale Long-Tailed Recognition in an Open World
• Generative Adversarial Minority Oversampling
• Class-Balanced Loss Based on Effective Number of Samples
• Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning
• Attention Is All You Need
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Factors in Finetuning Deep Model for object detection
• Deep Residual Learning for Image Recognition
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distributed Representations of Words and Phrases and their Compositionality
• ImageNet classification with deep convolutional neural networks
• Determinantal point processes for machine learning
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Related

• A Survey of Deep Long-Tail Classification Advancements
• Investigate the Essence of Long-Tailed Recognition from a Unified Perspective
• Solving the long-tailed distribution problem by exploiting the synergies and balance of different techniques
• LTRL: Boosting Long-tail Recognition via Reflective Learning
• Deep Long-Tailed Learning: A Survey