Fine-Grained Class-Conditional Distribution Balancing for Debiased Learning

2025/05/11 by Miaoyun Zhao, Qiang Zhang, Zhao, Miaoyun +2
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Face and Expression Recognition #Imbalanced Data Classification Techniques

paper · pdf · doi:10.48550/arxiv.2505.06831

openalex publication_date 2025/05/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Achieving group-robust generalization in the presence of spurious correlations remains a significant challenge, particularly when bias annotations are unavailable. Recent studies on Class-Conditional Distribution Balancing (CCDB) reveal that spurious correlations often stem from mismatches between the class-conditional and marginal distributions of bias attributes. They achieve promising results by addressing this issue through simple distribution matching in a bias-agnostic manner. However, CCDB approximates each distribution using a single Gaussian, which is overly simplistic and rarely holds in real-world applications. To address this limitation, we propose a novel Multi-stage data-Selective reTraining strategy (MST), which describes each distribution in greater detail using the hard confusion matrix. Building on these finer descriptions, we propose a fine-grained variant of CCDB, termed FG-CCDB, which enhances distribution matching through more precise confusion-cell-wise reweighting. FG-CCDB learns sample weights from a global perspective, effectively mitigating spurious correlations without incurring substantial storage or computational overhead. Extensive experiments demonstrate that MST serves as a reliable proxy for ground-truth bias annotations and can be seamlessly integrated with bias-supervised methods. Moreover, when combined with FG-CCDB, our method performs on par with bias-supervised approaches on binary classification tasks and significantly outperforms them in highly biased multi-class and multi-shortcut scenarios.

Citations

• Class-Conditional Distribution Balancing for Group Robust Classification
• Gradient Extrapolation for Debiased Representation Learning
• Diffusing DeBias: Synthetic Bias Amplification for Model Debiasing
• Mitigating Spurious Correlations via Disagreement Probability
• ExMap: Leveraging Explainability Heatmaps for Unsupervised Group Robustness to Spurious Correlations
• Group Robust Classification Without Any Group Information
• Discovering environments with XRM
• Towards Last-layer Retraining for Group Robustness with Fewer Annotations
• Bias Amplification Enhances Minority Group Performance
• Don't blame Dataset Shift! Shortcut Learning due to Gradients and Cross Entropy
• Discovering and Mitigating Visual Biases through Keyword Explanation
• Model Agnostic Sample Reweighting for Out-of-Distribution Learning
• A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Others
• Avoiding spurious correlations via logit correction
• Stable Learning via Sparse Variable Independence
• MaskTune: Mitigating Spurious Correlations by Forcing to Explore
• Discovering Bugs in Vision Models using Off-the-shelf Image Generation and Captioning
• Discover and Mitigate Unknown Biases with Debiasing Alternate Networks
• Distilling Model Failures as Directions in Latent Space
• Revisiting the Importance of Amplifying Bias for Debiasing
• Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations
• Improving Out-of-Distribution Robustness via Selective Augmentation
• Learning Debiased and Disentangled Representations for Semantic Segmentation
• Just Train Twice: Improving Group Robustness without Training Group\n Information
• Invariant Information Bottleneck for Domain Generalization
• Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization
• Deep Stable Learning for Out-Of-Distribution Generalization
• Learning Transferable Visual Models From Natural Language Supervision
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems
• The Pitfalls of Simplicity Bias in Neural Networks
• Shortcut learning in deep neural networks
• Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
• Learning De-biased Representations with Biased Representations
• Invariant Risk Minimization
• Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
• Nuanced Metrics for Measuring Unintended Bias with Real Data for Text\n Classification
• Recognition in Terra Incognita
• Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
• Equality of Opportunity in Supervised Learning
• Deep Residual Learning for Image Recognition
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Related

• Class-Conditional Distribution Balancing for Group Robust Classification
• Group Robust Classification Without Any Group Information
• Mitigating Spurious Correlation via Distributionally Robust Learning with Hierarchical Ambiguity Sets
• Mitigating Clever Hans Strategies in Image Classifiers through Generating Counterexamples
• The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlations