Spread Spurious Attribute: Improving Worst-group Accuracy with Spurious Attribute Estimation

2022/04/05 by Junhyun Nam, Nam, Junhyun, Jaehyung Kim +5 · 13 citations
Computer Science · #Anomaly Detection Techniques and Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Imbalanced Data Classification Techniques #Machine Learning (cs.LG) #Machine Learning and Data Classification #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.2204.02070

ICLR 2022 camera ready

arxiv created 2022/04/05 · openalex publication_date 2022/04/05 · arxiv updated 2022/04/06 · openalex created_date 2022/05/05 · openalex updated_date 2026/07/28

Abstract

The paradigm of worst-group loss minimization has shown its promise in avoiding to learn spurious correlations, but requires costly additional supervision on spurious attributes. To resolve this, recent works focus on developing weaker forms of supervision -- e.g., hyperparameters discovered with a small number of validation samples with spurious attribute annotation -- but none of the methods retain comparable performance to methods using full supervision on the spurious attribute. In this paper, instead of searching for weaker supervisions, we ask: Given access to a fixed number of samples with spurious attribute annotations, what is the best achievable worst-group loss if we "fully exploit" them? To this end, we propose a pseudo-attribute-based algorithm, coined Spread Spurious Attribute (SSA), for improving the worst-group accuracy. In particular, we leverage samples both with and without spurious attribute annotations to train a model to predict the spurious attribute, then use the pseudo-attribute predicted by the trained model as supervision on the spurious attribute to train a new robust model having minimal worst-group loss. Our experiments on various benchmark datasets show that our algorithm consistently outperforms the baseline methods using the same number of validation samples with spurious attribute annotations. We also demonstrate that the proposed SSA can achieve comparable performances to methods using full (100%) spurious attribute supervision, by using a much smaller number of annotated samples -- from 0.6% and up to 1.5%, depending on the dataset.

Cited by

• Let Samples Speak: Mitigating Spurious Correlation by Exploiting the Clusterness of Samples
• SAGE: Spuriousness-Aware Guided Prompt Exploration for Mitigating Multimodal Bias
• Mitigating Spurious Correlation via Distributionally Robust Learning with Hierarchical Ambiguity Sets
• GroupCoOp: Group-robust Fine-tuning via Group Prompt Learning
• ACE and Diverse Generalization via Selective Disagreement
• Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning
• Controllable Feature Whitening for Hyperparameter-Free Bias Mitigation
• Improving Group Robustness on Spurious Correlation via Evidential Alignment
• NeuronTune: Towards Self-Guided Spurious Bias Mitigation
• ShortcutProbe: Probing Prediction Shortcuts for Learning Robust Models
• An XAI-based Analysis of Shortcut Learning in Neural Networks
• Invisible Shortcuts: Why Vision Encoders Know Your Camera
• Steering CLIP's vision transformer with sparse autoencoders

Related

• Improving Group Robustness on Spurious Correlation Requires Preciser Group Inference
• Examining and Combating Spurious Features under Distribution Shift
• Trained Models Tell Us How to Make Them Robust to Spurious Correlation without Group Annotation
• Out of Spuriousity: Improving Robustness to Spurious Correlations without Group Annotations
• Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations