Mitigating Spurious Correlations in NLI via LLM-Synthesized Counterfactuals and Dynamic Balanced Sampling

2025/12/20 by Christopher Román Jaimes, Jaimes, Christopher Román
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling

paper · doi:10.48550/arxiv.2512.18462

openalex publication_date 2025/12/20 · openalex created_date 2025/12/24 · openalex updated_date 2026/07/28

Abstract

Natural Language Inference (NLI) models frequently rely on spurious correlations rather than semantic reasoning. Existing mitigation strategies often incur high annotation costs or trigger catastrophic forgetting during fine-tuning. We propose an automated, scalable pipeline to address these limitations. First, we introduce Log-Frequency LMI (LF-LMI) to accurately detect semantic artifacts. Second, we generate a high-quality synthetic contrast set via an LLM-synthesis pipeline with multi-judge verification. Finally, we introduce Dynamic Balanced Sampling, a training strategy that rotates the original data distribution to prevent forgetting. Our method improves consistency on a challenging benchmark from 63.5% to 81.0% while maintaining 88.4% in-domain accuracy, significantly outperforming naive fine-tuning.

Citations

• An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
• Towards Debiasing Fact Verification Models
• Hypothesis Only Baselines in Natural Language Inference
• Annotation Artifacts in Natural Language Inference Data
• A large annotated corpus for learning natural language inference

Related

• LLMs for Generating and Evaluating Counterfactuals: A Comprehensive Study
• Neutralizing Bias in LLM Reasoning using Entailment Graphs
• Mitigating Spurious Correlations in LLMs via Causality-Aware Post-Training
• Multi-Scales Data Augmentation Approach In Natural Language Inference For Artifacts Mitigation And Pre-Trained Model Optimization
• Enhancing NLP Robustness and Generalization through LLM-Generated Contrast Sets: A Scalable Framework for Systematic Evaluation and Adversarial Training