Environment Inference for Invariant Learning

2020/10/14 by Elliot Creager, Jörn-Henrik Jacobsen, Creager, Elliot +3 · 22 citations
Computer Science · #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Algorithms #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.2010.07249

openalex publication_date 2020/10/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Learning models that gracefully handle distribution shifts is central to research on domain generalization, robust optimization, and fairness. A promising formulation is domain-invariant learning, which identifies the key issue of learning which features are domain-specific versus domain-invariant. An important assumption in this area is that the training examples are partitioned into "domains" or "environments". Our focus is on the more common setting where such partitions are not provided. We propose EIIL, a general framework for domain-invariant learning that incorporates Environment Inference to directly infer partitions that are maximally informative for downstream Invariant Learning. We show that EIIL outperforms invariant learning methods on the CMNIST benchmark without using environment labels, and significantly outperforms ERM on worst-group performance in the Waterbirds and CivilComments datasets. Finally, we establish connections between EIIL and algorithmic fairness, which enables EIIL to improve accuracy and calibration in a fair prediction problem.

Citations

• Equality of Opportunity in Supervised Learning
• Fairness and Abstraction in Sociotechnical Systems
• LLM-Supported Manufacturing Mapping Generation
• Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
• Shortcut learning in deep neural networks

Cited by

• PISA: Prioritized Invariant Subgraph Aggregation
• Harnessing Diffusion-Generated Synthetic Images for Fair Image Classification
• Spurious Correlation-Aware Embedding Regularization for Worst-Group Robustness
• Evolving Graph Learning for Out-of-Distribution Generalization in Non-stationary Environments
• ODP-Bench: Benchmarking Out-of-Distribution Performance Prediction
• Mitigating Spurious Correlation via Distributionally Robust Learning with Hierarchical Ambiguity Sets
• Latent Space Factorization in LoRA
• Rectifying Shortcut Behaviors in Preference-based Reward Learning
• Tackling Time-Series Forecasting Generalization via Mitigating Concept Drift
• Universal Adaptive Environment Discovery
• Does LLM Focus on the Right Words? Mitigating Context Bias in LLM-based Recommenders
• BLADE: Bias-Linked Adaptive DEbiasing
• Spatially Grounded Concept-Based Image Classification
• Fairness without Demographics through Adversarially Reweighted Learning
• Learning the Wrong Lessons: Syntactic-Domain Spurious Correlations in Language Models
• Shortcut to Nowhere: Demystifying Deep Spurious Regression
• ACE and Diverse Generalization via Selective Disagreement
• Measuring Bias or Measuring the Task: Understanding the Brittle Nature of LLM Gender Biases
• Multi Attribute Bias Mitigation via Representation Learning
• Face4FairShifts: A Large Image Benchmark for Fairness and Robust Learning across Visual Domains
• Superclass-Guided Representation Disentanglement for Spurious Correlation Mitigation
• Algorithmic Fairness amid Social Determinants: Reflection, Characterization, and Approach
• I³-MRec: Invariant Learning with Information Bottleneck for Incomplete Modality Recommendation

Related

• Repeated Environment Inference for Invariant Learning
• Invariant Learning with Annotation-free Environments
• Decorr: Environment Partitioning for Invariant Learning and OOD Generalization
• ZIN: When and How to Learn Invariance Without Environment Partition?
• On the benefits of representation regularization in invariance based domain generalization