Domain Generalization via Invariant Feature Representation

2013/01/10 by Krikamol Muandet, David Balduzzi, Muandet, Krikamol +2 · 65 citations
Computer Science · Biochemistry, Genetics and Molecular Biology · #Domain Adaptation and Few-Shot Learning #Cancer-related molecular mechanisms research #Machine Learning and ELM

paper · pdf · doi:10.48550/arxiv.1301.2115

Abstract

This paper investigates domain generalization: How to take knowledge acquired from an arbitrary number of related domains and apply it to previously unseen domains? We propose Domain-Invariant Component Analysis (DICA), a kernel-based optimization algorithm that learns an invariant transformation by minimizing the dissimilarity across domains, whilst preserving the functional relationship between input and output variables. A learning-theoretic analysis shows that reducing dissimilarity improves the expected generalization ability of classifiers on new domains, motivating the proposed algorithm. Experimental results on synthetic and real-world datasets demonstrate that DICA successfully learns invariant features and improves classifier performance in practice.

Cited by

• Warmer for Less: A Cost-Efficient Strategy for Cold-Start Recommendations at Pinterest
• Learning to See Through a Baby's Eyes: Early Visual Diets Enable Robust Visual Intelligence in Humans and Machines
• S2R-DepthNet: Learning a Generalizable Depth-specific Structural Representation
• Quantifying and Improving Transferability in Domain Generalization
• Domain Adaptation as a Problem of Inference on Graphical Models
• Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation
• Towards Universal Representation Learning for Deep Face Recognition
• Collaborating Foundation Models for Domain Generalized Semantic Segmentation
• Nonlinear Invariant Risk Minimization: A Causal Approach
• Anomalous Example Detection in Deep Learning: A Survey
• Self-Challenging Improves Cross-Domain Generalization
• Embracing the Dark Knowledge: Domain Generalization Using Regularized Knowledge Distillation
• The Connection between Out-of-Distribution Generalization and Privacy of ML Models
• Fundamental Limits and Tradeoffs in Invariant Representation Learning
• Performative Learning Theory
• An Analysis of Causal Effect Estimation using Outcome Invariant Data Augmentation
• Balance-Subsampled Stable Prediction
• AdvBlur: Adversarial Blur for Robust Diabetic Retinopathy Classification and Cross-Domain Generalization
• Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach
• SCMD: A Kernel-Based Distance for Structural Causal Models to Quantify Transferability Across Environments
• Moment-Based Domain Adaptation: Learning Bounds and Algorithms
• Recent Advances in Transfer Learning for Cross-Dataset Visual Recognition: A Problem-Oriented Perspective
• Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalization
• Self-Training with Dynamic Weighting for Robust Gradual Domain Adaptation
• Effect-Invariant Mechanisms for Policy Generalization
• Exploring Data Aggregation and Transformations to Generalize across Visual Domains
• Sequential Learning for Domain Generalization
• Domain Generalization Under Posterior Drift
• Learning explanations that are hard to vary
• Learning to Diversify for Single Domain Generalization
• Progressive Domain Expansion Network for Single Domain Generalization
• Dual-supervised Asymmetric Co-training for Semi-supervised Medical Domain Generalization
• Learning to see across Domains and Modalities
• Selecting Data Augmentation for Simulating Interventions
• An Information-theoretic Approach to Distribution Shifts
• Domain Generalization via Multidomain Discriminant Analysis
• FSDR: Frequency Space Domain Randomization for Domain Generalization
• Transfer Metric Learning: Algorithms, Applications and Outlooks
• SWAD: Domain Generalization by Seeking Flat Minima
• "A 6 or a 9?": Ensemble Learning Through the Multiplicity of Performant Models and Explanations
• Robust and Adaptive Spectral Method for Representation Multi-Task Learning with Contamination
• Respecting Domain Relations: Hypothesis Invariance for Domain Generalization
• Generalizable Model-agnostic Semantic Segmentation via Target-specific Normalization
• Single Domain Generalization in Diabetic Retinopathy: A Neuro-Symbolic Learning Approach
• Towards Out-Of-Distribution Generalization: A Survey
• Quantization Meets OOD: Generalizable Quantization-aware Training from a Flatness Perspective
• AdaRNN: Adaptive Learning and Forecasting of Time Series
• Domain Generalization in-the-Wild: Disentangling Classification from Domain-Aware Representations
• A Review of Single-Source Deep Unsupervised Visual Domain Adaptation
• Stable Prediction across Unknown Environments
• Learning to Learn Single Domain Generalization
• In Search of Lost Domain Generalization
• Boosting Unconstrained Face Recognition with Auxiliary Unlabeled Data
• Global Convergence and Generalization Bound of Gradient-Based Meta-Learning with Deep Neural Nets
• Domain Agnostic Learning for Unbiased Authentication
• Hierarchical Adaptive networks with Task vectors for Test-Time Adaptation
• Domain Generalization of Pathological Image Segmentation by Patch-Level and WSI-Level Contrastive Learning
• Confounder Identification-free Causal Visual Feature Learning
• Affective Image Content Analysis: Two Decades Review and New Perspectives
• Multimodal Causal-Driven Representation Learning for Generalizable Medical Image Segmentation
• A Fourier-based Framework for Domain Generalization
• Where and How to Enhance: Discovering Bit-Width Contribution for Mixed Precision Quantization
• Two Sides of Meta-Learning Evaluation: In vs. Out of Distribution
• Cross-Architecture Distillation Made Simple with Redundancy Suppression
• Federated Learning with Domain Generalization

Related

• Learning to Balance Specificity and Invariance for In and Out of Domain Generalization
• Respecting Domain Relations: Hypothesis Invariance for Domain Generalization
• Domain Generalization via Multidomain Discriminant Analysis
• Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalization
• Cross-Domain Feature Augmentation for Domain Generalization