Bridging Theory and Algorithm for Domain Adaptation

2019/04/11 by Yuchen Zhang, Tianle Liu, Zhang, Yuchen +5 · 18 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and ELM #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.1904.05801

openalex publication_date 2019/04/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper addresses the problem of unsupervised domain adaption from theoretical and algorithmic perspectives. Existing domain adaptation theories naturally imply minimax optimization algorithms, which connect well with the domain adaptation methods based on adversarial learning. However, several disconnections still exist and form the gap between theory and algorithm. We extend previous theories (Mansour et al., 2009c; Ben-David et al., 2010) to multiclass classification in domain adaptation, where classifiers based on the scoring functions and margin loss are standard choices in algorithm design. We introduce Margin Disparity Discrepancy, a novel measurement with rigorous generalization bounds, tailored to the distribution comparison with the asymmetric margin loss, and to the minimax optimization for easier training. Our theory can be seamlessly transformed into an adversarial learning algorithm for domain adaptation, successfully bridging the gap between theory and algorithm. A series of empirical studies show that our algorithm achieves the state of the art accuracies on challenging domain adaptation tasks.

Citations

• Deep Domain Confusion: Maximizing for Domain Invariance
• Domain Adaptation: Learning Bounds and Algorithms
• VisDA: The Visual Domain Adaptation Challenge
• Adaptation Algorithm and Theory Based on Generalized Discrepancy

Cited by

• C-DGPA: Class-Centric Dual-Alignment Generative Prompt Adaptation
• Enhancing Node-Level Graph Domain Adaptation by Alleviating Local Dependency
• Free Lunch to Meet the Gap: Intermediate Domain Reconstruction for Cross-Domain Few-Shot Learning
• Unsupervised Robust Domain Adaptation: Paradigm, Theory and Algorithm
• Cross-domain Self-supervised Learning for Domain Adaptation with Few Source Labels
• Dual Adversarial Domain Adaptation
• Robust Cross-Domain Adaptation in Texture Features Transferring for Wood Chip Moisture Content Prediction
• VirDA: Reusing Backbone for Unsupervised Domain Adaptation with Visual Reprogramming
• Beyond Real Data: Synthetic Data through the Lens of Regularization
• Domain Conditioned Adaptation Network
• On the Theoretical Equivalence of Several Trade-Off Curves Assessing Statistical Proximity
• Cross-Domain Attribute Alignment with CLIP: A Rehearsal-Free Approach for Class-Incremental Unsupervised Domain Adaptation
• Domain Adaptation via Feature Refinement
• Information-theoretic analysis for transfer learning
• Against Adversarial Learning: Naturally Distinguish Known and Unknown in Open Set Domain Adaptation
• TRUST: Leveraging Text Robustness for Unsupervised Domain Adaptation
• Bridging Simulation and Experiment: A Self-Supervised Domain Adaptation Framework for Concrete Damage Classification
• Rein++: Efficient Generalization and Adaptation for Semantic Segmentation with Vision Foundation Models

Related

• Adaptation Algorithm and Theory Based on Generalized Discrepancy
• A survey on domain adaptation theory: learning bounds and theoretical guarantees
• A General Upper Bound for Unsupervised Domain Adaptation
• A Brief Review of Domain Adaptation
• Unsupervised Visual Domain Adaptation: A Deep Max-Margin Gaussian Process Approach