f-Domain-Adversarial Learning: Theory and Algorithms

2021/06/21 by David Acuna, Acuna, David, Guojun Zhang +5 · 8 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #cs.AI #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.2106.11344

ICML 2021

arxiv created 2021/06/21 · arxiv updated 2021/06/23

Abstract

Unsupervised domain adaptation is used in many machine learning applications where, during training, a model has access to unlabeled data in the target domain, and a related labeled dataset. In this paper, we introduce a novel and general domain-adversarial framework. Specifically, we derive a novel generalization bound for domain adaptation that exploits a new measure of discrepancy between distributions based on a variational characterization of f-divergences. It recovers the theoretical results from Ben-David et al. (2010a) as a special case and supports divergences used in practice. Based on this bound, we derive a new algorithmic framework that introduces a key correction in the original adversarial training method of Ganin et al. (2016). We show that many regularizers and ad-hoc objectives introduced over the last years in this framework are then not required to achieve performance comparable to (if not better than) state-of-the-art domain-adversarial methods. Experimental analysis conducted on real-world natural language and computer vision datasets show that our framework outperforms existing baselines, and obtains the best results for f-divergences that were not considered previously in domain-adversarial learning.

Cited by

• DAMA: A Unified Accelerated Approach for Decentralized Nonconvex Minimax Optimization-Part I: Algorithm Development and Results
• GREAT: Generalizable Representation Enhancement via Auxiliary Transformations for Zero-Shot Environmental Prediction
• Confidence and Dispersity as Signals: Unsupervised Model Evaluation and Ranking
• Euclid's Gift: Enhancing Spatial Perception and Reasoning in Vision-Language Models via Geometric Surrogate Tasks
• Feature-Space Planes Searcher: A Universal Domain Adaptation Framework for Interpretability and Computational Efficiency
• On the Hardness of Unsupervised Domain Adaptation: Optimal Learners and Information-Theoretic Perspective
• IDAL: Improved Domain Adaptive Learning for Natural Images Dataset
• Reliably Detecting Model Failures in Deployment Without Labels

Related

• On f-Divergence Principled Domain Adaptation: An Improved Framework
• DART: A Principled Approach to Adversarially Robust Unsupervised Domain Adaptation
• Adversarial-Learned Loss for Domain Adaptation
• A General Upper Bound for Unsupervised Domain Adaptation
• Multi-step domain adaptation by adversarial attack to H ΔH-divergence