PAC-Bayesian Theorems for Domain Adaptation with Specialization to Linear Classifiers

2015/03/24 by Pascal Germain, Germain, Pascal, Amaury Habrard +5
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.1503.06944

openalex publication_date 2015/03/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we provide two main contributions in PAC-Bayesian theory for domain adaptation where the objective is to learn, from a source distribution, a well-performing majority vote on a different target distribution. On the one hand, we propose an improvement of the previous approach proposed by Germain et al. (2013), that relies on a novel distribution pseudodistance based on a disagreement averaging, allowing us to derive a new tighter PAC-Bayesian domain adaptation bound for the stochastic Gibbs classifier. We specialize it to linear classifiers, and design a learning algorithm which shows interesting results on a synthetic problem and on a popular sentiment annotation task. On the other hand, we generalize these results to multisource domain adaptation allowing us to take into account different source domains. This study opens the door to tackle domain adaptation tasks by making use of all the PAC-Bayesian tools.

Citations

• Domain Adaptation: Learning Bounds and Algorithms
• A Note on the PAC Bayesian Theorem
• A PAC-Bayesian bound for Lifelong Learning

Related

• A New PAC-Bayesian Perspective on Domain Adaptation
• An Improvement to the Domain Adaptation Bound in a PAC-Bayesian context
• PAC-Bayesian Domain Adaptation Bounds for Multi-view learning
• PAC-Bayesian Domain Adaptation Bounds for Multiclass Learners
• PAC-Bayesian inductive and transductive learning