Learning Bound for Parameter Transfer Learning

2016/10/27 by Wataru Kumagai, Kumagai, Wataru · 3 citations
Computer Science · Engineering · Mathematics · #Domain Adaptation and Few-Shot Learning #Machine Learning and ELM #Sparse and Compressive Sensing Techniques #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1610.08696

This paper was accepted at NIPS 2016 as a poster presentation

arxiv created 2017/01/18 · arxiv updated 2017/01/19

Abstract

We consider a transfer-learning problem by using the parameter transfer approach, where a suitable parameter of feature mapping is learned through one task and applied to another objective task. Then, we introduce the notion of the local stability and parameter transfer learnability of parametric feature mapping,and thereby derive a learning bound for parameter transfer algorithms. As an application of parameter transfer learning, we discuss the performance of sparse coding in self-taught learning. Although self-taught learning algorithms with plentiful unlabeled data often show excellent empirical performance, their theoretical analysis has not been studied. In this paper, we also provide the first theoretical learning bound for self-taught learning.

Cited by

• Recent Advances in Transfer Learning for Cross-Dataset Visual Recognition: A Problem-Oriented Perspective
• Few-shot Domain Adaptation by Causal Mechanism Transfer
• G2R Bound: A Generalization Bound for Supervised Learning from GAN-Synthetic Data

Related

• Theoretical Guarantees of Transfer Learning
• Limits of Transfer Learning
• Generalization Performance of Transfer Learning: Overparameterized and Underparameterized Regimes
• Feasibility of Transfer Learning: A Mathematical Framework
• Scaling Laws for Transfer