Invariant Meta Learning for Out-of-Distribution Generalization

2023/01/26 by Penghao Jiang, Xin Ke, Jiang, Penghao +5
Computer Science · Biochemistry, Genetics and Molecular Biology · #Domain Adaptation and Few-Shot Learning #Speech Recognition and Synthesis #Cancer-related molecular mechanisms research

paper · pdf · doi:10.48550/arxiv.2301.11779

Abstract

Modern deep learning techniques have illustrated their excellent capabilities in many areas, but relies on large training data. Optimization-based meta-learning train a model on a variety tasks, such that it can solve new learning tasks using only a small number of training samples.However, these methods assumes that training and test dataare identically and independently distributed. To overcome such limitation, in this paper, we propose invariant meta learning for out-of-distribution tasks. Specifically, invariant meta learning find invariant optimal meta-initialization,and fast adapt to out-of-distribution tasks with regularization penalty. Extensive experiments demonstrate the effectiveness of our proposed invariant meta learning on out-of-distribution few-shot tasks.

Related

• An Information-Theoretic Analysis of Out-of-Distribution Generalization in Meta-Learning with Applications to Meta-RL
• Robust Meta Learning for Image based tasks
• Meta Dropout: Learning to Perturb Features for Generalization
• Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution Generalization
• Generating meta-learning tasks to evolve parametric loss for classification learning