A Nested Bi-level Optimization Framework for Robust Few Shot Learning

2020/11/13 by Killamsetty, Krishnateja, Li, Changbin, Zhao, Chen +2
#FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2011.06782

Abstract

Model-Agnostic Meta-Learning (MAML), a popular gradient-based meta-learning framework, assumes that the contribution of each task or instance to the meta-learner is equal. Hence, it fails to address the domain shift between base and novel classes in few-shot learning. In this work, we propose a novel robust meta-learning algorithm, NestedMAML, which learns to assign weights to training tasks or instances. We consider weights as hyper-parameters and iteratively optimize them using a small set of validation tasks set in a nested bi-level optimization approach (in contrast to the standard bi-level optimization in MAML). We then apply NestedMAML in the meta-training stage, which involves (1) several tasks sampled from a distribution different from the meta-test task distribution, or (2) some data samples with noisy labels. Extensive experiments on synthetic and real-world datasets demonstrate that NestedMAML efficiently mitigates the effects of "unwanted" tasks or instances, leading to significant improvement over the state-of-the-art robust meta-learning methods.

Related

• Gradient-Based Meta-Learning Using Uncertainty to Weigh Loss for Few-Shot Learning
• Repurposing Pretrained Models for Robust Out-of-domain Few-Shot Learning
• Fast Adaptation with Kernel and Gradient based Meta Leaning
• Stress Testing of Meta-learning Approaches for Few-shot Learning
• Task-Agnostic Meta-Learning for Few-shot Learning