When does MAML Work the Best? An Empirical Study on Model-Agnostic Meta-Learning in NLP Applications

2020/05/24 by Zequn Liu, Ruiyi Zhang, Liu, Zequn +5
Computer Science · #Topic Modeling #Domain Adaptation and Few-Shot Learning #Natural Language Processing Techniques

paper · pdf · doi:10.48550/arxiv.2005.11700

Abstract

Model-Agnostic Meta-Learning (MAML), a model-agnostic meta-learning method, is successfully employed in NLP applications including few-shot text classification and multi-domain low-resource language generation. Many impacting factors, including data quantity, similarity among tasks, and the balance between general language model and task-specific adaptation, can affect the performance of MAML in NLP, but few works have thoroughly studied them. In this paper, we conduct an empirical study to investigate these impacting factors and conclude when MAML works the best based on the experimental results.

Related

• Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Tasks
• Does MAML Only Work via Feature Re-use? A Data Centric Perspective
• MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learning
• On the Importance of Attention in Meta-Learning for Few-Shot Text Classification
• Meta-Learning for Natural Language Understanding under Continual Learning Framework