The Effect of Diversity in Meta-Learning

2022/01/27 by Ramnath Kumar, Kumar, Ramnath, Tristan Deleu +3 · 1 citation
Computer Science · #Domain Adaptation and Few-Shot Learning #Machine Learning and Data Classification #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.2201.11775

Abstract

Recent studies show that task distribution plays a vital role in the meta-learner's performance. Conventional wisdom is that task diversity should improve the performance of meta-learning. In this work, we find evidence to the contrary; (i) our experiments draw into question the efficacy of our learned models: similar manifolds can be learned with a subset of the data (lower task diversity). This finding questions the advantage of providing more data to the model, and (ii) adding diversity to the task distribution (higher task diversity) sometimes hinders the model and does not lead to a significant improvement in performance as previously believed. To strengthen our findings, we provide both empirical and theoretical evidence.

Cited by

• Statistical Insight into Meta-Learning via Predictor Subspace Characterization and Quantification of Task Diversity

Related

• The Curse of Zero Task Diversity: On the Failure of Transfer Learning to Outperform MAML and their Empirical Equivalence
• Towards Task Sampler Learning for Meta-Learning
• Statistical Insight into Meta-Learning via Predictor Subspace Characterization and Quantification of Task Diversity
• The Curse of Low Task Diversity: On the Failure of Transfer Learning to Outperform MAML and Their Empirical Equivalence
• Meta-Learning and representation learner: A short theoretical note