A Channel Coding Benchmark for Meta-Learning

2021/07/15 by Rui Li, Ondrej Bohdal, Li, Rui +13 · 4 citations
Computer Science · Engineering · Mathematics · #Artificial Intelligence (cs.AI) #Artificial intelligence #Benchmark (surveying) #Channel (broadcasting) #Coding (social sciences) #Computer science #Domain Adaptation and Few-Shot Learning #Engineering #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning and Data Classification #Machine learning #Mathematics #Meta learning (computer science) #Multi-task learning #Multimodal Machine Learning Applications #Task (project management) #cs.AI #cs.IT #cs.LG #math.IT

paper · pdf · doi:10.48550/arxiv.2107.07579

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2021/07/15 · arxiv created 2021/12/02 · arxiv updated 2021/12/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

Meta-learning provides a popular and effective family of methods for data-efficient learning of new tasks. However, several important issues in meta-learning have proven hard to study thus far. For example, performance degrades in real-world settings where meta-learners must learn from a wide and potentially multi-modal distribution of training tasks; and when distribution shift exists between meta-train and meta-test task distributions. These issues are typically hard to study since the shape of task distributions, and shift between them are not straightforward to measure or control in standard benchmarks. We propose the channel coding problem as a benchmark for meta-learning. Channel coding is an important practical application where task distributions naturally arise, and fast adaptation to new tasks is practically valuable. We use our MetaCC benchmark to study several aspects of meta-learning, including the impact of task distribution breadth and shift, which can be controlled in the coding problem. Going forward, MetaCC provides a tool for the community to study the capabilities and limitations of meta-learning, and to drive research on practically robust and effective meta-learners.

Citations

• On Information and Sufficiency
• Human-level concept learning through probabilistic program induction
• Prototypical Networks for Few-shot Learning
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments
• mixup: Beyond Empirical Risk Minimization
• On First-Order Meta-Learning Algorithms
• Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation
• Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
• BOIL: Towards Representation Change for Few-shot Learning

Related

• Advances and Challenges in Meta-Learning: A Technical Review
• Towards Task Sampler Learning for Meta-Learning
• Improving Generalization in Meta-learning via Task Augmentation
• Diverse Distributions of Self-Supervised Tasks for Meta-Learning in NLP
• Task-Distributionally Robust Data-Free Meta-Learning