Meta-Learning with Differentiable Convex Optimization

2019/04/07 by Lee, Kwonjoon, Maji, Subhransu, Ravichandran, Avinash +1 · 17 citations
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.1904.03758

Abstract

Many meta-learning approaches for few-shot learning rely on simple base learners such as nearest-neighbor classifiers. However, even in the few-shot regime, discriminatively trained linear predictors can offer better generalization. We propose to use these predictors as base learners to learn representations for few-shot learning and show they offer better tradeoffs between feature size and performance across a range of few-shot recognition benchmarks. Our objective is to learn feature embeddings that generalize well under a linear classification rule for novel categories. To efficiently solve the objective, we exploit two properties of linear classifiers: implicit differentiation of the optimality conditions of the convex problem and the dual formulation of the optimization problem. This allows us to use high-dimensional embeddings with improved generalization at a modest increase in computational overhead. Our approach, named MetaOptNet, achieves state-of-the-art performance on miniImageNet, tieredImageNet, CIFAR-FS, and FC100 few-shot learning benchmarks. Our code is available at https://github.com/kjunelee/MetaOptNet.

Cited by

• Adaptive receptive field-based spatial-frequency feature reconstruction network for fine-grained few-shot image classification
• CausalFSFG: Rethinking Few-Shot Fine-Grained Visual Categorization from Causal Perspective
• MetaTPT: Meta Test-time Prompt Tuning for Vision-Language Models
• Neural Coherence : Find higher performance to out-of-distribution tasks from few samples
• A Fully First-Order Layer for Differentiable Optimization
• Toward Better Generalization in Few-Shot Learning through the Meta-Component Combination
• Parameter-Free Hypergraph Neural Network for Few-Shot Node Classification
• Neural Variational Dropout Processes
• Connecting Giants: Synergistic Knowledge Transfer of Large Multimodal Models for Few-Shot Learning
• Instance Relation Learning Network with Label Knowledge Propagation for Few-shot Multi-label Intent Detection
• Task-Level Contrastiveness for Cross-Domain Few-Shot Learning
• VT-FSL: Bridging Vision and Text with LLMs for Few-Shot Learning
• Learnable Loss Geometries with Mirror Descent for Scalable and Convergent Meta-Learning
• ICM-Fusion: In-Context Meta-Optimized LoRA Fusion for Multi-Task Adaptation
• Where and How to Enhance: Discovering Bit-Width Contribution for Mixed Precision Quantization
• Color as the Impetus: Transforming Few-Shot Learner
• MetaLab: Few-Shot Game Changer for Image Recognition

Related

• Fast Few-Shot Classification by Few-Iteration Meta-Learning
• Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition
• Unraveling Meta-Learning: Understanding Feature Representations for Few-Shot Tasks
• Beyond Simple Meta-Learning: Multi-Purpose Models for Multi-Domain, Active and Continual Few-Shot Learning
• Complementing Representation Deficiency in Few-shot Image Classification: A Meta-Learning Approach