On Enforcing Better Conditioned Meta-Learning for Rapid Few-Shot Adaptation

2022/06/15 by M. Hiller, Mehrtash Harandi, Hiller, Markus +3 · 1 citation
Biochemistry, Genetics and Molecular Biology · Computer Science · #Advanced Neural Network Applications #Cancer-related molecular mechanisms research #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2206.07260

openalex publication_date 2022/06/15 · openalex created_date 2022/06/18 · openalex updated_date 2026/07/28

Abstract

Inspired by the concept of preconditioning, we propose a novel method to increase adaptation speed for gradient-based meta-learning methods without incurring extra parameters. We demonstrate that recasting the optimization problem to a non-linear least-squares formulation provides a principled way to actively enforce a well-conditioned parameter space for meta-learning models based on the concepts of the condition number and local curvature. Our comprehensive evaluations show that the proposed method significantly outperforms its unconstrained counterpart especially during initial adaptation steps, while achieving comparable or better overall results on several few-shot classification tasks -- creating the possibility of dynamically choosing the number of adaptation steps at inference time.

Cited by

• Learnable Loss Geometries with Mirror Descent for Scalable and Convergent Meta-Learning

Related

• Fast Adaptation with Kernel and Gradient based Meta Leaning
• Is Fast Adaptation All You Need?
• Meta-Learning with Latent Embedding Optimization
• Subspace Adaptation Prior for Few-Shot Learning
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks