Negative Margin Matters: Understanding Margin in Few-shot Classification

2020/03/26 by Bin Liu, Liu, Bin, Yue Cao +11 · 4 citations
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Human Pose and Action Recognition #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2003.12060

openalex publication_date 2020/03/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper introduces a negative margin loss to metric learning based few-shot learning methods. The negative margin loss significantly outperforms regular softmax loss, and achieves state-of-the-art accuracy on three standard few-shot classification benchmarks with few bells and whistles. These results are contrary to the common practice in the metric learning field, that the margin is zero or positive. To understand why the negative margin loss performs well for the few-shot classification, we analyze the discriminability of learned features w.r.t different margins for training and novel classes, both empirically and theoretically. We find that although negative margin reduces the feature discriminability for training classes, it may also avoid falsely mapping samples of the same novel class to multiple peaks or clusters, and thus benefit the discrimination of novel classes. Code is available at https://github.com/bl0/negative-margin.few-shot.

Citations

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual\n Recognition
• ImageNet classification with deep convolutional neural networks
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Large-Margin Softmax Loss for Convolutional Neural Networks
• Prototypical Networks for Few-shot Learning
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal\n Networks
• Few-Shot Learning with Graph Neural Networks
• On First-Order Meta-Learning Algorithms

Cited by

• Enhancing Few-Shot Classification of Benchmark and Disaster Imagery with ATTBHFA-Net
• Shot2Tactic-Caption: Multi-Scale Captioning of Badminton Videos for Tactical Understanding
• MoTiC: Momentum Tightness and Contrast for Few-Shot Class-Incremental Learning
• Sculpting Margin Penalty: Intra-Task Adapter Merging and Classifier Calibration for Few-Shot Class-Incremental Learning

Related

• Boosting Few-Shot Learning With Adaptive Margin Loss
• Large Margin Few-Shot Learning
• A Transductive Maximum Margin Classifier for Few-Shot Learning
• Beyond Max-Margin: Class Margin Equilibrium for Few-shot Object Detection
• DiffKendall: A Novel Approach for Few-Shot Learning with Differentiable Kendall's Rank Correlation