DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning

2020/03/15 by Chi Zhang, Yujun Cai, Zhang, Chi +5 · 5 citations
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #FOS: Electrical engineering #Human Pose and Action Recognition #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2003.06777

openalex publication_date 2020/03/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this work, we develop methods for few-shot image classification from a new perspective of optimal matching between image regions. We employ the Earth Mover's Distance (EMD) as a metric to compute a structural distance between dense image representations to determine image relevance. The EMD generates the optimal matching flows between structural elements that have the minimum matching cost, which is used to calculate the image distance for classification. To generate the important weights of elements in the EMD formulation, we design a cross-reference mechanism, which can effectively alleviate the adverse impact caused by the cluttered background and large intra-class appearance variations. To implement k-shot classification, we propose to learn a structured fully connected layer that can directly classify dense image representations with the EMD. Based on the implicit function theorem, the EMD can be inserted as a layer into the network for end-to-end training. Our extensive experiments validate the effectiveness of our algorithm which outperforms state-of-the-art methods by a significant margin on five widely used few-shot classification benchmarks, namely, miniImageNet, tieredImageNet, Fewshot-CIFAR100 (FC100), Caltech-UCSD Birds-200-2011 (CUB), and CIFAR-FewShot (CIFAR-FS). We also demonstrate the effectiveness of our method on the image retrieval task in our experiments.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• ImageNet Large Scale Visual Recognition Challenge
• The Earth Mover's Distance as a Metric for Image Retrieval
• ImageNet classification with deep convolutional neural networks
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• Few-Shot Learning with Graph Neural Networks
• Meta-Learning with Implicit Gradients
• Differentiable Convex Optimization Layers
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Adaptive receptive field-based spatial-frequency feature reconstruction network for fine-grained few-shot image classification
• SPAGS: Sparse-View Articulated Object Reconstruction from Single State via Planar Gaussian Splatting
• DetailSemNet: Elevating Signature Verification through Detail-Semantic Integration
• Graph-Radiomic Learning (GrRAiL) Descriptor to Characterize Imaging Heterogeneity in Confounding Tumor Pathologies
• Shallow Deep Learning Can Still Excel in Fine-Grained Few-Shot Learning

Related

• DeepEMD: A Transformer-based Fast Estimation of the Earth Mover's Distance
• Relaxed Earth Mover's Distances for Chain- and Tree-connected Spaces and their use as a Loss Function in Deep Learning
• Deep Metric Learning for Few-Shot Image Classification: A Review of Recent Developments
• Few-shot Image Classification: Just Use a Library of Pre-trained Feature Extractors and a Simple Classifier
• Few-shot Metric Learning: Online Adaptation of Embedding for Retrieval