ML-Decoder: Scalable and Versatile Classification Head

2021/11/25 by Tal Ridnik, Gilad Sharir, Ridnik, Tal +7 · 12 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Text and Document Classification Technologies

paper · pdf · doi:10.48550/arxiv.2111.12933

openalex publication_date 2021/11/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we introduce ML-Decoder, a new attention-based classification head. ML-Decoder predicts the existence of class labels via queries, and enables better utilization of spatial data compared to global average pooling. By redesigning the decoder architecture, and using a novel group-decoding scheme, ML-Decoder is highly efficient, and can scale well to thousands of classes. Compared to using a larger backbone, ML-Decoder consistently provides a better speed-accuracy trade-off. ML-Decoder is also versatile - it can be used as a drop-in replacement for various classification heads, and generalize to unseen classes when operated with word queries. Novel query augmentations further improve its generalization ability. Using ML-Decoder, we achieve state-of-the-art results on several classification tasks: on MS-COCO multi-label, we reach 91.4% mAP; on NUS-WIDE zero-shot, we reach 31.1% ZSL mAP; and on ImageNet single-label, we reach with vanilla ResNet50 backbone a new top score of 80.7%, without extra data or distillation. Public code is available at: https://github.com/Alibaba-MIIL/ML_Decoder

Cited by

• Multi-label Classification with Panoptic Context Aggregation Networks
• EMMA: Concept Erasure Benchmark with Comprehensive Semantic Metrics and Diverse Categories
• Self-Paced Learning for Images of Antinuclear Antibodies
• DiCaP: Distribution-Calibrated Pseudo-labeling for Semi-Supervised Multi-Label Learning
• MAT-Agent: Adaptive Multi-Agent Training Optimization
• OTTER: Open-Tagging via Text-Image Representation for Multi-modal Understanding
• Diffusion-Based Cross-Modal Feature Extraction for Multi-Label Classification
• Neural Collapse-Inspired Multi-Label Federated Learning under Label-Distribution Skew
• Towards Effective Prompt Stealing Attack against Text-to-Image Diffusion Models
• CXR-LT 2024: A MICCAI challenge on long-tailed, multi-label, and zero-shot disease classification from chest X-ray
• CKD-EHR:Clinical Knowledge Distillation for Electronic Health Records
• Any-Class Presence Likelihood for Robust Multi-Label Classification with Abundant Negative Data

Related

• MDMLP: Image Classification from Scratch on Small Datasets with MLP
• RedundancyLens: Revealing and Exploiting Visual Token Processing Redundancy for Efficient Decoder-Only MLLMs
• Sliceformer: Make Multi-head Attention as Simple as Sorting in Discriminative Tasks
• Catching the Details: Self-Distilled RoI Predictors for Fine-Grained MLLM Perception
• DS-Det: Single-Query Paradigm and Attention Disentangled Learning for Flexible Object Detection