Bridging Diversity and Uncertainty in Active learning with Self-Supervised Pre-Training

2024/03/06 by Paul Doucet, Benjamin Estermann, Doucet, Paul +5 · 2 citations
Social Sciences · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Problem and Project Based Learning

paper · pdf · doi:10.48550/arxiv.2403.03728

openalex publication_date 2024/03/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This study addresses the integration of diversity-based and uncertainty-based sampling strategies in active learning, particularly within the context of self-supervised pre-trained models. We introduce a straightforward heuristic called TCM that mitigates the cold start problem while maintaining strong performance across various data levels. By initially applying TypiClust for diversity sampling and subsequently transitioning to uncertainty sampling with Margin, our approach effectively combines the strengths of both strategies. Our experiments demonstrate that TCM consistently outperforms existing methods across various datasets in both low and high data regimes.

Cited by

• Cleaning the Pool: Progressive Filtering of Unlabeled Pools in Deep Active Learning
• MedCAL-Bench: A Comprehensive Benchmark on Cold-Start Active Learning with Foundation Models for Medical Image Analysis

Related

• Learning to Sample: an Active Learning Framework
• On weighted uncertainty sampling in active learning
• Active Learning Using Uncertainty Information
• Understanding Uncertainty-based Active Learning Under Model Mismatch
• TiDAL: Learning Training Dynamics for Active Learning