Active Labeling: Streaming Stochastic Gradients

2022/05/26 by Vivien Cabannes, Cabannes, Vivien, Francis Bach +5
Computer Science · Mathematics · #68T37 #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #G.3 #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Markov Chains and Monte Carlo Methods #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2205.13255

openalex publication_date 2022/05/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The workhorse of machine learning is stochastic gradient descent. To access stochastic gradients, it is common to consider iteratively input/output pairs of a training dataset. Interestingly, it appears that one does not need full supervision to access stochastic gradients, which is the main motivation of this paper. After formalizing the "active labeling" problem, which focuses on active learning with partial supervision, we provide a streaming technique that provably minimizes the ratio of generalization error over the number of samples. We illustrate our technique in depth for robust regression.

Related

• Neural Active Learning with Performance Guarantees
• From Weakly Supervised Learning to Active Learning
• Efficient and Parsimonious Agnostic Active Learning
• Algorithmic Connections Between Active Learning and Stochastic Convex Optimization
• A-Optimal Active Learning