BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning

2019/06/19 by Andreas Kirsch, Joost van Amersfoort, Kirsch, Andreas +3 · 38 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms

paper · pdf · doi:10.48550/arxiv.1906.08158

openalex publication_date 2019/06/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We develop BatchBALD, a tractable approximation to the mutual information between a batch of points and model parameters, which we use as an acquisition function to select multiple informative points jointly for the task of deep Bayesian active learning. BatchBALD is a greedy linear-time 1 - (1)/(e)-approximate algorithm amenable to dynamic programming and efficient caching. We compare BatchBALD to the commonly used approach for batch data acquisition and find that the current approach acquires similar and redundant points, sometimes performing worse than randomly acquiring data. We finish by showing that, using BatchBALD to consider dependencies within an acquisition batch, we achieve new state of the art performance on standard benchmarks, providing substantial data efficiency improvements in batch acquisition.

Citations

• An analysis of approximations for maximizing submodular set functions—I
• Semi-Supervised Learning with Deep Generative Models
• Practical Bayesian Optimization of Machine Learning Algorithms
• Batch Bayesian Optimization via Local Penalization
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

Cited by

• AdaKP: Online Adaptive Knowledge-Point Selection for Reasoning-Oriented Reinforcement Learning
• Uncertainty Quantification for Machine Learning: One Size Does Not Fit All
• Hierarchical Dataset Selection for High-Quality Data Sharing
• Towards Active Synthetic Data Generation for Finetuning Language Models
• Active3D: Active High-Fidelity 3D Reconstruction via Hierarchical Uncertainty Quantification
• Active Learning with Selective Time-Step Acquisition for PDEs
• Bridging VLMs and Embodied Intelligence with Deliberate Practice Policy Optimization
• Constraint-Informed Active Learning for End-to-End ACOPF Optimization Proxies
• SynQuE: Estimating Synthetic Dataset Quality Without Annotations
• Geometric Data Valuation via Leverage Scores
• Uncertainty Quantification for Regression: A Unified Framework based on kernel scores
• Calibrated Uncertainty Sampling for Active Learning
• Informed Initialization for Bayesian Optimization and Active Learning
• Adaptive Data Selection for Multi-Layer Perceptron Training: A Sub-linear Value-Driven Method
• Image augmentation with invertible networks in interactive satellite image change detection
• Zero-Shot Performance Prediction for Probabilistic Scaling Laws
• Adaptive Individual Uncertainty under Out-Of-Distribution Shift with Expert-Routed Conformal Prediction
• Market-Driven Subset Selection for Budgeted Training
• Budget-constrained Active Learning to Effectively De-censor Survival Data
• Preference-driven Knowledge Distillation for Few-shot Node Classification
• Myopic Bayesian Decision Theory for Batch Active Learning with Partial Batch Label Sampling
• Multi-fidelity Batch Active Learning for Gaussian Process Classifiers
• Label-frugal satellite image change detection with generative virtual exemplar learning
• Uncertainty Quantification for Regression using Proper Scoring Rules
• Causal-EPIG: A Prediction-Oriented Active Learning Framework for CATE Estimation
• SubZeroCore: A Submodular Approach with Zero Training for Coreset Selection
• LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet?
• Towards Open-Ended Discovery for Low-Resource NLP
• Vendi Information Gain for Active Learning and its Application to Ecology
• Deep Active Learning for Lung Disease Severity Classification from Chest X-rays: Learning with Less Data in the Presence of Class Imbalance
• OmViD: Omni-supervised active learning for video action detection
• An AI-driven workflow for the accelerated optimization of cell-free protein synthesis
• Prediction-Oriented Subsampling from Data Streams
• Efficient dataset construction using active learning and uncertainty-aware neural networks for plasma turbulent transport surrogate models
• Neural Active Learning with Performance Guarantees
• Low-Regret Active learning
• CLEVER: Stream-based Active Learning for Robust Semantic Perception from Human Instructions
• Modern Bayesian Experimental Design

Related

• Scalable Batch Acquisition for Deep Bayesian Active Learning
• Stochastic Batch Acquisition: A Simple Baseline for Deep Active Learning
• Speeding Up BatchBALD: A k-BALD Family of Approximations for Active Learning
• Big Batch Bayesian Active Learning by Considering Predictive Probabilities
• Scalable Batch-Mode Deep Bayesian Active Learning via Equivalence Class Annealing