When does Active Learning Work?

2014/08/06 by Lewis P. G. Evans, Evans, Lewis, Niall M. Adams +3 · 1 citation
Computer Science · Engineering · #Machine Learning and Algorithms #Machine Learning and Data Classification #Fault Detection and Control Systems

paper · pdf · doi:10.48550/arxiv.1408.1319

Abstract

Active Learning (AL) methods seek to improve classifier performance when labels are expensive or scarce. We consider two central questions: Where does AL work? How much does it help? To address these questions, a comprehensive experimental simulation study of Active Learning is presented. We consider a variety of tasks, classifiers and other AL factors, to present a broad exploration of AL performance in various settings. A precise way to quantify performance is needed in order to know when AL works. Thus we also present a detailed methodology for tackling the complexities of assessing AL performance in the context of this experimental study.

Cited by

• Survey of Active Learning Hyperparameters: Insights from a Large-Scale Experimental Grid

Related

• Active Learning Methods for Efficient Data Utilization and Model Performance Enhancement
• Limitations of Assessing Active Learning Performance at Runtime
• The Role of Active Learning in Modern Machine Learning
• A Cross-Domain Benchmark for Active Learning
• Navigating the Pitfalls of Active Learning Evaluation: A Systematic Framework for Meaningful Performance Assessment