2018/08/16 by Aditya Siddhant, Zachary C. Lipton, Siddhant, Aditya +1 · 2 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Natural Language Processing Techniques #Topic Modeling
paper · pdf · doi:10.48550/arxiv.1808.05697
openalex publication_date 2018/08/16 · openalex created_date 2022/08/04 · openalex updated_date 2026/07/28
Several recent papers investigate Active Learning (AL) for mitigating the\ndata dependence of deep learning for natural language processing. However, the\napplicability of AL to real-world problems remains an open question. While in\nsupervised learning, practitioners can try many different methods, evaluating\neach against a validation set before selecting a model, AL affords no such\nluxury. Over the course of one AL run, an agent annotates its dataset\nexhausting its labeling budget. Thus, given a new task, an active learner has\nno opportunity to compare models and acquisition functions. This paper provides\na large scale empirical study of deep active learning, addressing multiple\ntasks and, for each, multiple datasets, multiple models, and a full suite of\nacquisition functions. We find that across all settings, Bayesian active\nlearning by disagreement, using uncertainty estimates provided either by\nDropout or Bayes-by Backprop significantly improves over i.i.d. baselines and\nusually outperforms classic uncertainty sampling.\n