Pre-trained Language Model Based Active Learning for Sentence Matching

2020/10/12 by Guirong Bai, Bai, Guirong, Shizhu He +7
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning and Algorithms #Natural Language Processing Techniques #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.2010.05522

Accepted by the conference of coling 2020

arxiv created 2020/10/12 · openalex publication_date 2020/10/12 · arxiv updated 2020/10/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Active learning is able to significantly reduce the annotation cost for data-driven techniques. However, previous active learning approaches for natural language processing mainly depend on the entropy-based uncertainty criterion, and ignore the characteristics of natural language. In this paper, we propose a pre-trained language model based active learning approach for sentence matching. Differing from previous active learning, it can provide linguistic criteria to measure instances and help select more efficient instances for annotation. Experiments demonstrate our approach can achieve greater accuracy with fewer labeled training instances.

Citations

• XLNet: Generalized Autoregressive Pretraining for Language Understanding
• Bilateral Multi-Perspective Matching for Natural Language Sentences
• A large annotated corpus for learning natural language inference

Related

• On the Importance of Effectively Adapting Pretrained Language Models for Active Learning
• A Survey of Active Learning for Natural Language Processing
• Active Learning for Natural Language Generation
• Active Learning for New Domains in Natural Language Understanding
• Active Learning for NLP with Large Language Models