TOKEN is a MASK: Few-shot Named Entity Recognition with Pre-trained Language Models

2022/06/15 by Ali Davody, Davody, Ali, David Ifeoluwa Adelani +5
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.2206.07841

Accepted to 25th International Conference on Text, Speech and Dialogue (TSD 2022)

arxiv created 2022/06/15 · openalex publication_date 2022/06/15 · arxiv updated 2022/06/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Transferring knowledge from one domain to another is of practical importance for many tasks in natural language processing, especially when the amount of available data in the target domain is limited. In this work, we propose a novel few-shot approach to domain adaptation in the context of Named Entity Recognition (NER). We propose a two-step approach consisting of a variable base module and a template module that leverages the knowledge captured in pre-trained language models with the help of simple descriptive patterns. Our approach is simple yet versatile and can be applied in few-shot and zero-shot settings. Evaluating our lightweight approach across a number of different datasets shows that it can boost the performance of state-of-the-art baselines by 2-5% F1-score.

Related

• Few-Shot Named Entity Recognition: A Comprehensive Study
• Learning from Language Description: Low-shot Named Entity Recognition via Decomposed Framework
• Formulating Few-shot Fine-tuning Towards Language Model Pre-training: A Pilot Study on Named Entity Recognition
• Show Less, Instruct More: Enriching Prompts with Definitions and Guidelines for Zero-Shot NER
• Evaluating Named Entity Recognition Using Few-Shot Prompting with Large Language Models