Few-shot Named Entity Recognition with Cloze Questions

2021/11/24 by Valerio La Gatta, La Gatta, Valerio, Vincenzo Moscato +6
Computer Science · #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling #cs.AI #cs.CL #cs.IR

paper · pdf · doi:10.48550/arxiv.2111.12421

arxiv created 2021/11/24 · arxiv updated 2021/11/25

Abstract

Despite the huge and continuous advances in computational linguistics, the lack of annotated data for Named Entity Recognition (NER) is still a challenging issue, especially in low-resource languages and when domain knowledge is required for high-quality annotations. Recent findings in NLP show the effectiveness of cloze-style questions in enabling language models to leverage the knowledge they acquired during the pre-training phase. In our work, we propose a simple and intuitive adaptation of Pattern-Exploiting Training (PET), a recent approach which combines the cloze-questions mechanism and fine-tuning for few-shot learning: the key idea is to rephrase the NER task with patterns. Our approach achieves considerably better performance than standard fine-tuning and comparable or improved results with respect to other few-shot baselines without relying on manually annotated data or distant supervision on three benchmark datasets: NCBI-disease, BC2GM and a private Italian biomedical corpus.

Related

• Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference
• Formulating Few-shot Fine-tuning Towards Language Model Pre-training: A Pilot Study on Named Entity Recognition
• Few-shot Learning for Named Entity Recognition in Medical Text
• TOKEN is a MASK: Few-shot Named Entity Recognition with Pre-trained Language Models
• CLLMFS: A Contrastive Learning enhanced Large Language Model Framework for Few-Shot Named Entity Recognition