Evaluating Pretrained Transformer Models for Entity Linking in Task-Oriented Dialog

2021/12/15 by Sai Muralidhar Jayanthi, Jayanthi, Sai Muralidhar, Varsha Embar +3
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech and dialogue systems #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2112.08327

openalex publication_date 2021/12/15 · openalex created_date 2022/05/05 · openalex updated_date 2026/07/28

Abstract

The wide applicability of pretrained transformer models (PTMs) for natural language tasks is well demonstrated, but their ability to comprehend short phrases of text is less explored. To this end, we evaluate different PTMs from the lens of unsupervised Entity Linking in task-oriented dialog across 5 characteristics -- syntactic, semantic, short-forms, numeric and phonetic. Our results demonstrate that several of the PTMs produce sub-par results when compared to traditional techniques, albeit competitive to other neural baselines. We find that some of their shortcomings can be addressed by using PTMs fine-tuned for text-similarity tasks, which illustrate an improved ability in comprehending semantic and syntactic correspondences, as well as some improvements for short-forms, numeric and phonetic variations in entity mentions. We perform qualitative analysis to understand nuances in their predictions and discuss scope for further improvements. Code can be found at https://github.com/murali1996/el_tod

Related

• DIET: Lightweight Language Understanding for Dialogue Systems
• Syntax-Infused Transformer and BERT models for Machine Translation and Natural Language Understanding
• E.T.: Entity-Transformers. Coreference augmented Neural Language Model for richer mention representations via Entity-Transformer blocks
• Syntax-Enhanced Pre-trained Model
• LMN at SemEval-2022 Task 11: A Transformer-based System for English Named Entity Recognition