TENER: Adapting Transformer Encoder for Named Entity Recognition

2019/11/10 by Hang Yan, Bocao Deng, Yan, Hang +5 · 4 citations
Computer Science · Decision Sciences · #Computation and Language (cs.CL) #Data Quality and Management #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1911.04474

openalex publication_date 2019/11/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/30

Abstract

The Bidirectional long short-term memory networks (BiLSTM) have been widely used as an encoder in models solving the named entity recognition (NER) task. Recently, the Transformer is broadly adopted in various Natural Language Processing (NLP) tasks owing to its parallelism and advantageous performance. Nevertheless, the performance of the Transformer in NER is not as good as it is in other NLP tasks. In this paper, we propose TENER, a NER architecture adopting adapted Transformer Encoder to model the character-level features and word-level features. By incorporating the direction and relative distance aware attention and the un-scaled attention, we prove the Transformer-like encoder is just as effective for NER as other NLP tasks.

Citations

• Long Short-Term Memory
• CAN-NER: Convolutional Attention Network for Chinese Named Entity Recognition
• Empower Sequence Labeling with Task-Aware Neural Language Model

Cited by

• Beyond Real: Imaginary Extension of Rotary Position Embeddings for Long-Context LLMs
• A Word is Worth 4-bit: Efficient Log Parsing with Binary Coded Decimal Recognition
• Deep Probabilistic Modeling of User Behavior for Anomaly Detection via Mixture Density Networks
• Cascaded Semantic and Positional Self-Attention Network for Document Classification

Related

• GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer
• Positional Attention for Efficient BERT-Based Named Entity Recognition
• Hierarchical Transformer Model for Scientific Named Entity Recognition
• Neural Architectures for Named Entity Recognition
• NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Data