Deep Semantic Role Labeling with Self-Attention

2017/12/05 by Zhixing Tan, Tan, Zhixing, Mingxuan Wang +7 · 3 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #Biomedical Text Mining and Ontologies #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1712.01586

openalex publication_date 2017/12/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Semantic Role Labeling (SRL) is believed to be a crucial step towards natural language understanding and has been widely studied. Recent years, end-to-end SRL with recurrent neural networks (RNN) has gained increasing attention. However, it remains a major challenge for RNNs to handle structural information and long range dependencies. In this paper, we present a simple and effective architecture for SRL which aims to address these problems. Our model is based on self-attention which can directly capture the relationships between two tokens regardless of their distance. Our single model achieves F₁=83.4 on the CoNLL-2005 shared task dataset and F₁=82.7 on the CoNLL-2012 shared task dataset, which outperforms the previous state-of-the-art results by 1.8 and 1.0 F₁ score respectively. Besides, our model is computationally efficient, and the parsing speed is 50K tokens per second on a single Titan X GPU.

Cited by

• Towards Open Intent Discovery for Conversational Text
• SemSketches-2021: experimenting with the machine processing of the pilot semantic sketches corpus
• Linguistically-Enriched and Context-Aware Zero-shot Slot Filling

Related

• Linguistically-Informed Self-Attention for Semantic Role Labeling
• Syntax-Enhanced Self-Attention-Based Semantic Role Labeling
• Towards Semi-Supervised Learning for Deep Semantic Role Labeling
• End-to-end Semantic Role Labeling with Neural Transition-based Model
• LLMs Can Also Do Well! Breaking Barriers in Semantic Role Labeling via Large Language Models