Semantic Tagging with Deep Residual Networks

2016/09/22 by Johannes Bjerva, Barbara Plank, Bjerva, Johannes +3
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1609.07053

openalex publication_date 2016/09/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose a novel semantic tagging task, sem-tagging, tailored for the purpose of multilingual semantic parsing, and present the first tagger using deep residual networks (ResNets). Our tagger uses both word and character representations and includes a novel residual bypass architecture. We evaluate the tagset both intrinsically on the new task of semantic tagging, as well as on Part-of-Speech (POS) tagging. Our system, consisting of a ResNet and an auxiliary loss function predicting our semantic tags, significantly outperforms prior results on English Universal Dependencies POS tagging (95.71% accuracy on UD v1.2 and 95.67% accuracy on UD v1.3).

Citations

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Text segmentation with character-level text embeddings
• Gradient-based learning applied to document recognition
• A Unified Tagging Solution: Bidirectional LSTM Recurrent Neural Network with Word Embedding
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Deep Residual Learning for Image Recognition

Related

• Deep or Simple Models for Semantic Tagging? It Depends on your Data [Experiments]
• Towards Universal Semantic Tagging
• The Best of Both Worlds: Lexical Resources To Improve Low-Resource Part-of-Speech Tagging
• What can we learn from Semantic Tagging?
• A General-Purpose Tagger with Convolutional Neural Networks