Read, Tag, and Parse All at Once, or Fully-neural Dependency Parsing

2016/09/12 by Jan Chorowski, Chorowski, Jan, Michał Zapotoczny +3
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1609.03441

openalex publication_date 2016/09/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present a dependency parser implemented as a single deep neural network that reads orthographic representations of words and directly generates dependencies and their labels. Unlike typical approaches to parsing, the model doesn't require part-of-speech (POS) tagging of the sentences. With proper regularization and additional supervision achieved with multitask learning we reach state-of-the-art performance on Slavic languages from the Universal Dependencies treebank: with no linguistic features other than characters, our parser is as accurate as a transition- based system trained on perfect POS tags.

Citations

• ADADELTA: An Adaptive Learning Rate Method
• Theano: new features and speed improvements
• Grammar as a Foreign Language
• Bidirectional recurrent neural networks
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• Exploring the Limits of Language Modeling
• Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation
• Transition-Based Dependency Parsing with Stack Long Short-Term Memory

Related

• On Multilingual Training of Neural Dependency Parsers
• Is POS Tagging Necessary or Even Helpful for Neural Dependency Parsing?
• Many Languages, One Parser
• A Fundamental Algorithm for Dependency Parsing (With Corrections)
• Dependency Parsing as Head Selection