Bi-directional Attention with Agreement for Dependency Parsing

2016/08/06 by Hao Cheng, Hao Fang, Cheng, Hao +7 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Speech and dialogue systems #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1608.02076

openalex publication_date 2016/08/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We develop a novel bi-directional attention model for dependency parsing, which learns to agree on headword predictions from the forward and backward parsing directions. The parsing procedure for each direction is formulated as sequentially querying the memory component that stores continuous headword embeddings. The proposed parser makes use of \it soft headword embeddings, allowing the model to implicitly capture high-order parsing history without dramatically increasing the computational complexity. We conduct experiments on English, Chinese, and 12 other languages from the CoNLL 2006 shared task, showing that the proposed model achieves state-of-the-art unlabeled attachment scores on 6 languages.

Citations

• Structured Training for Neural Network Transition-Based Parsing

Cited by

• In-Order Transition-based Constituent Parsing

Related

• Deep Biaffine Attention for Neural Dependency Parsing
• Dependency Parsing as Head Selection
• Incremental Parsing with Minimal Features Using Bi-Directional LSTM
• A Fundamental Algorithm for Dependency Parsing (With Corrections)
• Left-to-Right Dependency Parsing with Pointer Networks