Learning Structured Text Representations

2017/05/25 by Yang Liu, Mirella Lapata, Liu, Yang +1
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Text and Document Classification Technologies #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1705.09207

openalex publication_date 2017/05/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we focus on learning structure-aware document representations from data without recourse to a discourse parser or additional annotations. Drawing inspiration from recent efforts to empower neural networks with a structural bias, we propose a model that can encode a document while automatically inducing rich structural dependencies. Specifically, we embed a differentiable non-projective parsing algorithm into a neural model and use attention mechanisms to incorporate the structural biases. Experimental evaluation across different tasks and datasets shows that the proposed model achieves state-of-the-art results on document modeling tasks while inducing intermediate structures which are both interpretable and meaningful.

Citations

• Long Short-Term Memory
• A large annotated corpus for learning natural language inference

Related

• Improved Document Modelling with a Neural Discourse Parser
• Interpretable Structure-aware Document Encoders with Hierarchical Attention
• Language Model Pre-training for Hierarchical Document Representations
• Modelling the semantics of text in complex document layouts using graph transformer networks
• Vector representations of text data in deep learning