A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents

2018/04/16 by Cohan, Arman, Dernoncourt, Franck, Kim, Doo Soon +4 · 39 citations
#Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.1804.05685

Abstract

Neural abstractive summarization models have led to promising results in summarizing relatively short documents. We propose the first model for abstractive summarization of single, longer-form documents (e.g., research papers). Our approach consists of a new hierarchical encoder that models the discourse structure of a document, and an attentive discourse-aware decoder to generate the summary. Empirical results on two large-scale datasets of scientific papers show that our model significantly outperforms state-of-the-art models.

Cited by

• Every Time I Hire a Linguist, Inference Costs Go Down: On Linguistic Rules as Effective Prompt Compressors
• Short-Context Dominance: How Much Local Context Natural Language Actually Needs?
• Understanding LLM Reasoning for Abstractive Summarization
• Large Language Models for the Summarization of Czech Documents: From History to the Present
• SynClaimEval: A Framework for Evaluating the Utility of Synthetic Data in Long-Context Claim Verification
• From the Rock Floor to the Cloud: A Systematic Survey of State-of-the-Art NLP in Battery Life Cycle
• Decomposition-Enhanced Training for Post-Hoc Attributions In Language Models
• Simple Denoising Diffusion Language Models
• Encoder-Decoder Diffusion Language Models for Efficient Training and Inference
• Loopholing Discrete Diffusion: Deterministic Bypass of the Sampling Wall
• Latent-Augmented Discrete Diffusion Models
• Evaluating Medical LLMs by Levels of Autonomy: A Survey Moving from Benchmarks to Applications
• AcademicEval: Live Long-Context LLM Benchmark
• Mirror Speculative Decoding: Breaking the Serial Barrier in LLM Inference
• Glance for Context: Learning When to Leverage LLMs for Node-Aware GNN-LLM Fusion
• Learning What to Remember: Adaptive Probabilistic Memory Retention for Memory-Efficient Language Models
• From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefill
• Pragyaan: Designing and Curating High-Quality Cultural Post-Training Datasets for Indian Languages
• Paper Espresso: From Paper Overload to Research Insight
• SurveyGen: Quality-Aware Scientific Survey Generation with Large Language Models
• Trigger Where It Hurts: Unveiling Hidden Backdoors through Sensitivity with Sensitron
• Topic-Guided Reinforcement Learning with LLMs for Enhancing Multi-Document Summarization
• Efficient Decoding Methods for Language Models on Encrypted Data
• From Linear to Hierarchical: Evolving Tree-structured Thoughts for Efficient Alpha Mining
• Implicit Reasoning in Large Language Models: A Comprehensive Survey
• FActBench: A Benchmark for Fine-grained Automatic Evaluation of LLM-Generated Text in the Medical Domain
• Dynamic Sparse Attention on Mobile SoCs
• REFRAG: Rethinking RAG based Decoding
• Can Smaller LLMs do better? Unlocking Cross-Domain Potential through Parameter-Efficient Fine-Tuning for Text Summarization
• Benchmarking GPT-5 for biomedical natural language processing
• Evaluating the Evaluators: Are readability metrics good measures of readability?
• A Comparative Study of Decoding Strategies in Medical Text Generation
• SCOPE: A Generative Approach for LLM Prompt Compression
• Large Language Models for Summarizing Czech Historical Documents and Beyond
• DeCAL Tokenwise Compression
• MORE-CLEAR: Multimodal Offline Reinforcement learning for Clinical notes Leveraged Enhanced State Representation
• LLM Serving Optimization with Variable Prefill and Decode Lengths
• Let's Measure Information Step-by-Step: LLM-Based Evaluation Beyond Vibes
• MRGSEM-Sum: An Unsupervised Multi-document Summarization Framework based on Multi-Relational Graphs and Structural Entropy Minimization

Related

• A Neural Attention Model for Abstractive Sentence Summarization
• On Extractive and Abstractive Neural Document Summarization with Transformer Language Models
• Neural Abstractive Summarization with Structural Attention
• Abstractive Summarization Using Attentive Neural Techniques
• Abstractive Text Summarization by Incorporating Reader Comments