Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional\n Neural Networks for Extreme Summarization

2018/08/27 by Shashi Narayan, Shay B. Cohen, Narayan, Shashi +3 · 91 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #Advanced Text Analysis Techniques #Biomedical Text Mining and Ontologies #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1808.08745

openalex publication_date 2018/08/27 · openalex created_date 2022/08/19 · openalex updated_date 2026/07/28

Abstract

We introduce extreme summarization, a new single-document summarization task\nwhich does not favor extractive strategies and calls for an abstractive\nmodeling approach. The idea is to create a short, one-sentence news summary\nanswering the question "What is the article about?". We collect a real-world,\nlarge-scale dataset for this task by harvesting online articles from the\nBritish Broadcasting Corporation (BBC). We propose a novel abstractive model\nwhich is conditioned on the article's topics and based entirely on\nconvolutional neural networks. We demonstrate experimentally that this\narchitecture captures long-range dependencies in a document and recognizes\npertinent content, outperforming an oracle extractive system and\nstate-of-the-art abstractive approaches when evaluated automatically and by\nhumans.\n

Citations

• Contextual LSTM (CLSTM) models for Large scale NLP tasks

Cited by

• FaithLens: Detecting and Explaining Faithfulness Hallucination
• Short-Context Dominance: How Much Local Context Natural Language Actually Needs?
• Progress Ratio Embeddings: An Impatience Signal for Robust Length Control in Neural Text Generation
• Learning from Self Critique and Refinement for Faithful LLM Summarization
• DAMASHA: Detecting AI in Mixed Adversarial Texts via Segmentation with Human-interpretable Attribution
• SVRG and Beyond via Posterior Correction
• KVReviver: Reversible KV Cache Compression with Sketch-Based Token Reconstruction
• EduEval: A Hierarchical Cognitive Benchmark for Evaluating Large Language Models in Chinese Education
• Mosaic Pruning: A Hierarchical Framework for Generalizable Pruning of Mixture-of-Experts Models
• Exploiting the Experts: Unauthorized Compression in MoE-LLMs
• Leveraging Digitized Newspapers to Collect Summarization Data in Low-Resource Languages
• SnapAudit: Active Auditing of Differentially Private In-Context Learning via Snapshot-Based Simulation
• Steering Pretrained Drafters during Speculative Decoding
• PrefixNLI: Detecting Factual Inconsistencies as Soon as They Arise
• Synergy over Discrepancy: A Partition-Based Approach to Multi-Domain LLM Fine-Tuning
• HLPD: Aligning LLMs to Human Language Preference for Machine-Revised Text Detection
• Towards Resource-Efficient Multimodal Intelligence: Learned Routing among Specialized Expert Models
• Evaluating Cultural Knowledge Processing in Large Language Models: A Cognitive Benchmarking Framework Integrating Retrieval-Augmented Generation
• Prompt-R1: Collaborative Automatic Prompting Framework via End-to-end Reinforcement Learning
• A CPU-Centric Perspective on Agentic AI
• Training LLMs Beyond Next Token Prediction -- Filling the Mutual Information Gap
• BiSparse-AAS: Bilinear Sparse Attention and Adaptive Spans Framework for Scalable and Efficient Text Summarization
• LoRAQuant: Mixed-Precision Quantization of LoRA to Ultra-Low Bits
• Seeing Through the MiRAGE: Evaluating Multimodal Retrieval Augmented Generation
• Compressing Many-Shots in In-Context Learning
• Simple Context Compression: Mean-Pooling and Multi-Ratio Training
• Fast Inference via Hierarchical Speculative Decoding
• AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders
• DiSRouter: Distributed Self-Routing for LLM Selections
• Synera: Synergistic LLM Serving across Device and Cloud at Scale
• When Personalization Tricks Detectors: The Feature-Inversion Trap in Machine-Generated Text Detection
• F2LLM Technical Report: Matching SOTA Embedding Performance with 6 Million Open-Source Data
• Enhancing Faithfulness in Abstractive Summarization via Span-Level Fine-Tuning
• SIMBA UQ: Similarity-Based Aggregation for Uncertainty Quantification in Large Language Models
• Provable Training Data Identification for Large Language Models
• Active Model Selection for Large Language Models
• Black-Box Detection of LLM-Generated Text Using Generalized Jensen-Shannon Divergence
• Learning to Rewrite Prompts for Bootstrapping LLMs on Downstream Tasks
• InforME: Improving Informativeness of Abstractive Text Summarization With Informative Attention Guided by Named Entity Salience
• Gamma Mixture Modeling for Cosine Similarity in Small Language Models
• Toward a unified framework for data-efficient evaluation of large language models
• Large Language Models Hallucination: A Comprehensive Survey
• Inclusive Easy-to-Read Generation for Individuals with Cognitive Impairments
• Generalized Parallel Scaling with Interdependent Generations
• Facilitating Cognitive Accessibility with LLMs: A Multi-Task Approach to Easy-to-Read Text Generation
• HiSpec: Hierarchical Speculative Decoding for LLMs
• LoRAFusion: Efficient LoRA Fine-Tuning for LLMs
• Paired by the Teacher: Turning Unpaired Data into High-Fidelity Pairs for Low-Resource Text Generation
• BOE-XSUM: Extreme Summarization in Clear Language of Spanish Legal Decrees and Notifications
• AdaDetectGPT: Adaptive Detection of LLM-Generated Text with Statistical Guarantees
• Multidimensional Uncertainty Quantification via Optimal Transport
• Mixture of Detectors: A Compact View of Machine-Generated Text Detection
• Towards Transparent AI: A Survey on Explainable Language Models
• LLMTrace: A Corpus for Classification and Fine-Grained Localization of AI-Written Text
• BridgeAlign: Bridging Preference Alignment for Humanities and Social Sciences
• HSS-Synth: Humanities and Social Sciences Data Synthesis for LLMs
• ELF: Embedded Language Flows
• Diversity Boosts AI-Generated Text Detection
• TsqLoRA: Towards Sensitivity and Quality Low-Rank Adaptation for Efficient Fine-Tuning
• CoCoA: Confidence and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in Large Language Models
• Decoding Uncertainty: The Impact of Decoding Strategies for Uncertainty Estimation in Large Language Models
• Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from Token and Parameter Levels
• Pipeline Parallelism is All You Need for Optimized Early-Exit Based Self-Speculative Decoding
• DNA-DetectLLM: Unveiling AI-Generated Text via a DNA-Inspired Mutation-Repair Paradigm
• CARGO: A Framework for Confidence-Aware Routing of Large Language Models
• Deep learning and abstractive summarisation for radiological reports: an empirical study for adapting the PEGASUS models' family with scarce data
• Selective Risk Certification for LLM Outputs via Information-Lift Statistics: PAC-Bayes, Robustness, and Skeleton Design
• Query-Focused Extractive Summarization for Sentiment Explanation
• MALLM: Multi-Agent Large Language Models Framework
• DetectAnyLLM: Towards Generalizable and Robust Detection of Machine-Generated Text Across Domains and Models
• Introducing Spotlight: A Novel Approach for Generating Captivating Key Information from Documents
• Topic-Guided Reinforcement Learning with LLMs for Enhancing Multi-Document Summarization
• Breaking the Mirror: Activation-Based Mitigation of Self-Preference in LLM Evaluators
• EverTracer: Hunting Stolen Large Language Models via Stealthy and Robust Probabilistic Fingerprint
• MoSEs: Uncertainty-Aware AI-Generated Text Detection via Mixture of Stylistics Experts with Conditional Thresholds
• DSDE: Dynamic Speculative Decoding with KLD Stability for Real-World Serving
• Can Smaller LLMs do better? Unlocking Cross-Domain Potential through Parameter-Efficient Fine-Tuning for Text Summarization
• OpinioRAG: Towards Generating User-Centric Opinion Highlights from Large-scale Online Reviews
• InSQuAD: In-Context Learning for Efficient Retrieval via Submodular Mutual Information to Enforce Quality and Diversity
• VocabTailor: Dynamic Vocabulary Selection for Downstream Tasks in Small Language Models
• Reference and Document Aware Semantic Evaluation Methods for Korean Language Summarization
• Prompt-Based One-Shot Exact Length-Controlled Generation with LLMs
• TracSum: A New Benchmark for Aspect-Based Summarization with Sentence-Level Traceability in Medical Domain
• GLASS: Global-Local Aggregation for Inference-time Sparsification of LLMs
• RepreGuard: Detecting LLM-Generated Text by Revealing Hidden Representation Patterns
• A Survey on Training-free Alignment of Large Language Models
• Hide or Highlight: Understanding the Impact of Factuality Expression on User Trust
• Play Favorites: A Statistical Method to Measure Self-Bias in LLM-as-a-Judge
• Let's Measure Information Step-by-Step: LLM-Based Evaluation Beyond Vibes
• A Single Direction of Truth: An Observer Model's Linear Residual Probe Exposes and Steers Contextual Hallucinations
• Where to show Demos in Your Prompt: A Positional Bias of In-Context Learning

Related

• What is this Article about? Extreme Summarization with Topic-aware Convolutional Neural Networks
• TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising
• A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents
• Neural Summarization by Extracting Sentences and Words
• Efficient Extractive Text Summarization for Online News Articles Using Machine Learning