Hallucinate at the Last in Long Response Generation: A Case Study on Long Document Summarization

2025/05/21 by Joonho Yang, Yang, Joonho, Seunghyun Yoon +7 · 1 citation
Computer Science · #Advanced Text Analysis Techniques #Computation and Language (cs.CL) #FOS: Computer and information sciences #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2505.15291

openalex publication_date 2025/05/21 · openalex created_date 2025/10/19 · openalex updated_date 2026/07/28

Abstract

Large Language Models (LLMs) have significantly advanced text generation capabilities, including tasks like summarization, often producing coherent and fluent outputs. However, faithfulness to source material remains a significant challenge due to the generation of hallucinations. While extensive research focuses on detecting and reducing these inaccuracies, less attention has been paid to the positional distribution of hallucination within generated text, particularly in long outputs. In this work, we investigate where hallucinations occur in LLM-based long response generation, using long document summarization as a key case study. Focusing on the challenging setting of long context-aware long response generation, we find a consistent and concerning phenomenon: hallucinations tend to concentrate disproportionately in the latter parts of the generated long response. To understand this bias, we explore potential contributing factors related to the dynamics of attention and decoding over long sequences. Furthermore, we investigate methods to mitigate this positional hallucination, aiming to improve faithfulness specifically in the concluding segments of long outputs.

Citations

• How Does Response Length Affect Long-Form Factuality
• Gemma 3 Technical Report
• Shifting Long-Context LLMs Research from Input to Output
• Discourse-Driven Evaluation: Unveiling Factual Inconsistency in Long Document Summarization
• Toward Robust Hyper-Detailed Image Captioning: A Multiagent Approach and Dual Evaluation Metrics for Factuality and Coverage
• On Positional Bias of Faithfulness for Long-form Summarization
• Learning to Summarize from LLM-generated Feedback
• From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization
• Mitigating Hallucinations in Large Vision-Language Models via Summary-Guided Decoding
• AdaCAD: Adaptively Decoding to Balance Conflicts between Contextual and Parametric Knowledge
• LongGenBench: Benchmarking Long-Form Generation in Long Context LLMs
• LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs
• Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps
• MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention
• FineSurE: Fine-grained Summarization Evaluation using LLMs
• Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
• ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools
• RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthiness
• FIZZ: Factual Inconsistency Detection by Zoom-in Summary and Zoom-out Document
• MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents
• FABLES: Evaluating faithfulness and content selection in book-length summarization
• FENICE: Factuality Evaluation of summarization based on Natural language Inference and Claim Extraction
• Fine-Grained Natural Language Inference Based Faithfulness Evaluation for Diverse Summarisation Tasks
• A Thorough Examination of Decoding Methods in the Era of LLMs
• Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs
• Merging Generated and Retrieved Knowledge for Open-Domain QA
• BooookScore: A systematic exploration of book-length summarization in the era of LLMs
• Efficient Memory Management for Large Language Model Serving with PagedAttention
• Lost in the Middle: How Language Models Use Long Contexts
• Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
• Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model
• Trusting Your Evidence: Hallucinate Less with Context-aware Decoding
• FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
• HistAlign: Improving Context Dependency in Language Generation by Aligning with History
• NarraSum: A Large-Scale Dataset for Abstractive Narrative Summarization
• Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors
• Efficient Attentions for Long Document Summarization
• On-the-Fly Attention Modulation for Neural Generation
• Generating Long Sequences with Sparse Transformers
• Abstractive Summarization of Reddit Posts with Multi-level Memory Networks
• Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional\n Neural Networks for Extreme Summarization
• A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents
• Teaching Machines to Read and Comprehend

Cited by

• The Curious Case of Factual (Mis)Alignment between LLMs' Short- and Long-Form Answers

Related

• Hallucination Reduction in Long Input Text Summarization
• Don't Believe Everything You Read: Enhancing Summarization Interpretability through Automatic Identification of Hallucinations in Large Language Models
• From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization
• Analyzing LLM Behavior in Dialogue Summarization: Unveiling Circumstantial Hallucination Trends
• Evaluating LLMs' Assessment of Mixed-Context Hallucination Through the Lens of Summarization