ECoRAG: Evidentiality-guided Compression for Long Context RAG

2025/06/05 by Yeonseok Jeong, Jin-Su Kim, Jeong, Yeonseok +5
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Information Retrieval and Search Behavior #Multimodal Machine Learning Applications #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2506.05167

openalex publication_date 2025/06/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

Large Language Models (LLMs) have shown remarkable performance in Open-Domain Question Answering (ODQA) by leveraging external documents through Retrieval-Augmented Generation (RAG). To reduce RAG overhead, from longer context, context compression is necessary. However, prior compression methods do not focus on filtering out non-evidential information, which limit the performance in LLM-based RAG. We thus propose Evidentiality-guided RAG, or ECoRAG framework. ECoRAG improves LLM performance by compressing retrieved documents based on evidentiality, ensuring whether answer generation is supported by the correct evidence. As an additional step, ECoRAG reflects whether the compressed content provides sufficient evidence, and if not, retrieves more until sufficient. Experiments show that ECoRAG improves LLM performance on ODQA tasks, outperforming existing compression methods. Furthermore, ECoRAG is highly cost-efficient, as it not only reduces latency but also minimizes token usage by retaining only the necessary information to generate the correct answer. Code is available at https://github.com/ldilab/ECoRAG.

Citations

• Agent-as-Judge for Factual Summarization of Long Narratives
• Gemma 2: Improving Open Language Models at a Practical Size
• The Llama 3 Herd of Models
• CompAct: Compressing Retrieved Documents Actively for Question Answering
• When to Retrieve: Teaching LLMs to Utilize Information Retrieval Effectively
• RULER: What's the Real Context Size of Your Long-Context Language Models?
• Select and Summarize: Scene Saliency for Movie Script Summarization
• Long-context LLMs Struggle with Long In-context Learning
• LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression
• M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation
• CareerX: A Retrieval-Augmented Generation Framework for Personalized AI-Driven Career Guidance
• LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression
• C-Pack: Packed Resources For General Chinese Embeddings
• LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
• On Monotonic Aggregation for Open-domain QA
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• Lost in the Middle: How Language Models Use Long Contexts
• Augmented Large Language Models with Parametric Knowledge Guiding
• Can Large Language Models Be an Alternative to Human Evaluations?
• Learning to Compress Prompts with Gist Tokens
• Self-Refine: Iterative Refinement with Self-Feedback
• Large Language Models Can Be Easily Distracted by Irrelevant Context
• In-Context Retrieval-Augmented Language Models
• Generate rather than Retrieve: Large Language Models are Strong Context Generators
• Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks
• Unsupervised Dense Information Retrieval with Contrastive Learning
• Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks
• Robustifying Multi-hop QA through Pseudo-Evidentiality Training
• Language Models are Open Knowledge Graphs
• Leveraging Passage Retrieval with Generative Models for Open Domain\n Question Answering
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• A Self-Training Method for Machine Reading Comprehension with Soft Evidence Extraction
• Dense Passage Retrieval for Open-Domain Question Answering
• CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
• TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for\n Reading Comprehension

Related

• LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering
• Open-RAG: Enhanced Retrieval-Augmented Reasoning with Open-Source Large Language Models
• SAGE: A Framework of Precise Retrieval for RAG
• FIT-RAG: Black-Box RAG with Factual Information and Token Reduction
• BRIEF: Bridging Retrieval and Inference for Multi-hop Reasoning via Compression