Improving Consistency in Retrieval-Augmented Systems with Group Similarity Rewards

2025/10/05 by Faisal Hamman, Hamman, Faisal, Chenyang Zhu +11 · 1 citation
Computer Science · #Age of Information Optimization #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computers and Society (cs.CY) #FOS: Computer and information sciences #Information Retrieval and Search Behavior #Machine Learning (cs.LG) #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.2510.04392

openalex publication_date 2025/10/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

RAG systems are increasingly deployed in high-stakes domains where users expect outputs to be consistent across semantically equivalent queries. However, existing systems often exhibit significant inconsistencies due to variability in both the retriever and generator (LLM), undermining trust and reliability. In this work, we focus on information consistency, i.e., the requirement that outputs convey the same core content across semantically equivalent inputs. We introduce a principled evaluation framework that decomposes RAG consistency into retriever-level, generator-level, and end-to-end components, helping identify inconsistency sources. To improve consistency, we propose Paraphrased Set Group Relative Policy Optimization (PS-GRPO), an RL approach that leverages multiple rollouts across paraphrased set to assign group similarity rewards. We leverage PS-GRPO to achieve Information Consistent RAG (Con-RAG), training the generator to produce consistent outputs across paraphrased queries and remain robust to retrieval-induced variability. Because exact reward computation over paraphrase sets is computationally expensive, we also introduce a scalable approximation method that retains effectiveness while enabling efficient, large-scale training. Empirical evaluations across short-form, multi-hop, and long-form QA benchmarks demonstrate that Con-RAG significantly improves both consistency and accuracy over strong baselines, even in the absence of explicit ground-truth supervision. Our work provides practical solutions for evaluating and building reliable RAG systems for safety-critical deployments.

Citations

• On the Theoretical Limitations of Embedding-Based Retrieval
• Investigating the Robustness of Retrieval-Augmented Generation at the Query Level
• BLEUBERI: BLEU is a surprisingly effective reward for instruction following
• Consistency in Language Models: Current Landscape, Challenges, and Future Directions
• QE-RAG: A Robust Retrieval-Augmented Generation Benchmark for Query Entry Errors
• Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model
• Improving Consistency in Large Language Models through Chain of Guidance
• Benchmarking Prompt Sensitivity in Large Language Models
• Inference Scaling for Long-Context Retrieval Augmented Generation
• An Empirical Study of Retrieval Augmented Generation with Chain-of-Thought
• Quantifying Prediction Consistency Under Fine-Tuning Multiplicity in Tabular LLMs
• Improving the Robustness of Large Language Models via Consistency Alignment
• RAGGED: Towards Informed Design of Scalable and Stable RAG Systems
• Prompt Perturbation in Retrieval-Augmented Generation based Large Language Models
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• A Survey of Reinforcement Learning from Human Feedback
• Retrieval-Augmented Generation for Large Language Models: A Survey
• Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
• On Measuring Faithfulness or Self-consistency of Natural Language Explanations
• Predicting Question-Answering Performance of Large Language Models through Semantic Consistency
• Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation
• Text Embeddings by Weakly-Supervised Contrastive Pre-training
• Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference
• Evaluating the Factual Consistency of Large Language Models Through News Summarization
• Measuring Reliability of Large Language Models through Semantic Consistency
• Scaling Instruction-Finetuned Language Models
• MuSiQue: Multihop Questions via Single-hop Question Composition
• Measuring and Improving Consistency in Pretrained Language Models
• Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps
• KILT: a Benchmark for Knowledge Intensive Language Tasks
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• On Faithfulness and Factuality in Abstractive Summarization
• Dense Passage Retrieval for Open-Domain Question Answering
• Asking and Answering Questions to Evaluate the Factual Consistency of Summaries
• Logic-Guided Data Augmentation and Regularization for Consistent Question Answering
• REALM: Retrieval-Augmented Language Model Pre-Training
• A Logic-Driven Framework for Consistency of Neural Models
• ELI5: Long Form Question Answering
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
• TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for\n Reading Comprehension

Cited by

• Information-Consistent Language Model Recommendations through Group Relative Policy Optimization

Related

• Harmonizing Diverse Models: A Layer-wise Merging Strategy for Consistent Generation
• RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewards
• FAIR-RAG: Faithful Adaptive Iterative Refinement for Retrieval-Augmented Generation
• RAGGED: Towards Informed Design of Scalable and Stable RAG Systems
• Beyond Component Strength: Synergistic Integration and Adaptive Calibration in Multi-Agent RAG Systems