DecEx-RAG: Boosting Agentic Retrieval-Augmented Generation with Decision and Execution Optimization via Process Supervision

2025/10/07 by Yongqi Leng, Leng, Yongqi, Yikun Lei +16
Business, Management and Accounting · Computer Science · #Business Process Modeling and Analysis #Semantic Web and Ontologies #Big Data and Business Intelligence

paper · pdf · doi:10.48550/arxiv.2510.05691

Abstract

Agentic Retrieval-Augmented Generation (Agentic RAG) enhances the processing capability for complex tasks through dynamic retrieval and adaptive workflows. Recent advances (e.g., Search-R1) have shown that outcome-supervised reinforcement learning demonstrate strong performance. However, this approach still suffers from inefficient exploration, sparse reward signals, and ambiguous global reward feedback. To address these challenges, we propose DecEx-RAG, which models RAG as a Markov Decision Process (MDP) incorporating decision-making and execution, while introducing an efficient pruning strategy to optimize data expansion. Through comprehensive process-level policy optimization, DecEx-RAG significantly enhances the autonomous task decomposition, dynamic retrieval, and high-quality answer generation capabilities of large language models (LLMs). Experiments show that DecEx-RAG achieves an average absolute performance improvement of 6.2% across six datasets, significantly outperforming existing baselines. Moreover, the pruning strategy improves data construction efficiency by nearly 6 ×, providing an efficient solution for process-supervised RAG training. The code is available at https://github.com/sdsxdxl/DecEx-RAG.

Citations

• Process vs. Outcome Reward: Which is Better for Agentic RAG Reinforcement Learning
• Reinforced Internal-External Knowledge Synergistic Reasoning for Efficient Adaptive Search Agent
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
• Search-o1: Agentic Search-Enhanced Large Reasoning Models
• Large Language Model Safety: A Holistic Survey
• RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
• ChatSOP: An SOP-Guided MCTS Planning Framework for Controllable LLM Dialogue Agents
• Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs
• ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search
• AlphaMath Almost Zero: Process Supervision without Process
• Let's Verify Step by Step
• Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy
• Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
• When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
• Text Embeddings by Weakly-Supervised Contrastive Pre-training
• Measuring and Narrowing the Compositionality Gap in Language Models
• Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• AmbigQA: Answering Ambiguous Open-domain Questions
• Dense Passage Retrieval for Open-Domain Question Answering
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering

Related

• Process vs. Outcome Reward: Which is Better for Agentic RAG Reinforcement Learning
• Supervising the search process produces reliable and generalizable information-seeking agents
• HiPRAG: Hierarchical Process Rewards for Efficient Agentic Retrieval Augmented Generation
• TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Framework
• RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewards