Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions

2022/12/20 by Harsh Trivedi, Trivedi, Harsh, Niranjan Balasubramanian +5 · 165 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2212.10509

openalex publication_date 2022/12/20 · openalex created_date 2023/01/04 · openalex updated_date 2026/07/28

Abstract

Prompting-based large language models (LLMs) are surprisingly powerful at generating natural language reasoning steps or Chains-of-Thoughts (CoT) for multi-step question answering (QA). They struggle, however, when the necessary knowledge is either unavailable to the LLM or not up-to-date within its parameters. While using the question to retrieve relevant text from an external knowledge source helps LLMs, we observe that this one-step retrieve-and-read approach is insufficient for multi-step QA. Here, what to retrieve depends on what has already been derived, which in turn may depend on what was previously retrieved. To address this, we propose IRCoT, a new approach for multi-step QA that interleaves retrieval with steps (sentences) in a CoT, guiding the retrieval with CoT and in turn using retrieved results to improve CoT. Using IRCoT with GPT3 substantially improves retrieval (up to 21 points) as well as downstream QA (up to 15 points) on four datasets: HotpotQA, 2WikiMultihopQA, MuSiQue, and IIRC. We observe similar substantial gains in out-of-distribution (OOD) settings as well as with much smaller models such as Flan-T5-large without additional training. IRCoT reduces model hallucination, resulting in factually more accurate CoT reasoning. Code, data, and prompts are available at \urlhttps://github.com/stonybrooknlp/ircot

Cited by

• DMA: Online RAG Alignment with Human Feedback
• A New Role for Relevance: Guiding Corpus Interaction in Agentic Search
• VecTree-RAG: An Agentic Retrieval-Augmented Generation Framework Combining Vector and Tree Retrieval for Efficiency and Accuracy
• Co-Evolving Graph and Text Memory for Training-Free Multi-Hop Question Answering
• ScalableRAG: High-Quality RAG at Zero Ingestion Cost
• When Thinking Before Retrieval Hurts: TraceBound Diagnostics for Adaptive Knowledge-Graph Retrieval
• HyCE-RAG: Hypergraph Chain-of-Evidence Retrieval-Augmented Generation for Explainable Multi-hop Question Answering
• Structure Over Scale: Schema-Constrained Causal Graphs for RAG
• SF-AMS: Strategic Forgetting for Structured Memory in LLM Agent
• Language-Coupled Reinforcement Learning for Multilingual Retrieval-Augmented Generation
• QuCo-RAG: Quantifying Uncertainty from the Pre-training Corpus for Dynamic Retrieval-Augmented Generation
• AdaSearch: Balancing Parametric Knowledge and Search in Large Language Models via Reinforcement Learning
• Leveraging Spreading Activation for Improved Document Retrieval in Knowledge-Graph-Based RAG Systems
• CoDA: A Context-Decoupled Hierarchical Agent with Reinforcement Learning
• Hybrid Retrieval-Augmented Generation for Robust Multilingual Document Question Answering
• VERAFI: Verified Agentic Financial Intelligence through Neurosymbolic Policy Generation
• PathFinder: MCTS and LLM Feedback-based Path Selection for Multi-Hop Question Answering
• Cooperative Retrieval-Augmented Generation for Question Answering: Mutual Information Exchange and Ranking by Contrasting Layers
• KBQA-R1: Reinforcing Large Language Models for Knowledge Base Question Answering
• RouteRAG: Efficient Retrieval-Augmented Generation from Text and Graph via Reinforcement Learning
• Source Coverage and Citation Bias in LLM-based vs. Traditional Search Engines
• LightSearcher: Efficient DeepSearch via Experiential Memory
• ChipMind: Retrieval-Augmented Reasoning for Long-Context Circuit Design Specifications
• Agentic Policy Optimization via Instruction-Policy Co-Evolution
• EmoRAG: Evaluating RAG Robustness to Symbolic Perturbations
• Context-Aware Pragmatic Metacognitive Prompting for Sarcasm Detection
• Reducing Latency of LLM Search Agent via Speculation-based Algorithm-System Co-Design
• Agent-as-a-Graph: Knowledge Graph-Based Tool and Agent Retrieval for LLM Multi-Agent Systems
• Parametric Retrieval-Augmented Generation using Latent Routing of LoRA Adapters
• MuISQA: Multi-Intent Retrieval-Augmented Generation for Scientific Question Answering
• NeuroPath: Neurobiology-Inspired Path Tracking and Reflection for Semantically Coherent Retrieval
• RAGSmith: A Framework for Finding the Optimal Composition of Retrieval-Augmented Generation Methods Across Datasets
• Modeling Uncertainty Trends for Timely Retrieval in Dynamic RAG
• Thinking Forward and Backward: Multi-Objective Reinforcement Learning for Retrieval-Augmented Reasoning
• DeepSpecs: Expert-Level Questions Answering in 5G
• Thinker: Training LLMs in Hierarchical Thinking for Deep Search via Multi-Turn Interaction
• Think Before You Retrieve: Learning Test-Time Adaptive Search with Small Language Models
• LGM: Enhancing Large Language Models with Conceptual Meta-Relations and Iterative Retrieval
• MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learning
• Beyond Single Embeddings: Capturing Diverse Targets with Multi-Query Retrieval
• Tool-to-Agent Retrieval: Bridging Tools and Agents for Scalable LLM Multi-Agent Systems
• PathFinder: Efficiently Supporting Conjunctions and Disjunctions for Filtered Approximate Nearest Neighbor Search
• Test-time Scaling of LLMs: A Survey from A Subproblem Structure Perspective
• TreeQA: Enhanced LLM-RAG with logic tree reasoning for reliable and interpretable multi-hop question answering
• Interact-RAG: Reason and Interact with the Corpus, Beyond Black-Box Retrieval
• SIGMA: Search-Augmented On-Demand Knowledge Integration for Agentic Mathematical Reasoning
• MARAG-R1: Beyond Single Retriever via Reinforcement-Learned Multi-Tool Agentic Retrieval
• Empowering RepoQA-Agent based on Reinforcement Learning Driven by Monte-carlo Tree Search
• Towards Global Retrieval Augmented Generation: A Benchmark for Corpus-Level Reasoning
• GAP: Graph-Based Agent Planning with Parallel Tool Use and Reinforcement Learning
• BM25 Wins at Scale: A Scaling Study of Retrieval-Augmented Generation Paradigms
• CMT-RAG: Complementary Memory Traces for Multi-turn Multi-hop RAG
• Beyond Self-Knowledge: Propagating Uncertainty Across Reasoning and Retrieval in LLMs
• WikiLoop: Jointly Learning to Build and Navigate Agent-Native Wikis with Downstream Feedback
• GroupRAG: Cognitively Inspired Group-Aware Retrieval and Reasoning via Knowledge-Driven Problem Structuring
• Optimizing Retrieval for RAG via Reinforced Contrastive Learning
• Mitigating Hallucination in Large Language Models (LLMs): An Application-Oriented Survey on RAG, Reasoning, and Agentic Systems
• Metadata-Driven Retrieval-Augmented Generation for Financial Question Answering
• FAIR-RAG: Faithful Adaptive Iterative Refinement for Retrieval-Augmented Generation
• Bridging Language Gaps with Adaptive RAG: Improving Indonesian Language Question Answering
• GlobalRAG: Enhancing Global Reasoning in Multi-hop Question Answering via Reinforcement Learning
• Think Parallax: Solving Multi-Hop Problems via Multi-View Knowledge-Graph-Based Retrieval-Augmented Generation
• Think Straight, Stop Smart: Structured Reasoning for Efficient Multi-Hop RAG
• WebSeer: Training Deeper Search Agents through Reinforcement Learning with Self-Reflection
• AcademicEval: Live Long-Context LLM Benchmark
• SafeSearch: Do Not Trade Safety for Utility in LLM Search Agents
• FinSight: Towards Real-World Financial Deep Research
• A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications
• Cost-Aware Retrieval-Augmentation Reasoning Models with Adaptive Retrieval Depth
• EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle
• An Efficient Rubric-based Generative Verifier for Search-Augmented LLMs
• PluriHop: Exhaustive, Recall-Sensitive QA over Distractor-Rich Corpora
• Stop-RAG: Value-Based Retrieval Control for Iterative RAG
• PRISM: Agentic Retrieval with LLMs for Multi-Hop Question Answering
• Learnable Game-theoretic Policy Optimization for Data-centric Self-explanation Rationalization
• Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generation
• Understanding Parametric Knowledge Injection in Retrieval-Augmented Generation
• CTIArena: Benchmarking LLM Knowledge and Reasoning Across Heterogeneous Cyber Threat Intelligence
• Query-Specific GNN: A Comprehensive Graph Representation Learning Method for Retrieval Augmented Generation
• Uncertainty Quantification for Retrieval-Augmented Reasoning
• VeriCite: Towards Reliable Citations in Retrieval-Augmented Generation via Rigorous Verification
• LLM-Specific Utility: A New Perspective for Retrieval-Augmented Generation
• BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actions
• RECON: Reasoning with Condensation for Efficient Retrieval-Augmented Generation
• Beyond the limitation of a single query: Train your LLM for query expansion with Reinforcement Learning
• Meta-Harness: End-to-End Optimization of Model Harnesses
• STEPER: Step-wise Knowledge Distillation for Enhancing Reasoning Ability in Multi-Step Retrieval-Augmented Language Models
• HiPRAG: Hierarchical Process Rewards for Efficient Agentic Retrieval Augmented Generation
• QAgent: A modular Search Agent with Interactive Query Understanding
• A²Search: Ambiguity-Aware Question Answering with Reinforcement Learning
• Benchmarking Chinese Commonsense Reasoning with a Multi-hop Reasoning Perspective
• When Thoughts Meet Facts: Reusable Reasoning for Long-Context LMs
• Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluation
• TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Robotics
• Stratified GRPO: Handling Structural Heterogeneity in Reinforcement Learning of LLM Search Agents
• DecEx-RAG: Boosting Agentic Retrieval-Augmented Generation with Decision and Execution Optimization via Process Supervision
• CAM: A Constructivist View of Agentic Memory for LLM-Based Reading Comprehension
• RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problems
• Beyond Outcome Reward: Decoupling Search and Answering Improves LLM Agents
• Multi-Hop Question Answering: When Can Humans Help, and Where do They Struggle?
• TRAJECT-Bench:A Trajectory-Aware Benchmark for Evaluating Agentic Tool Use
• Large Language Models Hallucination: A Comprehensive Survey
• Less Diverse, Less Safe: The Indirect But Pervasive Risk of Test-Time Scaling in Large Language Models
• One More Question is Enough, Expert Question Decomposition (EQD) Model for Domain Quantitative Reasoning
• Learning to Route: A Rule-Driven Agent Framework for Hybrid-Source Retrieval-Augmented Generation
• RE-Searcher: Robust Agentic Search with Goal-oriented Planning and Self-reflection
• ReTAG: Retrieval-Enhanced, Topic-Augmented Graph-Based Global Sensemaking
• Hybrid Reward Normalization for Process-supervised Non-verifiable Agentic Tasks
• Beyond Static Retrieval: Opportunities and Pitfalls of Iterative Retrieval in GraphRAG
• AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforced Self-Play
• G-reasoner: Foundation Models for Unified Reasoning over Graph-structured Knowledge
• Not Wrong, But Untrue: LLM Overconfidence in Document-Based Queries
• PaperSearchQA: Learning to Search and Reason over Scientific Papers with RLVR
• Think-on-Graph 3.0: Efficient and Adaptive LLM Reasoning on Heterogeneous Graphs via Multi-Agent Dual-Evolving Context Retrieval
• What Should I Cite? A RAG Benchmark for Academic Citation Prediction
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning
• Harness-G: A Graph-Structured Harness for Search Agents
• Dr-DCI: Scaling Direct Corpus Interaction via Dynamic Workspace Expansion
• Retrieval Feedback Memory Enhancement Large Model Retrieval Generation Method
• Advances in Large Language Models for Medicine
• Understanding Benchmark Language Under Weakened Formal Semantics
• Enhancing Retrieval Augmentation via Adversarial Collaboration
• Who Taught the Lie? Responsibility Attribution for Poisoned Knowledge in Retrieval-Augmented Generation
• Metacognitive Reuse: Turning Recurring LLM Reasoning Into Concise Behaviors
• Deploying AI for Signal Processing education: Selected challenges and intriguing opportunities
• Learning from Diverse Reasoning Paths with Routing and Collaboration
• HANRAG: Heuristic Accurate Noise-resistant Retrieval-Augmented Generation for Multi-hop Question Answering
• Chain or tree? Re-evaluating complex reasoning from the perspective of a matrix of thought
• OPERA: A Reinforcement Learning--Enhanced Orchestrated Planner-Executor Architecture for Reasoning-Oriented Multi-Hop Retrieval
• KoBLEX: Open Legal Question Answering with Multi-hop Reasoning
• Towards Open-World Retrieval-Augmented Generation on Knowledge Graph: A Multi-Agent Collaboration Framework
• VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use
• MMAPG: A Training-Free Framework for Multimodal Multi-hop Question Answering via Adaptive Planning Graphs
• Multimodal Iterative RAG for Knowledge-Intensive Visual Question Answering
• Decomposing and Revising What Language Models Generate
• EviNote-RAG: Enhancing RAG Models via Answer-Supportive Evidence Notes
• Open Data Synthesis For Deep Research
• Exploring Reasoning-Infused Text Embedding with Large Language Models for Zero-Shot Dense Retrieval
• KG-CQR: Leveraging Structured Relation Representations in Knowledge Graphs for Contextual Query Retrieval
• AI-SearchPlanner: Modular Agentic Search via Pareto-Optimal Multi-Objective Reinforcement Learning
• TinyServe: Query-Aware Cache Selection for Efficient LLM Serving
• Can Compact Language Models Search Like Agents? Distillation-Guided Policy Optimization for Preserving Agentic RAG Capabilities
• Hybrid Deep Searcher: Integrating Parallel and Sequential Search Reasoning
• Test-time Corpus Feedback: From Retrieval to RAG
• Thinking Before You Speak: A Proactive Test-time Scaling Approach
• FLAIR: Feedback Learning for Adaptive Information Retrieval
• Cross-Granularity Hypergraph Retrieval-Augmented Generation for Multi-hop Question Answering
• MoNaCo: More Natural and Complex Questions for Reasoning Across Dozens of Documents
• ComoRAG: A Cognitive-Inspired Memory-Organized RAG for Stateful Long Narrative Reasoning
• Inductive Bias Extraction and Matching for LLM Prompts
• ParallelSearch: Train your LLMs to Decompose Query and Search Sub-queries in Parallel with Reinforcement Learning
• KG-o1: Enhancing Multi-hop Question Answering in Large Language Models via Knowledge Graph Integration
• You Don't Need Pre-built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures
• CoCoLex: Confidence-guided Copy-based Decoding for Grounded Legal Text Generation
• LAG: Logic-Augmented Generation from a Cartesian Perspective
• MultiRAG: A Knowledge-guided Framework for Mitigating Hallucination in Multi-source Retrieval Augmented Generation
• Beyond Chunks and Graphs: Retrieval-Augmented Generation through Triplet-Driven Thinking
• LMAR: Language Model Augmented Retriever for Domain-specific Knowledge Indexing
• CoCoA: Collaborative Chain-of-Agents for Parametric-Retrieved Knowledge Synergy
• A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challenges
• From Query to Logic: Ontology-Driven Multi-Hop Reasoning in LLMs
• Lucy: edgerunning agentic web search on mobile with machine generated task vectors
• From Sufficiency to Reflection: Reinforcement-Guided Thinking Quality in Retrieval-Augmented Reasoning for LLMs
• DeepSieve: Information Sieving via LLM-as-a-Knowledge-Router
• RAG in the Wild: On the (In)effectiveness of LLMs with Mixture-of-Knowledge Retrieval Augmentation

Related

• Knowledge-Driven CoT: Exploring Faithful Reasoning in LLMs for Knowledge-intensive Question Answering
• Graph Elicitation for Guiding Multi-Step Reasoning in Large Language Models
• KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoning
• Retrieval-augmented Multi-modal Chain-of-Thoughts Reasoning for Large Language Models
• Tree of Reviews: A Tree-based Dynamic Iterative Retrieval Framework for Multi-hop Question Answering