Dynamic and Parametric Retrieval-Augmented Generation

2025/06/07 by Weihang Su, Su, Weihang, Qingyao Ai +7 · 1 voice · 2 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Information Retrieval and Search Behavior #Multimodal Machine Learning Applications #Topic Modeling #cs.CL #cs.IR

paper · pdf · doi:10.48550/arxiv.2506.06704

openalex publication_date 2025/06/07 · arxiv published 2025/06/07 · arxiv updated 2025/06/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Retrieval-Augmented Generation (RAG) has become a foundational paradigm for equipping large language models (LLMs) with external knowledge, playing a critical role in information retrieval and knowledge-intensive applications. However, conventional RAG systems typically adopt a static retrieve-then-generate pipeline and rely on in-context knowledge injection, which can be suboptimal for complex tasks that require multihop reasoning, adaptive information access, and deeper integration of external knowledge. Motivated by these limitations, the research community has moved beyond static retrieval and in-context knowledge injection. Among the emerging directions, this tutorial delves into two rapidly growing and complementary research areas on RAG: Dynamic RAG and Parametric RAG. Dynamic RAG adaptively determines when and what to retrieve during the LLM's generation process, enabling real-time adaptation to the LLM's evolving information needs. Parametric RAG rethinks how retrieved knowledge should be injected into LLMs, transitioning from input-level to parameter-level knowledge injection for enhanced efficiency and effectiveness. This tutorial offers a comprehensive overview of recent advances in these emerging research areas. It also shares theoretical foundations and practical insights to support and inspire further research in RAG.

Citations

• Decoupling Reasoning and Knowledge Injection for In-Context Knowledge Editing
• Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement
• JuDGE: Benchmarking Judgment Document Generation for Chinese Legal System
• Parametric Retrieval Augmented Generation
• Knowledge Editing through Chain-of-Thought
• Qwen2.5 Technical Report
• LeKUBE: A Legal Knowledge Update BEnchmark
• Mitigating Entity-Level Hallucination in Large Language Models
• DRAGIN: Dynamic Retrieval Augmented Generation based on the Information Needs of Large Language Models
• Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Models
• Neuron-Level Knowledge Attribution in Large Language Models
• Caseformer: Pre-training for Legal Case Retrieval Based on Inter-Case Distinctions
• Knowledge Editing for Large Language Models: A Survey
• Lost in the Middle: How Language Models Use Long Contexts
• CaseEncoder: A Knowledge-enhanced Pre-trained Model for Legal Case Encoding
• Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer
• Survey of Hallucination in Natural Language Generation
• Condenser: a Pre-training Architecture for Dense Retrieval
• Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering
• Language Models are Few-Shot Learners
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Don't Stop Pretraining: Adapt Language Models to Domains and Tasks
• Dense Passage Retrieval for Open-Domain Question Answering
• CtrlA: Adaptive Retrieval-Augmented Generation via Inherent Control

Cited by

• Prior Knowledge Makes It Possible: From Sublinear Graph Algorithms to LLM Test-Time Methods
• SurGE: A Benchmark and Evaluation Framework for Scientific Survey Generation

Discussions

• Dynamic and Parametric Retrieval-Augmented Generation [hn, 1 points, 0 comments]

Related

• Parametric Retrieval Augmented Generation
• Retrieval-Augmented Generation for Large Language Models: A Survey
• Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement
• When Retrieval Succeeds and Fails: Rethinking Retrieval-Augmented Generation for LLMs
• Retrieval-Augmented Generation for Natural Language Processing: A Survey