RAGPulse: An Open-Source RAG Workload Trace to Optimize RAG Serving Systems

2025/11/17 by Z. Wang, Wang, Zhengchao, Yitao Hu +11
Computer Science · Materials Science · #Advanced Graph Neural Networks #Databases (cs.DB) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Materials Science #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2511.12979

openalex publication_date 2025/11/17 · openalex created_date 2025/11/19 · openalex updated_date 2026/07/28

Abstract

Retrieval-Augmented Generation (RAG) is a critical paradigm for building reliable, knowledge-intensive Large Language Model (LLM) applications. However, the multi-stage pipeline (retrieve, generate) and unique workload characteristics (e.g., knowledge dependency) of RAG systems pose significant challenges for serving performance optimization. Existing generic LLM inference traces fail to capture these RAG-specific dynamics, creating a significant performance gap between academic research and real-world deployment. To bridge this gap, this paper introduces RAGPulse, an open-source RAG workload trace dataset. This dataset was collected from an university-wide Q&A system serving that has served more than 40,000 students and faculties since April 2024. We detail RAGPulse's system architecture, its privacy-preserving hash-based data format, and provide an in-depth statistical analysis. Our analysis reveals that real-world RAG workloads exhibit significant temporal locality and a highly skewed hot document access pattern. RAGPulse provides a high-fidelity foundation for researchers to develop and validate novel optimization strategies for RAG systems, such as content-aware batching and retrieval caching, ultimately enhancing the efficiency and reliability of RAG services. The code is available at https://github.com/flashserve/RAGPulse.

Citations

• TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Text
• Llumnix: Dynamic Scheduling for Large Language Model Serving
• M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitions
• RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation
• BurstGPT: A Real-world Workload Dataset to Optimize LLM Serving Systems
• Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
• Qwen Technical Report
• Efficient Memory Management for Large Language Model Serving with PagedAttention
• METIS: Fast Quality-Aware RAG Systems with Configuration Adaptation

Related

• Patchwork: A Unified Framework for RAG Serving
• RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Serving
• RAG Foundry: A Framework for Enhancing LLMs for Retrieval Augmented Generation
• Know Your RAG: Dataset Taxonomy and Generation Strategies for Evaluating RAG Systems
• RAGGED: Towards Informed Design of Scalable and Stable RAG Systems