M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitions

2024/05/26 by Zheng Wang, Wang, Zheng, Shu Xian Teo +7 · 7 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2405.16420

openalex publication_date 2024/05/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by retrieving relevant memories from an external database. However, existing RAG methods typically organize all memories in a whole database, potentially limiting focus on crucial memories and introducing noise. In this paper, we introduce a multiple partition paradigm for RAG (called M-RAG), where each database partition serves as a basic unit for RAG execution. Based on this paradigm, we propose a novel framework that leverages LLMs with Multi-Agent Reinforcement Learning to optimize different language generation tasks explicitly. Through comprehensive experiments conducted on seven datasets, spanning three language generation tasks and involving three distinct language model architectures, we confirm that M-RAG consistently outperforms various baseline methods, achieving improvements of 11%, 8%, and 12% for text summarization, machine translation, and dialogue generation, respectively.

Cited by

• RAGPulse: An Open-Source RAG Workload Trace to Optimize RAG Serving Systems
• Grounded by Experience: Generative Healthcare Prediction Augmented with Hierarchical Agentic Retrieval
• A Decentralized Retrieval Augmented Generation System with Source Reliabilities Secured on Blockchain
• Separate the Wheat from the Chaff: Winnowing Down Divergent Views in Retrieval Augmented Generation
• ToM: Leveraging Tree-oriented MapReduce for Long-Context Reasoning in Large Language Models
• From Deferral to Learning: Online In-Context Knowledge Distillation for LLM Cascades
• TracSum: A New Benchmark for Aspect-Based Summarization with Sentence-Level Traceability in Medical Domain

Related

• RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewards
• Retrieval-Augmented Generation for Large Language Models: A Survey
• RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation
• A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
• A Survey on Retrieval-Augmented Text Generation for Large Language Models