KVComm: Enabling Efficient LLM Communication through Selective KV Sharing

2025/10/02 by Shi, Xiangyu, Marco Chiesa, Chiesa, Marco +4 · 4 citations
Computer Science · #Advanced Graph Neural Networks #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Healthcare #Multiagent Systems (cs.MA) #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2510.03346

openalex publication_date 2025/10/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large Language Models (LLMs) are increasingly deployed in multi-agent systems, where effective inter-model communication is crucial. Existing communication protocols either rely on natural language, incurring high inference costs and information loss, or on hidden states, which suffer from information concentration bias and inefficiency. To address these limitations, we propose KVComm, a novel communication framework that enables efficient communication between LLMs through selective sharing of KV pairs. KVComm leverages the rich information encoded in the KV pairs while avoiding the pitfalls of hidden states. We introduce a KV layer-wise selection strategy based on attention importance scores with a Gaussian prior to identify the most informative KV pairs for communication. Extensive experiments across diverse tasks and model pairs demonstrate that KVComm achieves comparable performance to the upper-bound method, which directly merges inputs to one model without any communication, while transmitting as few as 30% of layers' KV pairs. Our study highlights the potential of KV pairs as an effective medium for inter-LLM communication, paving the way for scalable and efficient multi-agent systems.

Citations

• KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems
• Communicating Activations Between Language Model Agents
• Qwen2.5 Technical Report
• DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving
• Compute Or Load KV Cache? Why Not Both?
• LayerKV: Optimizing Large Language Model Serving with Layer-wise KV Cache Management
• The Llama 3 Herd of Models
• InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management
• CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion
• MiniCache: KV Cache Compression in Depth Dimension for Large Language Models
• Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention
• Large Language Model based Multi-Agents: A Survey of Progress and Challenges
• The Falcon Series of Open Language Models
• Prompt Cache: Modular Attention Reuse for Low-Latency Inference
• CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving
• Let Models Speak Ciphers: Multiagent Debate through Embeddings
• Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs
• Efficient Memory Management for Large Language Model Serving with PagedAttention
• LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
• AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
• AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
• Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
• Improving Factuality and Reasoning in Language Models through Multiagent Debate
• LLM-Pruner: On the Structural Pruning of Large Language Models
• Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond
• CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society
• DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale
• FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
• Self-attention Does Not Need O(n²) Memory
• MuSiQue: Multihop Questions via Single-hop Question Composition
• A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers
• Transformer Feed-Forward Layers Are Key-Value Memories
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
• ChatDev: Communicative Agents for Software Development

Cited by

• Latent Collaboration in Multi-Agent Systems
• SmartGen: Seamless Disaggregated LLM Inference with Selective KV Cache Transfer
• Mixture-of-Translators: Translating KV Caches Across Heterogeneous Large Language Models
• When Does Latent Communication Pay? A Causal Audit of Relayed KV Caches in Multi-Agent LLMs

Related

• Q-KVComm: Efficient Multi-Agent Communication Via Adaptive KV Cache Compression
• Beyond KV Caching: Shared Attention for Efficient LLMs
• KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems
• A Systematic Study of Cross-Layer KV Sharing for Efficient LLM Inference
• Adapting LLM Agents with Universal Feedback in Communication