More Agents Is All You Need
2024/02/03 by Junyou Li, Qin Zhang, Li, Junyou +7 · 6 voices · 54 citations
Chemistry · Computer Science · #Business #Chemistry #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2402.05120
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2024/02/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Abstract
We find that, simply via a sampling-and-voting method, the performance of large language models (LLMs) scales with the number of agents instantiated. Also, this method, termed as Agent Forest, is orthogonal to existing complicated methods to further enhance LLMs, while the degree of enhancement is correlated to the task difficulty. We conduct comprehensive experiments on a wide range of LLM benchmarks to verify the presence of our finding, and to study the properties that can facilitate its occurrence. Our code is publicly available at: https://github.com/MoreAgentsIsAllYouNeed/AgentForest
Cited by
- When Agents Disagree: The Selection Bottleneck in Multi-Agent LLM Pipelines
- LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning
- When Do Multi-Agent Systems Help? An Information Bottleneck Perspective
- Language Model Teams as Distributed Systems
- Emergent Coordination in Multi-Agent Language Models
- A Taxonomy of Transcendence
- Scoring, Reasoning, and Selecting the Best! Ensembling Large Language Models via a Peer-Review Process
- LLMBoost: Make Large Language Models Stronger with Boosting
- State-dependent error correlations shape voting thresholds in committees of AI agents
- Towards a Science of Scaling Agent Systems
- Reliable agent engineering should integrate machine-compatible organizational principles
- WISE: Weighted Iterative Society-of-Experts for Robust Multimodal Multi-Agent Debate
- Beyond human gold standards: A multimodel framework for automated abstract classification and information extraction
- RETuning: Upgrading Inference-Time Scaling for Stock Movement Prediction with Large Language Models
- Teaming LLMs to Detect and Mitigate Hallucinations
- Securing Multi-Agent Systems Against Corruptions via Node Contribution Backpropagation
- Selecting and Combining Large Language Models for Scalable Code Clone Detection
- Agentic Inequality
- Effective Strategies for Asynchronous Software Engineering Agents
- Slm-mux: Orchestrating small language models for reasoning
- Bridging Reasoning to Learning: Unmasking Illusions using Complexity Out of Distribution Generalization
- Best-of-∞ -- Asymptotic Performance of Test-Time LLM Ensembling
- Escaping the Self-Confirmation Trap: An Execute-Distill-Verify Paradigm for Agentic Experience Learning
- System 0/1/2/3: Quad-Process Theory for Multitimescale Embodied Collective Cognitive Systems
- CARGO: A Framework for Confidence-Aware Routing of Large Language Models
- AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning
- Orchestrator: Active Inference for Multi-Agent Systems in Long-Horizon Tasks
- Transforming Agency. On the mode of existence of Large Language Models
- Intrinsic Memory Agents: Heterogeneous Multi-Agent LLM Systems through Structured Contextual Memory
- Yan: Foundational Interactive Video Generation
- MoMA: A Mixture-of-Multimodal-Agents Architecture for Enhancing Clinical Prediction Modelling
- CTTS: Collective Test-Time Scaling
- CoE-Ops: Collaboration of LLM-based Experts for AIOps Question-Answering
- Byzantine-Robust Decentralized Coordination of LLM Agents
- The Future is Agentic: Definitions, Perspectives, and Open Challenges of Multi-Agent Recommender Systems
- Online Multi-LLM Selection via Contextual Bandits under Unstructured Context Evolution
- Integrating Traditional Technical Analysis with AI: A Multi-Agent LLM-Based Approach to Stock Market Forecasting
- LLM-based Satisfiability Checking of String Requirements by Consistent Data and Checker Generation
- Incentivizing LLMs to Self-Verify Their Answers
- EL4NER: Ensemble Learning for Named Entity Recognition via Multiple Small-Parameter Large Language Models
- Efficient Leave-one-out Approximation in LLM Multi-agent Debate Based on Introspection
- Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaboration
- SecVulEval: Benchmarking LLMs for Real-World C/C++ Vulnerability Detection
- Cost-aware LLM-based Online Dataset Annotation
- When Does LLM Orchestration Pay Off? A Controlled Evaluation of Accuracy, Cost, and Task Difficulty
- Automated Meta Prompt Engineering for Alignment with the Theory of Mind
- Internet of Agents: Fundamentals, Applications, and Challenges
- When to Vote, When to Rewrite: Disagreement-Guided Strategy Routing for Test-Time Scaling
- Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets
- When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models
- Federation over Text: Insight Sharing for Multi-Agent Reasoning
- Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self-Improving OCR
- EMAFusion: A Self-Optimizing System for Seamless LLM Selection and Integration
- Hogwild! Inference: Parallel LLM Generation via Concurrent Attention
Discussions
- More Agents Is All You Need: LLMs performance scales with the number of agents [hn, 288 points, 206 comments]
- More Agents Is All You Need [hn, 2 points, 0 comments]
- Ensembling LLMs with sample and voting is found to improve response accuracy asymptomatically. arxiv.org/abs/2402.05120 [bsky, 0 points, 0 comments]
- This document demonstrates that adding more agents increases #LLMs accuracy using a simple sampling-and-voting technique. https://arxiv.org/abs/2402.05120v1?utm_medium=social&utm_source=twitter [bsky, 0 points, 0 comments]
- Sampling multiple LLM improves answers. [bsky, 0 points, 0 comments]
- Increasing number of agents in LLMs shows greater improvement for more complex tasks. Extensive tests across various LLM benchmarks confirm these findings. arxiv.org/abs/2402.05120 [bsky, 0 points, 0 comments]
Related