More Agents Is All You Need

2024/02/03 by Junyou Li, Qin Zhang, Li, Junyou +7 · 6 voices · 32 citations
Computer Science · #Topic Modeling #Multimodal Machine Learning Applications #Natural Language Processing Techniques

paper · pdf · doi:10.48550/arxiv.2402.05120

Abstract

We find that, simply via a sampling-and-voting method, the performance of large language models (LLMs) scales with the number of agents instantiated. Also, this method, termed as Agent Forest, is orthogonal to existing complicated methods to further enhance LLMs, while the degree of enhancement is correlated to the task difficulty. We conduct comprehensive experiments on a wide range of LLM benchmarks to verify the presence of our finding, and to study the properties that can facilitate its occurrence. Our code is publicly available at: https://github.com/MoreAgentsIsAllYouNeed/AgentForest

Cited by

• When Agents Disagree: The Selection Bottleneck in Multi-Agent LLM Pipelines
• LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning
• When Do Multi-Agent Systems Help? An Information Bottleneck Perspective
• Language Model Teams as Distributed Systems
• Emergent Coordination in Multi-Agent Language Models
• A Taxonomy of Transcendence
• Scoring, Reasoning, and Selecting the Best! Ensembling Large Language Models via a Peer-Review Process
• LLMBoost: Make Large Language Models Stronger with Boosting
• State-dependent error correlations shape voting thresholds in committees of AI agents
• Towards a Science of Scaling Agent Systems
• Reliable agent engineering should integrate machine-compatible organizational principles
• WISE: Weighted Iterative Society-of-Experts for Robust Multimodal Multi-Agent Debate
• Beyond human gold standards: A multimodel framework for automated abstract classification and information extraction
• RETuning: Upgrading Inference-Time Scaling for Stock Movement Prediction with Large Language Models
• Teaming LLMs to Detect and Mitigate Hallucinations
• Monitoring LLM-based Multi-Agent Systems Against Corruptions via Node Evaluation
• Selecting and Combining Large Language Models for Scalable Code Clone Detection
• Agentic Inequality
• Effective Strategies for Asynchronous Software Engineering Agents
• Slm-mux: Orchestrating small language models for reasoning
• Bridging Reasoning to Learning: Unmasking Illusions using Complexity Out of Distribution Generalization
• Best-of-∞ -- Asymptotic Performance of Test-Time Compute
• Escaping the Self-Confirmation Trap: An Execute-Distill-Verify Paradigm for Agentic Experience Learning
• System 0/1/2/3: Quad-Process Theory for Multitimescale Embodied Collective Cognitive Systems
• CARGO: A Framework for Confidence-Aware Routing of Large Language Models
• AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning
• Orchestrator: Active Inference for Multi-Agent Systems in Long-Horizon Tasks
• Transforming Agency. On the mode of existence of Large Language Models
• Intrinsic Memory Agents: Heterogeneous Multi-Agent LLM Systems through Structured Contextual Memory
• Yan: Foundational Interactive Video Generation
• MoMA: A Mixture-of-Multimodal-Agents Architecture for Enhancing Clinical Prediction Modelling
• CTTS: Collective Test-Time Scaling

Discussions

• More Agents Is All You Need: LLMs performance scales with the number of agents [hn, 288 points, 206 comments]
• More Agents Is All You Need [hn, 2 points, 0 comments]
• Ensembling LLMs with sample and voting is found to improve response accuracy asymptomatically. arxiv.org/abs/2402.05120 [bsky, 0 points, 0 comments]
• This document demonstrates that adding more agents increases #LLMs accuracy using a simple sampling-and-voting technique. https://arxiv.org/abs/2402.05120v1?utm_medium=social&utm_source=twitter [bsky, 0 points, 0 comments]
• Sampling multiple LLM improves answers. [bsky, 0 points, 0 comments]
• Increasing number of agents in LLMs shows greater improvement for more complex tasks. Extensive tests across various LLM benchmarks confirm these findings. arxiv.org/abs/2402.05120 [bsky, 0 points, 0 comments]

Related

• AgentBench: Evaluating LLMs as Agents
• AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents
• Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models
• Efficient Agents: Building Effective Agents While Reducing Cost
• Enhancing the General Agent Capabilities of Low-Parameter LLMs through Tuning and Multi-Branch Reasoning