AgentTuning: Enabling Generalized Agent Abilities for LLMs
2023/10/19 by Aohan Zeng, Mingdao Liu, Zeng, Aohan +11 · 87 citations
Computer Science · Medicine · #Topic Modeling #Natural Language Processing Techniques #Artificial Intelligence in Healthcare and Education
paper · pdf · doi:10.48550/arxiv.2310.12823
Abstract
Open large language models (LLMs) with great performance in various tasks have significantly advanced the development of LLMs. However, they are far inferior to commercial models such as ChatGPT and GPT-4 when acting as agents to tackle complex tasks in the real world. These agent tasks employ LLMs as the central controller responsible for planning, memorization, and tool utilization, necessitating both fine-grained prompting methods and robust LLMs to achieve satisfactory performance. Though many prompting methods have been proposed to complete particular agent tasks, there is lack of research focusing on improving the agent capabilities of LLMs themselves without compromising their general abilities. In this work, we present AgentTuning, a simple and general method to enhance the agent abilities of LLMs while maintaining their general LLM capabilities. We construct AgentInstruct, a lightweight instruction-tuning dataset containing high-quality interaction trajectories. We employ a hybrid instruction-tuning strategy by combining AgentInstruct with open-source instructions from general domains. AgentTuning is used to instruction-tune the Llama 2 series, resulting in AgentLM. Our evaluations show that AgentTuning enables LLMs' agent capabilities without compromising general abilities. The AgentLM-70B is comparable to GPT-3.5-turbo on unseen agent tasks, demonstrating generalized agent capabilities. We open source the AgentInstruct and AgentLM-7B, 13B, and 70B models at https://github.com/THUDM/AgentTuning, serving open and powerful alternatives to commercial LLMs for agent tasks.
Cited by
- MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles
- AWPO: Enhancing Tool-Use of Large Language Models through Adaptive Integration of Reasoning Rewards
- Learning Hierarchical Procedural Memory for LLM Agents through Bayesian Selection and Contrastive Refinement
- Reinforcement Learning for Self-Improving Agent with Skill Library
- Inference-Time Distillation: Cost-Efficient Agents Without Fine-Tuning or Manual Prompt Engineering
- COACH: Collaborative Agents for Contextual Highlighting -- A Multi-Agent Framework for Sports Video Analysis
- Co-Evolving Agents: Learning from Failures as Hard Negatives
- DEPO: Dual-Efficiency Preference Optimization for LLM Agents
- Simulating Environments with Reasoning Models for Agent Training
- Can a Small Model Learn to Look Before It Leaps? Dynamic Learning and Proactive Correction for Hallucination Detection
- Systematizing LLM Persona Design: A Four-Quadrant Technical Taxonomy for AI Companion Applications
- Delegated Authorization for Agents Constrained to Semantic Task-to-Scope Matching
- InfoFlow: Reinforcing Search Agent Via Reward Density Optimization
- RAPID: An Efficient Reinforcement Learning Algorithm for Small Language Models
- AgentSnare: Learning to Delay, Divert, and Defuse Autonomous Penetration Agents
- Agent Data Protocol: Unifying Datasets for Diverse, Effective Fine-tuning of LLM Agents
- WebLeaper: Empowering Efficiency and Efficacy in WebAgent via Enabling Info-Rich Seeking
- TEXT2DB: Integration-Aware Information Extraction with Large Language Model Agents
- DeepAgent: A General Reasoning Agent with Scalable Toolsets
- SALT: Step-level Advantage Assignment for Long-horizon Agents via Trajectory Graph
- Enhancing Language Agent Strategic Reasoning through Self-Play in Adversarial Games
- RoboGPT-R1: Enhancing Robot Planning with Reinforcement Learning
- KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems
- Self-Improving LLM Agents at Test-Time
- Chain-of-Trigger: An Agentic Backdoor that Paradoxically Enhances Agentic Robustness
- AlphaApollo: A System for Deep Agentic Reasoning
- AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Framework
- ReSeek: A Self-Correcting Framework for Search Agents with Instructive Rewards
- LogPilot: Intent-aware and Scalable Alert Diagnosis for Large-scale Online Service Systems
- Scaling Generalist Data-Analytic Agents
- A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memory
- LatentEvolve: Self-Evolving Test-Time Scaling in Latent Space
- Rethinking Reward Miscalibration of GRPO in Agentic RL
- Solving the Granularity Mismatch: Hierarchical Preference Learning for Long-Horizon LLM Agents
- From Scoring to Acting: Outcome-Verified Comparative Self-Distillation for LLM Agents
- AirQA: A Comprehensive QA Dataset for AI Research with Instance-Level Evaluation
- Generalizability of Large Language Model-Based Agents: A Comprehensive Survey
- Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems
- AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning
- Planning with Reasoning using Vision Language World Model
- S3LoRA: Safe Spectral Sharpness-Guided Pruning in Adaptation of Agent Planner
- ToolACE-MT: Non-Autoregressive Generation for Agentic Multi-Turn Interaction
- PilotRL: Training Language Model Agents via Global Planning-Guided Progressive Reinforcement Learning
- MMAT-1M: A Large Reasoning Dataset for Multimodal Agent Tuning
- Initial Steps in Integrating Large Reasoning and Action Models for Service Composition
- AgentFly: Extensible and Scalable Reinforcement Learning for LM Agents
- Compiling Agentic Workflows into LLM Weights: Near-Frontier Quality at Two Orders of Magnitude Less Cost
- CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios
- Your Agent Can Defend Itself against Backdoor Attacks
- DatasetAgent: A Novel Multi-Agent System for Auto-Constructing Datasets from Real-World Images
- Unleashing Embodied Task Planning Ability in LLMs via Reinforcement Learning
- SEEA-R1: Tree-Structured Reinforcement Fine-Tuning for Self-Evolving Embodied Agents
- Agent-RewardBench: Towards a Unified Benchmark for Reward Modeling across Perception, Planning, and Safety in Real-World Multimodal Agents
- Engineering RAG Systems for Real-World Applications: Design, Development, and Evaluation
- Why Do Open-Source LLMs Struggle with Data Analysis? A Systematic Empirical Study
- Distilling On-device Language Models for Robot Planning with Minimal Human Intervention
- MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents
- AutoMind: Adaptive Knowledgeable Agent for Automated Data Science
- Automated Skill Discovery for Language Agents through Exploration and Iterative Feedback
- VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agents
- PGPO: Enhancing Agent Reasoning via Pseudocode-style Planning Guided Preference Optimization
- VLM Q-Learning: Aligning Vision-Language Models for Interactive Decision-Making
- OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation
- LLM-ODDR: A Large Language Model Framework for Joint Order Dispatching and Driver Repositioning
- WebDancer: Towards Autonomous Information Seeking Agency
- SPA-RL: Reinforcing LLM Agents via Stepwise Progress Attribution
- Large Language Models for Planning: A Comprehensive and Systematic Survey
- Distilling LLM Agent into Small Models with Retrieval and Code Tools
- USTBench: Benchmarking and Dissecting Spatiotemporal Reasoning of LLMs as Urban Agents
- The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning
- Retrospex: Language Agent Meets Offline Reinforcement Learning Critic
- DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generation
- Do Language Model Agents Align with Humans in Rating Visualizations? An Empirical Study
- OpenResearcher: A Fully Open Pipeline for Long-Horizon Deep Research Trajectory Synthesis
- Towards Efficient Online Tuning of VLM Agents via Counterfactual Soft Reinforcement Learning
- Iterative Tool Usage Exploration for Multimodal Agents via Step-wise Preference Tuning
- SkillGen: Verified Inference-Time Agent Skill Synthesis
- From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review
- A Survey of Task-Oriented Knowledge Graph Reasoning: Status, Applications, and Prospects
- State2State: Environment-Derived Mid-Training for LLM Agents
- NAACL2025 Tutorial: Adaptation of Large Language Models
- Building LLM Agents by Incorporating Insights from Computer Systems
- a1: Steep Test-time Scaling Law via Environment Augmented Generation
- Exploring Expert Failures Improves LLM Agent Tuning
- ToolRL: Reward is All Tool Learning Needs
- Playpen: An Environment for Exploring Learning Through Conversational Interaction
- A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directions
Related