CoPS: Empowering LLM Agents with Provable Cross-Task Experience Sharing

2024/10/22 by Yang Chen, Chenyang Zhao, Yang, Chen +4 · 5 citations
Business, Management and Accounting · Computer Science · #Artificial Intelligence (cs.AI) #Business Process Modeling and Analysis #Cloud Computing and Resource Management #FOS: Computer and information sciences #Machine Learning (cs.LG) #Service-Oriented Architecture and Web Services

paper · pdf · doi:10.48550/arxiv.2410.16670

openalex publication_date 2024/10/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Sequential reasoning in agent systems has been significantly advanced by large language models (LLMs), yet existing approaches face limitations. Reflection-driven reasoning relies solely on knowledge in pretrained models, limiting performance in novel scenarios, while experience-assisted reasoning often depends on external experiences and lacks clear principles for selecting representative experiences. We address these limitations by proposing CoPS (Cross-Task Experience Sharing), a generalizable algorithm that enhances sequential reasoning by cross-task experience sharing and selection. In detail, CoPS leverages agents' experiences on previous tasks, selecting distribution-matched experiences via a provable pessimism-based strategy to maximize utility while minimizing risks from distribution shifts. Extensive experimental results on benchmarks like Alfworld, Webshop, and HotPotQA demonstrate that CoPS consistently outperforms state-of-the-art baselines, with superior sample efficiency suitable for resource-constrained scenarios. Theoretically, we show that the performance of our algorithm depends on both the quality of the pretrained LLM and the matching between the agent's task-dependent trial distribution and that generated by the LLM. Our work bridges the gap between existing sequential reasoning paradigms and validates the effectiveness of leveraging cross-task experiences, shedding light on the potential to improve agents' generalization and adaptability across diverse tasks. Our codes are available at \hrefhttps://github.com/uclaml/COPShttps://github.com/uclaml/COPS.

Cited by

• Unifying Dynamic Tool Creation and Cross-Task Experience Sharing through Cognitive Memory Architecture
• Fed-SE: Federated Self-Evolution for Privacy-Constrained Multi-Environment LLM Agents
• Escaping the Self-Confirmation Trap: An Execute-Distill-Verify Paradigm for Agentic Experience Learning
• DICE: Dynamic In-Context Example Selection in LLM Agents via Efficient Knowledge Transfer
• Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs

Related

• DICE: Dynamic In-Context Example Selection in LLM Agents via Efficient Knowledge Transfer
• Interactive Learning for LLM Reasoning
• Self-Generated In-Context Examples Improve LLM Agents for Sequential Decision-Making Tasks
• Cross-Task Experiential Learning on LLM-based Multi-Agent Collaboration
• Distilling LLM Agent into Small Models with Retrieval and Code Tools