FlowReasoner: Reinforcing Query-Level Meta-Agents

2025/04/21 by Gao, Hongcheng, Liu, Yue, He, Yufei +6 · 12 citations
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2504.15257

Abstract

This paper proposes a query-level meta-agent named FlowReasoner to automate the design of query-level multi-agent systems, i.e., one system per user query. Our core idea is to incentivize a reasoning-based meta-agent via external execution feedback. Concretely, by distilling DeepSeek R1, we first endow the basic reasoning ability regarding the generation of multi-agent systems to FlowReasoner. Then, we further enhance it via reinforcement learning (RL) with external execution feedback. A multi-purpose reward is designed to guide the RL training from aspects of performance, complexity, and efficiency. In this manner, FlowReasoner is enabled to generate a personalized multi-agent system for each user query via deliberative reasoning. Experiments on both engineering and competition code benchmarks demonstrate the superiority of FlowReasoner. Remarkably, it surpasses o1-mini by 10.52% accuracy across three benchmarks. The code is available at https://github.com/sail-sg/FlowReasoner.

Cited by

• Grammar Search for Multi-Agent Systems
• FutureWeaver: Planning Test-Time Compute for Multi-Agent Systems with Modularized Collaboration
• VideoChat-M1: Collaborative Policy Planning for Video Understanding via Multi-Agent Reinforcement Learning
• EvoTest: Evolutionary Test-Time Learning for Self-Improving Agentic Systems
• Agentic-KGR: Co-evolutionary Knowledge Graph Construction through Multi-Agent Reinforcement Learning
• CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards
• Efficient Tree-Structured Deep Research with Adaptive Resource Allocation
• ARM: Discovering Agentic Reasoning Modules for Generalizable Multi-Agent Systems
• MAS²: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systems
• RobustFlow: Towards Robust Agentic Workflow Generation
• AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?
• Graph-Augmented Large Language Model Agents: Current Progress and Future Prospects

Related

• MARS-SQL: A multi-agent reinforcement learning framework for Text-to-SQL
• MarsRL: Advancing Multi-Agent Reasoning System via Reinforcement Learning with Agentic Pipeline Parallelism
• Learning to Coordinate Multiple Reinforcement Learning Agents for Diverse Query Reformulation
• FlowSearch: Advancing deep research with dynamic structured knowledge flow
• MAO-ARAG: Multi-Agent Orchestration for Adaptive Retrieval-Augmented Generation