Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstruction
2025/10/16 by Shen, Xu, Zhang, Qi, Wang, Song +8
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences
paper · doi:10.48550/arxiv.2510.14319
Abstract
Large Language Model based multi-agent systems (MAS) excel at collaborative problem solving but remain brittle to cascading errors: a single faulty step can propagate across agents and disrupt the trajectory. In this paper, we present MASC, a metacognitive framework that endows MAS with real-time, unsupervised, step-level error detection and self-correction. MASC rethinks detection as history-conditioned anomaly scoring via two complementary designs: (1) Next-Execution Reconstruction, which predicts the embedding of the next step from the query and interaction history to capture causal consistency, and (2) Prototype-Guided Enhancement, which learns a prototype prior over normal-step embeddings and uses it to stabilize reconstruction and anomaly scoring under sparse context (e.g., early steps). When an anomaly step is flagged, MASC triggers a correction agent to revise the acting agent's output before information flows downstream. On the Who&When benchmark, MASC consistently outperforms all baselines, improving step-level error detection by up to 8.47% AUC-ROC ; When plugged into diverse MAS frameworks, it delivers consistent end-to-end gains across architectures, confirming that our metacognitive monitoring and targeted correction can mitigate error propagation with minimal overhead.
Citations
- EVADE: LLM-Based Explanation Generation and Validation for Error Detection in NLI
- Separate the Wheat from the Chaff: Winnowing Down Divergent Views in Retrieval Augmented Generation
- Agree, Disagree, Explain: Decomposing Human Label Variation in NLI through the Lens of Explanations
- Where LLM Agents Fail and How They can Learn From Failures
- Evaluating Large Language Models for Cross-Lingual Retrieval
- Aegis: Automated Error Generation and Attribution for Multi-Agent Systems
- Learning from Diverse Reasoning Paths with Routing and Collaboration
- BlindGuard: Safeguarding LLM-based Multi-Agent Systems under Unknown Attacks
- Assemble Your Crew: Automatic Multi-agent Communication Topology Design via Autoregressive Graph Generation
- AnyMAC: Cascading Flexible Multi-Agent Collaboration via Next-Agent Prediction
- To trust or not to trust: Attention-based Trust Management for LLM Multi-Agent Systems
- Understanding the Information Propagation Effects of Communication Topologies in LLM-based Multi-Agent Systems
- Multi-Agent Collaboration via Evolving Orchestration
- TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoning
- Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems
- Weak-for-Strong: Training Weak Meta-Agent to Harness Strong Executors
- Why Do Multi-Agent LLM Systems Fail?
- In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents
- Optimizing Model Selection for Compound AI Systems
- G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent Systems
- EvoFlow: Evolving Diverse Agentic Workflows On The Fly
- Multi-agent Architecture Search via Agentic Supernet
- Agent Laboratory: Using LLM Agents as Research Assistants
- Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents
- SMoA: Improving Multi-agent Large Language Models with Sparse Mixture-of-Agents
- NetSafe: Exploring the Topological Safety of Multi-agent Networks
- G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networks
- AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
- MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
- SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning
- AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases
- Scaling Large Language Model-based Multi-Agent Collaboration
- LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
- LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead
- LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision, and the Road Ahead
- Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimization
- InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents
- A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems
- R-Judge: Benchmarking Safety Risk Awareness for LLM Agents
- CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges
- ChatDev: Communicative Agents for Software Development
- Improving Factuality and Reasoning in Language Models through Multiagent Debate
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models
- CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society
- Automatic Chain of Thought Prompting in Large Language Models
- Distributing Accountability, Not Capability: Phase Separation and the LLM Workflow Quadrant in Autonomous AI Agent Architectures
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
- Training Verifiers to Solve Math Word Problems
- Evaluating Large Language Models Trained on Code
- BERT: A Review of Applications in Natural Language Processing and Understanding
- Are NLP Models really able to Solve Simple Math Word Problems?
- Measuring Massive Multitask Language Understanding
- Language Models are Few-Shot Learners
- Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems
- Solving General Arithmetic Word Problems
- AgentOrchestra: Orchestrating Hierarchical Multi-Agent Intelligence with the Tool-Environment-Agent(TEA) Protocol
Cited by
Related