Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding

2024/01/23 by Mirac Süzgün, Mirac Suzgun, Adam Tauman Kalai +2 · 2 voices · 23 citations
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Human-Computer Interaction (cs.HC) #Topic Modeling #cs.AI #cs.CL #cs.HC

paper · pdf · doi:10.48550/arxiv.2401.12954

openalex publication_date 2024/01/23 · arxiv published 2024/01/23 · arxiv updated 2024/01/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce meta-prompting, an effective scaffolding technique designed to enhance the functionality of language models (LMs). This approach transforms a single LM into a multi-faceted conductor, adept at managing and integrating multiple independent LM queries. By employing high-level instructions, meta-prompting guides the LM to break down complex tasks into smaller, more manageable subtasks. These subtasks are then handled by distinct "expert" instances of the same LM, each operating under specific, tailored instructions. Central to this process is the LM itself, in its role as the conductor, which ensures seamless communication and effective integration of the outputs from these expert models. It additionally employs its inherent critical thinking and robust verification processes to refine and authenticate the end result. This collaborative prompting approach empowers a single LM to simultaneously act as a comprehensive orchestrator and a panel of diverse experts, significantly enhancing its performance across a wide array of tasks. The zero-shot, task-agnostic nature of meta-prompting greatly simplifies user interaction by obviating the need for detailed, task-specific instructions. Furthermore, our research demonstrates the seamless integration of external tools, such as a Python interpreter, into the meta-prompting framework, thereby broadening its applicability and utility. Through rigorous experimentation with GPT-4, we establish the superiority of meta-prompting over conventional scaffolding methods: When averaged across all tasks, including the Game of 24, Checkmate-in-One, and Python Programming Puzzles, meta-prompting, augmented with a Python interpreter functionality, surpasses standard prompting by 17.1%, expert (dynamic) prompting by 17.3%, and multipersona prompting by 15.2%.

Cited by

• UCAgents: Unidirectional Convergence for Visual Evidence Anchored Multi-Agent Medical Decision-Making
• Monadic Context Engineering
• Reflection Pretraining Enables Token-Level Self-Correction in Biological Sequence Models
• The Meta-Prompting Protocol: Orchestrating LLMs via Adversarial Feedback Loops
• RecGPT-V2 Technical Report
• Instruction-Tuning Open-Weight Language Models for BPMN Model Generation
• ReasonBENCH: Benchmarking the (In)Stability of LLM Reasoning
• SAFENLIDB: A Privacy-Preserving Safety Alignment Framework for LLM-based Natural Language Database Interfaces
• Maestro: Learning to Collaborate via Conditional Listwise Policy Optimization for Multi-Agent LLMs
• The Era of Agentic Organization: Learning to Organize with Language Models
• DiLLS: Interactive Diagnosis of LLM-based Multi-agent Systems via Layered Summary of Agent Behaviors
• Verification-Aware Planning for Multi-Agent Systems
• CodeEvolve: An open source evolutionary coding agent for algorithm discovery and optimization
• Can LLMs Write Mathematics Papers? A Case Study in Reservoir Computing
• AutoLabs: Cognitive Multi-Agent Systems with Self-Correction for Autonomous Chemical Experimentation
• Fuzzy Reasoning Chain (FRC): An Innovative Reasoning Framework from Fuzziness to Clarity
• Federation of Agents: A Semantics-Aware Communication Fabric for Large-Scale Agentic AI
• AgentInit: Initializing LLM-based Multi-Agent Systems via Diversity and Expertise Orchestration for Effective and Efficient Collaboration
• Crash Report Enhancement with Large Language Models: An Empirical Study
• Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning
• Encouraging Good Processes Without the Need for Good Answers: Reinforcement Learning for LLM Agent Planning
• Adversarial Attacks against Neural Ranking Models via In-Context Learning
• Tuning LLM-based Code Optimization via Meta-Prompting: An Industrial Perspective

Discussions

• Meta Prompting by OpenAI and Mirac Suzgun Stanford [hn, 3 points, 0 comments]
• Been studying the meta-prompting research and applying it to optimize my Claude artifacts. Fascinating how it transforms LLMs into collaborative teams through expert-driven subtasks. Cut my project kn [bsky, 1 points, 1 comments]

Related

• Meta Prompting for AI Systems
• On Meta-Prompting
• Prompting as Probing: Using Language Models for Knowledge Base Construction
• Meta Reasoning for Large Language Models
• Meta-Task Prompting Elicits Embeddings from Large Language Models