Debate as Optimization: Adaptive Conformal Prediction and Diverse Retrieval for Event Extraction

2024/06/18 by Sijia Wang, Lifu Huang, Wang, Sijia +1 · 5 citations
Physics and Astronomy · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Opinion Dynamics and Social Influence

paper · pdf · doi:10.48550/arxiv.2406.12197

openalex publication_date 2024/06/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose a multi-agent debate as optimization (DAO) system for event extraction, where the primary objective is to iteratively refine the large language models (LLMs) outputs through debating without parameter tuning. In DAO, we introduce two novel modules: the Diverse-RAG (DRAG) module and the Adaptive Conformal Prediction (AdaCP) module. DRAG systematically retrieves supporting information that best fits the debate discussion, while AdaCP enhances the accuracy and reliability of event extraction by effectively rejecting less promising answers. Experimental results demonstrate a significant reduction in the performance gap between supervised approaches and tuning-free LLM-based methods by 18.1% and 17.8% on ACE05 and 17.9% and 15.2% on CASIE for event detection and argument extraction respectively.

Cited by

• Event Extraction in Large Language Model
• A Multi-Agent LLM Framework for Multi-Domain Low-Resource In-Context NER via Knowledge Retrieval, Disambiguation and Reflective Analysis
• Extracting Events Like Code: A Multi-Agent Programming Framework for Zero-Shot Event Extraction
• Multi-Agent Debate Strategies: Survey, Taxonomy, and Challenges
• Reflective Agreement: Combining Self-Mixture of Agents with a Sequence Tagger for Robust Event Extraction

Related

• Free-MAD: Consensus-Free Multi-Agent Debate
• Debate Only When Necessary: Adaptive Multiagent Collaboration for Efficient LLM Reasoning
• Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs
• iMAD: Intelligent Multi-Agent Debate for Efficient and Accurate LLM Inference
• InspireDebate: Multi-Dimensional Subjective-Objective Evaluation-Guided Reasoning and Optimization for Debating