A State-Update Prompting Strategy for Efficient and Robust Multi-turn Dialogue

2025/09/22 by Ziyi Liu, Liu, Ziyi
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multi-Agent Systems and Negotiation #Service-Oriented Architecture and Web Services #Speech and dialogue systems

paper · pdf · doi:10.48550/arxiv.2509.17766

openalex publication_date 2025/09/22 · openalex created_date 2025/10/16 · openalex updated_date 2026/07/28

Abstract

Large Language Models (LLMs) struggle with information forgetting and inefficiency in long-horizon, multi-turn dialogues. To address this, we propose a training-free prompt engineering method, the State-Update Multi-turn Dialogue Strategy. It utilizes "State Reconstruction" and "History Remind" mechanisms to effectively manage dialogue history. Our strategy shows strong performance across multiple multi-hop QA datasets. For instance, on the HotpotQA dataset, it improves the core information filtering score by 32.6%, leading to a 14.1% increase in the downstream QA score, while also reducing inference time by 73.1% and token consumption by 59.4%. Ablation studies confirm the pivotal roles of both components. Our work offers an effective solution for optimizing LLMs in long-range interactions, providing new insights for developing more robust Agents.

Citations

• LongVideoAgent: Multi-Agent Reasoning with Long Videos
• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
• LLMs Get Lost In Multi-Turn Conversation
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
• Qwen2.5 Technical Report
• Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
• Inference Scaling for Long-Context Retrieval Augmented Generation
• Retrieval-Augmented Generation for Large Language Models: A Survey
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• Lost in the Middle: How Language Models Use Long Contexts
• Active Retrieval Augmented Generation
• Toolformer: Language Models Can Teach Themselves to Use Tools
• Dense Text Retrieval based on Pretrained Language Models: A Survey
• Dense Text Retrieval Based on Pretrained Language Models: A Survey
• Distributing Accountability, Not Capability: Phase Separation and the LLM Workflow Quadrant in Autonomous AI Agent Architectures
• Training language models to follow instructions with human feedback
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps
• Language Models are Few-Shot Learners
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• QASC: A Dataset for Question Answering via Sentence Composition
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
• A Neural Network Approach to Context-Sensitive Generation of Conversational Responses

Related

• LoopServe: An Adaptive Dual-phase LLM Inference Acceleration System for Multi-Turn Dialogues
• Test-Time Policy Adaptation for Enhanced Multi-Turn Interactions with LLMs
• Intermittent Semi-working Mask: A New Masking Paradigm for LLMs
• D-SMART: Enhancing LLM Dialogue Consistency via Dynamic Structured Memory And Reasoning Tree
• Beyond Single-Turn: A Survey on Multi-Turn Interactions with Large Language Models