Tailored Conversations beyond LLMs: A RL-Based Dialogue Manager

2025/06/24 by Lucie Galland, Galland, Lucie, Catherine Pélachaud +3
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Semantic Web and Ontologies #Speech and dialogue systems

paper · pdf · doi:10.48550/arxiv.2506.19652

openalex publication_date 2025/06/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this work, we propose a novel framework that integrates large language models (LLMs) with an RL-based dialogue manager for open-ended dialogue with a specific goal. By leveraging hierarchical reinforcement learning to model the structured phases of dialogue and employ meta-learning to enhance adaptability across diverse user profiles, our approach enhances adaptability and efficiency, enabling the system to learn from limited data, transition fluidly between dialogue phases, and personalize responses to heterogeneous patient needs. We apply our framework to Motivational Interviews, aiming to foster behavior change, and demonstrate that the proposed dialogue manager outperforms a state-of-the-art LLM baseline in terms of reward, showing a potential benefit of conditioning LLMs to create open-ended dialogue systems with specific goals.

Citations

• Planning Like Human: A Dual-process Framework for Dialogue Planning
• Supporting Student Decisions on Learning Recommendations: An LLM-Based Chatbot with Knowledge Graph Contextualization for Conversational Explainability and Mentoring
• Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization
• Prompt-Based Monte-Carlo Tree Search for Goal-Oriented Dialogue Policy Planning
• Learning Goal-oriented Dialogue Policy with Opposite Agent Awareness
• Multi-Agent Task-Oriented Dialog Policy Learning with Role-Aware Reward Decomposition
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Deep Reinforcement Learning for Dialogue Generation

Related

• A Mixture-of-Expert Approach to RL-based Dialogue Management
• Interactive Dialogue Agents via Reinforcement Learning on Hindsight Regenerations
• Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversations
• Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk
• Enhancing Personalized Multi-Turn Dialogue with Curiosity Reward