Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning

2022/07/25 by Deborah A. Cohen, Cohen, Deborah, Moonkyung Ryu +19
Computer Science · #AI in Service Interactions #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Speech and dialogue systems #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2208.02294

openalex publication_date 2022/07/25 · openalex created_date 2022/08/06 · openalex updated_date 2026/07/28

Abstract

Despite recent advances in natural language understanding and generation, and decades of research on the development of conversational bots, building automated agents that can carry on rich open-ended conversations with humans "in the wild" remains a formidable challenge. In this work we develop a real-time, open-ended dialogue system that uses reinforcement learning (RL) to power a bot's conversational skill at scale. Our work pairs the succinct embedding of the conversation state generated using SOTA (supervised) language models with RL techniques that are particularly suited to a dynamic action space that changes as the conversation progresses. Trained using crowd-sourced data, our novel system is able to substantially exceeds the (strong) baseline supervised model with respect to several metrics of interest in a live experiment with real users of the Google Assistant.

Related

• Offline Reinforcement Learning for Mixture-of-Expert Dialogue Management
• Learning to Generate Prompts for Dialogue Generation through Reinforcement Learning
• The RLLChatbot: a solution to the ConvAI challenge
• A Review of Dialogue Systems: From Trained Monkeys to Stochastic Parrots
• CHAI: A CHatbot AI for Task-Oriented Dialogue with Offline Reinforcement Learning