Learning Dialog Policies from Weak Demonstrations

2020/04/23 by Gabriel Gordon-Hall, Gordon-Hall, Gabriel, Philip John Gorinski +3
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE) #cs.CL #cs.LG #cs.NE

paper · pdf · doi:10.48550/arxiv.2004.11054

9 pages + 2 pages references + 1 page appendices, 6 figures, 2 tables, 1 algorithm, accepted as long paper at ACL2020

arxiv created 2020/08/13 · arxiv updated 2020/08/14

Abstract

Deep reinforcement learning is a promising approach to training a dialog manager, but current methods struggle with the large state and action spaces of multi-domain dialog systems. Building upon Deep Q-learning from Demonstrations (DQfD), an algorithm that scores highly in difficult Atari games, we leverage dialog data to guide the agent to successfully respond to a user's requests. We make progressively fewer assumptions about the data needed, using labeled, reduced-labeled, and even unlabeled data to train expert demonstrators. We introduce Reinforced Fine-tune Learning, an extension to DQfD, enabling us to overcome the domain gap between the datasets and the environment. Experiments in a challenging multi-domain dialog system framework validate our approaches, and get high success rates even when trained on out-of-domain data.

Related

• Show Us the Way: Learning to Manage Dialog from Demonstrations
• Reinforcement Learning of Multi-Domain Dialog Policies Via Action Embeddings
• Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning
• Why Guided Dialog Policy Learning performs well? Understanding the role of adversarial learning and its alternative
• A Survey on Dialog Management: Recent Advances and Challenges