Budgeted Policy Learning for Task-Oriented Dialogue Systems

2019/06/02 by Zhirui Zhang, Xiujun Li, Zhang, Zhirui +5 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE) #cs.AI #cs.CL #cs.LG #cs.NE

paper · pdf · doi:10.48550/arxiv.1906.00499

10 pages, 7 figures, ACL 2019

arxiv created 2019/06/02 · arxiv updated 2019/06/04

Abstract

This paper presents a new approach that extends Deep Dyna-Q (DDQ) by incorporating a Budget-Conscious Scheduling (BCS) to best utilize a fixed, small amount of user interactions (budget) for learning task-oriented dialogue agents. BCS consists of (1) a Poisson-based global scheduler to allocate budget over different stages of training; (2) a controller to decide at each training step whether the agent is trained using real or simulated experiences; (3) a user goal sampling module to generate the experiences that are most effective for policy learning. Experiments on a movie-ticket booking task with simulated and real users show that our approach leads to significant improvements in success rate over the state-of-the-art baselines given the fixed budget.

Cited by

• Simulating Before Planning: Constructing Intrinsic User World Model for User-Tailored Dialogue Policy Planning

Related

• Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning
• Discriminative Deep Dyna-Q: Robust Planning for Dialogue Policy Learning
• Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning
• A User Simulator for Task-Completion Dialogues
• Switch-based Active Deep Dyna-Q: Efficient Adaptive Planning for Task-Completion Dialogue Policy Learning