Adversarial Advantage Actor-Critic Model for Task-Completion Dialogue Policy Learning

2017/10/31 by Baolin Peng, Peng, Baolin, Xiujun Li +11 · 1 citation
Computer Science · #Multimodal Machine Learning Applications #Reinforcement Learning in Robotics #Topic Modeling #cs.AI #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.1710.11277

5 pages, 3 figures, ICASSP 2018

arxiv created 2018/02/08 · arxiv updated 2018/02/09

Abstract

This paper presents a new method --- adversarial advantage actor-critic (Adversarial A2C), which significantly improves the efficiency of dialogue policy learning in task-completion dialogue systems. Inspired by generative adversarial networks (GAN), we train a discriminator to differentiate responses/actions generated by dialogue agents from responses/actions by experts. Then, we incorporate the discriminator as another critic into the advantage actor-critic (A2C) framework, to encourage the dialogue agent to explore state-action within the regions where the agent takes actions similar to those of the experts. Experimental results in a movie-ticket booking domain show that the proposed Adversarial A2C can accelerate policy exploration efficiently.

Citations

• Generative Adversarial Imitation Learning
• Iterative Policy Learning in End-to-End Trainable Task-Oriented Neural Dialog Models

Cited by

• Discriminative Deep Dyna-Q: Robust Planning for Dialogue Policy Learning

Related

• Sample Efficient Deep Reinforcement Learning for Dialogue Systems with Large Action Spaces
• Why Guided Dialog Policy Learning performs well? Understanding the role of adversarial learning and its alternative
• Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Management
• Integrating Pretrained Language Model for Dialogue Policy Learning
• Adversarial learning of neural user simulators for dialogue policy optimisation