Multi-Task Learning for Situated Multi-Domain End-to-End Dialogue Systems

2021/10/11 by Po-Nien Kung, Chung‐Cheng Chang, Kung, Po-Nien +9
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech and dialogue systems #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2110.05221

openalex publication_date 2021/10/11 · openalex created_date 2021/10/25 · openalex updated_date 2026/07/28

Abstract

Task-oriented dialogue systems have been a promising area in the NLP field. Previous work showed the effectiveness of using a single GPT-2 based model to predict belief states and responses via causal language modeling. In this paper, we leverage multi-task learning techniques to train a GPT-2 based model on a more challenging dataset with multiple domains, multiple modalities, and more diversity in output formats. Using only a single model, our method achieves better performance on all sub-tasks, across domains, compared to task and domain-specific models. Furthermore, we evaluated several proposed strategies for GPT-2 based dialogue systems with comprehensive ablation studies, showing that all techniques can further improve the performance.

Citations

• Multi-Task Deep Neural Networks for Natural Language Understanding
• A Simple Language Model for Task-Oriented Dialogue

Related

• A Simple Language Model for Task-Oriented Dialogue
• Hello, It's GPT-2 -- How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems
• DialogAct2Vec: Towards End-to-End Dialogue Agent by Multi-Task Representation Learning
• A Study on Prompt-based Few-Shot Learning Methods for Belief State Tracking in Task-oriented Dialog Systems
• Improving Multi-Domain Task-Oriented Dialogue System with Offline Reinforcement Learning