End to End Dialogue Transformer

2020/08/24 by Ondrej Mekota, Měkota, Ondřej, Memduh Gökırmak +3
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech and dialogue systems #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2008.10392

openalex publication_date 2020/08/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Dialogue systems attempt to facilitate conversations between humans and computers, for purposes as diverse as small talk to booking a vacation. We are here inspired by the performance of the recurrent neural network-based model Sequicity, which when conducting a dialogue uses a sequence-to-sequence architecture to first produce a textual representation of what is going on in the dialogue, and in a further step use this along with database findings to produce a reply to the user. We here propose a dialogue system based on the Transformer architecture instead of Sequicity's RNN-based architecture, that works similarly in an end-to-end, sequence-to-sequence fashion.

Citations

• Incorporating Copying Mechanism in Sequence-to-Sequence Learning
• A Network-based End-to-End Trainable Task-oriented Dialogue System
• MASS: Masked Sequence to Sequence Pre-training for Language Generation
• Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models
• Language Models are Few-Shot Learners

Related

• Dialogue Transformers
• Talking with Machines: A Comprehensive Survey of Emergent Dialogue Systems
• Generative Deep Neural Networks for Dialogue: A Short Review
• End-to-end Conversation Modeling Track in DSTC6
• A Neural Conversational Model