2020/09/28 by Shikib Mehri, Mehri, Shikib, Mihail Eric +4 · 4 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech and dialogue systems #Topic Modeling #cs.AI #cs.CL
paper · pdf · doi:10.48550/arxiv.2009.13570
Benchmark hosted on: https://evalai.cloudcv.org/web/challenges/challenge-page/708/
openalex publication_date 2020/09/28 · arxiv created 2020/10/01 · arxiv updated 2020/10/02 · openalex created_date 2020/10/08 · openalex updated_date 2026/07/28
A long-standing goal of task-oriented dialogue research is the ability to flexibly adapt dialogue models to new domains. To progress research in this direction, we introduce DialoGLUE (Dialogue Language Understanding Evaluation), a public benchmark consisting of 7 task-oriented dialogue datasets covering 4 distinct natural language understanding tasks, designed to encourage dialogue research in representation-based transfer, domain adaptation, and sample-efficient task learning. We release several strong baseline models, demonstrating performance improvements over a vanilla BERT architecture and state-of-the-art results on 5 out of 7 tasks, by pre-training on a large open-domain dialogue corpus and task-adaptive self-supervised training. Through the DialoGLUE benchmark, the baseline methods, and our evaluation scripts, we hope to facilitate progress towards the goal of developing more general task-oriented dialogue models.