2021/11/09 by David Alfonso-Hermelo, Alfonso-Hermelo, David, Ahmad Rashid +7
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Speech Recognition and Synthesis #Speech and dialogue systems #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2111.05196
openalex publication_date 2021/11/09 · openalex created_date 2022/05/05 · openalex updated_date 2026/07/28
Slot-filling and intent detection are the backbone of conversational agents\nsuch as voice assistants, and are active areas of research. Even though\nstate-of-the-art techniques on publicly available benchmarks show impressive\nperformance, their ability to generalize to realistic scenarios is yet to be\ndemonstrated. In this work, we present NATURE, a set of simple spoken-language\noriented transformations, applied to the evaluation set of datasets, to\nintroduce human spoken language variations while preserving the semantics of an\nutterance. We apply NATURE to common slot-filling and intent detection\nbenchmarks and demonstrate that simple perturbations from the standard\nevaluation set by NATURE can deteriorate model performance significantly.\nThrough our experiments we demonstrate that when NATURE operators are applied\nto evaluation set of popular benchmarks the model accuracy can drop by up to\n40%.\n