Data Augmentation for Voice-Assistant NLU using BERT-based Interchangeable Rephrase

2021/04/16 by Akhila Yerukola, Yerukola, Akhila, Mason Bretan +3
Computer Science · #Artificial intelligence #Boosting (machine learning) #Computation and Language (cs.CL) #Computer science #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine translation #Natural Language Processing Techniques #Natural language processing #Naturalness #Speech Recognition and Synthesis #Speech recognition #Topic Modeling #Transformer #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.2104.08268

published in arXiv (Cornell University) (Cornell University) · Accepted at EACL'21

arxiv created 2021/04/16 · openalex publication_date 2021/04/16 · arxiv updated 2021/04/19 · openalex created_date 2021/04/26 · openalex updated_date 2026/07/28

Abstract

We introduce a data augmentation technique based on byte pair encoding and a BERT-like self-attention model to boost performance on spoken language understanding tasks. We compare and evaluate this method with a range of augmentation techniques encompassing generative models such as VAEs and performance-boosting techniques such as synonym replacement and back-translation. We show our method performs strongly on domain and intent classification tasks for a voice assistant and in a user-study focused on utterance naturalness and semantic similarity.

Citations

• Skip-Thought Vectors
• THE DISTRIBUTION OF THE FLORA IN THE ALPINE ZONE.1
• LLM-Supported Manufacturing Mapping Generation

Related

• Retraining DistilBERT for a Voice Shopping Assistant by Using Universal Dependencies
• Multimodal Audio-textual Architecture for Robust Spoken Language Understanding
• Tokenwise Contrastive Pretraining for Finer Speech-to-BERT Alignment in End-to-End Speech-to-Intent Systems
• Spoken Language Understanding for Conversational AI: Recent Advances and Future Direction
• Explainable and Accurate Natural Language Understanding for Voice Assistants and Beyond