Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding

2022/05/02 by Chang, Ya-Hsin, Chen, Yun-Nung
#Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Sound (cs.SD) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2205.00693

Abstract

Spoken language understanding (SLU) is an essential task for machines to understand human speech for better interactions. However, errors from the automatic speech recognizer (ASR) usually hurt the understanding performance. In reality, ASR systems may not be easy to adjust for the target scenarios. Therefore, this paper focuses on learning utterance representations that are robust to ASR errors using a contrastive objective, and further strengthens the generalization ability by combining supervised contrastive learning and self-distillation in model fine-tuning. Experiments on three benchmark datasets demonstrate the effectiveness of our proposed approach.

Related

• ML-LMCL: Mutual Learning and Large-Margin Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding
• Task Oriented Dialogue as a Catalyst for Self-Supervised Automatic Speech Recognition
• Contrastive and Consistency Learning for Neural Noisy-Channel Model in Spoken Language Understanding
• A Reference-less Quality Metric for Automatic Speech Recognition via Contrastive-Learning of a Multi-Language Model with Self-Supervision
• An Approach to Improve Robustness of NLP Systems against ASR Errors