Transformer-based Model for ASR N-Best Rescoring and Rewriting

2024/06/12 by Iwen E. Kang, Kang, Iwen E., Christophe Van Gysel +3 · 2 citations
Computer Science · Engineering · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Fault Detection and Control Systems #Machine Learning (cs.LG) #Natural Language Processing Techniques #Sound (cs.SD) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2406.08207

openalex publication_date 2024/06/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Voice assistants increasingly use on-device Automatic Speech Recognition (ASR) to ensure speed and privacy. However, due to resource constraints on the device, queries pertaining to complex information domains often require further processing by a search engine. For such applications, we propose a novel Transformer based model capable of rescoring and rewriting, by exploring full context of the N-best hypotheses in parallel. We also propose a new discriminative sequence training objective that can work well for both rescore and rewrite tasks. We show that our Rescore+Rewrite model outperforms the Rescore-only baseline, and achieves up to an average 8.6% relative Word Error Rate (WER) reduction over the ASR system by itself.

Cited by

• Audio-Conditioned Diffusion LLMs for ASR and Deliberation Processing
• Phonetically-Augmented Discriminative Rescoring for Voice Search Error Correction

Related

• Correction of Automatic Speech Recognition with Transformer\n Sequence-to-sequence Model
• Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition
• An Empirical Study of Efficient ASR Rescoring with Transformers
• Transformer-based ASR Incorporating Time-reduction Layer and Fine-tuning\n with Self-Knowledge Distillation
• Audio-attention discriminative language model for ASR rescoring