Listening, Imagining & Refining: A Heuristic Optimized ASR Correction Framework with LLMs

2025/09/18 by Liu, Yutong, Ziyue Zhang, Zhang, Ziyue +10 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Semantic Web and Ontologies #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2509.15095

openalex publication_date 2025/09/18 · openalex created_date 2025/10/16 · openalex updated_date 2026/07/28

Abstract

Automatic Speech Recognition (ASR) systems remain prone to errors that affect downstream applications. In this paper, we propose LIR-ASR, a heuristic optimized iterative correction framework using LLMs, inspired by human auditory perception. LIR-ASR applies a "Listening-Imagining-Refining" strategy, generating phonetic variants and refining them in context. A heuristic optimization with finite state machine (FSM) is introduced to prevent the correction process from being trapped in local optima and rule-based constraints help maintain semantic fidelity. Experiments on both English and Chinese ASR outputs show that LIR-ASR achieves average reductions in CER/WER of up to 1.5 percentage points compared to baselines, demonstrating substantial accuracy gains in transcription.

Citations

• Whale: Large-Scale multilingual ASR model with w2v-BERT and E-Branchformer with large speech data
• Fewer Hallucinations, More Verification: A Three-Stage LLM-Based Framework for ASR Error Correction
• Dolphin: A Large-Scale Automatic Speech Recognition Model for Eastern Languages
• ASR Error Correction using Large Language Models
• The Llama 3 Herd of Models
• Evolutionary Prompt Design for LLM-Based Post-ASR Error Correction
• ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools
• Denoising LM: Pushing the Limits of Error Correction Models for Speech Recognition
• Listen Again and Choose the Right Answer: A New Paradigm for Automatic Speech Recognition with Large Language Models
• GPT-4 Technical Report
• Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages
• Robust Speech Recognition via Large-Scale Weak Supervision
• FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech
• FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition

Cited by

• Tibetan Language and AI: A Comprehensive Survey of Resources, Methods and Challenges

Related

• Fewer Hallucinations, More Verification: A Three-Stage LLM-Based Framework for ASR Error Correction
• ASR-EC Benchmark: Evaluating Large Language Models on Chinese ASR Error Correction
• Effective Text Adaptation for LLM-based ASR through Soft Prompt Fine-Tuning
• Leveraging Large Language Models for Exploiting ASR Uncertainty
• A Survey on Non-Intrusive ASR Refinement: From Output-Level Correction to Full-Model Distillation