Streaming Non-Autoregressive Model for Accent Conversion and Pronunciation Improvement

2025/06/19 by Nguyen, Tuan-Nam, Pham, Ngoc-Quan, Akti, Seymanur +1
Computer Science · Psychology · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Natural Language Processing Techniques #Phonetics and Phonology Research #Sound (cs.SD) #Speech Recognition and Synthesis #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2506.16580

openalex publication_date 2025/06/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose a first streaming accent conversion (AC) model that transforms non-native speech into a native-like accent while preserving speaker identity, prosody and improving pronunciation. Our approach enables stream processing by modifying a previous AC architecture with an Emformer encoder and an optimized inference mechanism. Additionally, we integrate a native text-to-speech (TTS) model to generate ideal ground-truth data for efficient training. Our streaming AC model achieves comparable performance to the top AC models while maintaining stable latency, making it the first AC system capable of streaming.

Citations

• Flex Attention: A Programming Model for Generating Optimized Attention Kernels
• Improving Pronunciation and Accent Conversion through Knowledge Distillation And Synthetic Ground-Truth from Native TTS
• Accent conversion using discrete units with parallel data synthesized from controllable accented TTS
• StreamVC: Real-Time Low-Latency Voice Conversion
• Zero-Shot Accent Conversion using Pseudo Siamese Disentanglement Network
• Voice-preserving Zero-shot Multiple Accent Conversion
• FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversion
• Code-Switching without Switching: Language Agnostic End-to-End Speech Translation
• Comparing Conventional Pitch Detection Algorithms with a Neural Network Approach
• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
• VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversion
• HuBERT: Self-Supervised Speech Representation Learning by Masked\n Prediction of Hidden Units
• Conditional Variational Autoencoder with Adversarial Learning for\n End-to-End Text-to-Speech
• Speech Activity Detection Based on Multilingual Speech Recognition\n System
• HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
• Super-Human Performance in Online Low-latency Recognition of\n Conversational Speech
• wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
• Relative Positional Encoding for Speech Recognition and Direct\n Translation
• Low-Latency Neural Speech Translation
• Attention Is All You Need
• WaveNet: A Generative Model for Raw Audio

Related

• Accent conversion using discrete units with parallel data synthesized from controllable accented TTS
• Transfer the linguistic representations from TTS to accent conversion with non-parallel data
• Non-autoregressive real-time Accent Conversion model with voice cloning
• Improving Pronunciation and Accent Conversion through Knowledge Distillation And Synthetic Ground-Truth from Native TTS
• Improving Accent Conversion with Reference Encoder and End-To-End Text-To-Speech