TMD-TTS: A Unified Tibetan Multi-Dialect Text-to-Speech Framework for Ü-Tsang, Amdo and Kham Speech Dataset Generation

2025/09/22 by Yutong Liu, Ziyue Zhang, Liu, Yutong +17 · 1 citation
Computer Science · Social Sciences · #Artificial Intelligence (cs.AI) #China's Ethnic Minorities and Relations #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech Recognition and Synthesis

paper · pdf · doi:10.48550/arxiv.2509.18060

openalex publication_date 2025/09/22 · openalex created_date 2025/10/16 · openalex updated_date 2026/07/28

Abstract

Tibetan is a low-resource language with limited parallel speech corpora spanning its three major dialects (Ü-Tsang, Amdo, and Kham), limiting progress in speech modeling. To address this issue, we propose TMD-TTS, a unified Tibetan multi-dialect text-to-speech (TTS) framework that synthesizes parallel dialectal speech from explicit dialect labels. Our method features a dialect fusion module and a Dialect-Specialized Dynamic Routing Network (DSDR-Net) to capture fine-grained acoustic and linguistic variations across dialects. Extensive objective and subjective evaluations demonstrate that TMD-TTS significantly outperforms baselines in dialectal expressiveness. We further validate the quality and utility of the synthesized speech through a challenging Speech-to-Speech Dialect Conversion (S2SDC) task.

Citations

• F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching
• CLAP: Isolating Content from Style through Contrastive Learning with Augmented Prompts
• Matcha-TTS: A fast TTS architecture with conditional flow matching
• BigVSAN: Enhancing GAN-based Neural Vocoders with Slicing Adversarial Network
• VITS2: Improving Quality and Efficiency of Single-Stage Text-to-Speech with Adversarial Learning and Architecture Design
• Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis
• NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers
• Scalable Diffusion Models with Transformers
• BigVGAN: A Universal Neural Vocoder with Large-Scale Training
• NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality
• Conditional Variational Autoencoder with Adversarial Learning for\n End-to-End Text-to-Speech
• MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement
• HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
• FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
• Normalizing Flows for Probabilistic Modeling and Inference
• SDR - half-baked or well done?
• Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions
• Adam: A Method for Stochastic Optimization

Cited by

• Tibetan Language and AI: A Comprehensive Survey of Resources, Methods and Challenges

Related

• FMSD-TTS: Few-shot Multi-Speaker Multi-Dialect Text-to-Speech Synthesis for Ü-Tsang, Amdo and Kham Speech Dataset Generation
• DiaMoE-TTS: A Unified IPA-Based Dialect TTS Framework with Mixture-of-Experts and Parameter-Efficient Zero-Shot Adaptation
• Bailing-TTS: Chinese Dialectal Speech Synthesis Towards Human-like Spontaneous Representation
• WenetSpeech-Chuan: A Large-Scale Sichuanese Corpus with Rich Annotation for Dialectal Speech Processing
• MnTTS2: An Open-Source Multi-Speaker Mongolian Text-to-Speech Synthesis Dataset