ARTI-6: Towards Six-dimensional Articulatory Speech Encoding

2025/09/25 by Lee, Jihwan, Foley, Sean, Lertpetchpun, Thanathai +6
#Artificial Intelligence (cs.AI) #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2509.21447

Abstract

We propose ARTI-6, a compact six-dimensional articulatory speech encoding framework derived from real-time MRI data that captures crucial vocal tract regions including the velum, tongue root, and larynx. ARTI-6 consists of three components: (1) a six-dimensional articulatory feature set representing key regions of the vocal tract; (2) an articulatory inversion model, which predicts articulatory features from speech acoustics leveraging speech foundation models, achieving a prediction correlation of 0.87; and (3) an articulatory synthesis model, which reconstructs intelligible speech directly from articulatory features, showing that even a low-dimensional representation can generate natural-sounding speech. Together, ARTI-6 provides an interpretable, computationally efficient, and physiologically grounded framework for advancing articulatory inversion, synthesis, and broader speech technology applications. The source code and speech samples are publicly available.

Citations

• A long-form single-speaker real-time MRI speech dataset and benchmark
• On the Relationship between Accent Strength and Articulatory Features
• Enhancing Acoustic-to-Articulatory Speech Inversion by Incorporating Nasality
• Towards disentangling the contributions of articulation and acoustics in multimodal phoneme recognition
• Articulatory Feature Prediction from Surface EMG during Speech Production
• LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus
• Miipher: A Robust Speech Restoration Model Integrating Self-Supervised Speech and Text Representations
• Speaker-Independent Acoustic-to-Articulatory Speech Inversion
• Robust Speech Recognition via Large-Scale Weak Supervision
• Deep Speech Synthesis from Articulatory Representations
• UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022
• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
• LoRA: Low-Rank Adaptation of Large Language Models
• HuBERT: Self-Supervised Speech Representation Learning by Masked\n Prediction of Hidden Units
• HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
• wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
• Attention Is All You Need
• Least Squares Generative Adversarial Networks
• Articulatory Phonology: An Overview

Related

• Deep Speech Synthesis from MRI-Based Articulatory Representations
• Deep Speech Synthesis from Articulatory Representations
• Reconstruction of the Complete Vocal Tract Contour Through Acoustic to Articulatory Inversion Using Real-Time MRI Data
• Decoding Vocal Articulations from Acoustic Latent Representations
• Acoustic to Articulatory Inversion of Speech; Data Driven Approaches, Challenges, Applications, and Future Scope