An Unsupervised Autoregressive Model for Speech Representation Learning

2019/04/05 by Yu-An Chung, Chung, Yu-An, Wei-Ning Hsu +5 · 8 citations
Computer Science · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.1904.03240

openalex publication_date 2019/04/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper proposes a novel unsupervised autoregressive neural model for learning generic speech representations. In contrast to other speech representation learning methods that aim to remove noise or speaker variabilities, ours is designed to preserve information for a wide range of downstream tasks. In addition, the proposed model does not require any phonetic or word boundary labels, allowing the model to benefit from large quantities of unlabeled data. Speech representations learned by our model significantly improve performance on both phone classification and speaker verification over the surface features and other supervised and unsupervised approaches. Further analysis shows that different levels of speech information are captured by our model at different layers. In particular, the lower layers tend to be more discriminative for speakers, while the upper layers provide more phonetic content.

Citations

• The information bottleneck method
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Discriminative Acoustic Word Embeddings: Recurrent Neural Network-Based Approaches
• A Theoretical Analysis of Contrastive Unsupervised Representation Learning

Cited by

• PASE: Leveraging the Phonological Prior of WavLM for Low-Hallucination Generative Speech Enhancement
• Adaptive vector steering: A training-free, layer-wise intervention for hallucination mitigation in large audio and multimodal models
• Learning Robust Spatial Representations from Binaural Audio through Feature Distillation
• Multilingual Dataset Integration Strategies for Robust Audio Deepfake Detection: A SAFE Challenge System
• EmoSLLM: Parameter-Efficient Adaptation of LLMs for Speech Emotion Recognition
• Representing Speech Through Autoregressive Prediction of Cochlear Tokens
• Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Models
• Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends

Related

• A Brief Overview of Unsupervised Neural Speech Representation Learning
• Generative Pre-Training for Speech with Autoregressive Predictive Coding
• Speech representation learning: Learning bidirectional encoders with single-view, multi-view, and multi-task methods
• Unsupervised neural and Bayesian models for zero-resource speech processing
• Efficiency-oriented approaches for self-supervised speech representation learning