Completely Unsupervised Phoneme Recognition by Adversarially Learning Mapping Relationships from Audio Embeddings

2018/04/01 by Da-Rong Liu, Liu, Da-Rong, Kuan‐Yu Chen +5 · 1 citation
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Music and Audio Processing #Speech Recognition and Synthesis #Speech and Audio Processing

paper · pdf · doi:10.48550/arxiv.1804.00316

openalex publication_date 2018/04/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Unsupervised discovery of acoustic tokens from audio corpora without annotation and learning vector representations for these tokens have been widely studied. Although these techniques have been shown successful in some applications such as query-by-example Spoken Term Detection (STD), the lack of mapping relationships between these discovered tokens and real phonemes have limited the down-stream applications. This paper represents probably the first attempt towards the goal of completely unsupervised phoneme recognition, or mapping audio signals to phoneme sequences without phoneme-labeled audio data. The basic idea is to cluster the embedded acoustic tokens and learn the mapping between the cluster sequences and the unknown phoneme sequences with a Generative Adversarial Network (GAN). An unsupervised phoneme recognition accuracy of 36% was achieved in the preliminary experiments.

Citations

• Categorical Reparameterization with Gumbel-Softmax
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

Cited by

• Towards Unsupervised Speech Recognition at the Syllable-Level

Related

• Learning Phone Recognition from Unpaired Audio and Phone Sequences Based on Generative Adversarial Network
• Unsupervised Spoken Term Discovery on Untranscribed Speech
• Unsupervised Iterative Deep Learning of Speech Features and Acoustic Tokens with Applications to Spoken Term Detection
• Learning Word Embeddings from Speech
• Audio Word2Vec: Unsupervised Learning of Audio Segment Representations using Sequence-to-sequence Autoencoder