Spoken Word2Vec: Learning Skipgram Embeddings from Speech

2023/11/15 by Mohammad Amaan Sayeed, Hanan Aldarmaki, Sayeed, Mohammad Amaan +1
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Music and Audio Processing #Speech Recognition and Synthesis #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2311.09319

openalex publication_date 2023/11/15 · openalex created_date 2023/11/18 · openalex updated_date 2026/07/28

Abstract

Text word embeddings that encode distributional semantics work by modeling contextual similarities of frequently occurring words. Acoustic word embeddings, on the other hand, typically encode low-level phonetic similarities. Semantic embeddings for spoken words have been previously explored using analogous algorithms to Word2Vec, but the resulting vectors still mainly encoded phonetic rather than semantic features. In this paper, we examine the assumptions and architectures used in previous works and show experimentally how shallow skipgram-like algorithms fail to encode distributional semantics when the input units are acoustically correlated. We illustrate the potential of an alternative deep end-to-end variant of the model and examine the effects on the resulting embeddings, showing positive results of semantic relatedness in the embedding space.

Related

• Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speech
• Leveraging multilingual transfer for unsupervised semantic acoustic word embeddings
• Learning Word Embeddings from Speech
• Learned In Speech Recognition: Contextual Acoustic Word Embeddings
• Phonetic-and-Semantic Embedding of Spoken Words with Applications in Spoken Content Retrieval