Query-by-Example Keyword Spotting system using Multi-head Attention and Softtriple Loss

2021/02/14 by Jinmiao Huang, Waseem Gharbieh, Huang, Jinmiao +5 · 5 citations
Computer Science · #Advanced Text Analysis Techniques #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Text and Document Classification Technologies #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2102.07061

openalex publication_date 2021/02/14 · openalex created_date 2021/03/01 · openalex updated_date 2026/07/28

Abstract

This paper proposes a neural network architecture for tackling the query-by-example user-defined keyword spotting task. A multi-head attention module is added on top of a multi-layered GRU for effective feature extraction, and a normalized multi-head attention module is proposed for feature aggregation. We also adopt the softtriple loss - a combination of triplet loss and softmax loss - and showcase its effectiveness. We demonstrate the performance of our model on internal datasets with different languages and the public Hey-Snips dataset. We compare the performance of our model to a baseline system and conduct an ablation study to show the benefit of each component in our architecture. The proposed work shows solid performance while preserving simplicity.

Citations

• DONUT: CTC-based Query-by-Example Keyword Spotting

Cited by

• Synaspot: A Lightweight, Streaming Multi-modal Framework for Keyword Spotting with Audio-Text Synergy
• Keyword Spotting with Hyper-Matched Filters for Small Footprint Devices
• H-QuEST: Accelerating Query-by-Example Spoken Term Detection with Hierarchical Indexing
• MFA-KWS: Effective Keyword Spotting with Multi-head Frame-asynchronous Decoding
• Adversarial Deep Metric Learning for Cross-Modal Audio-Text Alignment in Open-Vocabulary Keyword Spotting

Related

• Attention-based End-to-End Models for Small-Footprint Keyword Spotting
• Multilingual Query-by-Example Keyword Spotting with Metric Learning and Phoneme-to-Embedding Mapping
• Orthogonality Constrained Multi-Head Attention For Keyword Spotting
• QbyE-MLPMixer: Query-by-Example Open-Vocabulary Keyword Spotting using MLPMixer
• Separable Temporal Convolution plus Temporally Pooled Attention for Lightweight High-performance Keyword Spotting