BEST-STD2.0: Balanced and Efficient Speech Tokenizer for Spoken Term Detection

2025/12/18 by Singh, Anup, Demuynck, Kris, Arora, Vipul
#Audio and Speech Processing (eess.AS) #FOS: Electrical engineering #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2512.16395

Abstract

Fast and accurate spoken content retrieval is vital for applications such as voice search. Query-by-Example Spoken Term Detection (STD) involves retrieving matching segments from an audio database given a spoken query. Token-based STD systems, which use discrete speech representations, enable efficient search but struggle with robustness to noise and reverberation, and with inefficient token utilization. We address these challenges by proposing a noise and reverberation-augmented training strategy to improve tokenizer robustness. In addition, we introduce optimal transport-based regularization to ensure balanced token usage and enhance token efficiency. To further speed up retrieval, we adopt a TF-IDF-based search mechanism. Empirical evaluations demonstrate that the proposed method outperforms STD baselines across various distortion levels while maintaining high search efficiency.

Citations

• BEST-STD: Bidirectional Mamba-Enhanced Speech Tokenization for Spoken Term Detection
• SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models
• Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networks
• HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
• SpeechBrain: A General-Purpose Speech Toolkit
• Segmental Dtw: A Parallelizable Alternative to Dynamic Time Warping
• Cross-lingual and Multilingual Spoken Term Detection for Low-Resource Indian Languages
• wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
• Neural Discrete Representation Learning
• Multi-view Recurrent Neural Acoustic Word Embeddings
• Audio Word2Vec: Unsupervised Learning of Audio Segment Representations using Sequence-to-sequence Autoencoder
• MUSAN: A Music, Speech, and Noise Corpus
• Deep convolutional acoustic word embeddings using word-pair side\n information

Related

• BEST-STD: Bidirectional Mamba-Enhanced Speech Tokenization for Spoken Term Detection
• H-QuEST: Accelerating Query-by-Example Spoken Term Detection with Hierarchical Indexing
• Cross-Lingual Query-by-Example Spoken Term Detection: A Transformer-Based Approach
• SpeechPrune: Context-aware Token Pruning for Speech Information Retrieval
• Query-by-example Spoken Term Detection using Attention-based Multi-hop Networks