Exploring the User Experience of AI-Assisted Sound Searching Systems for Creative Workflows

2025/04/22 by Haohe Liu, Liu, Haohe, Thomas Deacon +7 · 1 citation
Computer Science · #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Music Technology and Sound Studies #Sound (cs.SD) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2504.15575

openalex publication_date 2025/04/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Locating the right sound effect efficiently is an important yet challenging topic for audio production. Most current sound-searching systems rely on pre-annotated audio labels created by humans, which can be time-consuming to produce and prone to inaccuracies, limiting the efficiency of audio production. Following the recent advancement of contrastive language-audio pre-training (CLAP) models, we explore an alternative CLAP-based sound-searching system (CLAP-UI) that does not rely on human annotations. To evaluate the effectiveness of CLAP-UI, we conducted comparative experiments with a widely used sound effect searching platform, the BBC Sound Effect Library. Our study evaluates user performance, cognitive load, and satisfaction through ecologically valid tasks based on professional sound-searching workflows. Our result shows that CLAP-UI demonstrated significantly enhanced productivity and reduced frustration while maintaining comparable cognitive demands. We also qualitatively analyzed the participants' feedback, which offered valuable perspectives on the design of future AI-assisted sound search systems.

Citations

• The language of sound search: Examining User Queries in Audio Search Engines
• Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation
• Leveraging Pre-trained BERT for Audio Captioning
• HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection
• Audio Retrieval with Natural Language Queries: A Benchmark Study
• PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition
• BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
• Gradio: Hassle-Free Sharing and Testing of ML Models in the Wild
• A Tutorial on Kernel Density Estimation and Recent Advances
• Rank-Biserial Correlation
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Related

• Sound Search by Text Description or Vocal Imitation?
• The language of sound search: Examining User Queries in Audio Search Engines
• Quality Audio Prototyping: a prototype system for unified sound retrieval and procedural generation
• Audio-Language Models for Audio-Centric Tasks: A Systematic Survey
• FxSearcher: gradient-free text-driven audio transformation