Sound Event Detection Transformer: An Event-based End-to-End Model for Sound Event Detection

2021/10/05 by Zhirong Ye, Xiangdong Wang, Ye, Zhirong +11 · 3 citations
Computer Science · #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Music Technology and Sound Studies #Music and Audio Processing #Sound (cs.SD) #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2110.02011

openalex publication_date 2021/10/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Sound event detection (SED) has gained increasing attention with its wide application in surveillance, video indexing, etc. Existing models in SED mainly generate frame-level prediction, converting it into a sequence multi-label classification problem. A critical issue with the frame-based model is that it pursues the best frame-level prediction rather than the best event-level prediction. Besides, it needs post-processing and cannot be trained in an end-to-end way. This paper firstly presents the one-dimensional Detection Transformer (1D-DETR), inspired by Detection Transformer for image object detection. Furthermore, given the characteristics of SED, the audio query branch and a one-to-many matching strategy for fine-tuning the model are added to 1D-DETR to form Sound Event Detection Transformer (SEDT). To our knowledge, SEDT is the first event-based and end-to-end SED model. Experiments are conducted on the URBAN-SED dataset and the DCASE2019 Task4 dataset, and both show that SEDT can achieve competitive performance.

Citations

• SONYC: A System for the Monitoring, Analysis and Mitigation of Urban\n Noise Pollution
• Conformer: Convolution-augmented Transformer for Speech Recognition
• Deformable DETR: Deformable Transformers for End-to-End Object Detection

Cited by

• Mirage Fools the Ear, Mute Hides the Truth: Precise Targeted Adversarial Attacks on Polyphonic Sound Event Detection Systems
• SpeechR: A Benchmark for Speech Reasoning in Large Audio-Language Models
• Detect Any Sound: Open-Vocabulary Sound Event Detection with Multi-Modal Queries

Related

• SP-SEDT: Self-supervised Pre-training for Sound Event Detection Transformer
• AST-SED: An Effective Sound Event Detection Method Based on Audio Spectrogram Transformer
• A Hybrid System of Sound Event Detection Transformer and Frame-wise Model for DCASE 2022 Task 4
• DiffSED: Sound Event Detection with Denoising Diffusion
• FMSG-JLESS Submission for DCASE 2024 Task4 on Sound Event Detection with Heterogeneous Training Dataset and Potentially Missing Labels