Towards Human-in-the-Loop Onset Detection: A Transfer Learning Approach for Maracatu

2025/07/07 by Pinto, António Sá
#Artificial Intelligence (cs.AI) #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Sound (cs.SD) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2507.04858

Abstract

We explore transfer learning strategies for musical onset detection in the Afro-Brazilian Maracatu tradition, which features complex rhythmic patterns that challenge conventional models. We adapt two Temporal Convolutional Network architectures: one pre-trained for onset detection (intra-task) and another for beat tracking (inter-task). Using only 5-second annotated snippets per instrument, we fine-tune these models through layer-wise retraining strategies for five traditional percussion instruments. Our results demonstrate significant improvements over baseline performance, with F1 scores reaching up to 0.998 in the intra-task setting and improvements of over 50 percentage points in best-case scenarios. The cross-task adaptation proves particularly effective for time-keeping instruments, where onsets naturally align with beat positions. The optimal fine-tuning configuration varies by instrument, highlighting the importance of instrument-specific adaptation strategies. This approach addresses the challenges of underrepresented musical traditions, offering an efficient human-in-the-loop methodology that minimizes annotation effort while maximizing performance. Our findings contribute to more inclusive music information retrieval tools applicable beyond Western musical contexts.

Citations

• Few-Shot Drum Transcription in Polyphonic Music
• Score-informed syllable segmentation for a cappella singing voice with convolutional neural networks
• Transfer learning for music classification and regression tasks
• madmom: a new Python Audio and Music Signal Processing Library
• Music Information Retrieval: Recent Developments and Applications

Related

• Adapting Meter Tracking Models to Latin American Music
• Revisiting Meter Tracking in Carnatic Music using Deep Learning Approaches
• Towards an efficient deep learning model for musical onset detection
• MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuning
• Tatum-Level Drum Transcription Based on a Convolutional Recurrent Neural Network with Language Model-Based Regularized Training