Music Boomerang: Reusing Diffusion Models for Data Augmentation and Audio Manipulation

2025/07/07 by Fichtinger, Alexander, Schlüter, Jan, Widmer, Gerhard
#Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Sound (cs.SD) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2507.04864

Abstract

Generative models of music audio are typically used to generate output based solely on a text prompt or melody. Boomerang sampling, recently proposed for the image domain, allows generating output close to an existing example, using any pretrained diffusion model. In this work, we explore its application in the audio domain as a tool for data augmentation or content manipulation. Specifically, implementing Boomerang sampling for Stable Audio Open, we augment training data for a state-of-the-art beat tracker, and attempt to replace musical instruments in recordings. Our results show that the rhythmic structure of existing examples is mostly preserved, that it improves performance of the beat tracker, but only in scenarios of limited training data, and that it can accomplish text-based instrument replacement on monophonic inputs. We publish our implementation to invite experiments on data augmentation in other tasks and explore further applications.

Citations

• Beat This! Spectrograms for Beat and Downbeat Tracking
• Stable Audio Open
• Long-form music generation with latent diffusion
• Fast Timing-Conditioned Latent Audio Diffusion
• Masked Audio Generation using a Single Non-Autoregressive Transformer
• Consistency Models
• AudioLDM: Text-to-Audio Generation with Latent Diffusion Models
• DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models
• High Fidelity Neural Audio Compression
• Boomerang: Local sampling on image manifolds using diffusion models
• Musika! Fast Infinite Waveform Music Generation
• DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps
• Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models\n for Inverse Problems through Stochastic Contraction
• SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations
• Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech
• RoFormer: Enhanced Transformer with Rotary Position Embedding
• Score-Based Generative Modeling through Stochastic Differential Equations
• Denoising Diffusion Probabilistic Models
• Neural Networks Fail to Learn Periodic Functions and How to Fix It
• Jukebox: A Generative Model for Music
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• WaveNet: A Generative Model for Raw Audio
• U-Net: Convolutional Networks for Biomedical Image Segmentation

Related

• MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategies
• Noise2Music: Text-conditioned Music Generation with Diffusion Models
• Controllable Music Production with Diffusion Models and Guidance Gradients
• ERNIE-Music: Text-to-Waveform Music Generation with Diffusion Models
• Bass Accompaniment Generation via Latent Diffusion