GANStrument: Adversarial Instrument Sound Synthesis with Pitch-invariant Instance Conditioning

2022/11/10 by Gaku Narita, Narita, Gaku, Junichi Shimizu +3 · 1 citation
Computer Science · Physics and Astronomy · #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Music Technology and Sound Studies #Music and Audio Processing #Sound (cs.SD) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2211.05385

openalex publication_date 2022/11/10 · openalex created_date 2022/11/16 · openalex updated_date 2026/07/28

Abstract

We propose GANStrument, a generative adversarial model for instrument sound synthesis. Given a one-shot sound as input, it is able to generate pitched instrument sounds that reflect the timbre of the input within an interactive time. By exploiting instance conditioning, GANStrument achieves better fidelity and diversity of synthesized sounds and generalization ability to various inputs. In addition, we introduce an adversarial training scheme for a pitch-invariant feature extractor that significantly improves the pitch accuracy and timbre consistency. Experimental results show that GANStrument outperforms strong baselines that do not use instance conditioning in terms of generation quality and input editability. Qualitative examples are available online.

Cited by

• Pitch-Conditioned Instrument Sound Synthesis From an Interactive Timbre Latent Space

Related

• HyperGANStrument: Instrument Sound Synthesis and Editing with Pitch-Invariant Hypernetworks
• Pitch-Conditioned Instrument Sound Synthesis From an Interactive Timbre Latent Space
• DrumGAN: Synthesis of Drum Sounds With Timbral Feature Conditioning Using Generative Adversarial Networks
• Assisted Sound Sample Generation with Musical Conditioning in Adversarial Auto-Encoders
• Generating Sample-Based Musical Instruments Using Neural Audio Codec Language Models