2017/09/19 by Hao-Wen Dong, Wen-Yi Hsiao, Dong, Hao-Wen +6 · 151 citations
Computer Science · Engineering · Mathematics · #Acoustics #Art #Artificial Intelligence (cs.AI) #Artificial intelligence #Artificial neural network #Audio and Speech Processing (eess.AS) #Bass (fish) #Code (set theory) #Computer science #FOS: Computer and information sciences #FOS: Electrical engineering #Generative Adversarial Networks and Image Synthesis #Generative grammar #Generative model #MIDI #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Melody #Music Technology and Sound Studies #Music and Audio Processing #Musical #Piano #Polyphony #Programming language #Sound (cs.SD) #Speech recognition #Track (disk drive) #Visual arts #cs.AI #cs.LG #cs.SD #eess.AS #electronic engineering #information engineering #stat.ML
paper · pdf · doi:10.48550/arxiv.1709.06298
published in arXiv (Cornell University) (Cornell University) · to appear at AAAI 2018
openalex publication_date 2017/09/19 · arxiv created 2017/11/24 · openalex created_date 2019/07/30 · arxiv updated 2020/08/06 · openalex updated_date 2026/08/08
Generating music has a few notable differences from generating images and videos. First, music is an art of time, necessitating a temporal model. Second, music is usually composed of multiple instruments/tracks with their own temporal dynamics, but collectively they unfold over time interdependently. Lastly, musical notes are often grouped into chords, arpeggios or melodies in polyphonic music, and thereby introducing a chronological ordering of notes is not naturally suitable. In this paper, we propose three models for symbolic multi-track music generation under the framework of generative adversarial networks (GANs). The three models, which differ in the underlying assumptions and accordingly the network architectures, are referred to as the jamming model, the composer model and the hybrid model. We trained the proposed models on a dataset of over one hundred thousand bars of rock music and applied them to generate piano-rolls of five tracks: bass, drums, guitar, piano and strings. A few intra-track and inter-track objective metrics are also proposed to evaluate the generative results, in addition to a subjective user study. We show that our models can generate coherent music of four bars right from scratch (i.e. without human inputs). We also extend our models to human-AI cooperative music generation: given a specific track composed by human, we can generate four additional tracks to accompany it. All code, the dataset and the rendered audio samples are available at https://salu133445.github.io/musegan/ .