2018/09/20 by Gino Brunner, Andres Konrad, Brunner, Gino +5 · 6 citations
Computer Science · Engineering · Mathematics · Neuroscience · #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #H.5.5 #I.2.1 #I.2.4 #I.2.6 #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Music Technology and Sound Studies #Music and Audio Processing #Neuroscience and Music Perception #Sound (cs.SD) #cs.LG #cs.SD #eess.AS #electronic engineering #information engineering #stat.ML
paper · pdf · doi:10.48550/arxiv.1809.07600
Paper accepted at the 19th International Society for Music Information Retrieval Conference, ISMIR 2018, Paris, France
arxiv created 2018/09/20 · openalex publication_date 2018/09/20 · arxiv updated 2018/09/21 · openalex created_date 2019/06/27 · openalex updated_date 2026/07/28
We introduce MIDI-VAE, a neural network model based on Variational Autoencoders that is capable of handling polyphonic music with multiple instrument tracks, as well as modeling the dynamics of music by incorporating note durations and velocities. We show that MIDI-VAE can perform style transfer on symbolic music by automatically changing pitches, dynamics and instruments of a music piece from, e.g., a Classical to a Jazz style. We evaluate the efficacy of the style transfer by training separate style validation classifiers. Our model can also interpolate between short pieces of music, produce medleys and create mixtures of entire songs. The interpolations smoothly change pitches, dynamics and instrumentation to create a harmonic bridge between two music pieces. To the best of our knowledge, this work represents the first successful attempt at applying neural style transfer to complete musical compositions.