2021/07/13 by Gunjan Aggarwal, Aggarwal, Gunjan, Devi Parikh +1 · 1 citation
Computer Science · Engineering · #Human Motion and Animation #Music Technology and Sound Studies #Music and Audio Processing #cs.MM #cs.SD #eess.AS
paper · pdf · doi:10.48550/arxiv.2107.06252
arxiv created 2021/07/20 · arxiv updated 2021/07/21
Dance and music typically go hand in hand. The complexities in dance, music, and their synchronisation make them fascinating to study from a computational creativity perspective. While several works have looked at generating dance for a given music, automatically generating music for a given dance remains under-explored. This capability could have several creative expression and entertainment applications. We present some early explorations in this direction. We present a search-based offline approach that generates music after processing the entire dance video and an online approach that uses a deep neural network to generate music on-the-fly as the video proceeds. We compare these approaches to a strong heuristic baseline via human studies and present our findings. We have integrated our online approach in a live demo! A video of the demo can be found here: https://sites.google.com/view/dance2music/live-demo.