2024/10/07 by Ivan Rinaldi, Rinaldi, Ivan, Nicola Fanelli +5 · 1 citation
Computer Science · Engineering · #Audio and Speech Processing (eess.AS) #Computer Graphics and Visualization Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Human Motion and Animation #Multimedia (cs.MM) #Music Technology and Sound Studies #Sound (cs.SD) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2410.04906
openalex publication_date 2024/10/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Artificial Intelligence and generative models have revolutionized music creation, with many models leveraging textual or visual prompts for guidance. However, existing image-to-music models are limited to simple images, lacking the capability to generate music from complex digitized artworks. To address this gap, we introduce Art2Mus, a novel model designed to create music from digitized artworks or text inputs. Art2Mus extends the AudioLDM~2 architecture, a text-to-audio model, and employs our newly curated datasets, created via ImageBind, which pair digitized artworks with music. Experimental results demonstrate that Art2Mus can generate music that resonates with the input stimuli. These findings suggest promising applications in multimedia art, interactive installations, and AI-driven creative tools.