2020/12/15 by Matthew Joseph Adiletta, Adiletta, Matthew Joseph, Oliver Thomas +1
Agricultural and Biological Sciences · Psychology · #Color perception and design #FOS: Computer and information sciences #Multimedia (cs.MM) #Multisensory perception and integration #Sensory Analysis and Statistical Methods
paper · pdf · doi:10.48550/arxiv.2012.08034
openalex publication_date 2020/12/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
This project brings music to sight. Music can be a visual masterpiece. Some people naturally experience a visualization of audio - a condition called synesthesia. The type of synesthesia explored is when sounds create colors in the 'mind's eye.' Project included interviews with people who experience synesthesia, examination of prior art, and topic research to inform project design. Audio input, digital signal processing (including Fast Fourier Transforms (FFTs)) and data manipulation produce arguments required for our visualization. Arguments are then applied to a physics particle simulator which is re-purposed to model a synesthetic experience. The result of the project is a simulator in MAX 8, which generates a visual performance using particles by varying each particle's position, velocity, and color based on parameters extracted via digital processing of input audio.