2019/05/14 by Yu-Ding Lu, Hsin-Ying Lee, Lu, Yu-Ding +5 · 1 citation
Computer Science · Neuroscience · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Hearing Loss and Rehabilitation #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Music and Audio Processing #Sound (cs.SD) #Speech and Audio Processing
paper · pdf · doi:10.48550/arxiv.1905.05375
openalex publication_date 2019/05/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Spatial audio is an essential medium to audiences for 3D visual and auditory experience. However, the recording devices and techniques are expensive or inaccessible to the general public. In this work, we propose a self-supervised audio spatialization network that can generate spatial audio given the corresponding video and monaural audio. To enhance spatialization performance, we use an auxiliary classifier to classify ground-truth videos and those with audio where the left and right channels are swapped. We collect a large-scale video dataset with spatial audio to validate the proposed method. Experimental results demonstrate the effectiveness of the proposed model on the audio spatialization task.