2019/12/03 by Yin-Jyun Luo, Luo, Yin-Jyun, C.C. Hsu +6 · 4 citations
Computer Science · Engineering · Mathematics · #Acoustics #Artificial intelligence #Audio and Speech Processing (eess.AS) #Autoencoder #Computer science #Deep learning #Encoder #FOS: Computer and information sciences #FOS: Electrical engineering #Identity (music) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Music and Audio Processing #Pattern recognition (psychology) #Singing #Sound (cs.SD) #Spectrogram #Speech Recognition and Synthesis #Speech and Audio Processing #Speech recognition #Visualization #cs.LG #cs.SD #eess.AS #electronic engineering #information engineering #stat.ML
paper · pdf · doi:10.48550/arxiv.1912.02613
published in arXiv (Cornell University) (Cornell University) · Accepted to ICASSP 2020
openalex publication_date 2019/12/03 · openalex created_date 2019/12/13 · arxiv created 2020/02/25 · arxiv updated 2020/02/26 · openalex updated_date 2026/08/08
We propose a flexible framework that deals with both singer conversion and singers vocal technique conversion. The proposed model is trained on non-parallel corpora, accommodates many-to-many conversion, and leverages recent advances of variational autoencoders. It employs separate encoders to learn disentangled latent representations of singer identity and vocal technique separately, with a joint decoder for reconstruction. Conversion is carried out by simple vector arithmetic in the learned latent spaces. Both a quantitative analysis as well as a visualization of the converted spectrograms show that our model is able to disentangle singer identity and vocal technique and successfully perform conversion of these attributes. To the best of our knowledge, this is the first work to jointly tackle conversion of singer identity and vocal technique based on a deep learning approach.