2023/12/07 by Xiaoyu Lin, Lin, Xiaoyu, Laurent Girin +3
Biochemistry, Genetics and Molecular Biology · Computer Science · #Cancer-related molecular mechanisms research #FOS: Computer and information sciences #Machine Learning (cs.LG) #Speech Recognition and Synthesis #Speech and Audio Processing
paper · pdf · doi:10.48550/arxiv.2312.04167
openalex created_date 2023/06/20 · openalex publication_date 2023/12/07 · openalex updated_date 2026/07/30
In this paper, we propose a latent-variable generative model called mixture of dynamical variational autoencoders (MixDVAE) to model the dynamics of a system composed of multiple moving sources. A DVAE model is pre-trained on a single-source dataset to capture the source dynamics. Then, multiple instances of the pre-trained DVAE model are integrated into a multi-source mixture model with a discrete observation-to-source assignment latent variable. The posterior distributions of both the discrete observation-to-source assignment variable and the continuous DVAE variables representing the sources content/position are estimated using a variational expectation-maximization algorithm, leading to multi-source trajectories estimation. We illustrate the versatility of the proposed MixDVAE model on two tasks: a computer vision task, namely multi-object tracking, and an audio processing task, namely single-channel audio source separation. Experimental results show that the proposed method works well on these two tasks, and outperforms several baseline methods.