2015/06/03 by Umut Şimşekli, Şimşekli, Umut, Hazal Koptagel +9 · 2 citations
Computer Science · Mathematics · #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #Machine Learning (stat.ML) #Markov Chains and Monte Carlo Methods #Tensor decomposition and applications
paper · pdf · doi:10.48550/arxiv.1506.01418
openalex publication_date 2015/06/03 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/28
For large matrix factorisation problems, we develop a distributed Markov Chain Monte Carlo (MCMC) method based on stochastic gradient Langevin dynamics (SGLD) that we call Parallel SGLD (PSGLD). PSGLD has very favourable scaling properties with increasing data size and is comparable in terms of computational requirements to optimisation methods based on stochastic gradient descent. PSGLD achieves high performance by exploiting the conditional independence structure of the MF models to sub-sample data in a systematic manner as to allow parallelisation and distributed computation. We provide a convergence proof of the algorithm and verify its superior performance on various architectures such as Graphics Processing Units, shared memory multi-core systems and multi-computer clusters.