2016/09/19 by Priyank Jaini, Pascal Poupart, Jaini, Priyank +1 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Target Tracking and Data Fusion in Sensor Networks
paper · pdf · doi:10.48550/arxiv.1609.05881
openalex publication_date 2016/09/19 · openalex created_date 2016/09/30 · openalex updated_date 2026/07/28
The Gaussian mixture model is a classic technique for clustering and data modeling that is used in numerous applications. With the rise of big data, there is a need for parameter estimation techniques that can handle streaming data and distribute the computation over several processors. While online variants of the Expectation Maximization (EM) algorithm exist, their data efficiency is reduced by a stochastic approximation of the E-step and it is not clear how to distribute the computation over multiple processors. We propose a Bayesian learning technique that lends itself naturally to online and distributed computation. Since the Bayesian posterior is not tractable, we project it onto a family of tractable distributions after each observation by matching a set of sufficient moments. This Bayesian moment matching technique compares favorably to online EM in terms of time and accuracy on a set of data modeling benchmarks.