2016/05/25 by Michael I. Jordan, Jason D. Lee, Jordan, Michael I. +3
Computer Science · Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Gaussian Processes and Bayesian Inference #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Methodology (stat.ME) #Optimization and Control (math.OC) #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #cs.IT #cs.LG #math.IT #math.OC #stat.ME #stat.ML
paper · pdf · doi:10.48550/arxiv.1605.07689
openalex publication_date 2016/05/25 · arxiv created 2016/11/06 · arxiv updated 2016/11/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We present a Communication-efficient Surrogate Likelihood (CSL) framework for solving distributed statistical inference problems. CSL provides a communication-efficient surrogate to the global likelihood that can be used for low-dimensional estimation, high-dimensional regularized estimation and Bayesian inference. For low-dimensional estimation, CSL provably improves upon naive averaging schemes and facilitates the construction of confidence intervals. For high-dimensional regularized estimation, CSL leads to a minimax-optimal estimator with controlled communication cost. For Bayesian inference, CSL can be used to form a communication-efficient quasi-posterior distribution that converges to the true posterior. This quasi-posterior procedure significantly improves the computational efficiency of MCMC algorithms even in a non-distributed setting. We present both theoretical analysis and experiments to explore the properties of the CSL approximation.