2017/03/31 by Dan Zhang, Xiaohang Song, Zhang, Dan +7 · 16 citations
Computer Science · Mathematics · #Algorithm #Artificial intelligence #Bayesian Modeling and Causal Inference #Belief propagation #Blind Source Separation Techniques #Computer science #Connection (principal bundle) #Constraint (computer-aided design) #Decoding methods #Distributed computing #Energy (signal processing) #Error Correcting Code Techniques #FOS: Computer and information sciences #Graphical model #Inference #Information Theory (cs.IT) #Mathematics #Message passing #Minification #Perspective (graphical) #SIGNAL (programming language) #Scalability #Simple (philosophy) #Statistical inference #Theoretical computer science #cs.IT #math.IT
paper · pdf · doi:10.48550/arxiv.1703.10932
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2017/03/31 · openalex created_date 2017/04/14 · arxiv created 2021/01/19 · arxiv updated 2021/01/20 · openalex updated_date 2026/08/05
Variational message passing (VMP), belief propagation (BP) and expectation propagation (EP) have found their wide applications in complex statistical signal processing problems. In addition to viewing them as a class of algorithms operating on graphical models, this paper unifies them under an optimization framework, namely, Bethe free energy minimization with differently and appropriately imposed constraints. This new perspective in terms of constraint manipulation can offer additional insights on the connection between different message passing algorithms and is valid for a generic statistical model. It also founds a theoretical framework to systematically derive message passing variants. Taking the sparse signal recovery (SSR) problem as an example, a low-complexity EP variant can be obtained by simple constraint reformulation, delivering better estimation performance with lower complexity than the standard EP algorithm. Furthermore, we can resort to the framework for the systematic derivation of hybrid message passing for complex inference tasks. Notably, a hybrid message passing algorithm is exemplarily derived for joint SSR and statistical model learning with near-optimal inference performance and scalable complexity.