2020/06/11 by Miguel Lázaro-Gredilla, Wolfgang Lehrach, Lázaro-Gredilla, Miguel +9
Computer Science · #Bayesian Modeling and Causal Inference #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and Data Classification
paper · pdf · doi:10.48550/arxiv.2006.06803
openalex publication_date 2020/06/11 · openalex created_date 2022/09/08 · openalex updated_date 2026/07/28
Probabilistic graphical models (PGMs) provide a compact representation of\nknowledge that can be queried in a flexible way: after learning the parameters\nof a graphical model once, new probabilistic queries can be answered at test\ntime without retraining. However, when using undirected PGMS with hidden\nvariables, two sources of error typically compound in all but the simplest\nmodels (a) learning error (both computing the partition function and\nintegrating out the hidden variables is intractable); and (b) prediction error\n(exact inference is also intractable). Here we introduce query training (QT), a\nmechanism to learn a PGM that is optimized for the approximate inference\nalgorithm that will be paired with it. The resulting PGM is a worse model of\nthe data (as measured by the likelihood), but it is tuned to produce better\nmarginals for a given inference algorithm. Unlike prior works, our approach\npreserves the querying flexibility of the original PGM: at test time, we can\nestimate the marginal of any variable given any partial evidence. We\ndemonstrate experimentally that QT can be used to learn a challenging\n8-connected grid Markov random field with hidden variables and that it\nconsistently outperforms the state-of-the-art AdVIL when tested on three\nundirected models across multiple datasets.\n