2012/02/14 by Benjamin M. Marlin, Benjamin Marlin, Nando de Freitas +2 · 1 citation
Computer Science · Mathematics · #Bayesian Modeling and Causal Inference #Distributed Sensor Networks and Detection Algorithms #Error Correcting Code Techniques #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Statistical Methods and Inference #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.1202.3746
arxiv created 2012/02/14 · openalex publication_date 2012/02/14 · arxiv updated 2012/02/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Standard maximum likelihood estimation cannot be applied to discrete energy-based models in the general case because the computation of exact model probabilities is intractable. Recent research has seen the proposal of several new estimators designed specifically to overcome this intractability, but virtually nothing is known about their theoretical properties. In this paper, we present a generalized estimator that unifies many of the classical and recently proposed estimators. We use results from the standard asymptotic theory for M-estimators to derive a generic expression for the asymptotic covariance matrix of our generalized estimator. We apply these results to study the relative statistical efficiency of classical pseudolikelihood and the recently-proposed ratio matching estimator.