2016/05/24 by Alp Kucukelbir, David M. Blei, Kucukelbir, Alp +1
Computer Science · Mathematics · #Artificial Intelligence (cs.AI) #Artificial intelligence #Bayesian Methods and Mixture Models #Bayesian probability #Computation (stat.CO) #Computer science #Data mining #Demography #Dispersion (optics) #Econometrics #Economics #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (stat.ML) #Machine learning #Mathematics #Metric (unit) #Population #Posterior probability #Probabilistic logic #Statistical Methods and Bayesian Inference #Voting #cs.AI #stat.CO #stat.ML
paper · pdf · doi:10.48550/arxiv.1605.07604
published in arXiv (Cornell University) (Cornell University)
arxiv created 2016/05/24 · openalex publication_date 2016/05/24 · arxiv updated 2016/05/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06
Probabilistic modeling is cyclical: we specify a model, infer its posterior, and evaluate its performance. Evaluation drives the cycle, as we revise our model based on how it performs. This requires a metric. Traditionally, predictive accuracy prevails. Yet, predictive accuracy does not tell the whole story. We propose to evaluate a model through posterior dispersion. The idea is to analyze how each datapoint fares in relation to posterior uncertainty around the hidden structure. We propose a family of posterior dispersion indices (PDI) that capture this idea. A PDI identifies rich patterns of model mismatch in three real data examples: voting preferences, supermarket shopping, and population genetics.