2018/12/26 by Lior Bracha, Bracha, Lior, Gal Chechik +1
Computer Science · #Advanced Image and Video Retrieval Techniques #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Image Retrieval and Classification Techniques #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #cs.AI #cs.CV #cs.LG
paper · pdf · doi:10.48550/arxiv.1812.10358
arxiv created 2018/12/26 · openalex publication_date 2018/12/26 · arxiv updated 2018/12/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Capturing the interesting components of an image is a key aspect of image understanding. When a speaker annotates an image, selecting labels that are informative greatly depends on the prior knowledge of a prospective listener. Motivated by cognitive theories of categorization and communication, we present a new unsupervised approach to model this prior knowledge and quantify the informativeness of a description. Specifically, we compute how knowledge of a label reduces uncertainty over the space of labels and utilize this to rank candidate labels for describing an image. While the full estimation problem is intractable, we describe an efficient algorithm to approximate entropy reduction using a tree-structured graphical model. We evaluate our approach on the open-images dataset using a new evaluation set of 10K ground-truth ratings and find that it achieves ~65% agreement with human raters, largely outperforming other unsupervised baseline approaches.