2015/03/31 by Xirong Li, Tiberio Uricchio, Lamberto Ballan +3 · 2 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Artificial intelligence #Bridge (graph theory) #Computer science #Construct (python library) #Context (archaeology) #Data science #Exploit #Image (mathematics) #Image Retrieval and Classification Techniques #Image retrieval #Information retrieval #Key (lock) #Multimodal Machine Learning Applications #Relevance (law) #Semantic gap #Taxonomy (biology) #cs.CV #cs.IR #cs.MM #cs.SI
paper · pdf · doi:10.1145/2906152
published as ACM Computing Surveys, Volume 49 Issue 1, 14:1-14:39, June 2016 · to appear in ACM Computing Surveys
arxiv created 2016/03/23 · openalex publication_date 2016/06/06 · arxiv updated 2016/06/10 · openalex created_date 2022/10/03 · openalex updated_date 2026/08/05
Where previous reviews on content-based image retrieval emphasize what can be seen in an image to bridge the semantic gap, this survey considers what people tag about an image. A comprehensive treatise of three closely linked problems (i.e., image tag assignment, refinement, and tag-based image retrieval) is presented. While existing works vary in terms of their targeted tasks and methodology, they rely on the key functionality of tag relevance, that is, estimating the relevance of a specific tag with respect to the visual content of a given image and its social context. By analyzing what information a specific method exploits to construct its tag relevance function and how such information is exploited, this article introduces a two-dimensional taxonomy to structure the growing literature, understand the ingredients of the main works, clarify their connections and difference, and recognize their merits and limitations. For a head-to-head comparison with the state of the art, a new experimental protocol is presented, with training sets containing 10,000, 100,000, and 1 million images, and an evaluation on three test sets, contributed by various research groups. Eleven representative works are implemented and evaluated. Putting all this together, the survey aims to provide an overview of the past and foster progress for the near future.