Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Feature

2015/05/05 by Babak Saleh, Ahmed Elgammal, Saleh, Babak +1 · 25 citations
Computer Science · Neuroscience · Physics and Astronomy · #Aesthetic Perception and Analysis #Color Science and Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Image Retrieval and Classification Techniques #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Multimedia (cs.MM)

paper · pdf · doi:10.48550/arxiv.1505.00855

openalex publication_date 2015/05/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In the past few years, the number of fine-art collections that are digitized and publicly available has been growing rapidly. With the availability of such large collections of digitized artworks comes the need to develop multimedia systems to archive and retrieve this pool of data. Measuring the visual similarity between artistic items is an essential step for such multimedia systems, which can benefit more high-level multimedia tasks. In order to model this similarity between paintings, we should extract the appropriate visual features for paintings and find out the best approach to learn the similarity metric based on these features. We investigate a comprehensive list of visual features and metric learning approaches to learn an optimized similarity measure between paintings. We develop a machine that is able to make aesthetic-related semantic-level judgments, such as predicting a painting's style, genre, and artist, as well as providing similarity measures optimized based on the knowledge available in the domain of art historical interpretation. Our experiments show the value of using this similarity measure for the aforementioned prediction tasks.

Citations

• Gradient-based learning applied to document recognition

Cited by

• Training-free Conditional Image Embedding Framework Leveraging Large Vision Language Models
• Generative Latent Coding for Ultra-Low Bitrate Image Compression
• EmoStyle: Emotion-Driven Image Stylization
• Artographer: a Curatorial Interface for Art Space Exploration
• Insert In Style: A Zero-Shot Generative Framework for Harmonious Cross-Domain Object Composition
• Cultural Alien Sampler: Open-ended art generation balancing originality and coherence
• MATRIX: Multimodal Agent Tuning for Robust Tool-Use Reasoning
• Learning What Matters: Steering Diffusion via Spectrally Anisotropic Forward Noise
• Dirichlet-Prior Shaping: Guiding Expert Specialization in Upcycled MoEs
• Erased, But Not Forgotten: Erased Rectified Flow Transformers Still Remain Unsafe Under Concept Attack
• Revoking Amnesia: RL-based Trajectory Optimization to Resurrect Erased Concepts in Diffusion Models
• Few-Shot Pattern Detection via Template Matching and Regression
• Decoupled Proxy Alignment: Mitigating Language Prior Conflict for Multimodal Alignment in MLLM
• SpotTune: Transfer Learning through Adaptive Fine-tuning
• SuMa: A Subspace Mapping Approach for Robust and Effective Concept Erasure in Text-to-Image Diffusion Models
• OmniArt: Multi-task Deep Learning for Artistic Data Analysis
• Leveraging Diffusion Models for Stylization using Multiple Style Images
• MultiRef: Controllable Image Generation with Multiple Visual References
• SCFlow: Implicitly Learning Style and Content Disentanglement with Flow Models
• Adaptive Sparse Softmax: An Effective and Efficient Softmax Variant
• Artificial Intelligence and Misinformation in Art: Can Vision Language Models Judge the Hand or the Machine Behind the Canvas?
• The Cow of Rembrandt - Analyzing Artistic Prompt Interpretation in Text-to-Image Models
• ZIUM: Zero-Shot Intent-Aware Adversarial Attack on Unlearned Models
• ArtSeek: Deep artwork understanding via multimodal in-context reasoning and late interaction retrieval
• AIComposer: Any Style and Content Image Composition via Feature Integration

Related

• Toward Automated Discovery of Artistic Influence
• Insights From A Large-Scale Database of Material Depictions In Paintings
• Have Large Vision-Language Models Mastered Art History?
• Machine: The New Art Connoisseur
• The Shape of Art History in the Eyes of the Machine