Towards Fine-grained Visual Representations by Combining Contrastive Learning with Image Reconstruction and Attention-weighted Pooling

2021/04/09 by Dippel, Jonas, Vogler, Steffen, Höhne, Johannes
#Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2104.04323

Abstract

This paper presents Contrastive Reconstruction, ConRec - a self-supervised learning algorithm that obtains image representations by jointly optimizing a contrastive and a self-reconstruction loss. We showcase that state-of-the-art contrastive learning methods (e.g. SimCLR) have shortcomings to capture fine-grained visual features in their representations. ConRec extends the SimCLR framework by adding (1) a self-reconstruction task and (2) an attention mechanism within the contrastive learning task. This is accomplished by applying a simple encoder-decoder architecture with two heads. We show that both extensions contribute towards an improved vector representation for images with fine-grained visual features. Combining those concepts, ConRec outperforms SimCLR and SimCLR with Attention-Pooling on fine-grained classification datasets.

Related

• A Simple Framework for Contrastive Learning of Visual Representations
• Decoupled Contrastive Learning
• Understand and Improve Contrastive Learning Methods for Visual Representation: A Review
• A simple, efficient and scalable contrastive masked autoencoder for learning visual representations
• G-SimCLR : Self-Supervised Contrastive Learning with Guided Projection\n via Pseudo Labelling