Repaint: Improving the Generalization of Down-Stream Visual Tasks by Generating Multiple Instances of Training Examples

2021/10/20 by Amin Banitalebi-Dehkordi, Yong Zhang, Banitalebi-Dehkordi, Amin +1 · 4 citations
Computer Science · Mathematics · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Artificial intelligence #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Computer vision #Convolutional neural network #FOS: Computer and information sciences #Generalization #Generative Adversarial Networks and Image Synthesis #Image (mathematics) #Image Enhancement Techniques #Machine Learning (cs.LG) #Mathematics #Object (grammar) #Pattern recognition (psychology) #Process (computing) #Set (abstract data type) #Texture (cosmology) #Training set #cs.AI #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.2110.10366

published in arXiv (Cornell University) (Cornell University) · BMVC 2021

arxiv created 2021/10/20 · openalex publication_date 2021/10/20 · arxiv updated 2021/10/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Convolutional Neural Networks (CNNs) for visual tasks are believed to learn both the low-level textures and high-level object attributes, throughout the network depth. This paper further investigates the `texture bias' in CNNs. To this end, we regenerate multiple instances of training examples from each original image, through a process we call `repainting'. The repainted examples preserve the shape and structure of the regions and objects within the scenes, but diversify their texture and color. Our method can regenerate a same image at different daylight, season, or weather conditions, can have colorization or de-colorization effects, or even bring back some texture information from blacked-out areas. The in-place repaint allows us to further use these repainted examples for improving the generalization of CNNs. Through an extensive set of experiments, we demonstrate the usefulness of the repainted examples in training, for the tasks of image classification (ImageNet) and object detection (COCO), over several state-of-the-art network architectures at different capacities, and across different data availability regimes.

Citations

• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• An overview of gradient descent optimization algorithms
• Improved Regularization of Convolutional Neural Networks with Cutout
• DropBlock: A regularization method for convolutional networks
• GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks
• Implicit Semantic Data Augmentation for Deep Networks
• EfficientDet: Scalable and Efficient Object Detection

Related

• Dynamic Association Learning of Self-Attention and Convolution in Image Restoration
• Distort-and-Recover: Color Enhancement using Deep Reinforcement Learning
• Texture Networks: Feed-forward Synthesis of Textures and Stylized Images
• ImageNet-trained CNNs are not biased towards texture: Revisiting feature reliance through controlled suppression
• Restore from Restored: Single-image Inpainting