Learning to Generate Chairs, Tables and Cars with Convolutional Networks

2014/11/21 by Alexey Dosovitskiy, Dosovitskiy, Alexey, Jost Tobias Springenberg +5 · 1 citation
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Handwritten Text Recognition Techniques #Human Pose and Action Recognition #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.1411.5928

openalex publication_date 2014/11/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

We train generative 'up-convolutional' neural networks which are able to generate images of objects given object style, viewpoint, and color. We train the networks on rendered 3D models of chairs, tables, and cars. Our experiments show that the networks do not merely learn all images by heart, but rather find a meaningful representation of 3D models allowing them to assess the similarity of different models, interpolate between given views to generate the missing ones, extrapolate views, and invent new objects not present in the training set by recombining training instances, or even two different object classes. Moreover, we show that such generative networks can be used to find correspondences between different objects from the dataset, outperforming existing approaches on this task.

Citations

• Deep Convolutional Inverse Graphics Network
• Random Walk Initialization for Training Very Deep Feedforward Networks
• Auto-Encoding Variational Bayes
• Semi-Supervised Learning with Deep Generative Models
• Deep Generative Stochastic Networks Trainable by Backprop
• A Fast Learning Algorithm for Deep Belief Nets
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual\n Recognition
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• ShapeNet: An Information-Rich 3D Model Repository
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

Cited by

• GeneGAN: Learning Object Transfiguration and Attribute Subspace from Unpaired Data

Related

• Multi-view 3D Models from Single Images with a Convolutional Network
• Learning to Generate Chairs with Generative Adversarial Nets
• Warp and Learn: Novel Views Generation for Vehicles and Other Objects
• Visual Object Networks: Image Generation with Disentangled 3D Representation
• Semantic Pose using Deep Networks Trained on Synthetic RGB-D