Learning FRAME Models Using CNN Filters

2015/09/28 by Yang Lu, Lu, Yang, Song‐Chun Zhu +3 · 4 citations
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Image Processing and 3D Reconstruction #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.1509.08379

openalex publication_date 2015/09/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The convolutional neural network (ConvNet or CNN) has proven to be very successful in many tasks such as those in computer vision. In this conceptual paper, we study the generative perspective of the discriminative CNN. In particular, we propose to learn the generative FRAME (Filters, Random field, And Maximum Entropy) model using the highly expressive filters pre-learned by the CNN at the convolutional layers. We show that the learning algorithm can generate realistic and rich object and texture patterns in natural scenes. We explain that each learned model corresponds to a new CNN unit at a layer above the layer of filters employed by the model. We further show that it is possible to learn a new layer of CNN units using a generative CNN model, which is a product of experts model, and the learning algorithm admits an EM interpretation with binary latent variables.

Citations

• DRAW: A Recurrent Neural Network For Image Generation
• Auto-Encoding Variational Bayes
• Gradient-based learning applied to document recognition
• Training Products of Experts by Minimizing Contrastive Divergence
• ImageNet classification with deep convolutional neural networks
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

Cited by

• Learning Latent Space Energy-Based Prior Model
• Implicit Generation and Generalization in Energy-Based Models
• Patch redundancy in images: a statistical testing framework and some\n applications
• Macrocanonical Models for Texture Synthesis

Related

• Generative Modeling of Convolutional Neural Networks
• A Bayesian Perspective of Convolutional Neural Networks through a Deconvolutional Generative Model
• A Theory of Generative ConvNet
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Curriculum By Smoothing