Learning FRAME Models Using CNN Filters

2015/09/28 by Yang Lu, Song‐Chun Zhu, Song-Chun Zhu +5 · 26 citations
Computer Science · Mathematics · #Artificial intelligence #Binary number #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Convolutional neural network #Discriminative model #FOS: Computer and information sciences #Frame (networking) #Generative Adversarial Networks and Image Synthesis #Generative grammar #Generative model #Image Processing and 3D Reconstruction #Interpretation (philosophy) #Layer (electronics) #Machine learning #Mathematics #Neural Networks and Applications #Pattern recognition (psychology) #cs.CV

paper · pdf · doi:10.48550/arxiv.1509.08379

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2015/09/28 · arxiv created 2015/12/07 · arxiv updated 2015/12/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/08

Abstract

The convolutional neural network (ConvNet or CNN) has proven to be very successful in many tasks such as those in computer vision. In this conceptual paper, we study the generative perspective of the discriminative CNN. In particular, we propose to learn the generative FRAME (Filters, Random field, And Maximum Entropy) model using the highly expressive filters pre-learned by the CNN at the convolutional layers. We show that the learning algorithm can generate realistic and rich object and texture patterns in natural scenes. We explain that each learned model corresponds to a new CNN unit at a layer above the layer of filters employed by the model. We further show that it is possible to learn a new layer of CNN units using a generative CNN model, which is a product of experts model, and the learning algorithm admits an EM interpretation with binary latent variables.

Citations

• DRAW: A Recurrent Neural Network For Image Generation
• Auto-Encoding Variational Bayes
• Gradient-based learning applied to document recognition
• Training Products of Experts by Minimizing Contrastive Divergence
• ImageNet classification with deep convolutional neural networks
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

Cited by

• Learning Latent Space Energy-Based Prior Model
• Implicit Generation and Generalization in Energy-Based Models
• Patch redundancy in images: a statistical testing framework and some applications
• Macrocanonical Models for Texture Synthesis
• Divergence Triangle for Joint Training of Generator Model, Energy-based Model, and Inference Model
• On quantitative Laplace-type convergence results for some exponential probability measures, with two applications

Related

• Generative Modeling of Convolutional Neural Networks
• A Bayesian Perspective of Convolutional Neural Networks through a Deconvolutional Generative Model
• A Theory of Generative ConvNet
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Curriculum By Smoothing