Generating Images with Perceptual Similarity Metrics based on Deep Networks

2016/02/08 by Alexey Dosovitskiy, Thomas Brox, Dosovitskiy, Alexey +1 · 12 citations
Computer Science · #Advanced Image Processing Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Image Retrieval and Classification Techniques #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.1602.02644

openalex publication_date 2016/02/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Image-generating machine learning models are typically trained with loss functions based on distance in the image space. This often leads to over-smoothed results. We propose a class of loss functions, which we call deep perceptual similarity metrics (DeePSiM), that mitigate this problem. Instead of computing distances in the image space, we compute distances between image features extracted by deep neural networks. This metric better reflects perceptually similarity of images and thus leads to better results. We show three applications: autoencoder training, a modification of a variational autoencoder, and inversion of deep convolutional networks. In all cases, the generated images look sharp and resemble natural images.

Cited by

• Residual Prior Diffusion: A Probabilistic Framework Integrating Coarse Latent Priors with Diffusion Models
• Dynamical modeling of nonlinear latent factors in multiscale neural activity with real-time inference
• A Probabilistic Framework for Temporal Distribution Generalization in Industry-Scale Recommender Systems
• Early Lung Cancer Diagnosis from Virtual Follow-up LDCT Generation via Correlational Autoencoder and Latent Flow Matching
• Align & Invert: Solving Inverse Problems with Diffusion and Flow-based Models via Representation Alignment
• Generative AI in Depth: A Survey of Recent Advances, Model Variants, and Real-World Applications
• Video Prediction of Dynamic Physical Simulations With Pixel-Space Spatiotemporal Transformers
• Seeing Through the Blur: Unlocking Defocus Maps for Deepfake Detection
• Overcoming Output Dimension Collapse: When Sparsity Enables Zero-shot Brain-to-Image Reconstruction at Small Data Scales
• Equivariant Flow Matching for Symmetry-Breaking Bifurcation Problems
• From Pixel to Mask: A Survey of Out-of-Distribution Segmentation
• Generative Artificial Intelligence in Medical Imaging: Foundations, Progress, and Clinical Translation

Related

• Learning to Generate Images with Perceptual Similarity Metrics
• Identifying and Mitigating Flaws of Deep Perceptual Similarity Metrics
• The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
• A Loss Function for Generative Neural Networks Based on Watson's Perceptual Model
• DR-KFS: A Differentiable Visual Similarity Metric for 3D Shape Reconstruction