A Loss Function for Generative Neural Networks Based on Watson's Perceptual Model

2020/06/26 by Steffen Czolbe, Oswin Krause, Czolbe, Steffen +5 · 4 citations
Computer Science · #Advanced Image Processing Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Generative Adversarial Networks and Image Synthesis #Image and Signal Denoising Methods #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2006.15057

openalex publication_date 2020/06/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

To train Variational Autoencoders (VAEs) to generate realistic imagery requires a loss function that reflects human perception of image similarity. We propose such a loss function based on Watson's perceptual model, which computes a weighted distance in frequency space and accounts for luminance and contrast masking. We extend the model to color images, increase its robustness to translation by using the Fourier Transform, remove artifacts due to splitting the image into blocks, and make it differentiable. In experiments, VAEs trained with the new loss function generated realistic, high-quality image samples. Compared to using the Euclidean distance and the Structural Similarity Index, the images were less blurry; compared to deep neural network based losses, the new approach required less computational resources and generated images with less artifacts.

Citations

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• Gradient-based learning applied to document recognition
• The JPEG still picture compression standard
• ImageNet classification with deep convolutional neural networks
• E-LPIPS: Robust Perceptual Image Similarity via Random Transformation Ensembles

Cited by

• Pixel Seal: Adversarial-only training for invisible image and video watermarking
• RePack then Refine: Efficient Diffusion Transformer with Vision Foundation Model
• RAVQ-HoloNet: Rate-Adaptive Vector-Quantized Hologram Compression
• An Efficient and Adaptive Watermark Detection System with Tile-based Error Correction

Related

• Generating Images with Perceptual Similarity Metrics based on Deep Networks
• Adversarial Training of Variational Auto-encoders for High Fidelity Image Generation
• Generate High Resolution Images With Generative Variational Autoencoder
• Perceptual Losses for Real-Time Style Transfer and Super-Resolution
• Autoencoding beyond pixels using a learned similarity metric