Learned transform compression with optimized entropy encoding

2021/04/07 by Magda Gregorová, Gregorová, Magda, Marc Desaules +3
Computer Science · #Advanced Image and Video Retrieval Techniques #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2104.03305

openalex publication_date 2021/04/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We consider the problem of learned transform compression where we learn both, the transform as well as the probability distribution over the discrete codes. We utilize a soft relaxation of the quantization operation to allow for back-propagation of gradients and employ vector (rather than scalar) quantization of the latent codes. Furthermore, we apply similar relaxation in the code probability assignments enabling direct optimization of the code entropy. To the best of our knowledge, this approach is completely novel. We conduct a set of proof-of concept experiments confirming the potency of our approaches.

Citations

• Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
• Neural Discrete Representation Learning
• Hierarchical Quantized Autoencoders
• Learning Multiple Layers of Features from Tiny Images

Related

• Learned Compression of Encoding Distributions
• Optimizing Learned Image Compression on Scalar and Entropy-Constraint Quantization
• Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
• Latent-Shift: Gradient of Entropy Helps Neural Codecs
• Entropy-Constrained Training of Deep Neural Networks