Towards Better & Faster Autoregressive Image Generation: From the Perspective of Entropy

2025/10/10 by Ma Xiaoxiao, Ma, Xiaoxiao, Feng Zhao +14 · 3 citations
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Digital Media Forensic Detection #FOS: Computer and information sciences #Face recognition and analysis #Generative Adversarial Networks and Image Synthesis

paper · pdf · doi:10.48550/arxiv.2510.09012

openalex publication_date 2025/10/10 · openalex created_date 2025/10/14 · openalex updated_date 2026/07/28

Abstract

In this work, we first revisit the sampling issues in current autoregressive (AR) image generation models and identify that image tokens, unlike text tokens, exhibit lower information density and non-uniform spatial distribution. Accordingly, we present an entropy-informed decoding strategy that facilitates higher autoregressive generation quality with faster synthesis speed. Specifically, the proposed method introduces two main innovations: 1) dynamic temperature control guided by spatial entropy of token distributions, enhancing the balance between content diversity, alignment accuracy, and structural coherence in both mask-based and scale-wise models, without extra computational overhead, and 2) entropy-aware acceptance rules in speculative decoding, achieving near-lossless generation at about 85% of the inference cost of conventional acceleration methods. Extensive experiments across multiple benchmarks using diverse AR image generation models demonstrate the effectiveness and generalizability of our approach in enhancing both generation quality and sampling speed.

Citations

• VideoMAR: Autoregressive Video Generatio with Continuous Tokens
• UniToken: Harmonizing Multimodal Understanding and Generation through Unified Visual Encoding
• Training-Free Text-Guided Image Editing with Visual Autoregressive Model
• Perceive, Understand and Restore: Real-World Image Super-Resolution with Autoregressive Multimodal Generative Models
• Neighboring Autoregressive Modeling for Efficient Visual Generation
• V2Flow: Unifying Visual Tokenization and Large Language Model Vocabularies for Autoregressive Image Generation
• Frequency Autoregressive Image Generation with Continuous Tokens
• UniTok: A Unified Tokenizer for Visual Generation and Understanding
• Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling
• rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking
• Parallelized Autoregressive Visual Generation
• Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis
• TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation
• Randomized Autoregressive Visual Generation
• HART: Efficient Visual Generation with Hybrid Autoregressive Transformer
• Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis
• LANTERN: Accelerating Visual Autoregressive Models with Relaxed Speculative Decoding
• Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decoding
• Emu3: Next-Token Prediction is All You Need
• Show-o: One Single Transformer to Unify Multimodal Understanding and Generation
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
• Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining
• ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generation
• An Image is Worth 32 Tokens for Reconstruction and Generation
• Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation
• SED: Self-Evaluation Decoding Enhances Large Language Models for Better Generation
• Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction
• Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
• OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation
• Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation
• PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis
• DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• Planting a SEED of Vision in Large Language Model
• SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
• Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis
• GPT-4 Technical Report
• LLaMA: Open and Efficient Foundation Language Models
• Accelerating Large Language Model Decoding with Speculative Sampling
• Scalable Diffusion Models with Transformers
• Fast Inference from Transformers via Speculative Decoding
• Scaling Autoregressive Models for Content-Rich Text-to-Image Generation
• Autoregressive Image Generation using Residual Quantization
• A Contrastive Framework for Neural Text Generation
• MaskGIT: Masked Generative Image Transformer
• High-Resolution Image Synthesis with Latent Diffusion Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• Vector-quantized Image Modeling with Improved VQGAN
• CogView: Mastering Text-to-Image Generation via Transformers
• Zero-Shot Text-to-Image Generation
• Taming Transformers for High-Resolution Image Synthesis
• If beam search is the answer, what was the question?
• CTRL: A Conditional Transformer Language Model for Controllable Generation
• The Curious Case of Neural Text Degeneration
• Conditional Image Generation with PixelCNN Decoders
• Microsoft COCO: Common Objects in Context

Cited by

• FR-TTS: Test-Time Scaling for NTP-based Image Generation with Effective Filling-based Reward Signal
• Diversity Has Always Been There in Your Visual Autoregressive Models
• Semantic Context Matters: Improving Conditioning for Autoregressive Models

Related

• Fast-ARDiff: An Entropy-informed Acceleration Framework for Continuous Space Autoregressive Generation
• DPAR: Dynamic Patchification for Efficient Autoregressive Visual Generation
• Hawk: Leveraging Spatial Context for Faster Autoregressive Text-to-Image Generation
• Generation then Reconstruction: Accelerating Masked Autoregressive Models via Two-Stage Sampling
• Distilled Decoding 1: One-step Sampling of Image Auto-regressive Models with Flow Matching