The Describe-Then-Generate Bottleneck: How VLM Descriptions Alter Image Generation Outcomes

2025/09/18 by Kodathala, Sai Varun, Rakesh Vunnam, Vunnam, Rakesh
Computer Science · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Data Visualization and Analytics #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.2509.18179

openalex publication_date 2025/09/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

With the increasing integration of multimodal AI systems in creative workflows, understanding information loss in vision-language-vision pipelines has become important for evaluating system limitations. However, the degradation that occurs when visual content passes through textual intermediation remains poorly quantified. In this work, we provide empirical analysis of the describe-then-generate bottleneck, where natural language serves as an intermediate representation for visual information. We generated 150 image pairs through the describe-then-generate pipeline and applied existing metrics (LPIPS, SSIM, and color distance) to measure information preservation across perceptual, structural, and chromatic dimensions. Our evaluation reveals that 99.3% of samples exhibit substantial perceptual degradation and 91.5% demonstrate significant structural information loss, providing empirical evidence that the describe-then-generate bottleneck represents a measurable and consistent limitation in contemporary multimodal systems.

Citations

• Lost in Embeddings: Information Loss in Vision-Language Models
• Principled Multimodal Representation Learning
• Aligning Multimodal Representations through an Information Bottleneck
• VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoning
• Learning Optimal Multimodal Information Bottleneck Representations
• Multimodal Conditional Information Bottleneck for Generalizable AI-Generated Image Detection
• Uncovering Cultural Representation Disparities in Vision-Language Models
• Identifying and Mitigating Position Bias of Multi-image Vision-Language Models
• DCBM: Data-Efficient Visual Concept Bottleneck Models
• Cross-modal Information Flow in Multimodal Large Language Models
• Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance
• Free-VSC: Free Semantics from Visual Foundation Models for Unsupervised Video Semantic Compression
• See It from My Perspective: How Language Affects Cultural Bias in Image Understanding
• DeCo: Decoupling Token Compression from Semantic Abstraction in Multimodal Large Language Models
• The Right Losses for the Right Gains: Improving the Semantic Consistency of Deep Text-to-Image Generation with Distribution-Sensitive Losses
• Extending Context Window of Large Language Models via Semantic Compression
• Dynamic Multimodal Information Bottleneck for Multimodality Classification
• Hierarchical Text-Conditional Image Generation with CLIP Latents
• Geometric Multimodal Contrastive Representation Learning
• High-Resolution Image Synthesis with Latent Diffusion Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• Learning Transferable Visual Models From Natural Language Supervision
• Concept Bottleneck Models
• Improving Image Autoencoder Embeddings with Perceptual Loss
• The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
• Perceptual Losses for Real-Time Style Transfer and Super-Resolution
• Image quality assessment: from error visibility to structural similarity
• The information bottleneck method

Related

• Hidden in plain sight: VLMs overlook their visual representations
• Synthesize, Diagnose, and Optimize: Towards Fine-Grained Vision-Language Understanding
• How Well Can Vision Language Models See Image Details?
• Probing Visual Language Priors in VLMs
• Demystifying the Visual Quality Paradox in Multimodal Large Language Models