T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generation

2025/08/24 by Sun, Kaiyue, Fang, Rongyao, Duan, Chengqi +2 · 15 citations
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2508.17472

Abstract

We propose T2I-ReasonBench, a benchmark evaluating reasoning capabilities of text-to-image (T2I) models. It consists of four dimensions: Idiom Interpretation, Textual Image Design, Entity-Reasoning and Scientific-Reasoning. We propose a two-stage evaluation protocol to assess the reasoning accuracy and image quality. We benchmark various T2I generation models, and provide comprehensive analysis on their performances.

Citations

• TIIF-Bench: How Does Your T2I Model Follow Your Instructions?
• R2I-Bench: Benchmarking Reasoning-Driven Text-to-Image Generation
• GoT-R1: Unleashing Reasoning Capability of MLLM for Visual Generation with Reinforcement Learning
• Emerging Properties in Unified Multimodal Pretraining
• POSTA: A Go-to Framework for Customized Artistic Poster Generation
• GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing
• WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation
• Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation
• Qwen2.5-VL Technical Report
• Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling
• DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
• A High-Quality Text-Rich Image Instruction Tuning Dataset via Hybrid Instruction Generation
• GPT-4o System Card
• Emu3: Next-Token Prediction is All You Need
• ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficulty
• Show-o: One Single Transformer to Unify Multimodal Understanding and Generation
• Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining
• The Llama 3 Herd of Models
• GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation
• PhyBench: A Physical Commonsense Benchmark for Evaluating Text-to-Image Models
• Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?
• Chameleon: Mixed-Modal Early-Fusion Foundation Models
• Evaluating Text-to-Visual Generation with Image-to-Text Generation
• ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment
• Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
• Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generation
• GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment
• T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generation
• TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering
• LAION-5B: An open large-scale dataset for training next generation image-text models
• Scaling Autoregressive Models for Content-Rich Text-to-Image Generation
• Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• High-Resolution Image Synthesis with Latent Diffusion Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
• CogView: Mastering Text-to-Image Generation via Transformers
• Diffusion Models Beat GANs on Image Synthesis
• InfographicVQA
• Zero-Shot Text-to-Image Generation
• Denoising Diffusion Probabilistic Models
• Neural Discrete Representation Learning

Cited by

• GenEval 2: Addressing Benchmark Drift in Text-to-Image Evaluation
• Position: Universal Aesthetic Alignment Narrows Artistic Expression
• Envision: Benchmarking Unified Understanding & Generation for Causal World Process Insights
• Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forward
• ImAgent: A Unified Multimodal Agent Framework for Test-Time Scalable Image Generation
• UniREditBench: A Unified Reasoning-based Image Editing Benchmark
• UniGenBench++: A Unified Semantic Evaluation Benchmark for Text-to-Image Generation
• PICABench: How Far Are We from Physically Realistic Image Editing?
• Generative Universal Verifier as Multimodal Meta-Reasoner
• SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models
• Factuality Matters: When Image Generation and Editing Meet Structured Visuals
• RealUnify: Do Unified Models Truly Benefit from Unification? A Comprehensive Benchmark
• GenExam: A Multidisciplinary Text-to-Image Exam
• FLUX-Reason-6M & PRISM-Bench: A Million-Scale Text-to-Image Reasoning Dataset and Comprehensive Benchmark
• TIIF-Bench: How Does Your T2I Model Follow Your Instructions?
• Simile Understanding in Text-to-Image Models: An Evaluation Framework

Related

• R2I-Bench: Benchmarking Reasoning-Driven Text-to-Image Generation
• T2I-FactualBench: Benchmarking the Factuality of Text-to-Image Models with Knowledge-Intensive Concepts
• WorldGenBench: A World-Knowledge-Integrated Benchmark for Reasoning-Driven Text-to-Image Generation
• FLUX-Reason-6M & PRISM-Bench: A Million-Scale Text-to-Image Reasoning Dataset and Comprehensive Benchmark
• Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage, but Not Direct the Play?