Learning Multi-dimensional Human Preference for Text-to-Image Generation

2024/05/23 by Sixian Zhang, Bohan Wang, Zhang, Sixian +11 · 28 citations
Engineering · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Motion and Animation

paper · pdf · doi:10.48550/arxiv.2405.14705

openalex publication_date 2024/05/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Current metrics for text-to-image models typically rely on statistical metrics which inadequately represent the real preference of humans. Although recent work attempts to learn these preferences via human annotated images, they reduce the rich tapestry of human preference to a single overall score. However, the preference results vary when humans evaluate images with different aspects. Therefore, to learn the multi-dimensional human preferences, we propose the Multi-dimensional Preference Score (MPS), the first multi-dimensional preference scoring model for the evaluation of text-to-image models. The MPS introduces the preference condition module upon CLIP model to learn these diverse preferences. It is trained based on our Multi-dimensional Human Preference (MHP) Dataset, which comprises 918,315 human preference choices across four dimensions (i.e., aesthetics, semantic alignment, detail quality and overall assessment) on 607,541 images. The images are generated by a wide range of latest text-to-image models. The MPS outperforms existing scoring methods across 3 datasets in 4 dimensions, enabling it a promising metric for evaluating and improving text-to-image generation.

Cited by

• FinPercep-RM: A Fine-grained Reward Model and Co-evolutionary Curriculum for RL-based Real-world Super-Resolution
• Co-GRPO: Co-Optimized Group Relative Policy Optimization for Masked Diffusion Model
• Harm is not Universal: Community-Specific Toxicity Detection is Urgently Needed
• Multi-dimensional Preference Alignment by Conditioning Reward Itself
• Position: Universal Aesthetic Alignment Narrows Artistic Expression
• I2I-Bench: A Comprehensive Benchmark Suite for Image-to-Image Editing Models
• PaCo-RL: Advancing Reinforcement Learning for Consistent Image Generation with Pairwise Reward Modeling
• JarvisEvo: Towards a Self-Evolving Photo Editing Agent with Synergistic Editor-Evaluator Optimization
• A Reason-then-Describe Instruction Interpreter for Controllable Video Generation
• MapReduce LoRA: Advancing the Pareto Front in Multi-Preference Optimization for Generative Models
• Flash-DMD: Towards High-Fidelity Few-Step Image Generation with Efficient Distillation and Joint Reinforcement Learning
• The Image as Its Own Reward: Reinforcement Learning with Adversarial Reward for Image Generation
• Personalized Reward Modeling for Text-to-Image Generation
• PairHuman: A High-Fidelity Photographic Dataset for Customized Dual-Person Generation
• Reg-DPO: SFT-Regularized Direct Preference Optimization with GT-Pair for Improving Video Generation
• Evolutionary Optimization Trumps Adam Optimization on Embedding Space Exploration
• PreferThinker: Reasoning-based Personalized Image Preference Assessment
• Cost Savings from Automatic Quality Assessment of Generated Images
• Dense2MoE: Restructuring Diffusion Transformer to MoE for Efficient Text-to-Image Generation
• TIT-Score: Evaluating Long-Prompt Based Text-to-Image Alignment via Text-to-Image-to-Text Consistency
• EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modeling
• GeoRemover: Removing Objects and Their Causal Visual Artifacts
• Directly Aligning the Full Diffusion Trajectory with Fine-Grained Human Preference
• Composition and Alignment of Diffusion Models using Constrained Learning
• Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models
• Multimodal learning with next-token prediction for large multimodal models
• HPSv3: Towards Wide-Spectrum Human Preference Score
• Trade-offs in Image Generation: How Do Different Dimensions Interact?

Related

• Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis
• HPSv3: Towards Wide-Spectrum Human Preference Score
• Human Preference Score: Better Aligning Text-to-Image Models with Human Preference
• ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation
• PrefGen: Multimodal Preference Learning for Preference-Conditioned Image Generation