Physics-Based Benchmarking Metrics for Multimodal Synthetic Images

2025/11/19 by Gupta, Kishor Datta, Kamal, Marufa, Rahman, Md. Mahfuzur +3
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Multimodal Machine Learning Applications

paper · doi:10.48550/arxiv.2511.15204

openalex publication_date 2025/11/19 · openalex created_date 2025/11/23 · openalex updated_date 2026/07/28

Abstract

Current state of the art measures like BLEU, CIDEr, VQA score, SigLIP-2 and CLIPScore are often unable to capture semantic or structural accuracy, especially for domain-specific or context-dependent scenarios. For this, this paper proposes a Physics-Constrained Multimodal Data Evaluation (PCMDE) metric combining large language models with reasoning, knowledge based mapping and vision-language models to overcome these limitations. The architecture is comprised of three main stages: (1) feature extraction of spatial and semantic information with multimodal features through object detection and VLMs; (2) Confidence-Weighted Component Fusion for adaptive component-level validation; and (3) physics-guided reasoning using large language models for structural and relational constraints (e.g., alignment, position, consistency) enforcement.

Citations

• LLM Reasoning for Machine Translation: Synthetic Data Generation over Thinking Tokens
• HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion Transformer
• Synthetic Data Generation & Multi-Step RL for Reasoning & Tool Use
• Multimodal Fusion and Vision-Language Models: A Survey for Robot Vision
• Synthetic Data is an Elegant GIFT for Continual Vision-Language Models
• SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
• YOLOv12: Attention-Centric Real-Time Object Detectors
• Text-to-Image Generation Via Energy-Based CLIP
• SynthVLM: Towards High-Quality and Efficient Synthesis of Image-Caption Datasets for Vision-Language Models
• PhyBench: A Physical Commonsense Benchmark for Evaluating Text-to-Image Models
• FLEUR: An Explainable Reference-Free Evaluation Metric for Image Captioning Using a Large Multimodal Model
• Evaluating Text-to-Visual Generation with Image-to-Text Generation
• Taiyi-Diffusion-XL: Advancing Bilingual Text-to-Image Generation with Large Vision-Language Model Support
• SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
• BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
• Coarse-to-Fine Reasoning for Visual Question Answering
• CIDEr-R: Robust Consensus-based Image Description Evaluation
• Learning Transferable Visual Models From Natural Language Supervision
• BERTScore: Evaluating Text Generation with BERT
• Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
• SPICE: Semantic Propositional Image Caption Evaluation

Related

• Pixels to Principles: Probing Intuitive Physics Understanding in Multimodal Language Models
• PhysicsArena: The First Multimodal Physics Reasoning Benchmark Exploring Variable, Process, and Solution Dimensions
• MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models
• PhysUniBench: A Multi-Modal Physics Reasoning Benchmark at Undergraduate Level
• GRASP: A novel benchmark for evaluating language GRounding And Situated Physics understanding in multimodal language models