LM Fight Arena: Benchmarking Large Multimodal Models via Game Competition

2025/10/10 by Yushuo Zheng, Zicheng Zhang, Zheng, Yushuo +6 · 2 citations
Computer Science · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Games #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2510.08928

openalex publication_date 2025/10/10 · openalex created_date 2025/10/14 · openalex updated_date 2026/07/28

Abstract

Existing benchmarks for large multimodal models (LMMs) often fail to capture their performance in real-time, adversarial environments. We introduce LM Fight Arena (Large Model Fight Arena), a novel framework that evaluates LMMs by pitting them against each other in the classic fighting game Mortal Kombat II, a task requiring rapid visual understanding and tactical, sequential decision-making. In a controlled tournament, we test six leading open- and closed-source models, where each agent operates controlling the same character to ensure a fair comparison. The models are prompted to interpret game frames and state data to select their next actions. Unlike static evaluations, LM Fight Arena provides a fully automated, reproducible, and objective assessment of an LMM's strategic reasoning capabilities in a dynamic setting. This work introduces a challenging and engaging benchmark that bridges the gap between AI evaluation and interactive entertainment.

Citations

• Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
• AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents
• InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
• MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
• MMBench: Is Your Multi-modal Model an All-around Player?
• MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• Visual Instruction Tuning
• GPT-4 Technical Report
• BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
• Flamingo: a Visual Language Model for Few-Shot Learning
• Dota 2 with Large Scale Deep Reinforcement Learning
• Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering
• Microsoft COCO: Common Objects in Context
• Deep Blue
• VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks

Cited by

• GeoX-Bench: Benchmarking Cross-View Geo-Localization and Pose Estimation Capabilities of Large Multimodal Models
• Evaluating from Benign to Dynamic Adversarial: A Squid Game for Large Language Models

Related

• FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning
• GVGAI-LLM: Evaluating Large Language Model Agents with Infinite Games
• Creating Pro-Level AI for a Real-Time Fighting Game Using Deep Reinforcement Learning
• Gaming the Arena: AI Model Evaluation and the Viral Capture of Attention
• Evaluating Large Language Models with Grid-Based Game Competitions: An Extensible LLM Benchmark and Leaderboard