G1: Bootstrapping Perception and Reasoning Abilities of Vision-Language Model via Reinforcement Learning

2025/05/19 by Liang Chen, Chen, Liang, Hongcheng Gao +13 · 11 citations
Computer Science · Psychology · Social Sciences · #Multimodal Machine Learning Applications #Language, Metaphor, and Cognition #Language and cultural evolution

paper · pdf · doi:10.48550/arxiv.2505.13426

Abstract

Vision-Language Models (VLMs) excel in many direct multimodal tasks but struggle to translate this prowess into effective decision-making within interactive, visually rich environments like games. This ``knowing-doing'' gap significantly limits their potential as autonomous agents, as leading VLMs often performing badly in simple games. To address this, we introduce VLM-Gym, a curated reinforcement learning (RL) environment featuring diverse visual games with unified interfaces and adjustable, compositional difficulty, specifically designed for scalable multi-game parallel training. Leveraging VLM-Gym, we train G0 models using pure RL-driven self-evolution, which demonstrate emergent perception and reasoning patterns. To further mitigate challenges arising from game diversity, we develop G1 models. G1 incorporates a perception-enhanced cold start prior to RL fine-tuning. Our resulting G1 models consistently surpass their teacher across all games and outperform leading proprietary models like Claude-3.7-Sonnet-Thinking. Systematic analysis reveals an intriguing finding: perception and reasoning abilities mutually bootstrap each other throughout the RL training process. Source code including VLM-Gym and RL training are released at https://github.com/chenllliang/G1 to foster future research in advancing VLMs as capable interactive agents.

Cited by

• CAST: Game Solvers as Turn-Level Teachers for LLM Agents
• More Than the Final Answer: Improving Visual Extraction and Logical Consistency in Vision-Language Models
• Thinking by Doing: Building Efficient World Model Reasoning in LLMs via Multi-turn Interaction
• Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMs
• Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learning
• HoneyBee: Data Recipes for Vision-Language Reasoners
• Spotlight on Token Perception for Multimodal Reinforcement Learning
• Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play
• Explain Before You Answer: A Survey on Compositional Visual Reasoning
• UAV-VL-R1: Generalizing Vision-Language Models via Supervised Fine-Tuning and Multi-Stage GRPO for UAV Visual Reasoning
• Reinforcement Learning for Large Model: A Survey

Related

• Game-RL: Synthesizing Multimodal Verifiable Game Data to Boost VLMs' General Reasoning
• Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learning
• Play to Generalize: Learning to Reason Through Game Play
• Enhancing Vision-Language Model Training with Reinforcement Learning in Synthetic Worlds for Real-World Success
• Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play