Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning

2025/04/23 by Peiyu Wang, Yichen Wei, Wang, Peiyu +22 · 12 citations
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Constraint Satisfaction and Optimization #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2504.16656

openalex publication_date 2025/04/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present Skywork R1V2, a next-generation multimodal reasoning model and a major leap forward from its predecessor, Skywork R1V. At its core, R1V2 introduces a hybrid reinforcement learning paradigm that jointly leverages the Mixed Preference Optimization (MPO) and the Group Relative Policy Optimization (GRPO), which harmonizes reward-model guidance with rule-based strategies, thereby addressing the long-standing challenge of balancing sophisticated reasoning capabilities with broad generalization. To further enhance training efficiency, we propose the Selective Sample Buffer (SSB) mechanism, which effectively addresses the vanishing advantages dilemma inherent in GRPO by prioritizing high-value samples throughout the optimization process. Notably, we observe that excessive reinforcement signals can induce visual hallucinations--a phenomenon we systematically monitor and mitigate through calibrated reward thresholds throughout the training process. Empirical results affirm the exceptional capability of R1V2, with benchmark-leading performances such as 62.6 on OlympiadBench, 78.9 on AIME2024, 63.6 on LiveCodeBench, and 73.6 on MMMU. These results underscore R1V2's superiority over existing open-source models and demonstrate significant progress in closing the performance gap with premier proprietary systems, including Gemini 2.5 and OpenAI-o4-mini. The Skywork R1V2 model weights have been publicly released to promote openness and reproducibility https://huggingface.co/Skywork/Skywork-R1V2-38B.

Cited by

• AMUSE: Audio-Visual Benchmark and Alignment Framework for Agentic Multi-Speaker Understanding
• More Than the Final Answer: Improving Visual Extraction and Logical Consistency in Vision-Language Models
• Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearch
• ReVSeg: Incentivizing the Reasoning Chain for Video Segmentation with Reinforcement Learning
• Boosting Reasoning in Large Multimodal Models via Activation Replay
• A Survey on Agentic Multimodal Large Language Models
• Spotlight on Token Perception for Multimodal Reinforcement Learning
• Clarification as Supervision: Reinforcement Learning for Vision-Language Interfaces
• VTPerception-R1: Enhancing Multimodal Reasoning via Explicit Visual and Textual Perceptual Grounding
• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
• Reinforcement Learning for Large Model: A Survey
• VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoning

Related

• Skywork-VL Reward: An Effective Reward Model for Multimodal Understanding and Reasoning
• Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought
• Skywork-R1V3 Technical Report
• Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models
• Perception-Aware Policy Optimization for Multimodal Reasoning