CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal Reasoning

2025/12/22 by Yongxin Wang, Wang, Yongxin, Zhicheng Yang +13 · 1 citation
Computer Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications

paper · doi:10.48550/arxiv.2512.19554

openalex publication_date 2025/12/22 · openalex created_date 2025/12/24 · openalex updated_date 2026/07/28

Abstract

Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures. When all rollouts are wrong, gradients stall; when one happens to be correct, the update usually ignores why the others are close-but-wrong, and credit can be misassigned to spurious chains. We present CARE (Contrastive Anchored REflection), a failure-centric post-training framework for multimodal reasoning that turns errors into supervision. CARE combines: (i) an anchored-contrastive objective that forms a compact subgroup around the best rollout and a set of semantically proximate hard negatives, performs within-subgroup z-score normalization with negative-only scaling, and includes an all-negative rescue to prevent zero-signal batches; and (ii) Reflection-Guided Resampling (RGR), a one-shot structured self-repair that rewrites a representative failure and re-scores it with the same verifier, converting near-misses into usable positives without any test-time reflection. CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures. On Qwen2.5-VL-7B, CARE lifts macro-averaged accuracy by 4.6 points over GRPO across six verifiable visual-reasoning benchmarks; with Qwen3-VL-8B it reaches competitive or state-of-the-art results on MathVista and MMMU-Pro under an identical evaluation protocol.

Citations

• GaussianDWM: 3D Gaussian Driving World Model for Unified Scene Understanding and Multi-Modal Generation
• OpenAI GPT-5 System Card
• STARE-VLA: Progressive Stage-Aware Reinforcement for Fine-Tuning Vision-Language-Action Models
• What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Models
• MedEyes: Learning Dynamic Visual Focus for Medical Progressive Diagnosis
• Qwen3-VL Technical Report
• Spotlight on Token Perception for Multimodal Reinforcement Learning
• ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level Entropy Shaping
• EditGRPO: Reinforcement Learning with Post-Rollout Edits for Clinically Accurate Chest X-Ray Report Generation
• MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources
• Kwai Keye-VL 1.5 Technical Report
• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
• Depth-Breadth Synergy in RLVR: Unlocking LLM Reasoning Gains with Adaptive Exploration
• UI-AGILE: Advancing GUI Agents with Effective Reinforcement Learning and Precise Inference-Time Grounding
• M2-Reasoning: Empowering MLLMs with Unified General and Spatial Reasoning
• Scaling RL to Long Videos
• R1-Track: Direct Application of MLLMs to Visual Object Tracking via Reinforcement Learning
• Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMs
• MiMo-VL Technical Report
• Grounded Reinforcement Learning for Visual Reasoning
• SeePhys: Does Seeing Help Thinking? -- Benchmarking Vision-Based Physics Reasoning
• VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use
• SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Reward
• NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation
• GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents
• VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning
• Understanding R1-Zero-Like Training: A Critical Perspective
• MetaSpatial: Reinforcing 3D Spatial Reasoning in VLMs for the Metaverse
• DAPO: An Open-Source LLM Reinforcement Learning System at Scale
• R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization
• Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models
• Reinforcement Learning with Verifiable Rewards: GRPO's Effective Loss, Dynamics, and Success Amplification
• Qwen2.5-VL Technical Report
• Temporal Working Memory: Query-Guided Segment Refinement for Enhanced Multimodal Understanding
• LLaVA-CoT: Let Vision Language Models Reason Step-by-Step
• GPT-4o System Card
• MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark
• LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models
• M³CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought
• MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?
• Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
• Visual Instruction Tuning
• ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning
• Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning

Cited by

• GaussianDWM: 3D Gaussian Driving World Model for Unified Scene Understanding and Multi-Modal Generation
• UI-AGILE: Advancing GUI Agents with Effective Reinforcement Learning and Precise Inference-Time Grounding
• Music's Multimodal Complexity in AVQA: Why We Need More than General Multimodal LLMs

Related

• GRPO-CARE: Consistency-Aware Reinforcement Learning for Multimodal Reasoning
• SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning
• Perception-Aware Policy Optimization for Multimodal Reasoning
• Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective
• When RLVR Shrinks the Reasoning Boundary: Diagnosing Pass@k Inversion