Can GRPO Boost Complex Multimodal Table Understanding?

2025/09/21 by Xi Kang, Shilin Wu, Kang, Xiaoqiang +16 · 3 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Reinforcement Learning in Robotics #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2509.16889

openalex publication_date 2025/09/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Existing table understanding methods face challenges due to complex table structures and intricate logical reasoning. While supervised finetuning (SFT) dominates existing research, reinforcement learning (RL), such as Group Relative Policy Optimization (GRPO), has shown promise but struggled with low initial policy accuracy and coarse rewards in tabular contexts. In this paper, we introduce Table-R1, a three-stage RL framework that enhances multimodal table understanding through: (1) Warm-up that prompts initial perception and reasoning capabilities, (2) Perception Alignment GRPO (PA-GRPO), which employs continuous Tree-Edit-Distance Similarity (TEDS) rewards for recognizing table structures and contents, and (3) Hint-Completion GRPO (HC-GRPO), which utilizes fine-grained rewards of residual steps based on the hint-guided question. Extensive experiments demonstrate that Table-R1 can boost the model's table reasoning performance obviously on both held-in and held-out datasets, outperforming SFT and GRPO largely. Notably, Qwen2-VL-7B with Table-R1 surpasses larger specific table understanding models (e.g., Table-LLaVA 13B), even achieving comparable performance to the closed-source model GPT-4o on held-in datasets, demonstrating the efficacy of each stage of Table-R1 in overcoming initialization bottlenecks and reward sparsity, thereby advancing robust multimodal table understanding.

Citations

• Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought
• DAPO: An Open-Source LLM Reinforcement Learning System at Scale
• R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization
• Visual-RFT: Visual Reinforcement Fine-Tuning
• Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs
• SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
• Does Table Source Matter? Benchmarking and Improving Multimodal Scientific Table Understanding and Reasoning
• DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding
• Reinforcement Learning Enhanced LLMs: A Survey
• Vision-Language Models Can Self-Improve Reasoning via Reflection
• Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
• Knowledge-Aware Reasoning over Multimodal Semi-structured Tables
• ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search
• TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy
• Why Tabular Foundation Models Should Be a Research Priority
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• Self-Rewarding Language Models
• ReFT: Reasoning with Reinforced Fine-Tuning
• Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
• Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Models
• mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration
• Improved Baselines with Visual Instruction Tuning
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model
• Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning
• Training language models to follow instructions with human feedback
• TableFormer: Robust Transformer Modeling for Table-Text Encoding
• FinQA: A Dataset of Numerical Reasoning over Financial Data
• HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation
• TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance
• INFOTABS: Inference on Tables as Semi-structured Data
• Image-based table recognition: data, model, and evaluation
• ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
• TabFact: A Large-scale Dataset for Table-based Fact Verification
• Proximal Policy Optimization Algorithms
• Compositional Semantic Parsing on Semi-Structured Tables

Cited by

• CheXPO-v2: Preference Optimization for Chest X-ray VLMs with Knowledge Graph Consistency
• IRPO: Boosting Image Restoration via Post-training GRPO
• Table Question Answering in the Era of Large Language Models: A Comprehensive Survey of Tasks, Methods, and Evaluation

Related

• Table-R1: Region-based Reinforcement Learning for Table Understanding
• Strengthening LLMs for Tabular Prediction with Structural Priors
• Reasoning-Table: Exploring Reinforcement Learning for Table Reasoning
• TableGPT-R1: Advancing Tabular Reasoning Through Reinforcement Learning
• Table-R1: Inference-Time Scaling for Table Reasoning