Self-Improvement in Multimodal Large Language Models: A Survey

2025/10/03 by Shijian Deng, Kai Wang, Deng, Shijian +7 · 1 citation
Computer Science · Social Sciences · #Computation and Language (cs.CL) #Computational and Text Analysis Methods #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2510.02665

openalex publication_date 2025/10/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recent advancements in self-improvement for Large Language Models (LLMs) have efficiently enhanced model capabilities without significantly increasing costs, particularly in terms of human effort. While this area is still relatively young, its extension to the multimodal domain holds immense potential for leveraging diverse data sources and developing more general self-improving models. This survey is the first to provide a comprehensive overview of self-improvement in Multimodal LLMs (MLLMs). We provide a structured overview of the current literature and discuss methods from three perspectives: 1) data collection, 2) data organization, and 3) model optimization, to facilitate the further development of self-improvement in MLLMs. We also include commonly used evaluations and downstream applications. Finally, we conclude by outlining open challenges and future research directions.

Citations

• Self-Improving Embodied Foundation Models
• Learning Active Perception via Self-Evolving Preference Optimization for GUI Grounding
• Improving Large Vision and Language Models by Learning from a Panel of Peers
• Reinforcement Learning for Large Model: A Survey
• A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems
• SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience
• A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence
• Self-Improvement for Audio Large Language Model using Unlabeled Speech
• C2-Evo: Co-Evolving Multimodal Data and Model for Self-Improving Reasoning
• MMSearch-R1: Incentivizing LMMs to Search
• VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning
• The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search
• Video-R1: Reinforcing Video Reasoning in MLLMs
• PHYSICS: Benchmarking Foundation Models on University-Level Physics Problem Solving
• Qwen2.5-Omni Technical Report
• Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning
• MLLM-For3D: Adapting Multimodal Large Language Model for 3D Reasoning Segmentation
• R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization
• DeepPerception: Advancing R1-like Cognitive Visual Perception in MLLMs for Knowledge-Intensive Visual Grounding
• PiSA: A Self-Augmented Data Engine and Training Strategy for 3D Understanding with Large Models
• R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization
• Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learning
• R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learning
• R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model
• SHAPE : Self-Improved Visual Preference Alignment by Iteratively Generating Holistic Winner
• Visual-RFT: Visual Reinforcement Fine-Tuning
• OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preference
• ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy
• Vision-Language Model Dialog Games for Self-Improvement
• Probing Visual Language Priors in VLMs
• Diving into Self-Evolving Training for Multimodal Reasoning
• Beyond Human Data: Aligning Multimodal Large Language Models by Iterative Self-Evolution
• A Survey on LLM Inference-Time Self-Improvement
• Efficient Self-Improvement in Multimodal Large Language Models: A Model-Level Judge-Free Approach
• Vision-Language Models Can Self-Improve Reasoning via Reflection
• AutoBench-V: Can Large Vision-Language Models Benchmark Themselves?
• Fine-Grained Verifiers: Preference Modeling as Next-token Prediction in Vision-Language Alignment
• VLFeedback: A Large-Scale AI Feedback Dataset for Large Vision-Language Models Alignment
• Self-Correction is More than Refinement: A Learning Framework for Visual and Language Reasoning Tasks
• MLLM as Retriever: Interactively Learning Multimodal Retrieval for Embodied Agents
• SELU: Self-Learning Embodied MLLMs in Unknown Environments
• MMEvol: Empowering Multimodal Large Language Models with Evol-Instruct
• CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs
• The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
• COMET: "Cone of experience" enhanced large multimodal model for mathematical problem generation
• MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Engine
• Video-STaR: Self-Training Enables Video Instruction Tuning with Any Supervision
• Understanding Alignment in Multimodal LLMs: A Comprehensive Study
• STLLaVA-Med: Self-Training Large Language and Vision Assistant for Medical Question-Answering
• Enhancing Large Vision Language Models with Self-Training on Image Comprehension
• Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvement
• Automated Multi-level Preference for MLLMs
• Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback
• Hear Me, See Me, Understand Me: Audio-Visual Autism Behavior Recognition
• SQ-LLaVA: Self-Questioning for Large Vision-Language Assistant
• Aligning Modalities in Vision Large Language Models via Preference Fine-tuning
• Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedback
• SceMQA: A Scientific College Entrance Level Multimodal Question Answering Benchmark
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• Comprehensive Exploration of Synthetic Data Generation: A Survey
• G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model
• Silkie: Preference Distillation for Large Visual Language Models
• RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback
• Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding
• Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization
• MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
• Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
• AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation
• MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
• Aligning Large Multimodal Models with Factually Augmented RLHF
• VIGC: Visual Instruction Generation and Correction
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model
• InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
• Large Language Models Can Self-Improve
• Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
• Training language models to follow instructions with human feedback
• A Survey of Data Augmentation Approaches for NLP
• SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answering
• PathVQA: 30000+ Questions for Medical Visual Question Answering
• ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering
• From Recognition to Cognition: Visual Commonsense Reasoning
• Object Hallucination in Image Captioning
• TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering
• OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cycles
• ISR-DPO: Aligning Large Multimodal Models for Videos by Iterative Self-Retrospective DPO
• Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond

Cited by

• Instrumental goals in advanced AI systems: Features to be managed and not failures to be eliminated?

Related

• Personalized Multimodal Large Language Models: A Survey
• Efficient Self-Improvement in Multimodal Large Language Models: A Model-Level Judge-Free Approach
• MM-LLMs: Recent Advances in MultiModal Large Language Models
• Can We Edit Multimodal Large Language Models?
• The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective