Visual Instruction Tuning

2023/04/17 by Haotian Liu, Liu, Haotian, Chunyuan Li +5 · 1347 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #Natural Language Processing Techniques

paper · pdf · doi:10.48550/arxiv.2304.08485

openalex publication_date 2023/04/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/30

Abstract

Instruction tuning large language models (LLMs) using machine-generated instruction-following data has improved zero-shot capabilities on new tasks, but the idea is less explored in the multimodal field. In this paper, we present the first attempt to use language-only GPT-4 to generate multimodal language-image instruction-following data. By instruction tuning on such generated data, we introduce LLaVA: Large Language and Vision Assistant, an end-to-end trained large multimodal model that connects a vision encoder and LLM for general-purpose visual and language understanding.Our early experiments show that LLaVA demonstrates impressive multimodel chat abilities, sometimes exhibiting the behaviors of multimodal GPT-4 on unseen images/instructions, and yields a 85.1% relative score compared with GPT-4 on a synthetic multimodal instruction-following dataset. When fine-tuned on Science QA, the synergy of LLaVA and GPT-4 achieves a new state-of-the-art accuracy of 92.53%. We make GPT-4 generated visual instruction tuning data, our model and code base publicly available.

Cited by

• DeCoRAG: Cognitive Decoupling and Semantic-Aware Cropping for Complex Document Understanding
• DecoupleMix: Decoupled Ratio Search and Convex Allocation for Scalable VLM Data Recipes
• UCAgents: Unidirectional Convergence for Visual Evidence Anchored Multi-Agent Medical Decision-Making
• Unison: A Fully Automatic, Task-Universal, and Low-Cost Framework for Unified Understanding and Generation
• ProGuard: Towards Proactive Multimodal Safeguard
• Instruction-Following Evaluation of Large Vision-Language Models
• D-OPSD: On-Policy Self-Distillation for Continuously Tuning Step-Distilled Diffusion Models
• CoFi-Dec: Hallucination-Resistant Decoding via Coarse-to-Fine Generative Feedback in Large Vision-Language Models
• Video Understanding: From Geometry and Semantics to Unified Models
• SpatialMosaic: A Multiview VLM Dataset for Partial Visibility
• An Architecture-Led Hybrid Report on Body Language Detection Project
• Embodied Robot Manipulation in the Era of Foundation Models: Planning and Learning Perspectives
• Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbone
• VULCAN: Tool-Augmented Multi Agents for Iterative 3D Object Arrangement
• Masking Teacher and Reinforcing Student for Distilling Vision-Language Models
• Meta-information Guided Cross-domain Synergistic Diffusion Model for Low-dose PET Reconstruction
• VideoScaffold: Elastic-Scale Visual Hierarchies for Streaming Video Understanding in MLLMs
• Open-Source Multimodal Moxin Models with Moxin-VLM and Moxin-VLA
• Gaze-to-text Generation: Beyond Categorical Decoding of Human Attention
• Towards High-Level Semantic Intelligence
• MemVLN: Episodic and Procedural Memory for Vision-and-Language Navigation
• RayRoPE: Projective Ray Positional Encoding for Multi-view Attention
• Hybrid Advantage Estimation with Unified Critic for VLM Agentic Reinforcement Learning
• Do Diagrams Help Large Language Models Reason? Evidence from Syllogistic Reasoning
• PathScale-R1: Cross-scale Reasoning for Pathological Image Analysis
• UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling
• MAViE: A Multi-scale Adaptive Vision Encoder for Fine-grained Visual Perception and Efficient Multimodal Reasoning
• Structured Redundancy Modeling for Efficient Visual Token Pruning in High-Resolution MLLMs
• What CLIP Knows but Cannot Say: Recovering Negation from Frozen Intermediate Features
• WaveZip: Wavelet-Driven Space-Time Decoupling for Video Token Condensation
• Self-Boosting Vision-Language Models with Noisy Student On-Policy Self-Distillation
• MEDit-Bench: A Dataset for Evaluating Message-Driven Narrative Video Editing
• Salient Knowledge Pathways: Sparse Cross-Modal Routing for Efficient Knowledge-Intensive Multimodal Question Answering
• Architectural Backdoors in Vision-Language Model Supply Chains via Representation Steering
• Noise-Free One-Step LoRA for Task-Driven Image Restoration with Diffusion Priors
• LENS: Adaptive Spatio-Temporal Zooming for Keyframe Sampling in Long-Form Videos
• Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model
• Harm is not Universal: Community-Specific Toxicity Detection is Urgently Needed
• LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integration
• A Reconstruction-Based Framework for Caption Evaluation Beyond Reference Captions
• Controlling Embedding Spaces with Text-Conditioned Transformations
• StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design
• RRS-10K: A Multitask Vision-Language Model Benchmark for Rare Remote Sensing Image Interpretation
• An LMM for Precisely Grounding Elements in Documents
• ReSAGE-PAR: Representational Similarity Assessment for Generative Expansion in Pedestrian Attribute Recognition
• Aligning Quantum Operators with Large Language Models
• Reasoning with Memory: A Temporal Granularity-Adaptive Framework for Training-Free Long Video Understanding
• NEXT: Reasoning-Driven Video Recommendation via a Vision-Language Model
• Reason Before You Retrieve: Agentic Planning for Multi-modal RAG
• DocAnnot -- Accelerating the Creation of Key Information Extraction Datasets with GenAI-Powered Auto-annotation
• Personalize Your Large Vision-language Models With In-context Prompt Tuning
• When Sinks Help or Hurt: Unified Framework for Attention Sink in Large Vision-Language Models
• Why Does Grounding Hurt Medical VQA? Benchmarking, Diagnosis, and Fine-Tuning of Vision-Language Models
• MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence
• LanteRn: Latent Visual Structured Reasoning
• StAR: Segment Anything Reasoner
• Compressing LLMs with MoP: Mixture of Pruners
• CLIP Is Shortsighted: Paying Attention Beyond the First Sentence
• Training-free Conditional Image Embedding Framework Leveraging Large Vision Language Models
• ImagineNav++: Prompting Vision-Language Models as Embodied Navigator through Scene Imagination
• A Medical Multimodal Diagnostic Framework Integrating Vision-Language Models and Logic Tree Reasoning
• EraseLoRA: MLLM-Driven Foreground Exclusion and Background Subtype Aggregation for Dataset-Free Object Removal
• Fixed-Budget Parameter-Efficient Training with Frozen Encoders Improves Multimodal Chest X-Ray Classification
• RLLaVA: An RL-central Framework for Language and Vision Assistants
• Streaming Video Instruction Tuning
• Fast SAM2 with Text-Driven Token Pruning
• Latent Implicit Visual Reasoning
• ORCA: Object Recognition and Comprehension for Archiving Marine Species
• MarineEval: Assessing the Marine Intelligence of Vision-Language Models
• Language-Guided Grasp Detection with Coarse-to-Fine Learning for Robotic Manipulation
• Learning from Next-Frame Prediction: Autoregressive Video Modeling Encodes Effective Representations
• MMSRARec: Summarization and Retrieval Augumented Sequential Recommendation Based on Multimodal Large Language Model
• PanoGrounder: Bridging 2D and 3D with Panoramic Scene Representations for VLM-based 3D Visual Grounding
• Input-Adaptive Visual Preprocessing for Efficient Fast Vision-Language Model Inference
• FlashVLM: Text-Guided Visual Token Selection for Large Multimodal Models
• Multi-Grained Text-Guided Image Fusion for Multi-Exposure and Multi-Focus Scenarios
• M³KG-RAG: Multi-hop Multimodal Knowledge Graph-enhanced Retrieval-Augmented Generation
• Generative Digital Twins: Vision-Language Simulation Models for Executable Industrial Systems
• SegEarth-R2: Towards Comprehensive Language-guided Segmentation for Remote Sensing Images
• SpatialTree: How Spatial Abilities Branch Out in MLLMs
• Pushing the Frontier of Audiovisual Perception with Large-Scale Multimodal Correspondence Learning
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal Reasoning
• Event Extraction in Large Language Model
• QuantiPhy: A Quantitative Benchmark Evaluating Physical Reasoning Abilities of Vision-Language Models
• ReasonCD: A Multimodal Reasoning Large Model for Implicit Change-of-Interest Semantic Mining
• Generative Human-Object Interaction Detection via Differentiable Cognitive Steering of Multi-modal LLMs
• IPCV: Information-Preserving Compression for MLLM Visual Encoders
• Delta-LLaVA: Base-then-Specialize Alignment for Token-Efficient Vision-Language Models
• Thinking Beyond Labels: Vocabulary-Free Fine-Grained Recognition using Reasoning-Augmented LMMs
• EchoMotion: Unified Human Video and Motion Generation via Dual-Modality Diffusion Transformer
• Revealing Perception and Generation Dynamics in LVLMs: Mitigating Hallucinations via Validated Dominance Correction
• LLaViDA: A Large Language Vision Driving Assistant for Explicit Reasoning and Enhanced Trajectory Planning
• M³-Verse: A "Spot the Difference" Challenge for Large Multimodal Models
• OpenView: Empowering MLLMs with Out-of-view VQA
• Enhancing 3D Semantic Scene Completion with a Refinement Module
• MoE Pathfinder: Trajectory-driven Expert Pruning
• HyDRA: Hierarchical and Dynamic Rank Adaptation for Mobile Vision Language Model
• SLIM: Semantic-based Low-bitrate Image compression for Machines by leveraging diffusion
• External Hippocampus: Topological Cognitive Maps for Guiding Large Language Model Reasoning
• FPBench: A Comprehensive Benchmark of Multimodal Large Language Models for Fingerprint Analysis
• Robotic VLA Benefits from Joint Learning with Motion Image Diffusion
• Both Semantics and Reconstruction Matter: Making Representation Encoders Ready for Text-to-Image Generation and Editing
• AnyTask: an Automated Task and Data Generation Framework for Advancing Sim-to-Real Policy Learning
• GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation
• MMLANDMARKS: a Cross-View Instance-Level Benchmark for Geo-Spatial Understanding
• Xiaomi MiMo-VL-Miloco Technical Report
• RadImageNet-VQA: A Large-Scale CT and MRI Dataset for Radiologic Visual Question Answering
• Are Vision Language Models Cross-Cultural Theory of Mind Reasoners?
• A Benchmark for Ultra-High-Resolution Remote Sensing MLLMs
• Deep But Reliable: Advancing Multi-turn Reasoning for Thinking with Images
• Vision-Language Model Guided Image Restoration
• CheXPO-v2: Preference Optimization for Chest X-ray VLMs with Knowledge Graph Consistency
• ABE-CLIP: Training-Free Attribute Binding Enhancement for Compositional Image-Text Matching
• 4D-RGPT: Toward Region-level 4D Understanding via Perceptual Distillation
• Differences That Matter: Auditing Models for Capability Gap Discovery and Rectification
• VIVA: VLM-Guided Instruction-Based Video Editing with Reward Optimization
• CitySeeker: How Do VLMS Explore Embodied Urban Navigation With Implicit Human Needs?
• Monte Carlo Query Search: Active Capability Assessment of AI Agents
• Sketch-in-Latents: Eliciting Unified Reasoning in MLLMs
• N3D-VLM: Native 3D Grounding Enables Accurate Spatial Reasoning in Vision-Language Models
• Guiding Perception-Reasoning Closer to Human in Blind Image Quality Assessment
• AMUSE: Audio-Visual Benchmark and Alignment Framework for Agentic Multi-Speaker Understanding
• Scaling Spatial Reasoning in MLLMs through Programmatic Data Synthesis
• Seeing Beyond Words: Self-Supervised Visual Learning for Multimodal Large Language Models
• MRG-R1: Reinforcement Learning for Clinically Aligned Medical Report Generation
• Sceniris: A Fast Procedural Scene Generation Framework
• Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future
• The Perceptual Observatory Characterizing Robustness and Grounding in MLLMs
• DiffusionVL: Translating Any Autoregressive Models into Diffusion Vision Language Models
• Skyra: AI-Generated Video Detection via Grounded Artifact Reasoning
• Large Video Planner Enables Generalizable Robot Control
• An Efficient and Effective Encoder Model for Vision and Language Tasks in the Remote Sensing Domain
• VLA-AN: An Efficient and Onboard Vision-Language-Action Framework for Aerial Navigation in Complex Environments
• On-device Large Multi-modal Agent for Human Activity Recognition
• EmoCaliber: Advancing Reliable Visual Emotion Comprehension via Confidence Verbalization and Calibration
• Parameter Efficient Multimodal Instruction Tuning for Romanian Vision Language Models
• EVOLVE-VLA: Test-Time Training from Environment Feedback for Vision-Language-Action Models
• Focus: A Streaming Concentration Architecture for Efficient Vision-Language Models
• JMMMU-Pro: Image-based Japanese Multi-discipline Multimodal Understanding Benchmark via Vibe Benchmark Construction
• Unified Semantic Transformer for 3D Scene Understanding
• Vector Prism: Animating Vector Graphics by Stratifying Semantic Structure
• Zoom-Zero: Reinforced Coarse-to-Fine Video Understanding via Temporal Zoom-in
• ChartAgent: A Chart Understanding Framework with Tool Integrated Reasoning
• Autonomous Construction-Site Safety Inspection Using Mobile Robots: A Multilayer VLM-LLM Pipeline
• SuperCLIP: CLIP with Simple Classification Supervision
• A4-Agent: An Agentic Framework for Zero-Shot Affordance Reasoning
• Beyond surface form: A pipeline for semantic analysis in Alzheimer's Disease detection from spontaneous speech
• AgentIAD: Agentic Industrial Anomaly Detection via Adaptive Memory Augmentation
• ShowTable: Unlocking Creative Table Visualization with Collaborative Reflection and Refinement
• Reflective Preference Optimization (RPO): Enhancing On-Policy Alignment via Hint-Guided Reflection
• MMDrive: Interactive Scene Understanding Beyond Vision with Multi-representational Fusion
• Why Text Prevails: Vision May Undermine Multimodal Medical Decision Making
• Lemon: A Unified and Scalable 3D Multimodal Model for Universal Spatial Understanding
• FysicsWorld: A Unified Full-Modality Benchmark for Any-to-Any Understanding, Generation, and Reasoning
• Efficient Vision-Language Reasoning via Adaptive Token Pruning
• Content-Aware Ad Banner Layout Generation with Two-Stage Chain-of-Thought in Vision Language Models
• WeDetect: Fast Open-Vocabulary Object Detection as Retrieval
• EditMGT: Unleashing Potentials of Masked Generative Transformers in Image Editing
• VideoARM: Agentic Reasoning over Hierarchical Memory for Long-Form Video Understanding
• VEGAS: Mitigating Hallucinations in Large Vision-Language Models via Vision-Encoder Attention Guided Adaptive Steering
• Exploring MLLM-Diffusion Information Transfer with MetaCanvas
• SmokeBench: Evaluating Multimodal Large Language Models for Wildfire Smoke Detection
• HFS: Holistic Query-Aware Frame Selection for Efficient Video Reasoning
• Cross-modal Context-aware Learning for Visual Prompt Guided Multimodal Image Understanding in Remote Sensing
• DentalGPT: Incentivizing Multimodal Complex Reasoning in Dentistry
• VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing
• UFVideo: Towards Unified Fine-Grained Video Cooperative Understanding with Large Language Models
• MeViS: A Multi-Modal Dataset for Referring Motion Expression Video Segmentation
• VL-JEPA: Joint Embedding Predictive Architecture for Vision-language
• BabyVLM-V2: Toward Developmentally Grounded Pretraining and Benchmarking of Vision Foundation Models
• Interpretable and Steerable Concept Bottleneck Sparse Autoencoders
• SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving
• Beyond Pixels: A Training-Free, Text-to-Text Framework for Remote Sensing Image Retrieval
• Blink: Dynamic Visual Token Resolution for Enhanced Multimodal Understanding
• Towards Fine-Grained Recognition with Large Visual Language Models: Benchmark and Optimization Strategies
• Visual Funnel: Resolving Contextual Blindness in Multimodal Large Language Models
• Multilingual VLM Training: Adapting an English-Trained VLM to French
• Investigating The Functional Roles of Attention Heads in Vision Language Models: Evidence for Reasoning Modules
• Grounding Everything in Tokens for Multimodal Large Language Models
• Mull-Tokens: Modality-Agnostic Latent Thinking
• Visual Species Recognition with Large Multimodal Models as Post-Hoc Correctors
• FlipLLM: Efficient Bit-Flip Attacks on Multimodal LLMs using Reinforcement Learning
• IF-Bench: Benchmarking and Enhancing MLLMs for Infrared Images with Generative Visual Prompting
• Investigate the Low-level Visual Perception in Vision-Language based Image Quality Assessment
• Building Reasonable Inference for Vision-Language Models in Blind Image Quality Assessment
• GAIR: GUI Automation via Information-Joint Reasoning and Group Reflection
• Video-QTR: Query-Driven Temporal Reasoning Framework for Lightweight Video Understanding
• FoundIR-v2: Optimizing Pre-Training Data Mixtures for Image Restoration Foundation Model
• GLACIA: Instance-Aware Positional Reasoning for Glacial Lake Segmentation via Multimodal Large Language Model
• Explaining the Unseen: Multimodal Vision-Language Reasoning for Situational Awareness in Underground Mining Disasters
• Same Content, Different Answers: Cross-Modal Inconsistency in MLLMs
• Unified Diffusion Transformer for High-fidelity Text-Aware Image Restoration
• SATGround: A Spatially-Aware Approach for Visual Grounding in Remote Sensing
• InfiniteVL: Synergizing Linear and Sparse Attention for Highly-Efficient, Unlimited-Input Vision-Language Models
• A Multi-Robot Platform for Robotic Triage Combining Onboard Sensing and Foundation Models
• Towards Lossless Ultimate Vision Token Compression for VLMs
• The Unseen Bias: How Norm Discrepancy in Pre-Norm MLLMs Leads to Visual Information Loss
• PAVAS: Physics-Aware Video-to-Audio Synthesis
• HybridToken-VLM: Hybrid Token Compression for Vision-Language Models
• VisKnow: Constructing Visual Knowledge Base for Object Understanding
• Embodied Tree of Thoughts: Deliberate Manipulation Planning with Embodied World Model
• CVP: Central-Peripheral Vision-Inspired Multimodal Model for Spatial Reasoning
• Instance-Aware Test-Time Segmentation for Continual Domain Shifts
• LUNA: Linear Universal Neural Attention with Generalization Guarantees
• Relational Visual Similarity
• One Layer Is Enough: Adapting Pretrained Visual Encoders for Image Generation
• SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagery
• SAVE: Sparse Autoencoder-Driven Visual Information Enhancement for Mitigating Object Hallucination
• HalluShift++: Bridging Language and Vision through Internal Representation Shifts for Hierarchical Hallucinations in MLLMs
• Venus: An Efficient Edge Memory-and-Retrieval System for VLM-based Online Video Understanding
• All You Need Are Random Visual Tokens? Demystifying Token Pruning in VLLMs
• Toward More Reliable Artificial Intelligence: Reducing Hallucinations in Vision-Language Models
• Zero-Shot Textual Explanations via Translating Decision-Critical Features
• Geo3DVQA: Evaluating Vision-Language Models for 3D Geospatial Reasoning from Aerial Imagery
• START: Spatial and Textual Learning for Chart Understanding
• SPACE: Noise Contrastive Estimation Stabilizes Self-Play Fine-Tuning for Large Language Models
• Think-Reflect-Revise: A Policy-Guided Reflective Framework for Safety Alignment in Large Vision Language Models
• See Once, Then Act: Vision-Language-Action Model with Task Learning from One-Shot Video Demonstrations
• Pay Less Attention to Function Words for Free Robustness of Vision-Language Models
• MulCLIP: A Multi-level Alignment Framework for Enhancing Fine-grained Long-context CLIP
• RVLF: A Reinforcing Vision-Language Framework for Gloss-Free Sign Language Translation
• ParaUni: Enhance Generation in Unified Multimodal Model with Reinforcement-driven Hierarchical Parallel Information Interaction
• NeuroABench: A Multimodal Evaluation Benchmark for Neurosurgical Anatomy Identification
• Less Is More, but Where? Dynamic Token Compression via LLM-Guided Keyframe Prior
• Decouple to Generalize: Context-First Self-Evolving Learning for Data-Scarce Vision-Language Reasoning
• CoT4Det: A Chain-of-Thought Framework for Perception-Oriented Vision-Language Tasks
• The Role of Entropy in Visual Grounding: Analysis and Optimization
• VideoVLA: Video Generators Can Be Generalizable Robot Manipulators
• Scaling Zero-Shot Reference-to-Video Generation
• Personalized Image Descriptions from Attention Sequences
• MedGRPO: Multi-Task Reinforcement Learning for Heterogeneous Medical Video Understanding
• VG-Refiner: Towards Tool-Refined Referring Grounded Reasoning via Agentic Reinforcement Learning
• Unleashing the Intrinsic Visual Representation Capability of Multimodal Large Language Models
• Knowing the Answer Isn't Enough: Fixing Reasoning Path Failures in LVLMs
• WAM-Flow: Parallel Coarse-to-Fine Motion Planning via Discrete Flow Matching for Autonomous Driving
• BeLLA: End-to-End Birds Eye View Large Language Assistant for Autonomous Driving
• ARGUS: Defending Against Multimodal Indirect Prompt Injection via Steering Instruction-Following Behavior
• Evaluating Concept Filtering Defenses against Child Sexual Abuse Material Generation by Text-to-Image Models
• 2K-Characters-10K-Stories: A Quality-Gated Stylized Narrative Dataset with Disentangled Control and Sequence Consistency
• Concept-based Explainable Data Mining with VLM for 3D Detection
• LoC-Path: Learning to Compress for Pathology Multimodal Large Language Models
• Visual Reasoning Tracer: Object-Level Grounded Reasoning Benchmark
• FASTer: Toward Efficient Autoregressive Vision Language Action Modeling via Neural Action Tokenization
• Can Multi-Modal LLMs Provide Live Step-by-Step Task Guidance?
• Qwen3.5-Omni Technical Report
• OmniScaleSR: Unleashing Scale-Controlled Diffusion Prior for Faithful and Realistic Arbitrary-Scale Image Super-Resolution
• Mitigating Object and Action Hallucinations in Multimodal LLMs via Self-Augmented Contrastive Alignment
• VideoMem: Enhancing Ultra-Long Video Understanding via Adaptive Memory Management
• dVLM-AD: Enhance Diffusion Vision-Language-Model for Driving via Controllable Reasoning
• SEASON: Mitigating Temporal Hallucination in Video Large Language Models via Self-Diagnostic Contrastive Decoding
• BiTAgent: A Task-Aware Modular Framework for Bidirectional Coupling between Multimodal Large Language Models and World Models
• Jina-VLM: Small Multilingual Vision Language Model
• C3G: Learning Compact 3D Representations with 2K Gaussians
• DIQ-H: Evaluating Hallucination Persistence in VLMs Under Temporal Visual Degradation
• TempR1: Improving Temporal Understanding of MLLMs via Temporal-Aware Multi-Task Reinforcement Learning
• UniMo: Unifying 2D Video and 3D Human Motion with an Autoregressive Framework
• Tutorial on Large Language Model-Enhanced Reinforcement Learning for Wireless Networks
• Colon-X: Advancing Intelligent Colonoscopy toward Clinical Reasoning
• VAT: Vision Action Transformer by Unlocking Full Representation of ViT
• V-ITI: Mitigating Hallucinations in Multimodal Large Language Models via Visual Inference-Time Intervention
• Text-Printed Image: Bridging the Image-Text Modality Gap for Text-centric Training of Large Vision-Language Models
• YOLOA: Real-Time Affordance Detection via LLM Adapter
• Large Language Models as Generalist Policies for Network Optimization
• Too Late to Recall: Explaining the Two-Hop Problem in Multimodal Knowledge Retrieval
• Hierarchical Process Reward Models are Symbolic Vision Learners
• Mitigating Intra- and Inter-modal Forgetting in Continual Learning of Unified Multimodal Models
• CoT4AD: A Vision-Language-Action Model with Explicit Chain-of-Thought Reasoning for Autonomous Driving
• GeoZero: Incentivizing Reasoning from Scratch on Geospatial Scenes
• MRD: Multi-resolution Retrieval-Detection Fusion for High-Resolution Image Understanding
• PPTBench: Towards Holistic Evaluation of Large Language Models for PowerPoint Layout and Design Understanding
• PGP-DiffSR: Phase-Guided Progressive Pruning for Efficient Diffusion-based Image Super-Resolution
• Does Hearing Help Seeing? Investigating Audio-Video Joint Denoising for Video Generation
• See, Think, Learn: A Self-Taught Multimodal Reasoner
• VACoT: Rethinking Visual Data Augmentation with VLMs
• Understanding and Harnessing Sparsity in Unified Multimodal Models
• VLM-Pruner: Buffering for Spatial Sparsity in an Efficient VLM Centrifugal Token Pruning Paradigm
• Progressive Image Restoration via Text-Conditioned Video Generation
• TUNA: Taming Unified Visual Representations for Native Unified Multimodal Models
• LLM CHESS: Benchmarking Reasoning and Instruction-Following in LLMs through Chess
• PAI-Bench: A Comprehensive Benchmark For Physical AI
• Script: Graph-Structured and Query-Conditioned Semantic Token Pruning for Multimodal Large Language Models
• Med-VCD: Mitigating Hallucination for Medical Large Vision Language Models through Visual Contrastive Decoding
• OpenREAD: Reinforced Open-Ended Reasoning for End-to-End Autonomous Driving with LLM-as-Critic
• Generative Action Tell-Tales: Assessing Human Motion in Synthesized Videos
• DiG-Flow: Discrepancy-Guided Flow Matching for Robust VLA Models
• LEC: Linear Expectation Constraints for Selection-Conditioned Risk Control in Selective Prediction and Routing Systems
• NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Prediction
• MCAT: Scaling Many-to-Many Speech-to-Text Translation with MLLMs to 70 Languages
• S²-MLLM: Boosting Spatial Reasoning Capability of MLLMs for 3D Visual Grounding with Structural Guidance
• ViRectify: A Challenging Benchmark for Video Reasoning Correction with Multimodal Large Language Models
• SocialDriveGen: Generating Diverse Traffic Scenarios with Controllable Social Interactions
• InternVideo-Next: Towards General Video Foundation Models without Video-Text Supervision
• TokenPure: Watermark Removal through Tokenized Appearance and Structural Guidance
• DefenSee: Dissecting Threat from Sight and Text -- A Multi-View Defensive Pipeline for Multi-modal Jailbreaks
• KidSpeak: A General Multi-purpose LLM for Kids' Speech Recognition and Screening
• Assimilation Matters: Model-level Backdoor Detection in Vision-Language Pretrained Models
• LLM2Fx-Tools: Tool Calling For Music Post-Production
• See, Hear, and Understand: Benchmarking Audiovisual Human Speech Understanding in Multimodal Large Language Models
• LISA-3D: Lifting Language-Image Segmentation to 3D via Multi-View Consistency
• PhotoFramer: Multi-modal Image Composition Instruction
• Efficient and Scalable Monocular Human-Object Interaction Motion Reconstruction
• Table as a Modality for Large Language Models
• AFRAgent : An Adaptive Feature Renormalization Based High Resolution Aware GUI agent
• Minimal neuron ablation triggers catastrophic collapse in the language core of Large Vision-Language Models
• Multilingual Training-Free Remote Sensing Image Captioning
• Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning
• BioPro: Towards Difference-Aware Gender Fairness for Vision-Language Models
• DEJIMA: A Novel Large-scale Japanese Dataset for Image Captioning and Visual Question Answering
• Concept-Guided Backdoor Attack on Vision Language Models
• Optimizing LVLMs with On-Policy Data for Effective Hallucination Mitigation
• Low-Bitrate Video Compression through Semantic-Conditioned Diffusion
• Better, Stronger, Faster: Tackling the Trilemma in MLLM-based Segmentation with Simultaneous Textual Mask Prediction
• CourseTimeQA: A Lecture-Video Benchmark and a Latency-Constrained Cross-Modal Fusion Method for Timestamped QA
• ChartPoint: Guiding MLLMs with Grounding Reflection for Chart Reasoning
• BioArc: Discovering Optimal Neural Architectures for Biological Foundation Models
• DialBench: Towards Accurate Reading Recognition of Pointer Meter using Large Foundation Models
• DenseScan: Advancing 3D Scene Understanding with 2D Dense Annotation
• Mammo-FM: Breast-specific foundational model for Integrated Mammographic Diagnosis, Prognosis, and Reporting
• Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Models
• Video-CoM: Interactive Video Reasoning via Chain of Manipulations
• DEAL-300K: Diffusion-based Editing Area Localization with a 300K-Scale Dataset and Frequency-Prompted Baseline
• Optimizing Multimodal Language Models through Attention-based Interpretability
• REVEAL: Reasoning-enhanced Forensic Evidence Analysis for Explainable AI-generated Image Detection
• ReAG: Reasoning-Augmented Generation for Knowledge-based Visual Question Answering
• SpaceMind: Camera-Guided Modality Fusion for Spatial Reasoning in Vision-Language Models
• Buffer replay enhances the robustness of multimodal learning under missing-modality
• HMR3D: Hierarchical Multimodal Representation for 3D Scene Understanding with Large Vision-Language Model
• Contrastive Heliophysical Image Pretraining for Solar Dynamics Observatory Records
• Mechanistic Finetuning of Vision-Language-Action Models via Few-Shot Demonstrations
• Revisiting the Necessity of Lengthy Chain-of-Thought in Vision-centric Reasoning Generalization
• RoadSceneBench: A Lightweight Benchmark for Mid-Level Road Scene Understanding
• Advancing Aesthetic Image Generation via Composition Transfer
• Beyond Real versus Fake Towards Intent-Aware Video Analysis
• GEO-Detective: Unveiling Location Privacy Risks in Images with LLM Agents
• INSIGHT: An Interpretable Neural Vision-Language Framework for Reasoning of Generative Artifacts
• Unexplored flaws in multiple-choice VQA evaluations
• Multi-Crit: Benchmarking Multimodal Judges on Pluralistic Criteria-Following
• VacuumVLA: Boosting VLA Capabilities via a Unified Suction and Gripping Tool for Complex Robotic Manipulation
• SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognition
• SurgMLLMBench: A Multimodal Large Language Model Benchmark Dataset for Surgical Scene Understanding
• Co-Training Vision Language Models for Remote Sensing Multi-task Learning
• GuardTrace-VL: Detecting Unsafe Multimodel Reasoning via Iterative Safety Supervision
• Exploring Diagnostic Prompting Approach for Multimodal LLM-based Visual Complexity Assessment: A Case Study of Amazon Search Result Pages
• TrafficLens: Multi-Camera Traffic Video Analysis Using LLMs
• Towards Reasoning-Preserving Unlearning in Multimodal Large Language Models
• Dynamic Test-Time Compute Scaling in Control Policy: Difficulty-Aware Stochastic Interpolant Policy
• Text-Guided Semantic Image Encoder
• LocateAnything3D: Vision-Language 3D Detection with Chain-of-Sight
• The Consistency Critic: Correcting Inconsistencies in Generated Images via Reference-Guided Attentive Alignment
• AlignBench: Benchmarking Fine-Grained Image-Text Alignment with Synthetic Image-Caption Pairs
• Look Where It Matters: Training-Free Ultra-HR Remote Sensing VQA via Adaptive Zoom Search
• Object-Centric Vision Token Pruning for Vision Language Models
• Thinking in 360°: Humanoid Visual Search in the Wild
• V-Attack: Targeting Disentangled Value Features for Controllable Adversarial Attacks on LVLMs
• Semantic Router: On the Feasibility of Hijacking MLLMs via a Single Adversarial Perturbation
• Boosting Reasoning in Large Multimodal Models via Activation Replay
• CoC-VLA: Delving into Adversarial Domain Transfer for Explainable Autonomous Driving via Chain-of-Causality Visual-Language-Action Model
• VeriSciQA: An Auto-Verified Dataset for Scientific Visual Question Answering
• It Hears, It Sees too: Multi-Modal LLM for Depression Detection By Integrating Visual Understanding into Audio Language Models
• MonoSR: Open-Vocabulary Spatial Reasoning from Monocular Images
• CropVLM: Learning to Zoom for Fine-Grained Vision-Language Perception
• Distilling Counterfactual Reasoning from Language to Vision: Causal Graph Guided Post-Training for Video Understanding
• Harmonious Parameter Adaptation in Continual Visual Instruction Tuning for Safety-Aligned MLLMs
• Fara-7B: An Efficient Agentic Model for Computer Use
• Be My Eyes: Extending Large Language Models to New Modalities Through Multi-Agent Collaboration
• HunyuanOCR Technical Report
• Evaluating Dataset Watermarking for Fine-tuning Traceability of Customized Diffusion Models: A Comprehensive Benchmark and Removal Approach
• Cross Domain Evaluation of Multimodal Chain-of-Thought Reasoning of different datasets into the Amazon CoT Framework
• LAST: LeArning to Think in Space and Time for Generalist Vision-Language Models
• Percept-WAM: Perception-Enhanced World-Awareness-Action Model for Robust End-to-End Autonomous Driving
• RAVEN++: Pinpointing Fine-Grained Violations in Advertisement Videos with Active Reinforcement Reasoning
• Think First, Assign Next (ThiFAN-VQA): A Two-stage Chain-of-Thought Framework for Post-Disaster Damage Assessment
• EEG-VLM: A Hierarchical Vision-Language Model with Multi-Level Feature Alignment and Visually Enhanced Language-Guided Reasoning for EEG Image-Based Sleep Stage Prediction
• Collaborative Learning with Multiple Foundation Models for Source-Free Domain Adaptation
• AVA-VLA: Improving Vision-Language-Action models with Active Visual Attention
• BackdoorVLM: A Benchmark for Backdoor Attacks on Vision-Language Models
• Parallel Vision Token Scheduling for Fast and Accurate Multimodal LMMs Inference
• Understanding Task Transfer in Vision-Language Models
• Thinking Ahead: Foresight Intelligence in MLLMs and World Models
• Yo'City: Personalized and Boundless 3D Realistic City Scene Generation via Self-Critic Expansion
• PhysGS: Bayesian-Inferred Gaussian Splatting for Physical Property Estimation
• Extreme Model Compression for Edge Vision-Language Models: Sparse Temporal Token Fusion and Adaptive Neural Compression
• SineProject: Machine Unlearning for Stable Vision Language Alignment
• DocPTBench: Benchmarking End-to-End Photographed Document Parsing and Translation
• ChineseVideoBench: Benchmarking Multi-modal Large Models for Chinese Video Question Answering
• Synthetic Curriculum Reinforces Compositional Text-to-Image Generation
• MASS: Motion-Aware Spatial-Temporal Grounding for Physics Reasoning and Comprehension in Vision-Language Models
• Weakly-supervised Latent Models for Task-specific Visual-Language Control
• DiVE-k: Differential Visual Reasoning for Fine-grained Image Recognition
• ViMix-14M: A Curated Multi-Source Video-Text Dataset with Long-Form, High-Quality Captions and Crawl-Free Access
• SafeGRPO: Self-Rewarded Multimodal Safety Alignment via Rule-Governed Policy Optimization
• ArtiWorld: LLM-Driven Articulation of 3D Objects in Scenes
• Bias Is a Subspace, Not a Coordinate: A Geometric Rethinking of Post-hoc Debiasing in Vision-Language Models
• VCU-Bridge: Hierarchical Visual Connotation Understanding via Semantic Bridging
• Consolidating Diffusion-Generated Video Detection with Unified Multimodal Forgery Learning
• Multi-speaker Attention Alignment for Multimodal Social Interaction
• SPINE: Token-Selective Test-Time Reinforcement Learning with Entropy-Band Regularization
• MobileVLA-R1: Reinforcing Vision-Language-Action for Mobile Robots
• VITAL: Vision-Encoder-centered Pre-training for LMMs in Visual Quality Assessment
• Show Me: Unifying Instructional Image and Video Generation with Diffusion Models
• Attention Guided Alignment in Efficient Vision-Language Models
• A cross-species neural foundation model for end-to-end speech decoding
• CORA: Consistency-Guided Semi-Supervised Framework for Reasoning Segmentation
• RynnVLA-002: A Unified Vision-Language-Action and World Model
• Video-R4: Reinforcing Text-Rich Video Reasoning with Visual Rumination
• SMILE: A Composite Lexical-Semantic Metric for Question-Answering Evaluation
• Q-MLLM: Vector Quantization for Robust Multimodal Large Language Model Security
• VLA-4D: Embedding 4D Awareness into Vision-Language-Action Models for SpatioTemporally Coherent Robotic Manipulation
• Bridging the Semantic-Action Gap in Visual Token Pruning for Efficient VLA Inference
• Do Vision-Language Models Understand Visual Persuasiveness?
• UAM: A Unified Attention-Mamba Backbone of Multimodal Framework for Tumor Cell Classification
• SPEAR-1: Scaling Beyond Robot Demonstrations via 3D Understanding
• SpatialGeo:Boosting Spatial Reasoning in Multimodal LLMs via Geometry-Semantics Fusion
• VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning
• Revisiting Audio-language Pretraining for Learning General-purpose Audio Representation
• TeamPath: Building MultiModal Pathology Experts with Reasoning AI Copilots
• TimeViper: A Hybrid Mamba-Transformer Vision-Language Model for Efficient Long Video Understanding
• Mantis: A Versatile Vision-Language-Action Model with Disentangled Visual Foresight
• TS-PEFT: Unveiling Token-Level Redundancy in Parameter-Efficient Fine-Tuning
• LLMs-based Augmentation for Domain Adaptation in Long-tailed Food Datasets
• Boosting Medical Visual Understanding From Multi-Granular Language Learning
• An Image Is Worth Ten Thousand Words: Verbose-Text Induction Attacks on VLMs
• Fairness in Multi-modal Medical Diagnosis with Demonstration Selection
• Thinking-while-Generating: Interleaving Textual Reasoning throughout Visual Generation
• Joint Semantic-Channel Coding and Modulation for Token Communications
• HV-Attack: Hierarchical Visual Attack for Multimodal Retrieval Augmented Generation
• Parameter Importance-Driven Continual Learning for Foundation Models
• Octopus: Agentic Multimodal Reasoning with Six-Capability Orchestration
• Instruction-Guided Lesion Segmentation for Chest X-rays with Automatically Generated Large-Scale Dataset
• Multimodal Continual Instruction Tuning with Dynamic Gradient Guidance
• MoDES: Accelerating Mixture-of-Experts Multimodal Large Language Models via Dynamic Expert Skipping
• Look, Zoom, Understand: The Robotic Eyeball for Embodied Perception
• OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Models
• Let Language Constrain Geometry: Vision-Language Models as Semantic and Spatial Critics for 3D Generation
• Orion: A Unified Visual Agent for Multimodal Perception, Advanced Visual Reasoning and Execution
• SMART: Shot-Aware Multimodal Video Moment Retrieval with Audio-Enhanced MLLM
• Stealth Fine-Tuning: Efficiently Breaking Alignment in RVLMs Using Self-Generated CoT
• Jailbreaking Large Vision Language Models in Intelligent Transportation Systems
• Weakly Supervised Ephemeral Gully Detection In Remote Sensing Images Using Vision Language Models
• Learning Skill-Attributes for Transferable Assessment in Video
• VLMs Guided Interpretable Decision Making for Autonomous Driving
• Unlocking the Forgery Detection Potential of Vanilla MLLMs: A Novel Training-Free Pipeline
• Moving Pictures of Thought: Extracting Visual Knowledge in Charles S. Peirce's Manuscripts with Vision-Language Models
• Dual-LoRA and Quality-Enhanced Pseudo Replay for Multimodal Continual Food Learning
• Video Finetuning Improves Reasoning Between Frames
• MMD-Thinker: Adaptive Multi-Dimensional Thinking for Multimodal Misinformation Detection
• ViSS-R1: Self-Supervised Reinforcement Video Reasoning
• From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Models
• Explore More, Learn Better: Parallel MLLM Embeddings under Mutual Information Minimization
• PixelVLA: Advancing Pixel-level Understanding in Vision-Language-Action Model
• Black-Box Membership Inference Attack for LVLMs via Prior Knowledge-Calibrated Memory Probing
• Direct Visual Grounding by Directing Attention of Visual Tokens
• FSDAM: Few-Shot Driving Attention Modeling via Vision-Language Coupling
• AERMANI-VLM: Structured Prompting and Reasoning for Aerial Manipulation with Vision Language Models
• HMVLM: Human Motion-Vision-Lanuage Model via MoE LoRA
• Can large language models be a cardinality estimator? An empirical study
• Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Data
• CoTBox-TTT: Grounding Medical VQA with Visual Chain-of-Thought Boxes During Test-time Training
• SpaceVLM: Sub-Space Modeling of Negation in Vision-Language Models
• RoboAfford++: A Generative AI-Enhanced Dataset for Multimodal Affordance Learning in Robotic Manipulation and Navigation
• Fast Reasoning Segmentation for Images and Videos
• Suppressing VLM Hallucinations with Spectral Representation Filtering
• MediRound: Multi-Round Entity-Level Reasoning Segmentation in Medical Images
• TopoPerception: A Shortcut-Free Evaluation of Global Visual Perception in Large Vision-Language Models
• PAS : Prelim Attention Score for Detecting Object Hallucinations in Large Vision--Language Models
• EcoAlign: An Economically Rational Framework for Efficient LVLM Alignment
• MAFM³: Modular Adaptation of Foundation Models for Multi-Modal Medical AI
• Large Language Models and 3D Vision for Intelligent Robotic Perception and Autonomy
• Hindsight Distillation Reasoning with Knowledge Encouragement Preference for Knowledge-based Visual Question Answering
• VIDEOP2R: Video Understanding from Perception to Reasoning
• Draft and Refine with Visual Experts
• Binary Verification for Zero-Shot Vision
• Viper-F1: Fast and Fine-Grained Multimodal Understanding with Cross-Modal State-Space Modulation
• Towards Trustworthy Dermatology MLLMs: A Benchmark and Multimodal Evaluator for Diagnostic Narratives
• TransactionGPT
• "It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with VLMs
• Anatomy-VLM: A Fine-grained Vision-Language Model for Medical Interpretation
• VectorSynth: Fine-Grained Satellite Image Synthesis with Structured Semantics
• Remodeling Semantic Relationships in Vision-Language Fine-Tuning
• SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards
• VADER: Towards Causal Video Anomaly Understanding with Relation-Aware Large Language Models
• NOVO: Bridging LLaVA and SAM with Visual-only Prompts for Reasoning Segmentation
• How Do VLAs Effectively Inherit from VLMs?
• MVU-Eval: Towards Multi-Video Understanding Evaluation for Multimodal LLMs
• Rep2Text: Decoding Full Text from a Single LLM Token Representation
• Interaction-Centric Knowledge Infusion and Transfer for Open-Vocabulary Scene Graph Generation
• MoRA: Missing Modality Low-Rank Adaptation for Visual Recognition
• VLDrive: Vision-Augmented Lightweight MLLMs for Efficient Language-grounded Autonomous Driving
• S²LM: Towards Semantic Steganography via Large Language Models
• LiveStar: Live Streaming Assistant for Real-World Online Video Understanding
• TwinVLA: Data-Efficient Bimanual Manipulation with Twin Single-Arm Vision-Language-Action Models
• Medical Referring Image Segmentation via Next-Token Mask Prediction
• DeepEyesV2: Toward Agentic Multimodal Model
• Visual Spatial Tuning
• Cambrian-S: Towards Spatial Supersensing in Video
• SIMS-V: Simulated Instruction-Tuning for Spatial Video Understanding
• IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs
• Embodiment Transfer Learning for Vision-Language-Action Models
• Seeing Straight: Document Orientation Detection for Efficient OCR
• An LLM-based Framework for Human-Swarm Teaming Cognition in Disaster Search and Rescue
• Part-Aware Bottom-Up Group Reasoning for Fine-Grained Social Interaction Detection
• Seeing What You Say: Expressive Image Generation from Speech
• Fine-Tuning Vision-Language Models for Multimodal Polymer Property Prediction
• XR-1: Towards Versatile Vision-Language-Action Models via Learning Unified Vision-Motion Representations
• Can Visual Input Be Compressed? A Visual Token Compression Benchmark for Large Multimodal Models
• TRACE: Textual Reasoning for Affordance Coordinate Extraction
• In-Context Adaptation of VLMs for Few-Shot Cell Detection in Optical Microscopy
• SAIL-RL: Guiding MLLMs in When and How to Think via Dual-Reward RL Tuning
• LACY: A Vision-Language Model-based Language-Action Cycle for Self-Improving Robotic Manipulation
• Personalized Decision Modeling: Utility Optimization or Textualized-Symbolic Reasoning
• When One Modality Sabotages the Others: A Diagnostic Lens on Multimodal Reasoning
• UniChange: Unifying Change Detection with Multimodal Large Language Model
• OmniVLA: Physically-Grounded Multimodal VLA with Unified Multi-Sensor Perception for Robotic Manipulation
• GeoToken: Hierarchical Geolocalization of Images via Next Token Prediction
• VesSAM: Efficient Multi-Prompting for Segmenting Complex Vessel
• Fleming-VL: Towards Universal Medical Visual Reasoning with Multimodal LLMs
• GraphGeo: Multi-Agent Debate Framework for Visual Geo-localization with Heterogeneous Graph Neural Networks
• Leveraging Hierarchical Image-Text Misalignment for Universal Fake Image Detection
• UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddings
• Rethinking Facial Expression Recognition in the Era of Multimodal Large Language Models: Benchmark, Datasets, and Beyond
• Text-guided Fine-Grained Video Anomaly Understanding
• LGCA: Enhancing Semantic Representation via Progressive Expansion
• FedReplay: A Feature Replay Assisted Federated Transfer Learning Framework for Efficient and Privacy-Preserving Smart Agriculture
• CompAgent: An Agentic Framework for Visual Compliance Verification
• Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challenges
• FLoC: Facility Location-Based Efficient Visual Token Compression for Long Video Understanding
• RzenEmbed: Towards Comprehensive Multimodal Retrieval
• ODP-Bench: Benchmarking Out-of-Distribution Performance Prediction
• Languages are Modalities: Cross-Lingual Alignment via Encoder Injection
• Generating Accurate and Detailed Captions for High-Resolution Images
• NAUTILUS: A Large Multimodal Model for Underwater Scene Understanding
• MM-OPERA: Benchmarking Open-ended Association Reasoning for Large Vision-Language Models
• ChartAB: A Benchmark for Chart Grounding & Dense Alignment
• Do Vision-Language Models Measure Up? Benchmarking Visual Measurement Reading with MeasureBench
• CATCH: A Modular Cross-domain Adaptive Template with Hook
• Towards Fine-Grained Vision-Language Alignment for Few-Shot Anomaly Detection
• ConceptScope: Characterizing Dataset Bias via Disentangled Visual Concepts
• Dynamic VLM-Guided Negative Prompting for Diffusion Models
• Robotic Assistant: Completing Collaborative Tasks with Dexterous Vision-Language-Action Models
• PureKV: Plug-and-Play KV Cache Optimization with Spatial-Temporal Sparse Attention for Vision-Language Large Models
• FlowMM: Cross-Modal Information Flow Guided KV Cache Merging for Efficient Multimodal Context Inference
• Instruction-based image editing: a survey on data, models, evaluation, and applications
• OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models
• MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities
• Rad-JEPA 3D: Radiology Joint-Embedding Predictive Model for 3D Computed Tomography
• Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models
• TS-Reasoner: Aligning Time Series Foundation Models with LLM Reasoning
• InstructPLM-mu: 1-Hour Fine-Tuning of ESM2 Beats ESM3 in Protein Mutation Predictions
• Don't Just Chase "Highlighted Tokens" in MLLMs: Revisiting Visual Holistic Context Retention
• MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learning
• Mitigating Modal Imbalance in Multimodal Reasoning
• Prior Directions: Why GUI Grounding Gets Locked in the Past
• Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation
• Compressed Video Aggregator: Content-driven Module for Efficient Micro-Video Recommendation
• Facial-Expression-Aware Prompting for Empathetic LLM Tutoring
• Deep Expert Injection for Anchoring Retinal VLMs with Domain-Specific Knowledge
• VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMs
• SHIELD: Suppressing Hallucinations In LVLM Encoders via Bias and Vulnerability Defense
• RL makes MLLMs see better than SFT
• PRISMM-Bench: A Benchmark of Peer-Review Grounded Multimodal Inconsistencies
• EDVD-LLaMA: Explainable Deepfake Video Detection via Multimodal Large Language Model Reasoning
• Large Language Model for Verilog Code Generation: Literature Review and the Road Ahead
• EnzyControl: Adding Functional and Substrate-Specific Control for Enzyme Backbone Generation
• NanoVLA: Routing Decoupled Vision-Language Understanding for Nano-sized Generalist Robotic Policies
• FruitProm: Probabilistic Maturity Estimation and Detection of Fruits and Vegetables
• OpenLVLM-MIA: A Controlled Benchmark Revealing the Limits of Membership Inference Attacks on Large Vision-Language Models
• QSVD: Efficient Low-rank Approximation for Unified Query-Key-Value Weight Compression in Low-Precision Vision-Language Models
• REALM: An MLLM-Agent Framework for Open World 3D Reasoning Segmentation and Editing on Gaussian Splatting
• Kernelized Sparse Fine-Tuning with Bi-level Parameter Competition for Vision Models
• SCOPE: Saliency-Coverage Oriented Token Pruning for Efficient Multimodel LLMs
• BLM₁: A Boundless Large Model for Cross-Space, Cross-Task, and Cross-Embodiment Learning
• Enhancing Vision-Language Models for Autonomous Driving through Task-Specific Prompting and Spatial Reasoning
• SafeVision: Efficient Image Guardrail with Robust Policy Adherence and Explainability
• Improving Visual Discriminability of CLIP for Training-Free Open-Vocabulary Semantic Segmentation
• Invoice Information Extraction: Methods and Performance Evaluation
• RAVEN: Robust Advertisement Video Violation Temporal Grounding via Reinforcement Reasoning
• Modeling Expert Interactions in Sparse Mixture of Experts via Graph Structures
• Decentralized Multi-Agent Goal Assignment for Path Planning using Large Language Models
• Explainable Detection of AI-Generated Images with Artifact Localization Using Faster-Than-Lies and Vision-Language Models for Edge Devices
• RoboOmni: Proactive Robot Manipulation in Omni-modal Context
• PixelRefer: A Unified Framework for Spatio-Temporal Object Referring with Arbitrary Granularity
• A Survey on Efficient Vision-Language-Action Models
• Lookahead Anchoring: Preserving Character Identity in Audio-Driven Human Animation
• UrbanVLA: A Vision-Language-Action Model for Urban Micromobility
• More Than Generation: Unifying Generation and Depth Estimation via Text-to-Image Diffusion Models
• Dexbotic: Open-Source Vision-Language-Action Toolbox
• MergeMix: A Unified Augmentation Paradigm for Visual and Multi-Modal Understanding
• Large language model-based task planning for service robots: A review
• Positional Preservation Embedding for Multimodal Large Language Models
• LLM-based Fusion of Multi-modal Features for Commercial Memorability Prediction
• Understanding What Is Not Said:Referring Remote Sensing Image Segmentation with Scarce Expressions
• Windsock is Dancing: Adaptive Multimodal Retrieval-Augmented Generation
• RoboSVG: A Unified Framework for Interactive SVG Generation with Multi-modal Guidance
• SRSR: Enhancing Semantic Accuracy in Real-World Image Super-Resolution with Spatially Re-Focused Text-Conditioning
• DynaSolidGeo: A Dynamic Benchmark for Genuine Spatial Mathematical Reasoning of VLMs in Solid Geometry
• HARMONY: Hidden Activation Representations and Model Output-Aware Uncertainty Estimation for Vision-Language Models
• Mitigating Coordinate Prediction Bias from Positional Encoding Failures
• Capturing Gaze Shifts for Guidance: Cross-Modal Fusion Enhancement for VLM Hallucination Mitigation
• OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLM
• Integrating Genomics into Multimodal EHR Foundation Models
• Head Pursuit: Probing Attention Specialization in Multimodal Transformers
• FineRS: Fine-grained Reasoning and Segmentation of Small Objects with Reinforcement Learning
• REMONI: An Autonomous System Integrating Wearables and Multimodal Large Language Models for Enhanced Remote Health Monitoring
• Vision Language Models for Dynamic Human Activity Recognition in Healthcare Settings
• VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Set
• Enhanced MLLM Black-Box Jailbreaking Attacks and Defenses
• NoisyGRPO: Incentivizing Multimodal CoT Reasoning via Noise Injection and Bayesian Estimation
• SafetyPairs: Isolating Safety Critical Image Features with Counterfactual Image Generation
• PhysVLM-AVR: Active Visual Reasoning for Multimodal Large Language Models in Physical Environments
• Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videos
• LLM-Integrated Bayesian State Space Models for Multimodal Time-Series Forecasting
• ARGenSeg: Image Segmentation with Autoregressive Image Generation Model
• C-NAV: Towards Self-Evolving Continual Object Navigation in Open World
• GranViT: A Fine-Grained Vision Model With Autoregressive Perception For MLLMs
• HyperET: Efficient Training in Hyperbolic Space for Multi-modal Large Language Models
• MCA: Modality Composition Awareness for Robust Composed Multimodal Retrieval
• BioCAP: Exploiting Synthetic Captions Beyond Labels in Biological Foundation Models
• StableSketcher: Enhancing Diffusion Model for Pixel-based Sketch Generation via Visual Question Answering Feedback
• Fake-in-Facext: Towards Fine-Grained Explainable DeepFake Analysis
• Mixing Importance with Diversity: Joint Optimization for KV Cache Compression in Large Vision-Language Models
• From Masks to Worlds: A Hitchhiker's Guide to World Models
• Better Tokens for Better 3D: Advancing Vision-Language Modeling in 3D Medical Imaging
• GenColorBench: A Color Evaluation Benchmark for Text-to-Image Generation Models
• TowerVision: Understanding and Improving Multilinguality in Vision-Language Models
• CoSense-LLM: Semantics at the Edge with Cost- and Uncertainty-Aware Cloud-Edge Cooperation
• From Forecasting to Planning: Policy World Model for Collaborative State-Action Prediction
• Modal Aphasia: Can Unified Multimodal Models Describe Images From Memory?
• Decomposed Attention Fusion in MLLMs for Training-Free Video Reasoning Segmentation
• GigaBrain-0: A World Model-Powered Vision-Language-Action Model
• From Denoising to Refining: A Corrective Framework for Vision-Language Diffusion Model
• PruneHal: Reducing Hallucinations in Multi-modal Large Language Models through Adaptive KV Cache Pruning
• Tibetan Language and AI: A Comprehensive Survey of Resources, Methods and Challenges
• See, Think, Act: Online Shopper Behavior Simulation with VLM Agents
• Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes
• Exploring a Unified Vision-Centric Contrastive Alternatives on Multi-Modal Web Documents
• Automated urban waterlogging assessment and early warning through a mixture of foundation models
• Beyond Single Models: Mitigating Multimodal Hallucinations via Adaptive Token Ensemble Decoding
• Learning with Dual-level Noisy Correspondence for Multi-modal Entity Alignment
• VLSU: Mapping the Limits of Joint Multimodal Understanding for AI Safety
• Grasp Any Region: Towards Precise, Contextual Pixel Understanding for Multimodal LLMs
• VFM-VAE: Vision Foundation Models Can Be Good Tokenizers for Latent Diffusion Models
• SafeCoop: Unravelling Full Stack Safety in Agentic Collaborative Driving
• Accelerating Vision Transformers with Adaptive Patch Sizes
• Unbiased Gradient Low-Rank Projection
• SparseVILA: Decoupling Visual Sparsity for Efficient VLM Inference
• VERA-V: Variational Inference Framework for Jailbreaking Vision-Language Models
• Multilingual Text-to-Image Person Retrieval via Bidirectional Relation Reasoning and Aligning
• Token-Level Inference-Time Alignment for Vision-Language Models
• Multimodal Safety Is Asymmetric: Cross-Modal Exploits Unlock Black-Box MLLMs Jailbreaks
• FineVision: Open Data Is All You Need
• SimpleVSF: VLM-Scoring Fusion for Trajectory Prediction of End-to-End Autonomous Driving
• AION-1: Omnimodal Foundation Model for Astronomical Sciences
• MERIT: Modular Framework for Multimodal Misinformation Detection with Web-Grounded Reasoning
• Xiaoice: Training-Free Video Understanding via Self-Supervised Spatio-Temporal Clustering of Semantic Features
• From Spatial to Actions: Grounding Vision-Language-Action Model in Spatial Foundation Priors
• ZSPAPrune: Zero-Shot Prompt-Aware Token Pruning for Vision-Language Models
• Efficient Vision-Language-Action Models for Embodied Manipulation: A Systematic Survey
• SAKE: Towards Editing Auditory Attribute Knowledge of Large Audio-Language Models
• SceneCOT: Eliciting Grounded Chain-of-Thought Reasoning in 3D Scenes
• Segmentation as A Plug-and-Play Capability for Frozen Multimodal LLMs
• MultiVerse: A Multi-Turn Conversation Benchmark for Evaluating Large Vision and Language Models
• AUGUSTUS: An LLM-Driven Multimodal Agent System with Contextualized User Memory
• Train a Unified Multimodal Data Quality Classifier with Synthetic Data
• Comprehensive language-image pre-training for 3D medical image understanding
• Composition-Grounded Instruction Synthesis for Visual Reasoning
• From Pixels to Words -- Towards Native Vision-Language Primitives at Scale
• Learning an Image Editing Model without Image Editing Pairs
• ChangingGrounding: 3D Visual Grounding in Changing Scenes
• CaptionFormer: Unified Segmentation, Tracking, and Captioning for Spatio-Temporal Objects
• VLA²: Empowering Vision-Language-Action Models with an Agentic Framework for Unseen Concept Manipulation
• You May Speak Freely: Improving the Fine-Grained Visual Recognition Capabilities of Multimodal Large Language Models with Answer Extraction
• Benchmarking Multimodal Large Language Models for Face Recognition
• Backdoor Unlearning by Linear Task Decomposition
• Cross-Scenario Unified Modeling of User Interests at Billion Scale
• xLLM Technical Report
• GOPLA: Generalizable Object Placement Learning via Synthetic Augmentation of Human Arrangement
• Efficient Video Sampling: Pruning Temporally Redundant Tokens for Faster VLM Inference
• Oracle-RLAIF: An Improved Fine-Tuning Framework for Multi-modal Video Models using Reinforcement Learning from Ranking Feedback
• Exploring Cross-Modal Flows for Few-Shot Learning
• Hi-Agent: Hierarchical Vision-Language Agents for Mobile Device Control
• Can MLLMs Absorb Math Reasoning Abilities from LLMs as Free Lunch?
• Spatial Preference Rewarding for MLLMs Spatial Understanding
• Vision-Centric Activation and Coordination for Multimodal Large Language Models
• IAD-GPT: Advancing Visual Knowledge in Multimodal Large Language Model for Industrial Anomaly Detection
• Sequential Comics for Jailbreaking Multimodal Large Language Models via Structured Visual Storytelling
• Vgent: Graph-based Retrieval-Reasoning-Augmented Generation For Long Video Understanding
• Efficient Few-Shot Learning in Remote Sensing: Fusing Vision and Vision-Language Models
• InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialogue
• InfraGPT Smart Infrastructure: An End-to-End VLM-Based Framework for Detecting and Managing Urban Defects
• End-to-End Multi-Modal Diffusion Mamba
• Map the Flow: Revealing Hidden Pathways of Information in VideoLLMs
• OralGPT: A Two-Stage Vision-Language Model for Oral Mucosal Disease Diagnosis and Description
• Rethinking the Simulation vs. Rendering Dichotomy: No Free Lunch in Spatial World Modelling
• Self-Aug: Query and Entropy Adaptive Decoding for Large Vision-Language Models
• Model-agnostic Adversarial Attack and Defense for Vision-Language-Action Models
• The Mechanistic Emergence of Symbol Grounding in Language Models
• Generative Universal Verifier as Multimodal Meta-Reasoner
• VisCoP: Visual Probing for Video Domain Adaptation of Vision Language Models
• NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matching
• Vision Language Models Map Logos to Text via Semantic Entanglement in the Visual Projector
• Reflection-Based Task Adaptation for Self-Improving VLA
• Towards Cross-Modal Error Detection with Tables and Images
• VideoLucy: Deep Memory Backtracking for Long Video Understanding
• Evolution of meta's llama models and parameter-efficient fine-tuning of large language models: a survey
• MetaCaptioner: Towards Generalist Visual Captioning with Open-source Suites
• ImageSentinel: Protecting Visual Datasets from Unauthorized Retrieval-Augmented Image Generation
• Unifying Vision-Language Latents for Zero-label Image Caption Enhancement
• SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models
• ViCO: A Training Strategy towards Semantic Aware Dynamic High-Resolution
• VQArt-Bench: A semantically rich VQA Benchmark for Art and Cultural Heritage
• Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoning
• Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Model
• UALM: Unified Audio Language Model for Understanding, Generation and Reasoning
• Evaluating Open-Source Vision-Language Models for Multimodal Sarcasm Detection
• Scaling Language-Centric Omnimodal Representation Learning
• InternSVG: Towards Unified SVG Tasks with Multimodal Large Language Models
• REACT3D: Recovering Articulations for Interactive Physical 3D Scenes
• CoPRS: Learning Positional Prior from Chain-of-Thought for Reasoning Segmentation
• LSVOS 2025 Challenge Report: Recent Advances in Complex Video Object Segmentation
• CoDefend: Cross-Modal Collaborative Defense via Diffusion Purification and Prompt Optimization
• Enhancing Zero-Shot Anomaly Detection: CLIP-SAM Collaboration with Cascaded Prompts
• Vlaser: Vision-Language-Action Model with Synergistic Embodied Reasoning
• A Survey on Agentic Multimodal Large Language Models
• Topological Alignment of Shared Vision-Language Embedding Space
• Prompt-Guided Spatial Understanding with RGB-D Transformers for Fine-Grained Object Relation Reasoning
• FG-CLIP 2: A Bilingual Fine-grained Vision-Language Alignment Model
• RefineShot: Rethinking Cinematography Understanding with Foundational Skill Evaluation
• OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs
• Image-to-Video Transfer Learning based on Image-Language Foundation Models: A Comprehensive Survey
• ViSurf: Visual Supervised-and-Reinforcement Fine-Tuning for Large Vision-and-Language Models
• Large Language Model-Empowered Channel Prediction and Predictive Beamforming for LEO Satellite Communications
• Unified Open-World Segmentation with Multi-Modal Prompts
• Towards Self-Refinement of Vision-Language Models with Triangular Consistency
• X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model
• Color3D: Controllable and Consistent 3D Colorization with Personalized Colorizer
• Think Twice to See More: Iterative Visual Reasoning in Medical VLMs
• Complementary and Contrastive Learning for Audio-Visual Segmentation
• Efficient Onboard Vision-Language Inference in UAV-Enabled Low-Altitude Economy Networks via LLM-Enhanced Optimization
• MIMO: A medical vision language model with visual referring multimodal input and pixel grounding multimodal output
• RAG-IGBench: Innovative Evaluation for RAG-based Interleaved Generation in Open-domain Question Answering
• Patch-as-Decodable-Token: Towards Unified Multi-Modal Vision Tasks in MLLMs
• Tapered Language Models
• Nav-EE: Navigation-Guided Early Exiting for Efficient Vision-Language Models in Autonomous Driving
• CoIDO: Efficient Data Selection for Visual Instruction Tuning via Coupled Importance-Diversity Optimization
• Holistic Order Prediction in Natural Scenes
• MedQ-Bench: Evaluating and Exploring Medical Image Quality Assessment Abilities in MLLMs
• VITA-VLA: Efficiently Teaching Vision-Language Models to Act via Action Expert Distillation
• Interpretable Generative and Discriminative Learning for Multimodal and Incomplete Clinical Data
• Entropy Meets Importance: A Unified Head Importance-Entropy Score for Stable and Efficient Transformer Pruning
• Zero-shot image privacy classification with Vision-Language Models
• RO-Bench: Large-scale robustness evaluation of MLLMs with text-driven counterfactual videos
• Constructive Distortion: Improving MLLMs with Attention-Guided Image Warping
• Provable Training Data Identification for Large Language Models
• LM Fight Arena: Benchmarking Large Multimodal Models via Game Competition
• HandEval: Taking the First Step Towards Hand Quality Evaluation in Generated Images
• VisuoAlign: Safety Alignment of LVLMs with Multimodal Tree Search
• BLINK-Twice: You see, but do you observe? A Reasoning Benchmark on Visual Perception
• Multimodal Prompt Optimization: Why Not Leverage Multiple Modalities for MLLMs
• Q-Router: Agentic Video Quality Assessment with Expert Model Routing and Artifact Localization
• NovaFlow: Zero-Shot Manipulation via Actionable Flow from Generated Videos
• MATRIX: Multimodal Agent Tuning for Robust Tool-Use Reasoning
• How to Teach Large Multimodal Models New Skills
• Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generation
• To Sink or Not to Sink: Visual Information Pathways in Large Vision-Language Models
• Evaluating Small Vision-Language Models on Distance-Dependent Traffic Perception
• LTCA: Long-range Temporal Context Attention for Referring Video Object Segmentation
• UniMMVSR: A Unified Multi-Modal Framework for Cascaded Video Super-Resolution
• MARC: Memory-Augmented RL Token Compression for Efficient Video Understanding
• Towards Proprioception-Aware Embodied Planning for Dual-Arm Humanoid Robots
• IntentionVLA: Generalizable and Efficient Embodied Intention Reasoning for Human-Robot Interaction
• Multimodal Safety Evaluation in Generative Agent Social Simulations
• Towards Unified World Models for Visual Navigation via Memory-Augmented Planning and Foresight
• NaViL: Rethinking Scaling Properties of Native Multimodal Large Language Models under Data Constraints
• VisualDAN: Exposing Vulnerabilities in VLMs with Visual-Driven DAN Commands
• CIR-CoT: Towards Interpretable Composed Image Retrieval via End-to-End Chain-of-Thought Reasoning
• RetouchLLM: Training-free Code-based Image Retouching with Vision Language Models
• VideoCanvas: Unified Video Completion from Arbitrary Spatiotemporal Patches via In-Context Conditioning
• SciVideoBench: Benchmarking Scientific Video Reasoning in Large Multimodal Models
• MLLM4TS: Leveraging Vision and Multimodal Language Models for General Time-Series Analysis
• TrackVLA++: Unleashing Reasoning and Memory Capabilities in VLA Models for Embodied Visual Tracking
• Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
• ImageNet-Think-250K: A Large-Scale Synthetic Dataset for Multimodal Reasoning for Vision Language Models
• Growing Visual Generative Capacity for Pre-Trained MLLMs
• Efficient Discriminative Joint Encoders for Large Scale Vision-Language Reranking
• TTRV: Test-Time Reinforcement Learning for Vision Language Models
• StaR-KVQA: Structured Reasoning Traces for Implicit-Knowledge Visual Question Answering
• VLA-R1: Enhancing Reasoning in Vision-Language-Action Models
• Get RICH or Die Scaling: Profitably Trading Inference Compute for Robustness
• AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs
• SpotDiff: Spotting and Disentangling Interference in Feature Space for Subject-Preserving Image Generation
• Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understanding
• Deforming Videos to Masks: Flow Matching for Referring Video Segmentation
• Detection and Measurement of Hailstones with Multimodal Large Language Models
• Gradient-Sign Masking for Task Vector Transport Across Pre-Trained Models
• VCoT-Grasp: Grasp Foundation Models with Visual Chain-of-Thought Reasoning for Language-driven Grasp Generation
• Syn-Diag: An LLM-based Synergistic Framework for Generalizable Few-shot Fault Diagnosis on the Edge
• SD-MVSum: Script-Driven Multimodal Video Summarization Method and Datasets
• Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Models
• Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AI
• Bridging Reasoning to Learning: Unmasking Illusions using Complexity Out of Distribution Generalization
• Beyond the Seen: Bounded Distribution Estimation for Open-Vocabulary Learning
• ContextNav: Towards Agentic Multimodal In-Context Learning
• Activation Quantization of Vision Encoders Needs Prefixing Registers
• More Than Meets the Eye? Uncovering the Reasoning-Planning Disconnect in Training Vision-Language Driving Models
• VaseVQA-3D: Benchmarking 3D VLMs on Ancient Greek Pottery
• Contrastive Representation Regularization for Vision-Language-Action Models
• Your Vision-Language Model Can't Even Count to 20: Exposing the Failures of VLMs in Compositional Counting
• A.I.R.: Enabling Adaptive, Iterative, and Reasoning-based Frame Selection For Video Question Answering
• Zephyrus: An Agentic Framework for Weather Science
• AgentTypo: Adaptive Typographic Prompt Injection Attacks against Black-box Multimodal Agents
• Learning from All: Concept Alignment for Autonomous Distillation from Multiple Drifting MLLMs
• RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learning
• AgriGPT-VL: Agricultural Vision-Language Understanding Suite
• SITCOM: Scaling Inference-Time COMpute for VLAs
• RetiBridge: Bridging Quantitative Retinal Biomarkers and Qualitative Diagnosis with a Knowledge-Guided Multimodal Large Language Model
• MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering
• ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Context
• Spatial CAPTCHA: Generatively Benchmarking Spatial Reasoning for Human-Machine Differentiation
• UGround: Towards Unified Visual Grounding with Unrolled Transformers
• Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Models
• Efficient Test-Time Scaling for Small Vision-Language Models
• Improving GUI Grounding with Explicit Position-to-Coordinate Mapping
• Multimodal Carotid Risk Stratification with Large Vision-Language Models: Benchmarking, Fine-Tuning, and Clinical Insights
• Retrv-R1: A Reasoning-Driven MLLM Framework for Universal and Efficient Multimodal Retrieval
• Brain-Language Model Alignment: Insights into the Platonic Hypothesis and Intermediate-Layer Advantage
• Backdoor Attacks Against Speech Language Models
• TextCAM: Explaining Class Activation Map with Text
• Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs
• Plug-and-Play Prompt Refinement via Latent Feedback for Diffusion Model Alignment
• PAL-UI: Planning with Active Look-back for Vision-Based GUI Agents
• TsLLM: Augmenting LLMs for General Time Series Understanding and Prediction
• Efficient Multi-modal Large Language Models via Progressive Consistency Distillation
• Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed
• Data Selection for Fine-tuning Vision Language Models via Cross Modal Alignment Trajectories
• BioVERSE: Representation Alignment of Biomedical Modalities to LLMs for Multi-Modal Reasoning
• Glaucoma Detection and Structured OCT Report Generation via a Fine-tuned Multimodal Large Language Model
• Apriel-1.5-15b-Thinker
• Retrieval-Augmented Generation for Electrocardiogram-Language Models
• Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training
• Stable Cinemetrics : Structured Taxonomy and Evaluation for Professional Video Generation
• Attention over Scene Graphs: Indoor Scene Representations Toward CSAI Classification
• Catalog-Native LLM: Speaking Item-ID Dialect with Less Entanglement for Recommendation
• EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing
• ProfVLM: A Lightweight Video-Language Model for Multi-View Proficiency Estimation
• IMG: Calibrating Diffusion Models via Implicit Multimodal Guidance
• Human-MME: A Holistic Evaluation Benchmark for Human-Centric Multimodal Large Language Models
• OWL: Geometry-Aware Spatial Reasoning for Audio Large Language Models
• PatchVSR: Breaking Video Diffusion Resolution Limits with Patch-wise Video Super-Resolution
• Learning Egocentric In-Hand Object Segmentation through Weak Supervision from Human Narrations
• NuRisk: A Visual Question Answering Dataset for Agent-Level Risk Assessment in Autonomous Driving
• Understanding the Mixture-of-Experts with Nadaraya-Watson Kernel
• Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations
• Logo-VGR: Visual Grounded Reasoning for Open-world Logo Recognition
• Self-Evolving Vision-Language Models for Image Quality Assessment via Voting and Ranking
• Importance Sampling for Multi-Negative Multimodal Direct Preference Optimization
• OmniNav: A Unified Framework for Prospective Exploration and Visual-Language Navigation
• dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thought
• DescribeEarth: Describe Anything for Remote Sensing Images
• Effective Model Pruning
• Towards Reliable and Holistic Visual In-Context Learning Prompt Selection
• PRPO: Paragraph-level Policy Optimization for Vision-Language Deepfake Detection
• VLM-FO1: Bridging the Gap Between High-Level Reasoning and Fine-Grained Perception in VLMs
• Judging by Appearances? Auditing and Intervening Vision-Language Models for Bail Prediction
• Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play
• Defeating Cerberus: Concept-Guided Privacy-Leakage Mitigation in Multimodal Language Models
• Saliency Guided Longitudinal Medical Visual Question Answering
• From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models
• VideoAnchor: Reinforcing Subspace-Structured Visual Cues for Coherent Visual-Spatial Reasoning
• LayerD: Decomposing Raster Graphic Designs into Layers
• MGM-Omni: Scaling Omni LLMs to Personalized Long-Horizon Speech
• OIG-Bench: A Multi-Agent Annotated Benchmark for Multimodal One-Image Guides Understanding
• From Code to Action: Hierarchical Learning of Diffusion-VLM Policies
• Training-Free Token Pruning via Zeroth-Order Gradient Estimation in Vision-Language Models
• Performance-Efficiency Trade-off for Fashion Image Retrieval
• Uni-X: Mitigating Modality Conflict with a Two-End-Separated Architecture for Unified Multimodal Models
• When MLLMs Meet Compression Distortion: A Coding Paradigm Tailored to MLLMs
• Latent Visual Reasoning
• World-Env: Leveraging World Model as a Virtual Environment for VLA Post-Training
• Skip-It? Theoretical Conditions for Layer Skipping in Vision-Language Models
• DepthLM: Metric Depth From Vision Language Models
• Euclid's Gift: Enhancing Spatial Perception and Reasoning in Vision-Language Models via Geometric Surrogate Tasks
• Mitigating Visual Hallucinations via Semantic Curriculum Preference Optimization in MLLMs
• FreeRet: MLLMs as Training-Free Retrievers
• GHOST: Hallucination-Inducing Image Generation for Multimodal LLMs
• GeoVLM-R1: Reinforcement Fine-Tuning for Improved Remote Sensing Reasoning
• VisualOverload: Probing Visual Understanding of VLMs in Really Dense Scenes
• FrameMind: Frame-Interleaved Video Reasoning via Reinforcement Learning
• Towards Redundancy Reduction in Diffusion Models for Efficient Video Super-Resolution
• Vision-Grounded Machine Interpreting: Improving the Translation Process through Visual Cues
• ColLab: A Collaborative Spatial Progressive Data Engine for Referring Expression Comprehension and Generation
• AutoPrune: Each Complexity Deserves a Pruning Policy
• Uni4D-LLM: A Unified SpatioTemporal-Aware VLM for 4D Understanding and Generation
• GUI-Shepherd: Reliable Process Reward and Verification for Long-Sequence GUI Tasks
• HomeSafeBench: A Benchmark for Embodied Vision-Language Models in Free-Exploration Home Safety Inspection
• PreScope: Unleashing the Power of Prefetching for Resource-Constrained MoE Inference
• VMDiff: Visual Mixing Diffusion for Limitless Cross-Object Synthesis
• HunyuanImage 3.0 Technical Report
• Assessing Visual Privacy Risks in Multimodal AI: A Novel Taxonomy-Grounded Evaluation of Vision-Language Models
• Uncovering Grounding IDs: How External Cues Shape Multimodal Binding
• RIV: Recursive Introspection Mask Diffusion Vision Language Model
• Multi-modal Data Spectrum: Multi-modal Datasets are Multi-dimensional
• LOTFormer: Doubly-Stochastic Linear Attention via Low-Rank Optimal Transport
• DentVLM: A Multimodal Vision-Language Model for Comprehensive Dental Diagnosis and Enhanced Clinical Practice
• Tree Reward-Aligned Search for TReASURe in Masked Diffusion Language Models
• Self-Consistency as a Free Lunch: Reducing Hallucinations in Vision-Language Models via Self-Reflection
• LAGEA: Language Guided Embodied Agents for Robotic Manipulation
• Uncovering Intrinsic Capabilities: A Paradigm for Data Curation in Vision-Language Models
• Understanding Language Prior of LVLMs by Contrasting Chain-of-Embedding
• Patient-specific Biomolecular Instruction Tuning
• MMPB: It's Time for Multi-Modal Personalization
• CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning
• UniMIC: Token-Based Multimodal Interactive Coding for Human-AI Collaboration
• Explaining multimodal LLMs via intra-modal token interactions
• RAU: Reference-based Anatomical Understanding with Vision Language Models
• RoboView-Bias: Benchmarking Visual Bias in Embodied Agents for Robotic Manipulation
• MultiMat: Multimodal Program Synthesis for Procedural Materials using Large Multimodal Models
• WAVE: Learning Unified & Versatile Audio-Visual Embeddings with Multimodal LLM
• From Bias to Balance: Exploring and Mitigating Spatial Bias in LVLMs
• Benchmarking and Mitigating Sycophancy in Medical Vision Language Models
• Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach
• DynaNav: Dynamic Feature and Layer Selection for Efficient Visual Navigation
• Spatial Reasoning in Foundation Models: Benchmarking Object-Centric Spatial Understanding
• Discrete Guidance Matching: Exact Guidance for Discrete Flow Matching
• Abductive Logical Rule Induction by Bridging Inductive Logic Programming and Multimodal Large Language Models
• Training-Free Multimodal Deepfake Detection via Graph Reasoning
• UniVid: Unifying Vision Tasks with Pre-trained Video Generation Models
• CoFFT: Chain of Foresight-Focus Thought for Visual Language Models
• VLCE: A Knowledge-Enhanced Framework for Image Description in Disaster Assessment
• CompareBench: A Benchmark for Visual Comparison Reasoning in Vision-Language Models
• Semantic Edge-Cloud Communication for Real-Time Urban Traffic Surveillance with ViT and LLMs over Mobile Networks
• TABLET: A Large-Scale Dataset for Robust Visual Table Understanding
• VideoChat-R1.5: Visual Test-Time Scaling to Reinforce Multimodal Reasoning by Iterative Perception
• GeoRef: Referring Expressions in Geometry via Task Formulation, Synthetic Supervision, and Reinforced MLLM-based Solutions
• FORCE: Transferable Visual Jailbreaking Attacks via Feature Over-Reliance CorrEction
• ArchGPT: Understanding the World's Architectures with Large Multimodal Models
• Null-Space Filtering for Data-Free Continual Model Merging: Preserving Transparency, Promoting Fidelity
• DeFacto: Counterfactual Thinking with Images for Enforcing Evidence-Grounded and Faithful Reasoning
• ImaginationPolicy: Towards Generalizable, Precise and Reliable End-to-End Policy for Robotic Manipulation
• EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning
• JaiLIP: Jailbreaking Vision-Language Models via Loss Guided Image Perturbation
• EchoBench: Benchmarking Sycophancy in Medical Large Vision-Language Models
• Embodied AI: From LLMs to World Models
• RAD: Towards Trustworthy Retrieval-Augmented Multi-modal Clinical Diagnosis
• OmniScene: Attention-Augmented Multimodal 4D Scene Understanding for Autonomous Driving
• Logics-Parsing Technical Report
• CAMILA: Context-Aware Masking for Image Editing with Language Alignment
• RSAVQ: Riemannian Sensitivity-Aware Vector Quantization for Large Language Models
• SIM-CoT: Supervised Implicit Chain-of-Thought
• Interpreting ResNet-based CLIP via Neuron-Attention Decomposition
• Unifying Adversarially Robust Model Experts in Vision-Language Models
• JigShape: Evaluating Visual-Geometric Reasoning in VLMs through Jigsaw Puzzles
• MMOOC: A Comprehensive Benchmark for Out-of-Context Evaluation in Multimodal Large Language Models
• Security of World-Model-Based Embodied AI: A Lifecycle of Threats, Defenses, and Evaluation
• LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledger
• ViewMind3D: Modular View-Aware Inference for Training-Free 3D-QA
• Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfer
• QCalEval: Benchmarking Vision-Language Models for Quantum Calibration Plot Understanding
• Energy-Driven Adaptive Visual Token Pruning for Efficient Vision-Language Models
• Harnessing Synthetic Data from Generative AI for Statistical Inference
• Propose and Rectify: A Forensics-Driven MLLM Framework for Image Manipulation Localization
• AVAM: Universal Training-free Adaptive Visual Anchoring Embedded into Multimodal Large Language Model for Multi-image Question Answering
• Alternating Training-based Label Smoothing Enhances Prompt Generalization
• PoRe: Position-Reweighted Visual Token Pruning for Vision Language Models
• The Platonic Universe: Do Foundation Models See the Same Sky?
• From Global to Local: Social Bias Transfer in CLIP
• Pure Vision Language Action (VLA) Models: A Comprehensive Survey
• How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective
• MAPO: Mixed Advantage Policy Optimization
• COLT: Enhancing Video Large Language Models with Continual Tool Usage
• Knowledge Transfer from Interaction Learning
• OSDA: A Framework for Open-Set Discovery and Automatic Interpretation of Land-cover in Remote Sensing Imagery
• Steering Multimodal Large Language Models Decoding for Context-Aware Safety
• OmniBridge: Unified Multimodal Understanding, Generation, and Retrieval via Latent Space Alignment
• The Photographer Eye: Teaching Multimodal Large Language Models to Understand Image Aesthetics like Photographers
• GRPO++: Enhancing Dermatological Reasoning under Low Resource Settings
• The 1st Solution for MOSEv2 Challenge 2025: Long-term and Concept-aware Video Segmentation via SeC
• Reading Images Like Texts: Sequential Image Understanding in Vision-Language Models
• ColorBlindnessEval: Can Vision-Language Models Pass Color Blindness Tests?
• Actions Speak Louder than Prompts: A Large-Scale Study of LLMs for Graph Inference
• Bi-VLM: Pushing Ultra-Low Precision Post-Training Quantization Boundaries in Vision-Language Models
• Does Audio Matter for Modern Video-LLMs and Their Benchmarks?
• UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoning
• V2V-GoT: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Models and Graph-of-Thoughts
• SD-VLM: Spatial Measuring and Understanding with Depth-Encoded Vision-Language Models
• Visual Instruction Pretraining for Domain-Specific Foundation Models
• Training-Free Label Space Alignment for Universal Domain Adaptation
• LLaVul: A Multimodal LLM for Interpretable Vulnerability Reasoning about Source Code
• UIPro: Unleashing Superior Interaction Capability For GUI Agents
• Qwen3-Omni Technical Report
• Virtual Consistency for Audio Editing
• I-FailSense: Towards General Robotic Failure Detection with Vision-Language Models
• SAEC: Scene-Aware Enhanced Edge-Cloud Collaborative Industrial Vision Inspection with Multimodal LLM
• The 1st Solution for 7th LSVOS RVOS Track: SaSaSa2VA
• Random Direct Preference Optimization for Radiography Report Generation
• MCTS-EP: Empowering Embodied Planning with Online Preference Optimization
• FitPro: A Zero-Shot Framework for Interactive Text-based Pedestrian Retrieval in Open World
• Eye Gaze Tells You Where to Compute: Gaze-Driven Efficient VLMs
• ADVEDM:Fine-grained Adversarial Attack against VLM-based Embodied Agents
• AHA -- Predicting What Matters Next: Online Highlight Detection Without Looking Ahead
• Causality-Induced Positional Encoding for Transformer-Based Representation Learning of Non-Sequential Features
• MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer
• CoReVLA: A Dual-Stage End-to-End Autonomous Driving Framework for Long-Tail Scenarios via Collect-and-Refine
• A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learning
• Vision-Language Models as Differentiable Semantic and Spatial Rewards for Text-to-3D Generation
• Pyramid Token Pruning for High-Resolution Large Vision-Language Models via Region, Token, and Instruction-Guided Importance
• Qianfan-VL: Domain-Enhanced Universal Vision-Language Models
• TennisTV: Do Multimodal Large Language Models Understand Tennis Rallies?
• EyePCR: A Comprehensive Benchmark for Fine-Grained Perception, Knowledge Comprehension and Clinical Reasoning in Ophthalmic Surgery
• Robust Object Detection for Autonomous Driving via Curriculum-Guided Group Relative Policy Optimization
• BaseReward: A Strong Baseline for Multimodal Reward Model
• Speech Language Models for Under-Represented Languages: Insights from Wolof
• RynnVLA-001: Using Human Demonstrations to Improve Robot Manipulation
• How Good are Foundation Models in Step-by-Step Embodied Reasoning?
• OmniMRI: A Unified Vision--Language Foundation Model for Generalist MRI Interpretation
• Copycat vs. Original: Multi-modal Pretraining and Variable Importance in Box-office Prediction
• VLA-LPAF: Lightweight Perspective-Adaptive Fusion for Vision-Language-Action to Enable More Unconstrained Robotic Manipulation
• Towards Human-like Multimodal Conversational Agent by Generating Engaging Speech
• VisMoDAl: Visual Analytics for Evaluating and Improving Corruption Robustness of Vision-Language Models
• Unleashing the Potential of Multimodal LLMs for Zero-Shot Spatio-Temporal Video Grounding
• A Framework for Generating Artificial Datasets to Validate Absolute and Relative Position Concepts
• M-PACE: Mother Child Framework for Multimodal Compliance
• CLAW: A Vision-Language-Action Framework for Weight-Aware Robotic Grasping
• EDITS: Enhancing Dataset Distillation with Implicit Textual Semantics
• MOCHA: Multi-modal Objects-aware Cross-arcHitecture Alignment
• Pre-Manipulation Alignment Prediction with Parallel Deep State-Space and Transformer Models
• Improving Generalized Visual Grounding with Instance-aware Joint Learning
• Re-purposing SAM into Efficient Visual Projectors for MLLM-Based Referring Image Segmentation
• GestOS: Advanced Hand Gesture Interpretation via Large Language Models to control Any Type of Robot
• Baseer: A Vision-Language Model for Arabic Document-to-Markdown OCR
• MARS2 2025 Challenge on Multimodal Reasoning: Datasets, Methods, Results, Discussion, and Outlook
• Towards Rationale-Answer Alignment of LVLMs via Self-Rationale Calibration
• Explain Before You Answer: A Survey on Compositional Visual Reasoning
• More performant and scalable: Rethinking contrastive vision-language pre-training of radiology in the LLM era
• WHU-STree: A Multi-modal Benchmark Dataset for Street Tree Inventory
• Defense-to-Attack: Bypassing Weak Defenses Enables Stronger Jailbreaks in Vision-Language Models
• AsyMoE: Leveraging Modal Asymmetry for Enhanced Expert Specialization in Large Vision-Language Models
• Data Scaling Laws for Radiology Foundation Models
• Large Language Model-Based Automatic Formulation for Stochastic Optimization Models
• MaRVL-QA: A Benchmark for Mathematical Reasoning over Visual Landscapes
• Cross-Layer Vision Smoothing: Enhancing Visual Understanding via Sustained Focus on Key Objects in Large Vision-Language Models
• Enhancing Video Large Language Models with Structured Multi-Video Collaborative Reasoning
• 3D Aware Region Prompted Vision Language Model
• Embodied Navigation Foundation Model
• Spec-LLaVA: Accelerating Vision-Language Models with Dynamic Tree-Based Speculative Decoding
• EgoMem: Lifelong Memory Agent for Full-duplex Omnimodal Models
• DRAG: Data Reconstruction Attack using Guided Diffusion
• DUAL-VAD: Dual Benchmarks and Anomaly-Focused Sampling for Video Anomaly Detection
• SpecVLM: Fast Speculative Decoding in Vision-Language Models
• Pathological Truth Bias in Vision-Language Models
• Enhancing Generalization in Vision-Language-Action Models by Preserving Pretrained Representations
• The System Description of CPS Team for Track on Driving with Language of CVPR 2024 Autonomous Grand Challenge
• Traffic-MLLM: Curiosity-Regularized Supervised Learning for Traffic Scenario Case-Based Reasoning
• OpenUrban3D: Annotation-Free Open-Vocabulary Semantic Segmentation of Large-Scale Urban Point Clouds
• Detecting Text Manipulation in Images using Vision Language Models
• Zero-Shot Referring Expression Comprehension via Vison-Language True/False Verification
• Adaptive Token Merging for Efficient Transformer Semantic Communication at the Edge
• Towards Understanding Visual Grounding in Visual Language Models
• A Modular and Multimodal Generative AI Framework for Urban Building Energy Data: Generating Synthetic Homes
• VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model
• Curriculum-Based Multi-Tier Semantic Exploration via Deep Reinforcement Learning
• Visual Programmability: A Guide for Code-as-Thought in Chart Understanding
• Towards Better Dental AI: A Multimodal Benchmark and Instruction Dataset for Panoramic X-ray Analysis
• Discovering Divergent Representations between Text-to-Image Models
• Generalized User-Oriented Image Semantic Coding Empowered by Large Vision-Language Model
• Calibrating MLLM-as-a-judge via Multimodal Bayesian Prompt Ensembles
• PlantExpertVQA: A Visual Question Answering Dataset for Benchmarking Vision-Language Models in Plant Science
• Prompt-Driven Image Analysis with Multimodal Generative AI: Detection, Segmentation, Inpainting, and Interpretation
• Retrieval-Augmented VLMs for Multimodal Melanoma Diagnosis
• MITS: A Large-Scale Multimodal Benchmark Dataset for Intelligent Traffic Surveillance
• Recurrence Meets Transformers for Universal Multimodal Retrieval
• RoboChemist: Long-Horizon and Safety-Compliant Robotic Chemical Experimentation
• Attribute-based Object Grounding and Robot Grasp Detection with Spatial Reasoning
• Visual Representation Alignment for Multimodal Large Language Models
• Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search
• Point Linguist Model: Segment Any Object via Bridged Large 3D-Language Model
• Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluation
• Fine-Tuning Vision-Language Models for Visual Navigation Assistance
• GLEAM: Learning to Match and Explain in Cross-View Geo-Localization
• Reconstruction Alignment Improves Unified Multimodal Models
• Aligning Large Vision-Language Models by Deep Reinforcement Learning and Direct Preference Optimization
• LLaDA-VLA: Vision Language Diffusion Action Models
• Focusing by Contrastive Attention: Enhancing VLMs' Visual Reasoning
• Teaching AI Stepwise Diagnostic Reasoning with Report-Guided Chain-of-Thought Learning
• Harnessing Object Grounding for Time-Sensitive Video Understanding
• Text4Seg++: Advancing Image Segmentation via Generative Language Modeling
• D-HUMOR: Dark Humor Understanding via Multimodal Open-ended Reasoning -- A Benchmark Dataset and Method
• An Explainable Deep Neural Network with Frequency-Aware Channel and Spatial Refinement for Flood Prediction in Sustainable Cities
• Compression Beyond Pixels: Semantic Compression with Multimodal Foundation Models
• BTCChat: Advancing Remote Sensing Bi-temporal Change Captioning with Multimodal Large Language Model
• PictOBI-20k: Unveiling Large Multimodal Models in Visual Decipherment for Pictographic Oracle Bone Characters
• SuMa: A Subspace Mapping Approach for Robust and Effective Concept Erasure in Text-to-Image Diffusion Models
• SpecPrune-VLA: Accelerating Vision-Language-Action Models via Action-Aware Self-Speculative Pruning
• LatticeWorld: A Multimodal Large Language Model-Empowered Framework for Interactive Complex World Generation
• Semantic-guided LoRA Parameters Generation
• PropVG: End-to-End Proposal-Driven Visual Grounding with Multi-Granularity Discrimination
• Dual-Domain Perspective on Degradation-Aware Fusion: A VLM-Guided Robust Infrared and Visible Image Fusion Framework
• Towards Open World Detection: A Survey
• AnomalyLMM: Bridging Generative Knowledge and Discriminative Retrieval for Text-Based Person Anomaly Search
• Inverse IFEval: Can LLMs Unlearn Stubborn Training Conventions to Follow Real Instructions?
• ANTS: Adaptive Negative Textual Space Shaping for OOD Detection via Test-Time MLLM Understanding and Reasoning
• A Foundation Model for Chest X-ray Interpretation with Grounded Reasoning via Online Reinforcement Learning
• Vehicle-to-Infrastructure Collaborative Spatial Perception via Multimodal Large Language Models
• Visible Yet Unreadable: A Systematic Blind Spot of Vision Language Models Across Writing Systems
• Sample-efficient Integration of New Modalities into Large Language Models
• OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generation
• Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage, but Not Direct the Play?
• VLMs-in-the-Wild: Bridging the Gap Between Academic Benchmarks and Enterprise Reality
• Mitigating Multimodal Hallucinations via Gradient-based Self-Reflection
• OmniActor: A Generalist GUI and Embodied Agent for 2D&3D Worlds
• Structuring GUI Elements through Vision Language Models: Towards Action Space Generation
• Behavioral Fingerprinting of Large Language Models
• Draw-In-Mind: Rebalancing Designer-Painter Roles in Unified Multimodal Models Benefits Image Editing
• Structure-aware Contrastive Learning for Diagram Understanding of Multimodal Models
• AutoDrive-R²: Incentivizing Reasoning and Self-Reflection Capacity for VLA Model in Autonomous Driving
• An Investigation of Visual Foundation Models Robustness
• PixFoundation 2.0: Do Video Multi-Modal LLMs Use Motion in Visual Grounding?
• Improving Large Vision and Language Models by Learning from a Panel of Peers
• Street-Level Geolocalization Using Multimodal Large Language Models and Retrieval-Augmented Generation
• Analysing the Language of Neural Audio Codecs
• OpenVision 2: A Family of Generative Pretrained Visual Encoders for Multimodal Learning
• Variation-aware Vision Token Dropping for Faster Large Vision-Language Models
• OmniDPO: A Preference Optimization Framework to Address Omni-Modal Hallucination
• Fusion to Enhance: Fusion Visual Encoder to Enhance Multimodal Language Model
• CARIS: A Context-Adaptable Robot Interface System for Personalized and Scalable Human-Robot Interaction
• LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Model
• Discrete Prompt Tuning via Recursive Utilization of Black-box Multimodal Large Language Model for Personalized Visual Emotion Recognition
• Make me an Expert: Distilling from Generalist Black-Box Models into Specialized Models for Semantic Segmentation
• Two Causes, Not One: Rethinking Omission and Fabrication Hallucinations in MLLMs
• SurgLLM: A Versatile Large Multimodal Model with Spatial Focus and Temporal Awareness for Surgical Video Understanding
• Target-Oriented Single Domain Generalization
• DriveQA: Passing the Driving Knowledge Test
• VoCap: Video Object Captioning and Segmentation from Any Prompt
• Domain Generalization in-the-Wild: Disentangling Classification from Domain-Aware Representations
• MM-SeR: Multimodal Self-Refinement for Lightweight Image Captioning
• UItron: Foundational GUI Agent with Advanced Perception and Planning
• Generalizable Object Re-Identification via Visual In-Context Prompting
• MedFoundationHub: A Lightweight and Secure Toolkit for Deploying Medical Vision Language Foundation Models
• StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding
• CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsification
• Veritas: Generalizable Deepfake Detection via Pattern-Aware Reasoning
• PathMR: Multimodal Visual Reasoning for Interpretable Pathology Diagnosis
• GRAFT: GRaPH and Table Reasoning for Textual Alignment -- A Benchmark for Structured Instruction Following and Visual Reasoning
• MobileCLIP2: Improving Multi-Modal Reinforced Training
• GLSim: Detecting Object Hallucinations in LVLMs via Global-Local Similarity
• Scalable Object Detection in the Car Interior With Vision Foundation Models
• MQAD: A Large-Scale Question Answering Dataset for Training Music Large Language Models
• Do MLLMs Really Understand the Charts?
• Mitigating Hallucinations in Multimodal LLMs via Object-aware Preference Optimization
• How Multimodal LLMs Solve Image Tasks: A Lens on Visual Grounding, Task Reasoning, and Answer Decoding
• MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulation
• MIDAS: Multimodal Interactive Digital-humAn Synthesis via Real-time Autoregressive Video Generation
• DeepMEL: A Multi-Agent Collaboration Framework for Multimodal Entity Linking
• ZeST: an LLM-based Zero-Shot Traversability Navigation for Unknown Environments
• Can we make NeRF-based visual localization privacy-preserving?
• Hidden Tail: Adversarial Image Causing Stealthy Resource Consumption in Vision-Language Models
• Tailored Teaching with Balanced Difficulty: Elevating Reasoning in Multimodal Chain-of-Thought via Prompt Curriculum
• PRISM: Robust VLM Alignment with Principled Reasoning for Integrated Safety in Multimodality
• Revisiting associative recall in modern recurrent models
• CLARIFY: A Specialist-Generalist Framework for Accurate and Lightweight Dermatological Visual Question Answering
• DemoBias: An Empirical Study to Trace Demographic Biases in Vision Foundation Models
• MMTok: Multimodal Coverage Maximization for Efficient Inference of VLMs
• Object Detection with Multimodal Large Vision-Language Models: An In-depth Review
• ArgusCogito: Chain-of-Thought for Cross-Modal Synergy and Omnidirectional Reasoning in Camouflaged Object Segmentation
• Unveiling Trust in Multimodal Large Language Models: Evaluation, Analysis, and Mitigation
• Virtual Community: An Open World for Humans, Robots, and Society
• Directed-Tokens: A Robust Multi-Modality Alignment Approach to Large Language-Vision Models
• RynnEC: Bringing MLLMs into Embodied World
• Enhancing Targeted Adversarial Attacks on Large Vision-Language Models via Intermediate Projector
• HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenes
• The 9th AI City Challenge
• Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey
• Learning to Steer: Input-dependent Steering for Multimodal LLMs
• Creative4U: MLLMs-based Advertising Creative Image Selector with Comparative Reasoning
• Multimodal Chain of Continuous Thought for Latent-Space Reasoning in Vision-Language Models
• Contrastive Representations for Temporal Reasoning
• Vision-G1: Towards General Vision Language Reasoning with Multi-Domain Data Curation
• LangVision-LoRA-NAS: Neural Architecture Search for Variable LoRA Rank in Vision Language Models
• Say It, See It: A Systematic Evaluation on Speech-Based 3D Content Generation Methods in Augmented Reality
• Region-Level Context-Aware Multimodal Understanding
• RadarQA: Multi-modal Quality Analysis of Weather Radar Forecasts
• Inverse-LLaVA: Rethinking Multimodal Alignment via Text-to-Vision Mapping
• Temporal Grounding as a Learning Signal for Referring Video Object Segmentation
• UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encoding
• OVG-HQ: Online Video Grounding with Hybrid-modal Queries
• Bongard-RWR+: Real-World Representations of Fine-Grained Concepts in Bongard Problems
• Language models align with brain regions that represent concepts across modalities
• ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Driving
• Probing the Representational Power of Sparse Autoencoders in Vision Models
• Audio Flamingo Sound-CoT Technical Report: Improving Chain-of-Thought Reasoning in Sound Understanding
• UAV-VL-R1: Generalizing Vision-Language Models via Supervised Fine-Tuning and Multi-Stage GRPO for UAV Visual Reasoning
• Are Large Pre-trained Vision Language Models Effective Construction Safety Inspectors?
• A Dataset for Distilling Knowledge Priors from Literature for Therapeutic Design
• Agentic Design Review System
• AddressVLM: Cross-view Alignment Tuning for Image Address Localization using Large Vision-Language Models
• Empowering Multimodal LLMs with External Tools: A Comprehensive Survey
• Contrast Sensitivity in Multimodal Large Language Models: A Psychophysics-Inspired Evaluation
• MAC: A Live Benchmark for Multimodal Large Language Models in Scientific Understanding
• MANGO: Multimodal Attention-based Normalizing Flow Approach to Fusion Learning
• Bridging Modality Gaps in e-Commerce Products via Vision-Language Alignment
• Echo-4o: Harnessing the Power of GPT-4o Synthetic Images for Improved Image Generation
• A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generation
• January Food Benchmark (JFB): A Public Benchmark Dataset and Evaluation Suite for Multimodal Food Analysis
• MoIIE: Mixture of Intra- and Inter-Modality Experts for Large Vision Language Models
• SHALE: A Scalable Benchmark for Fine-grained Hallucination Evaluation in LVLMs
• GoViG: Goal-Conditioned Visual Navigation Instruction Generation
• IAG: Input-aware Backdoor Attack on VLM-based Visual Grounding
• DAgger Diffusion Navigation: DAgger Boosted Diffusion Policy for Vision-Language Navigation
• Utilizing Multilingual Encoders to Improve Large Language Models for Low-Resource Languages
• VertexRegen: Mesh Generation with Continuous Level of Detail
• OmniVTLA: Vision-Tactile-Language-Action Model with Semantic-Aligned Tactile Sensing
• AMRG: Extend Vision Language Models for Automatic Mammography Report Generation
• Cowpox: Towards the Immunity of VLM-based Multi-Agent Systems
• Bridging Formal Language with Chain-of-Thought Reasoning to Geometry Problem Solving
• KFFocus: Highlighting Keyframes for Enhanced Video Understanding
• GeoVLA: Empowering 3D Representations in Vision-Language-Action Models
• Re:Verse -- Can Your VLM Read a Manga?
• TBAC-UniImage: Unified Understanding and Generation by Ladder-Side Diffusion Tuning
• ExpVG: Investigating the Design Space of Visual Grounding in Multimodal Large Language Model
• Selective Contrastive Learning for Weakly Supervised Affordance Grounding
• Being-M0.5: A Real-Time Controllable Vision-Language-Motion Model
• ACD-CLIP: Decoupling Representation and Dynamic Fusion for Zero-Shot Anomaly Detection
• Pose-RFT: Enhancing MLLMs for 3D Pose Generation via Hybrid Action Reinforcement Fine-Tuning
• MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Models
• CoT-Pose: Chain-of-Thought Reasoning for 3D Pose Generation from Abstract Prompts
• MolmoAct: Action Reasoning Models that can Reason in Space
• Planner-Refiner: Dynamic Space-Time Refinement for Vision-Language Alignment in Videos
• MCITlib: Multimodal Continual Instruction Tuning Library and Benchmark
• ForensicsSAM: Toward Robust and Unified Image Forgery Detection and Localization Resisting to Adversarial Attack
• Find Them All: Unveiling MLLMs for Versatile Person Re-identification
• eMotions: A Large-Scale Dataset and Audio-Visual Fusion Network for Emotion Analysis in Short-form Videos
• BASIC: Boosting Visual Alignment with Intrinsic Refined Embeddings in Multimodal Large Language Models
• Remote Sensing Image Intelligent Interpretation with the Language-Centered Perspective: Principles, Methods and Challenges
• SafePLUG: Empowering Multimodal LLMs with Pixel-Level Insight and Temporal Grounding for Traffic Accident Understanding
• Multimodal learning with next-token prediction for large multimodal models
• Text Embedded Swin-UMamba for DeepLesion Segmentation
• Shortcut Learning in Generalist Robot Policies: The Role of Dataset Diversity and Fragmentation
• Text-guided Visual Prompt DINO for Generic Segmentation
• CountQA: How Well Do MLLMs Count in the Wild?
• AnimateScene: Camera-controllable Animation in Any Scene
• Automatic Semantic Alignment of Flow Pattern Representations for Exploration with Large Language Models
• Fourier Compressor: Frequency-Domain Visual Token Compression for Vision-Language Models
• User-Intent-Driven Semantic Communication via Adaptive Deep Understanding
• MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenes
• Adapting Vision-Language Models Without Labels: A Comprehensive Survey
• MoMA: A Mixture-of-Multimodal-Agents Architecture for Enhancing Clinical Prediction Modelling
• Explaining Similarity in Vision-Language Encoders with Weighted Banzhaf Interactions
• VFlowOpt: A Token Pruning Framework for LMMs with Visual Information Flow-Guided Optimization
• SPEX: A Vision-Language Model for Land Cover Extraction on Spectral Remote Sensing Images
• A Survey on Video Temporal Grounding with Multimodal Large Language Model
• Speech LLMs in Low-Resource Scenarios: Data Volume Requirements and the Impact of Pretraining on High-Resource Languages
• Latent Expression Generation for Referring Image Segmentation and Grounding
• HOLODECK 2.0: Vision-Language-Guided 3D World Generation with Editing
• Enhancing Dialogue Annotation with Speaker Characteristics Leveraging a Frozen LLM
• SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience
• RoboTron-Sim: Improving Real-World Driving via Simulated Hard-Case
• Training-Free Multimodal Large Language Model Orchestration
• Knowledge to Sight: Reasoning over Visual Attributes via Knowledge Decomposition for Abnormality Grounding
• FrEVL: Leveraging Frozen Pretrained Embeddings for Efficient Vision-Language Understanding
• Boosting Visual Knowledge-Intensive Training for LVLMs Through Causality-Driven Visual Object Completion
• Decoding the Multimodal Maze: A Systematic Review on the Adoption of Explainability in Multimodal Attention-based Models
• S2M3: Split-and-Share Multi-Modal Models for Distributed Multi-Task Inference on the Edge
• From eye to AI: studying rodent social behavior in the era of machine Learning
• Continual Learning for VLMs: A Survey and Taxonomy Beyond Forgetting
• Scaling Up Audio-Synchronized Visual Animation: An Efficient Training Paradigm
• GeoShield: Safeguarding Geolocation Privacy from Vision-Language Models via Adversarial Perturbations
• FedPromo: Federated Lightweight Proxy Models at the Edge Bring New Domains to Foundation Models
• A Multi-Agent System for Complex Reasoning in Radiology Visual Question Answering
• Would you let a humanoid play storytelling with your child? A usability study on LLM-powered narrative Human-Robot Interaction
• VITRIX-CLIPIN: Enhancing Fine-Grained Visual Understanding in CLIP via Instruction Editing Data and Long Captions
• VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo
• FedVLA: Federated Vision-Language-Action Learning with Dual Gating Mixture-of-Experts for Robotic Manipulation
• Free-MoRef: Instantly Multiplexing Context Perception Capabilities of Video-MLLMs within Single Inference
• VLM4D: Towards Spatiotemporal Awareness in Vision Language Models
• Multimodal Large Language Models for End-to-End Affective Computing: Benchmarking and Boosting with Generative Knowledge Prompting
• MedVLThinker: Simple Baselines for Multimodal Medical Reasoning
• Bench2ADVLM: A Closed-Loop Benchmark for Vision-language Models in Autonomous Driving
• InspectVLM: Unified in Theory, Unreliable in Practice
• MLP Memory: A Retriever-Pretrained Memory for Large Language Models
• TimeExpert: An Expert-Guided Video LLM for Video Temporal Grounding
• LLaDA-MedV: Exploring Large Language Diffusion Models for Biomedical Image Understanding
• A Glimpse to Compress: Dynamic Visual Token Pruning for Large Vision-Language Models
• MAP: Mitigating Hallucinations in Large Vision-Language Models with Map-Level Attention Processing
• Simulated Ensemble Attack: Transferring Jailbreaks Across Fine-tuned Vision-Language Models
• M3LLM: Model Context Protocol-aided Mixture of Vision Experts For Multimodal LLMs in Networks
• SGCap: Decoding Semantic Group for Zero-shot Video Captioning
• MeshLLM: Empowering Large Language Models to Progressively Understand and Generate 3D Mesh
• ForenX: Towards Explainable AI-Generated Image Detection with Multimodal Large Language Models
• Trans-Adapter: A Plug-and-Play Framework for Transparent Image Inpainting
• ROVI: A VLM-LLM Re-Captioned Dataset for Open-Vocabulary Instance-Grounded Text-to-Image Generation
• How LLMs are Shaping the Future of Virtual Reality
• Contact-Aware Amodal Completion for Human-Object Interaction via Multi-Regional Inpainting
• Multimodal Referring Segmentation: A Survey
• Instruction-Grounded Visual Projectors for Continual Learning of Generative Vision-Language Models
• GeoExplorer: Active Geo-localization with Curiosity-Driven Exploration
• RAGNet: Large-scale Reasoning-based Affordance Segmentation Benchmark towards General Grasping
• TriP-LLM: A Tri-Branch Patch-wise Large Language Model Framework for Time-Series Anomaly Detection
• Efficient Masked Attention Transformer for Few-Shot Classification and Segmentation
• ART: Adaptive Relation Tuning for Generalized Relation Prediction
• Mitigating Resolution-Drift in Federated Learning: Case of Keypoint Detection
• UniEmo: Unifying Emotional Understanding and Generation with Learnable Expert Queries
• Bidirectional Likelihood Estimation with Multi-Modal Large Language Models for Text-Video Retrieval
• Unveiling Super Experts in Mixture-of-Experts Large Language Models
• Adversarial-Guided Diffusion for Multimodal LLM Attacks
• UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editing
• Towards Omnimodal Expressions and Reasoning in Referring Audio-Visual Segmentation
• ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agents
• Doctor Sun: A Bilingual Multimodal Large Language Model for Biomedical AI
• Hate in Plain Sight: On the Risks of Moderating AI-Generated Hateful Illusions
• DeltaVLM: Interactive Remote Sensing Image Change Analysis via Instruction-guided Difference Perception
• Invisible Injections: Exploiting Vision-Language Models Through Steganographic Prompt Embedding
• Vision-Language Cross-Attention for Real-Time Autonomous Driving
• SmartCLIP: Modular Vision-language Alignment with Identification Guarantees
• Compression Strategies for Efficient Multimodal LLMs in Medical Contexts
• MMAT-1M: A Large Reasoning Dataset for Multimodal Agent Tuning
• ArtSeek: Deep artwork understanding via multimodal in-context reasoning and late interaction retrieval
• MAGE: Multimodal Alignment and Generation Enhancement via Bridging Visual and Semantic Spaces
• Describe, Adapt and Combine: Empowering CLIP Encoders for Open-set 3D Object Retrieval
• Meta CLIP 2: A Worldwide Scaling Recipe
• TARS: MinMax Token-Adaptive Preference Strategy for Hallucination Reduction in MLLMs
• Learning Transferable Facial Emotion Representations from Large-Scale Semantically Rich Captions
• METEOR: Multi-Encoder Collaborative Token Pruning for Efficient Vision Language Models
• Advancing Compositional LLM Reasoning with Structured Task Relations in Interactive Multimodal Communications
• TransPrune: Token Transition Pruning for Efficient Large Vision-Language Model
• Enhancing Large Multimodal Models with Adaptive Sparsity and KV Cache Compression
• Annotation-Free Human Sketch Quality Assessment
• AgroBench: Vision-Language Model Benchmark in Agriculture
• Customize Multi-modal RAI Guardrails with Precedent-based predictions
• GPT-IMAGE-EDIT-1.5M: A Million-Scale, GPT-Generated Image Dataset
• RingMo-Agent: A Unified Remote Sensing Foundation Model for Multi-Platform and Multi-Modal Reasoning
• NeuroVoxel-LM: Language-Aligned 3D Perception via Dynamic Voxelization and Meta-Embedding
• LRR-Bench: Left, Right or Rotate? Vision-Language models Still Struggle With Spatial Understanding Tasks
• Trust the Model: Compact VLMs as In-Context Judges for Image-Text Data Quality
• A Survey of Token Compression for Efficient Multimodal Large Language Models
• CONCAP: Seeing Beyond English with Concepts Retrieval-Augmented Captioning
• The Devil is in the EOS: Sequence Training for Detailed Image Captioning
• TrackAny3D: Transferring Pretrained 3D Models for Category-unified 3D Point Cloud Tracking
• Object-centric Video Question Answering with Visual Grounding and Referring
• Closing the Modality Gap for Mixed Modality Search
• Seeing Beyond Frames: Zero-Shot Pedestrian Intention Prediction with Raw Temporal Video and Multimodal Cues
• A Survey of Multimodal Hallucination Evaluation and Detection
• Towards Effective Human-in-the-Loop Assistive AI Agents
• LMM-Det: Make Large Multimodal Models Excel in Object Detection
• Resource Consumption Red-Teaming for Large Vision-Language Models
• Captain Cinema: Towards Short Movie Generation
• DiagR1: A Vision-Language Model Trained via Reinforcement Learning for Digestive Pathology Diagnosis
• Adapting Large VLMs with Iterative and Manual Instructions for Generative Low-light Enhancement
• Emotion Recognition from Skeleton Data: A Comprehensive Survey

Related

• Generative Visual Instruction Tuning
• To See is to Believe: Prompting GPT-4V for Better Visual Instruction Tuning
• Position-Enhanced Visual Instruction Tuning for Multimodal Large Language Models
• StableLLaVA: Enhanced Visual Instruction Tuning with Synthesized Image-Dialogue Data
• VIGC: Visual Instruction Generation and Correction