BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
2022/01/28 by Junnan Li, Li, Junnan, Dongxu Li +5 · 572 citations
Computer Science · #Multimodal Machine Learning Applications #Domain Adaptation and Few-Shot Learning #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2201.12086
Abstract
Vision-Language Pre-training (VLP) has advanced the performance for many vision-language tasks. However, most existing pre-trained models only excel in either understanding-based tasks or generation-based tasks. Furthermore, performance improvement has been largely achieved by scaling up the dataset with noisy image-text pairs collected from the web, which is a suboptimal source of supervision. In this paper, we propose BLIP, a new VLP framework which transfers flexibly to both vision-language understanding and generation tasks. BLIP effectively utilizes the noisy web data by bootstrapping the captions, where a captioner generates synthetic captions and a filter removes the noisy ones. We achieve state-of-the-art results on a wide range of vision-language tasks, such as image-text retrieval (+2.7% in average recall@1), image captioning (+2.8% in CIDEr), and VQA (+1.6% in VQA score). BLIP also demonstrates strong generalization ability when directly transferred to video-language tasks in a zero-shot manner. Code, models, and datasets are released at https://github.com/salesforce/BLIP.
Cited by
- Direct Diffusion Score Preference Optimization via Stepwise Contrastive Policy-Pair Supervision
- Retrieve and Segment: Are a Few Examples Enough to Bridge the Supervision Gap in Open-Vocabulary Segmentation?
- Popularity Feedback Constrains Innovation in Cultural Markets
- Development of Vision-Language Model-based GNSS Spoofing Detection for Autonomous Vehicle Navigation
- Explaining BiomedCLIP with Weighted Banzhaf Interactions Supported by Tree-Gram Parsing
- WaveZip: Wavelet-Driven Space-Time Decoupling for Video Token Condensation
- The Spatial Blindspot of Vision-Language Models
- Architectural Backdoors in Vision-Language Model Supply Chains via Representation Steering
- LENS: Adaptive Spatio-Temporal Zooming for Keyframe Sampling in Long-Form Videos
- Diff-ID: Identity Consistent Facial Image Generation and Morphing via Diffusion Models
- A Reconstruction-Based Framework for Caption Evaluation Beyond Reference Captions
- ReSAGE-PAR: Representational Similarity Assessment for Generative Expansion in Pedestrian Attribute Recognition
- Why Does Grounding Hurt Medical VQA? Benchmarking, Diagnosis, and Fine-Tuning of Vision-Language Models
- Training-free Conditional Image Embedding Framework Leveraging Large Vision Language Models
- Hierarchy-Aware Fine-Tuning of Vision-Language Models
- Fixed-Budget Parameter-Efficient Training with Frozen Encoders Improves Multimodal Chest X-Ray Classification
- ORCA: Object Recognition and Comprehension for Archiving Marine Species
- MarineEval: Assessing the Marine Intelligence of Vision-Language Models
- Language-Guided Grasp Detection with Coarse-to-Fine Learning for Robotic Manipulation
- Learning from Next-Frame Prediction: Autoregressive Video Modeling Encodes Effective Representations
- Leveraging Lightweight Entity Extraction for Scalable Event-Based Image Retrieval
- Multi-Grained Text-Guided Image Fusion for Multi-Exposure and Multi-Focus Scenarios
- Generative Digital Twins: Vision-Language Simulation Models for Executable Industrial Systems
- Towards Natural Language-Based Document Image Retrieval: New Dataset and Benchmark
- Beyond Vision: Contextually Enriched Image Captioning with Multi-Modal Retrieval
- From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMs
- M3-Verse: A "Spot the Difference" Challenge for Large Multimodal Models
- Text2Graph VPR: A Text-to-Graph Expert System for Explainable Place Recognition in Changing Environments
- AmPLe: Supporting Vision-Language Models via Adaptive-Debiased Ensemble Multi-Prompt Learning
- Through the PRISm: Importance-Aware Scene Graphs for Image Retrieval
- Adversarial Robustness of Vision in Open Foundation Models
- InfSplign: Inference-Time Spatial Alignment of Text-to-Image Diffusion Models
- Deep But Reliable: Advancing Multi-turn Reasoning for Thinking with Images
- Vision-Language Model Guided Image Restoration
- UmniBench: Unified Understand and Generation Model Oriented Omni-dimensional Benchmark
- ABE-CLIP: Training-Free Attribute Binding Enhancement for Compositional Image-Text Matching
- Scaling Spatial Reasoning in MLLMs through Programmatic Data Synthesis
- SynthSeg-Agents: Multi-Agent Synthetic Data Generation for Zero-Shot Weakly Supervised Semantic Segmentation
- Null-LoRA: Low-Rank Adaptation on Null Space
- Focus: A Streaming Concentration Architecture for Efficient Vision-Language Models
- Semantic Mismatch and Perceptual Degradation: A New Perspective on Image Editing Immunity
- Enhancing Visual Programming for Visual Reasoning via Probabilistic Graphs
- UIXPOSE: Mobile Malware Detection via Intention-Behaviour Discrepancy Analysis
- Neurosymbolic Inference On Foundation Models For Remote Sensing Text-to-image Retrieval With Complex Queries
- HERBench: A Benchmark for Multi-Evidence Integration in Video Question Answering
- MMDrive: Interactive Scene Understanding Beyond Vision with Multi-representational Fusion
- UniVCD: A New Method for Unsupervised Change Detection in the Open-Vocabulary Era
- Lemon: A Unified and Scalable 3D Multimodal Model for Universal Spatial Understanding
- JointAVBench: A Benchmark for Joint Audio-Visual Reasoning Evaluation
- MetaTPT: Meta Test-time Prompt Tuning for Vision-Language Models
- Depth-Copy-Paste: Multimodal and Depth-Aware Compositing for Robust Face Detection
- VEGAS: Mitigating Hallucinations in Large Vision-Language Models via Vision-Encoder Attention Guided Adaptive Steering
- Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding
- VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing
- BabyVLM-V2: Toward Developmentally Grounded Pretraining and Benchmarking of Vision Foundation Models
- SoccerMaster: A Vision Foundation Model for Soccer Understanding
- Leveraging Text Guidance for Enhancing Demographic Fairness in Gender Classification
- Agile Deliberation: Concept Deliberation for Subjective Visual Classification
- IRG-MotionLLM: Interleaving Motion Generation, Assessment and Refinement for Text-to-Motion Generation
- Grounding Everything in Tokens for Multimodal Large Language Models
- An Automated Tip-and-Cue Framework for Optimized Satellite Tasking and Visual Intelligence
- Video-QTR: Query-Driven Temporal Reasoning Framework for Lightweight Video Understanding
- Explaining the Unseen: Multimodal Vision-Language Reasoning for Situational Awareness in Underground Mining Disasters
- AgentComp: From Agentic Reasoning to Compositional Mastery in Text-to-Image Models
- Towards Lossless Ultimate Vision Token Compression for VLMs
- How a Bit Becomes a Story: Semantic Steering via Differentiable Fault Injection
- ConceptPose: Training-Free Zero-Shot Object Pose Estimation using Concept Vectors
- Leveraging Machine Learning and Large Language Models for Automated Image Clustering and Description in Legal Discovery
- Zero-Shot Textual Explanations via Translating Decision-Critical Features
- Geometric Prior-Guided Federated Prompt Calibration
- Pay Less Attention to Function Words for Free Robustness of Vision-Language Models
- MulCLIP: A Multi-level Alignment Framework for Enhancing Fine-grained Long-context CLIP
- Rectifying Latent Space for Generative Single-Image Reflection Removal
- The Role of Entropy in Visual Grounding: Analysis and Optimization
- Evaluating and Preserving High-level Fidelity in Super-Resolution
- BeLLA: End-to-End Birds Eye View Large Language Assistant for Autonomous Driving
- SIMPACT: Simulation-Enabled Action Planning using Vision-Language Models
- Mitigating Bias with Words: Inducing Demographic Ambiguity in Face Recognition Templates by Text Encoding
- DEAR: Dataset for Evaluating the Aesthetics of Rendering
- Conscious Gaze: Adaptive Attention Mechanisms for Hallucination Mitigation in Vision-Language Models
- MIND: Multi-rationale INtegrated Discriminative Reasoning Framework for Multi-modal Large Models
- Light-X: Generative 4D Video Rendering with Camera and Illumination Control
- Can Multi-Modal LLMs Provide Live Step-by-Step Task Guidance?
- MemLoRA: Distilling Expert Adapters for On-Device Memory Systems
- Explainable Parkinsons Disease Gait Recognition Using Multimodal RGB-D Fusion and Large Language Models
- Think Before You Drive: World Model-Inspired Multimodal Grounding for Autonomous Vehicles
- Hierarchical Process Reward Models are Symbolic Vision Learners
- ClimaOoD: Improving Anomaly Segmentation via Physically Realistic Synthetic Data
- PopSim: Social Network Simulation for Social Media Popularity Prediction
- KidSpeak: A General Multi-purpose LLM for Kids' Speech Recognition and Screening
- CourseTimeQA: A Lecture-Video Benchmark and a Latency-Constrained Cross-Modal Fusion Method for Timestamped QA
- FiCoTS: Fine-to-Coarse LLM-Enhanced Hierarchical Cross-Modality Interaction for Time Series Forecasting
- DEAL-300K: Diffusion-based Editing Area Localization with a 300K-Scale Dataset and Frequency-Prompted Baseline
- Synthetic Industrial Object Detection: GenAI vs. Feature-Based Methods
- RoadSceneBench: A Lightweight Benchmark for Mid-Level Road Scene Understanding
- INSIGHT: An Interpretable Neural Vision-Language Framework for Reasoning of Generative Artifacts
- Unexplored flaws in multiple-choice VQA evaluations
- UNION: A Lightweight Target Representation for Efficient Zero-Shot Image-Guided Retrieval with Optional Textual Queries
- FIGROTD: A Friendly-to-Handle Dataset for Image Guided Retrieval with Optional Text
- Semantic Anchoring for Robust Personalization in Text-to-Image Diffusion Models
- PROMPTMINER: Black-Box Prompt Stealing against Text-to-Image Generative Models via Reinforcement Learning and Fuzz Optimization
- SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognition
- Training-Free Diffusion Priors for Text-to-Image Generation via Optimization-based Visual Inversion
- LocateAnything3D: Vision-Language 3D Detection with Chain-of-Sight
- Flash-DMD: Towards High-Fidelity Few-Step Image Generation with Efficient Distillation and Joint Reinforcement Learning
- AlignBench: Benchmarking Fine-Grained Image-Text Alignment with Synthetic Image-Caption Pairs
- Thinking in 360°: Humanoid Visual Search in the Wild
- IrisNet: Infrared Image Status Awareness Meta Decoder for Infrared Small Targets Detection
- V-Attack: Targeting Disentangled Value Features for Controllable Adversarial Attacks on LVLMs
- Intelligent Image Search Algorithms Fusing Visual Large Models
- Reasoning-VLA: A Fast and General Vision-Language-Action Reasoning Model for Autonomous Driving
- MonoSR: Open-Vocabulary Spatial Reasoning from Monocular Images
- Temporal-Visual Semantic Alignment: A Unified Architecture for Transferring Spatial Priors from Vision Models to Zero-Shot Temporal Tasks
- ReDirector: Creating Any-Length Video Retakes with Rotary Camera Encoding
- SAM-MI: A Mask-Injected Framework for Enhancing Open-Vocabulary Semantic Segmentation with SAM
- Prune-Then-Plan: Step-Level Calibration for Stable Frontier Exploration in Embodied Question Answering
- Are Neuro-Inspired Multi-Modal Vision-Language Models Resilient to Membership Inference Privacy Leakage?
- Be My Eyes: Extending Large Language Models to New Modalities Through Multi-Agent Collaboration
- Evaluating Dataset Watermarking for Fine-tuning Traceability of Customized Diffusion Models: A Comprehensive Benchmark and Removal Approach
- ReMatch: Boosting Representation through Matching for Multimodal Retrieval
- Cross Domain Evaluation of Multimodal Chain-of-Thought Reasoning of different datasets into the Amazon CoT Framework
- Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networks
- RAVEN++: Pinpointing Fine-Grained Violations in Advertisement Videos with Active Reinforcement Reasoning
- EEG-VLM: A Hierarchical Vision-Language Model with Multi-Level Feature Alignment and Visually Enhanced Language-Guided Reasoning for EEG Image-Based Sleep Stage Prediction
- EventSTU: Event-Guided Efficient Spatio-Temporal Understanding for Video Large Language Models
- Towards Efficient VLMs: Information-Theoretic Driven Compression via Adaptive Structural Pruning
- SineProject: Machine Unlearning for Stable Vision Language Alignment
- ChineseVideoBench: Benchmarking Multi-modal Large Models for Chinese Video Question Answering
- Exploring Weak-to-Strong Generalization for CLIP-based Classification
- Synthetic Curriculum Reinforces Compositional Text-to-Image Generation
- DiVE-k: Differential Visual Reasoning for Fine-grained Image Recognition
- ViMix-14M: A Curated Multi-Source Video-Text Dataset with Long-Form, High-Quality Captions and Crawl-Free Access
- Concept Regions Matter: Benchmarking CLIP with a New Cluster-Importance Approach
- Bias Is a Subspace, Not a Coordinate: A Geometric Rethinking of Post-hoc Debiasing in Vision-Language Models
- Consolidating Diffusion-Generated Video Detection with Unified Multimodal Forgery Learning
- RoadBench: Benchmarking MLLMs on Fine-Grained Spatial Understanding and Reasoning under Urban Road Scenarios
- VLA-4D: Embedding 4D Awareness into Vision-Language-Action Models for SpatioTemporally Coherent Robotic Manipulation
- Benchmarking Nighttime Traffic Sign Recognition with Illumination-Adaptive Detection and Semantic Attribute Reasoning
- VLM-Augmented Degradation Modeling for Image Restoration Under Adverse Weather Conditions
- UniModel: A Visual-Only Framework for Unified Multimodal Understanding and Generation
- A Little More Like This: Text-to-Image Retrieval with Vision-Language Models Using Relevance Feedback
- Contrastive vision-language learning with paraphrasing and negation
- When Alignment Fails: Multimodal Adversarial Attacks on Vision-Language-Action Models
- How Noise Benefits AI-generated Image Detection
- An Image Is Worth Ten Thousand Words: Verbose-Text Induction Attacks on VLMs
- Reasoning Guided Embeddings: Leveraging MLLM Reasoning for Improved Multimodal Retrieval
- GeoSceneGraph: Geometric Scene Graph Diffusion Model for Text-guided 3D Indoor Scene Synthesis
- DIR-TIR: Dialog-Iterative Refinement for Text-to-Image Retrieval
- Foundational Question Generation for Video Question Answering via an Embedding-Integrated Approach
- SweeperBot: Making 3D Browsing Accessible through View Analysis and Visual Question Answering
- Robust Defense Strategies for Multimodal Contrastive Learning: Efficient Fine-tuning Against Backdoor Attacks
- DGS-Net: Distillation-Guided Gradient Surgery for CLIP Fine-Tuning in AI-Generated Image Detection
- From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Models
- Explore More, Learn Better: Parallel MLLM Embeddings under Mutual Information Minimization
- FSDAM: Few-Shot Driving Attention Modeling via Vision-Language Coupling
- Privacy Preserving Ordinal-Meta Learning with VLMs for Fine-Grained Fruit Quality Prediction
- EVLP:Learning Unified Embodied Vision-Language Planner with Reinforced Supervised Fine-Tuning
- SEPS: Semantic-enhanced Patch Slimming Framework for fine-grained cross-modal alignment
- Appreciate the View: A Task-Aware Evaluation Framework for Novel View Synthesis
- Medical Knowledge Intervention Prompt Tuning for Medical Image Classification
- EmoVerse: A MLLMs-Driven Emotion Representation Dataset for Interpretable Visual Emotion Analysis
- MOON2.0: Dynamic Modality-balanced Multimodal Representation Learning for E-commerce Product Understanding
- SpaceVLM: Sub-Space Modeling of Negation in Vision-Language Models
- Genomic Next-Token Predictors are In-Context Learners
- Image-POSER: Reflective RL for Multi-Expert Image Generation and Editing
- Calibrated Multimodal Representation Learning with Missing Modalities
- VISTAv2: World Imagination for Indoor Vision-and-Language Navigation
- Language-Guided Graph Representation Learning for Video Summarization
- Regional Attention-Enhanced Swin Transformer for Clinically Relevant Medical Image Captioning
- ACT as Human: Multimodal Large Language Model Data Annotation with Critical Thinking
- SliderEdit: Continuous Image Editing with Fine-Grained Instruction Control
- Aligning by Misaligning: Boundary-aware Curriculum Learning for Multimodal Alignment
- Remodeling Semantic Relationships in Vision-Language Fine-Tuning
- Exploring the Underwater World Segmentation without Extra Training
- VectorSynth: Fine-Grained Satellite Image Synthesis with Structured Semantics
- Exploring the AI Obedience: Why is Generating a Pure Color Image Harder than CyberPunk?
- Adaptation of Foundation Models for Medical Image Analysis: Strategies, Challenges, and Future Directions
- PointCubeNet: 3D Part-level Reasoning with 3x3x3 Point Cloud Blocks
- VLAD-Grasp: Zero-shot Grasp Detection via Vision-Language Models
- S2LM: Towards Semantic Steganography via Large Language Models
- Thought-For-Food: Reasoning Chain Induced Food Visual Question Answering
- On the Brittleness of CLIP Text Encoders
- Caption Injection for Optimization in Generative Search Engine
- An LLM-based Framework for Human-Swarm Teaming Cognition in Disaster Search and Rescue
- SurgViVQA: Temporally-Grounded Video Question Answering for Surgical Scene Understanding
- DetectiumFire: A Comprehensive Multi-modal Dataset Bridging Vision and Language for Fire Understanding
- SLIP: Structural-aware Language-Image Pretraining for Vision-Language Alignment
- VesSAM: Efficient Multi-Prompting for Segmenting Complex Vessel
- GraphGeo: Multi-Agent Debate Framework for Visual Geo-localization with Heterogeneous Graph Neural Networks
- OmniBrainBench: A Comprehensive Multimodal Benchmark for Brain Imaging Analysis Across Multi-stage Clinical Tasks
- ToxicTextCLIP: Text-Based Poisoning and Backdoor Attacks on CLIP Pre-training
- Leveraging Hierarchical Image-Text Misalignment for Universal Fake Image Detection
- Text-guided Fine-Grained Video Anomaly Understanding
- LGCA: Enhancing Semantic Representation via Progressive Expansion
- From Evidence to Verdict: An Agent-Based Forensic Framework for AI-Generated Image Detection
- RzenEmbed: Towards Comprehensive Multimodal Retrieval
- FOCUS: Efficient Keyframe Selection for Long Video Understanding
- A Multi-Modal Neuro-Symbolic Approach for Spatial Reasoning-Based Visual Grounding in Robotics
- Dynamic VLM-Guided Negative Prompting for Diffusion Models
- Instance-Level Composed Image Retrieval
- A knowledge-enhanced network for joint multimodal entity-relation extraction
- Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models
- TS-Reasoner: Aligning Time Series Foundation Models with LLM Reasoning
- Don't Just Chase "Highlighted Tokens" in MLLMs: Revisiting Visual Holistic Context Retention
- GUing: A Mobile GUI Search Engine using a Vision-Language Model
- SG-CoT: An Ambiguity-Aware Robotic Planning Framework using Scene Graph Representations
- SHIELD: Suppressing Hallucinations In LVLM Encoders via Bias and Vulnerability Defense
- Enhancing Compositional Reasoning in CLIP via Reconstruction and Alignment of Text Descriptions
- NavQ: Learning a Q-Model for Foresighted Vision-and-Language Navigation
- Probing the Hidden Talent of ASR Foundation Models for L2 English Oral Assessment
- Visual Diversity and Region-aware Prompt Learning for Zero-shot HOI Detection
- DRIP: Dynamic patch Reduction via Interpretable Pooling
- FruitProm: Probabilistic Maturity Estimation and Detection of Fruits and Vegetables
- Eigenfunction Extraction for Ordered Representation Learning
- QSVD: Efficient Low-rank Approximation for Unified Query-Key-Value Weight Compression in Low-Precision Vision-Language Models
- A Unified Geometric Space Bridging AI Models and the Human Brain
- Few-Shot Remote Sensing Image Scene Classification with CLIP and Prompt Learning
- Enhancing CLIP Robustness via Cross-Modality Alignment
- Enhancing Pre-trained Representation Classifiability can Boost its Interpretability
- Mitigating Negative Transfer via Reducing Environmental Disagreement
- Diffusion Adaptive Text Embedding for Text-to-Image Diffusion Models
- RAVEN: Robust Advertisement Video Violation Temporal Grounding via Reinforcement Reasoning
- RefAtomNet++: Advancing Referring Atomic Video Action Recognition using Semantic Retrieval based Multi-Trajectory Mamba
- M3T2IBench: A Large-Scale Multi-Category, Multi-Instance, Multi-Relation Text-to-Image Benchmark
- MedXplain-VQA: Multi-Component Explainable Medical Visual Question Answering
- SRSR: Enhancing Semantic Accuracy in Real-World Image Super-Resolution with Spatially Re-Focused Text-Conditioning
- Vision and Structured-Language Pretraining for Cross-Modal Food Retrieval
- FlowOpt: Fast Optimization Through Whole Flow Processes for Training-Free Editing
- Bridging the gap to real-world language-grounded visual concept learning
- MUVR: A Multi-Modal Untrimmed Video Retrieval Benchmark with Multi-Level Visual Correspondence
- MCA: Modality Composition Awareness for Robust Composed Multimodal Retrieval
- BioCAP: Exploiting Synthetic Captions Beyond Labels in Biological Foundation Models
- Theoretical Refinement of CLIP by Utilizing Linear Structure of Optimal Similarity
- Tibetan Language and AI: A Comprehensive Survey of Resources, Methods and Challenges
- Exploring a Unified Vision-Centric Contrastive Alternatives on Multi-Modal Web Documents
- Sherlock Your Queries: Learning to Ask the Right Questions for Dialogue-Based Retrieval
- CovMatch: Cross-Covariance Guided Multimodal Dataset Distillation with Trainable Text Encoder
- FeatureFool: Zero-Query Fooling of Video Models via Feature Map
- Learning Human-Object Interaction as Groups
- MARIS: Marine Open-Vocabulary Instance Segmentation with Geometric Enhancement and Semantic Alignment
- Learning with Dual-level Noisy Correspondence for Multi-modal Entity Alignment
- Recurrent Attention-based Token Selection for Efficient Streaming Video-LLMs
- Region in Context: Text-condition Image editing with Human-like semantic reasoning
- Latent Diffusion Model without Variational Autoencoder
- EgMM-Corpus: A Multimodal Vision-Language Dataset for Egyptian Culture
- ReCon: Region-Controllable Data Augmentation with Rectification and Alignment for Object Detection
- Train a Unified Multimodal Data Quality Classifier with Synthetic Data
- CaptionFormer: Unified Segmentation, Tracking, and Captioning for Spatio-Temporal Objects
- Transcribe, Translate, or Transliterate: An Investigation of Intermediate Representations in Spoken Language Models
- Exploring Cross-Modal Flows for Few-Shot Learning
- Exploring Image Representation with Decoupled Classical Visual Descriptors
- Spatial Preference Rewarding for MLLMs Spatial Understanding
- Multimodal Function Vectors for Visual Relations
- End-to-End Multi-Modal Diffusion Mamba
- Self-Aug: Query and Entropy Adaptive Decoding for Large Vision-Language Models
- Reflection-Based Task Adaptation for Self-Improving VLA
- A Review of Longitudinal Radiology Report Generation: Dataset Composition, Methods, and Performance Evaluation
- FedMMKT:Co-Enhancing a Server Text-to-Image Model and Client Task Models in Multi-Modal Federated Learning
- Evolution of meta's llama models and parameter-efficient fine-tuning of large language models: a survey
- MetaCaptioner: Towards Generalist Visual Captioning with Open-source Suites
- ImageSentinel: Protecting Visual Datasets from Unauthorized Retrieval-Augmented Image Generation
- SAIL-Embedding Technical Report: Omni-modal Embedding Foundation Model
- Unifying Vision-Language Latents for Zero-label Image Caption Enhancement
- Scaling Language-Centric Omnimodal Representation Learning
- Massive Activations are the Key to Local Detail Synthesis in Diffusion Transformers
- Class Prototypes based Contrastive Learning for Classifying Multi-Label and Fine-Grained Educational Videos
- FlexAC: Towards Flexible Control of Associative Reasoning in Multimodal Large Language Models
- CoDefend: Cross-Modal Collaborative Defense via Diffusion Purification and Prompt Optimization
- Enhancing Zero-Shot Anomaly Detection: CLIP-SAM Collaboration with Cascaded Prompts
- microCLIP: Unsupervised CLIP Adaptation via Coarse-Fine Token Fusion for Fine-Grained Image Classification
- ST2HE: A Cross-Platform Framework for Virtual Histology and Annotation of High-Resolution Spatial Transcriptomics Data
- Direct Multi-Token Decoding
- Image-to-Video Transfer Learning based on Image-Language Foundation Models: A Comprehensive Survey
- OmniQuality-R: Advancing Reward Models Through All-Encompassing Quality Assessment
- UniFlow: A Unified Pixel Flow Tokenizer for Visual Understanding and Generation
- TCMA: Text-Conditioned Multi-granularity Alignment for Drone Cross-Modal Text-Video Retrieval
- Patch-as-Decodable-Token: Towards Unified Multi-Modal Vision Tasks in MLLMs
- StelLA: Subspace Learning in Low-rank Adaptation using Stiefel Manifold
- Accelerating Attention with Basis Decomposition
- Cluster-Aware Prompt Ensemble Learning for Few-Shot Vision-Language Model Adaptation
- PhysToolBench: Benchmarking Physical Tool Understanding for MLLMs
- Vision Language Models: A Survey of 26K Papers
- Q-Router: Agentic Video Quality Assessment with Expert Model Routing and Artifact Localization
- Towards Neurocognitive-Inspired Intelligence: From AI's Structural Mimicry to Human-Like Functional Cognition
- Multi-Condition Conformal Selection
- SatFusion: A Unified Framework for Enhancing Satellite IoT Images via Multi-Temporal and Multi-Source Data Fusion
- Approximate Domain Unlearning for Vision-Language Models
- NaViL: Rethinking Scaling Properties of Native Multimodal Large Language Models under Data Constraints
- MoA-VR: A Mixture-of-Agents System Towards All-in-One Video Restoration
- Unlocking 3D Affordance Segmentation with 2D Semantic Knowledge
- ImageNet-Think-250K: A Large-Scale Synthetic Dataset for Multimodal Reasoning for Vision Language Models
- Efficient Discriminative Joint Encoders for Large Scale Vision-Language Reranking
- NoiseShift: Resolution-Aware Noise Recalibration for Better Low-Resolution Image Generation
- Discrete Diffusion Models with MLLMs for Unified Medical Multimodal Generation
- FOCUS: Optimal Control for Multi-Entity World Modeling in Text-to-Image Generation
- Guided Query Refinement: Multimodal Hybrid Retrieval with Test-Time Optimization
- Knowledge Distillation Detection for Open-weights Models
- VaseVQA-3D: Benchmarking 3D VLMs on Ancient Greek Pottery
- Human Behavior Atlas: Benchmarking Unified Psychological and Social Behavior Understanding
- Zephyrus: An Agentic Framework for Weather Science
- Automating construction safety inspections using a multi-modal vision-language RAG framework
- Multimodal Learning with Augmentation Techniques for Natural Disaster Assessment
- Spatial CAPTCHA: Generatively Benchmarking Spatial Reasoning for Human-Machine Differentiation
- LoRA Patching: Exposing the Fragility of Proactive Defenses against Deepfakes
- Confidence and Dispersity as Signals: Unsupervised Model Evaluation and Ranking
- Improving Adversarial Robustness of Zero-Shot CLIP with Confidence-Aware Weighting
- Retrv-R1: A Reasoning-Driven MLLM Framework for Universal and Efficient Multimodal Retrieval
- Team Xiaomi EV-AD VLA: Caption-Guided Retrieval System for Cross-Modal Drone Navigation -- Technical Report for IROS 2025 RoboSense Challenge Track 4
- Visual Language Model as a Judge for Object Detection in Industrial Diagrams
- Brain-Language Model Alignment: Insights into the Platonic Hypothesis and Intermediate-Layer Advantage
- ImageDoctor: Diagnosing Text-to-Image Generation via Grounded Image Reasoning
- VIRTUE: Visual-Interactive Text-Image Universal Embedder
- From Videos to Indexed Knowledge Graphs -- Framework to Marry Methods for Multimodal Content Analysis and Understanding
- Video Object Segmentation-Aware Audio Generation
- PRPO: Paragraph-level Policy Optimization for Vision-Language Deepfake Detection
- IRIS: Intrinsic Reward Image Synthesis
- VideoAnchor: Reinforcing Subspace-Structured Visual Cues for Coherent Visual-Spatial Reasoning
- VT-FSL: Bridging Vision and Text with LLMs for Few-Shot Learning
- CORE-3D: Context-aware Open-vocabulary Retrieval by Embeddings in 3D
- Agentic Services Computing
- Semantic Editing with Coupled Stochastic Differential Equations
- World-Env: Leveraging World Model as a Virtual Environment for VLA Post-Training
- Skip-It? Theoretical Conditions for Layer Skipping in Vision-Language Models
- Mitigating Visual Hallucinations via Semantic Curriculum Preference Optimization in MLLMs
- FreeRet: MLLMs as Training-Free Retrievers
- AutoPrune: Each Complexity Deserves a Pruning Policy
- EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modeling
- Towards Fine-Grained Text-to-3D Quality Assessment: A Benchmark and A Two-Stage Rank-Learning Metric
- GroupCoOp: Group-robust Fine-tuning via Group Prompt Learning
- GenView++: Unifying Adaptive Generative Augmentation and Quality-Driven Supervision for Contrastive Representation Learning
- HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decoupling
- FUSAR-KLIP: Towards Multimodal Foundation Models for Remote Sensing
- No Concept Left Behind: Test-Time Optimization for Compositional Text-to-Image Generation
- Preventing Robotic Jailbreaking via Multimodal Domain Adaptation
- Follow-Your-Preference: Towards Preference-Aligned Image Inpainting
- CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning
- InfiMed-Foundation: Pioneering Advanced Multimodal Medical Models with Compute-Efficient Pre-Training and Multi-Stage Fine-Tuning
- SemanticControl: A Training-Free Approach for Handling Loosely Aligned Visual Conditions in ControlNet
- CoFFT: Chain of Foresight-Focus Thought for Visual Language Models
- MedVSR: Medical Video Super-Resolution with Cross State-Space Propagation
- SCRA-VQA: Summarized Caption-Rerank for Augmented Large Language Models in Visual Question Answering
- Boosting Zero-Shot VLN via Abstract Obstacle Map-Based Waypoint Prediction with TopoGraph-and-VisitInfo-Aware Prompting
- MeshFM: 2D Features Are All You Need for 3D Shape Understanding
- ViewMind3D: Modular View-Aware Inference for Training-Free 3D-QA
- VisualRouter: Query-Grounded Visual Sampling for Long Video Understanding
- DS@GT ARC at ImageCLEFmedical 2026: Architectural Diversity for Concept Detection and Foundation-Model Scaling for Caption Prediction in Medical Image Analysis
- Alternating Training-based Label Smoothing Enhances Prompt Generalization
- PoRe: Position-Reweighted Visual Token Pruning for Vision Language Models
- Pure Vision Language Action (VLA) Models: A Comprehensive Survey
- How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective
- MAPO: Mixed Advantage Policy Optimization
- Global Minimizers of Sigmoid Contrastive Loss
- What Makes You Unique? Attribute Prompt Composition for Object Re-Identification
- Training-Free Label Space Alignment for Universal Domain Adaptation
- Align Where the Words Look: Cross-Attention-Guided Patch Alignment with Contrastive and Transport Regularization for Bengali Captioning
- MLLM-Driven Semantic Identifier Generation for Generative Cross-Modal Retrieval
- MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interaction
- MDF-MLLM: Deep Fusion Through Cross-Modal Feature Alignment for Contextually Aware Fundoscopic Image Classification
- Automated Procedural Analysis via Video-Language Models for AI-assisted Nursing Skills Assessment
- FitPro: A Zero-Shot Framework for Interactive Text-based Pedestrian Retrieval in Open World
- MedCutMix: A Data-Centric Approach to Improve Radiology Vision-Language Pre-training with Disease Awareness
- Describe-to-Score: Text-Guided Efficient Image Complexity Assessment
- AHA -- Predicting What Matters Next: Online Highlight Detection Without Looking Ahead
- Causality-Induced Positional Encoding for Transformer-Based Representation Learning of Non-Sequential Features
- RACap: Relation-Aware Prompting for Lightweight Retrieval-Augmented Image Captioning
- ORIC: Benchmarking Object Recognition under Contextual Incongruity in Large Vision-Language Models
- SCENEFORGE: Enhancing 3D-text alignment with Structured Scene Compositions
- ORCA: Agentic Reasoning For Hallucination and Adversarial Robustness in Vision-Language Models
- OmniMRI: A Unified Vision--Language Foundation Model for Generalist MRI Interpretation
- OmniSegmentor: A Flexible Multi-Modal Learning Framework for Semantic Segmentation
- Copycat vs. Original: Multi-modal Pretraining and Variable Importance in Box-office Prediction
- VisMoDAl: Visual Analytics for Evaluating and Improving Corruption Robustness of Vision-Language Models
- SmolRGPT: Efficient Spatial Reasoning for Warehouse Environments with 600M Parameters
- CLAW: A Vision-Language-Action Framework for Weight-Aware Robotic Grasping
- SeqVLA: Sequential Task Execution for Long-Horizon Manipulation with Completion-Aware Vision-Language-Action Model
- Image Realness Assessment and Localization with Multimodal Features
- ResidualViT for Efficient Temporally Dense Video Encoding
- LadderSym: A Multimodal Interleaved Transformer for Music Practice Error Detection
- Bridging Vision Language Models and Symbolic Grounding for Video Question Answering
- Synthetic Captions for Open-Vocabulary Zero-Shot Segmentation
- LaV-CoT: Language-Aware Visual CoT with Multi-Aspect Reward Optimization for Real-World Multilingual VQA
- Adaptive Token Merging for Efficient Transformer Semantic Communication at the Edge
- Mixture of Semantics Transmission for Generative AI-Enabled Semantic Communication Systems
- Prompt Pirates Need a Map: Stealing Seeds helps Stealing Prompts
- FS-Diff: Semantic guidance and clarity-aware simultaneous multimodal image fusion and super-resolution
- Curriculum-Based Multi-Tier Semantic Exploration via Deep Reinforcement Learning
- Visual Programmability: A Guide for Code-as-Thought in Chart Understanding
- AWM-Fuse: Multi-Modality Image Fusion for Adverse Weather via Global and Local Text Perception
- VoxelFormer: Parameter-Efficient Multi-Subject Visual Decoding from fMRI
- PlantExpertVQA: A Visual Question Answering Dataset for Benchmarking Vision-Language Models in Plant Science
- Prompt-Driven Image Analysis with Multimodal Generative AI: Detection, Segmentation, Inpainting, and Interpretation
- Recurrence Meets Transformers for Universal Multimodal Retrieval
- SurgLaVi: Large-Scale Hierarchical Dataset for Surgical Vision-Language Representation Learning
- Attribute-based Object Grounding and Robot Grasp Detection with Spatial Reasoning
- MemoVis: A GenAI-Powered Tool for Creating Companion Reference Images for 3D Design Feedback
- Two Stage Context Learning with Large Language Models for Multimodal Stance Detection on Climate Change
- Data-Efficient Fine-Tuning of Vision-Language Models for Diagnosis of Alzheimer's Disease
- Testing chatbots on the creation of encoders for audio conditioned image generation
- XBusNet: Text-Guided Breast Ultrasound Segmentation via Multimodal Vision-Language Learning
- Embodied Hazard Mitigation using Vision-Language Models for Autonomous Mobile Robots
- D-HUMOR: Dark Humor Understanding via Multimodal Open-ended Reasoning -- A Benchmark Dataset and Method
- Prototype-Aware Multimodal Alignment for Open-Vocabulary Visual Grounding
- Effectively obtaining acoustic, visual and textual data from videos
- Semantic-guided LoRA Parameters Generation
- SemSteDiff: Generative Diffusion Model-based Coverless Semantic Steganography Communication
- Dual-Domain Perspective on Degradation-Aware Fusion: A VLM-Guided Robust Infrared and Visible Image Fusion Framework
- An Automated, Scalable Machine Learning Model Inversion Assessment Pipeline
- Reg3D: Reconstructive Geometry Instruction Tuning for 3D Scene Understanding
- TeRA: Rethinking Text-guided Realistic 3D Avatar Generation
- AutoDrive-R2: Incentivizing Reasoning and Self-Reflection Capacity for VLA Model in Autonomous Driving
- Hierarchical Vision-Language Reasoning for Multimodal Multiple-Choice Question Answering
- Reinforced Visual Perception with Tools
- Improving Large Vision and Language Models by Learning from a Panel of Peers
- Two-flow Feedback Multi-scale Progressive Generative Adversarial Network
- OpenVision 2: A Family of Generative Pretrained Visual Encoders for Multimodal Learning
- Target-Oriented Single Domain Generalization
- Safe-LLaVA: A Privacy-Preserving Vision-Language Dataset and Benchmark for Biometric Safety
- Category-level Text-to-Image Retrieval Improved: Bridging the Domain Gap with Diffusion Models and Vision Encoders
- BiListing: Modality Alignment for Listings
- OneReward: Unified Mask-Guided Image Generation via Multi-Task Human Preference Learning
- SUMMA: A Multimodal Large Language Model for Advertisement Summarization
- Plug-in Feedback Self-adaptive Attention in CLIP for Training-free Open-Vocabulary Segmentation
- CSRD2025: A Large-Scale Synthetic Radio Dataset for Spectrum Sensing in Wireless Communications
- Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignment
- On Surjectivity of Neural Networks: Can you elicit any behavior from your model?
- Hidden Tail: Adversarial Image Causing Stealthy Resource Consumption in Vision-Language Models
- CLARIFY: A Specialist-Generalist Framework for Accurate and Lightweight Dermatological Visual Question Answering
- SEAM: Semantically Equivalent Across Modalities Benchmark for Vision-Language Models
- Annotation-Free Open-Vocabulary Segmentation for Remote-Sensing Images
- Spiking Variational Graph Representation Inference for Video Summarization
- HiRQA: Hierarchical Ranking and Quality Alignment for Opinion-Unaware Image Quality Assessment
- Understanding Data Influence with Differential Approximation
- Ouroboros: Single-step Diffusion Models for Cycle-consistent Forward and Inverse Rendering
- Directed-Tokens: A Robust Multi-Modality Alignment Approach to Large Language-Vision Models
- Multimodal Data Storage and Retrieval for Embodied AI: A Survey
- Hierarchical Vision-Language Retrieval of Educational Metaverse Content in Agriculture
- Mitigating Easy Option Bias in Multiple-Choice Question Answering
- SAGA: Learning Signal-Aligned Distributions for Improved Text-to-Image Generation
- REVEAL -- Reasoning and Evaluation of Visual Evidence through Aligned Language
- VELVET-Med: Vision and Efficient Language Pre-training for Volumetric Imaging Tasks in Medicine
- UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encoding
- Recent Advances in Transformer and Large Language Models for UAV Applications
- LEARN: A Story-Driven Layout-to-Image Generation Framework for STEM Instruction
- SemPT: Semantic Prompt Tuning for Vision-Language Models
- Empowering Multimodal LLMs with External Tools: A Comprehensive Survey
- Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advances
- MANGO: Multimodal Attention-based Normalizing Flow Approach to Fusion Learning
- Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models
- GSFixer: Improving 3D Gaussian Splatting with Reference-Guided Video Diffusion Priors
- WeatherPrompt: Multi-modality Representation Learning for All-Weather Drone Visual Geo-Localization
- Semantic-Aware Reconstruction Error for Detecting AI-Generated Images
- SafeFix: Targeted Model Repair via Controlled Image Generation
- Unlocking the Potential of Diffusion Priors in Blind Face Restoration
- VSF: Simple, Efficient, and Effective Negative Guidance in Few-Step Image Generation Models By Value Sign Flip
- Re:Verse -- Can Your VLM Read a Manga?
- TRIDE: A Text-assisted Radar-Image weather-aware fusion network for Depth Estimation
- BadPromptFL: A Novel Backdoor Threat to Prompt-based Federated Learning in Multimodal Models
- OpenHAIV: A Framework Towards Practical Open-World Learning
- Towards Effective Prompt Stealing Attack against Text-to-Image Diffusion Models
- Remote Sensing Image Intelligent Interpretation with the Language-Centered Perspective: Principles, Methods and Challenges
- Adversarial Video Promotion Against Text-to-Video Retrieval
- The Trauma THOMPSON Dataset for Real-World Emergency AI
- CLIPin: A Non-contrastive Plug-in to CLIP for Multimodal Semantic Alignment
- VISTAR:A User-Centric and Role-Driven Benchmark for Text-to-Image Evaluation
- Text-guided Visual Prompt DINO for Generic Segmentation
- SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning
- Real-Time 3D Vision-Language Embedding Mapping
- SynSeg: Feature Synergy for Multi-Category Contrastive Learning in End-to-End Open-Vocabulary Semantic Segmentation
- Automatic Semantic Alignment of Flow Pattern Representations for Exploration with Large Language Models
- MoMA: A Mixture-of-Multimodal-Agents Architecture for Enhancing Clinical Prediction Modelling
- From Detection to Correction: Backdoor-Resilient Face Recognition via Vision-Language Trigger Detection and Noise-Based Neutralization
- UNCAGE: Contrastive Attention Guidance for Masked Generative Transformers in Text-to-Image Generation
- ReasoningTrack: Chain-of-Thought Reasoning for Long-term Vision-Language Tracking
- A Context-aware Attention and Graph Neural Network-based Multimodal Framework for Misogyny Detection
- ETTA: Efficient Test-Time Adaptation for Vision-Language Models through Dynamic Embedding Updates
- Robust Prompt Tuning for Vision-Language Models with Mild Semantic Noise
- FrEVL: Leveraging Frozen Pretrained Embeddings for Efficient Vision-Language Understanding
- Composed Object Retrieval: Object-level Retrieval via Composed Expressions
- Continual Learning for VLMs: A Survey and Taxonomy Beyond Forgetting
- HPSv3: Towards Wide-Spectrum Human Preference Score
- GeoShield: Safeguarding Geolocation Privacy from Vision-Language Models via Adversarial Perturbations
- Causal Disentanglement and Cross-Modal Alignment for Enhanced Few-Shot Learning
- Spatial 3D-LLM: Exploring Spatial Awareness in 3D Vision-Language Models
- Devil is in the Detail: Towards Injecting Fine Details of Image Prompt in Image Generation via Conflict-free Guidance and Stratified Attention
- IMoRe: Implicit Program-Guided Reasoning for Human Motion Q&A
- Enhancing Zero-Shot Brain Tumor Subtype Classification via Fine-Grained Patch-Text Alignment
- Set Pivot Learning: Redefining Generalized Segmentation with Vision Foundation Models
- Can3Tok: Canonical 3D Tokenization and Latent Modeling of Scene-Level 3D Gaussians
- Open-Attribute Person Retrieval: Finding People Through Distinctive and Novel Attributes
- NS-Net: Decoupling CLIP Semantic Information through NULL-Space for Generalizable AI-Generated Image Detection
- SaviorRec: Semantic-Behavior Alignment for Cold-Start Recommendation
- Discovering and using Spelke segments
- Training-Free Class Purification for Open-Vocabulary Semantic Segmentation
- Sel3DCraft: Interactive Visual Prompts for User-Friendly Text-to-3D Generation
- Decouple before Align: Visual Disentanglement Enhances Prompt Tuning
- From Generator to Embedder: Harnessing Innate Abilities of Multimodal LLMs via Building Zero-Shot Discriminative Embedding Model
- Boosting Vision Semantic Density with Anatomy Normality Modeling for Medical Vision-language Pre-training
- Multi-Prompt Progressive Alignment for Multi-Source Unsupervised Domain Adaptation
- Bidirectional Likelihood Estimation with Multi-Modal Large Language Models for Text-Video Retrieval
- Multi-Modal Motion Retrieval by Learning a Fine-Grained Joint Embedding Space
- Investigating the Invertibility of Multimodal Latent Spaces: Limitations of Optimization-Based Methods
- On the Reliability of Vision-Language Models Under Adversarial Frequency-Domain Perturbations
- DeltaVLM: Interactive Remote Sensing Image Change Analysis via Instruction-guided Difference Perception
- Invisible Injections: Exploiting Vision-Language Models Through Steganographic Prompt Embedding
- Multi-modal Relational Item Representation Learning for Inferring Substitutable and Complementary Items
- SmartCLIP: Modular Vision-language Alignment with Identification Guarantees
- Color as the Impetus: Transforming Few-Shot Learner
- HOLa: Zero-Shot HOI Detection with Low-Rank Decomposed VLM Feature Adaptation
- Foundation Models and Transformers for Anomaly Detection: A Survey
- MAGE: Multimodal Alignment and Generation Enhancement via Bridging Visual and Semantic Spaces
- Exploring the Link Between Bayesian Inference and Embodied Intelligence: Toward Open Physical-World Embodied AI Systems
- Quantifying and Narrowing the Unknown: Interactive Text-to-Video Retrieval via Uncertainty Minimization
- Group Relative Augmentation for Data Efficient Action Detection
- AgroBench: Vision-Language Model Benchmark in Agriculture
- LRR-Bench: Left, Right or Rotate? Vision-Language models Still Struggle With Spatial Understanding Tasks
- TAPS : Frustratingly Simple Test Time Active Learning for VLMs
- Region-based Cluster Discrimination for Visual Representation Learning
- Taking Language Embedded 3D Gaussian Splatting into the Wild
- CircuitProbe: Dissecting Spatiotemporal Visual Semantics with Circuit Tracing
- Negation-Aware Test-Time Adaptation for Vision-Language Models
- Enhancing Reward Models for High-quality Image Generation: Beyond Text-Image Alignment
- When Better Eyes Lead to Blindness: A Diagnostic Study of the Information Bottleneck in CNN-LSTM Image Captioning Models
- Towards Consistent Long-Term Pose Generation
- Resource Consumption Red-Teaming for Large Vision-Language Models
- Adapting Large VLMs with Iterative and Manual Instructions for Generative Low-light Enhancement
- E.A.R.T.H.: Structuring Creative Evolution through Model Error in Generative AI
- Med-GRIM: Enhanced Zero-Shot Medical VQA using prompt-embedded Multimodal Graph RAG
- U-MARVEL: Unveiling Key Factors for Universal Multimodal Retrieval via Embedding Learning with MLLMs
- MMCircuitEval: A Comprehensive Multimodal Circuit-Focused Benchmark for Evaluating LLMs
- Docopilot: Improving Multimodal Models for Document-Level Understanding
- Multimodal AI for Gastrointestinal Diagnostics: Tackling VQA in MEDVQA-GI 2025
- RaMen: Multi-Strategy Multi-Modal Learning for Bundle Construction
- PoemTale Diffusion: Minimising Information Loss in Poem to Image Generation with Multi-Stage Prompt Refinement
- SV3.3B: A Sports Video Understanding Model for Action Recognition
- RoadBench: A Vision-Language Foundation Model and Benchmark for Road Damage Understanding
- Principled Multimodal Representation Learning
- PARTE: Part-Guided Texturing for 3D Human Reconstruction from a Single Image
- FedVLM: Scalable Personalized Vision-Language Models through Federated Learning
- DNT: a Deeply Normalized Transformer that can be trained by Momentum SGD
- Enhancing LiDAR Point Features with Foundation Model Priors for 3D Object Detection
- Differential-informed Sample Selection Accelerates Multimodal Contrastive Learning
- FIQ: Fundamental Question Generation with the Integration of Question Embeddings for Video Question Answering
- AnyCap Project: A Unified Framework, Dataset, and Benchmark for Controllable Omni-modal Captioning
- Describe Anything Model for Visual Question Answering on Text-rich Images
- InstructFLIP: Exploring Unified Vision-Language Model for Face Anti-spoofing
- From Coarse to Nuanced: Cross-Modal Alignment of Fine-Grained Linguistic Cues and Visual Salient Regions for Dynamic Emotion Recognition
- QuRe: Query-Relevant Retrieval through Hard Negative Sampling in Composed Image Retrieval
- Mono-InternVL-1.5: Towards Cheaper and Faster Monolithic Multimodal Large Language Models
- MindJourney: Test-Time Scaling with World Models for Spatial Reasoning
- Spatially Grounded Explanations in Vision Language Models for Document Visual Question Answering
- Robust 3D-Masked Part-level Editing in 3D Gaussian Splatting with Regularized Score Distillation Sampling
- Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalities
- Text-Visual Semantic Constrained AI-Generated Image Quality Assessment
- Beyond Graph Model: Reliable VLM Fine-Tuning via Random Graph Adapter
- DisCo: Towards Distinct and Coherent Visual Encapsulation in Video MLLMs
- FIX-CLIP: Dual-Branch Hierarchical Contrastive Learning via Synthetic Captions for Better Understanding of Long Text
- Cross-modal Associations in Vision and Language Models: Revisiting the Bouba-Kiki Effect
- Towards Fine-Grained Adaptation of CLIP via a Self-Trained Alignment Score
- Large Multi-modal Model Cartographic Map Comprehension for Textual Locality Georeferencing
- From Enhancement to Understanding: Build a Generalized Bridge for Low-light Vision via Semantically Consistent Unsupervised Fine-tuning
- Multi-modal Mutual-Guidance Conditional Prompt Learning for Vision-Language Models
- Multi-Granular Spatio-Temporal Token Merging for Training-Free Acceleration of Video LLMs
- Beyond the Linear Separability Ceiling: Aligning Representations in VLMs
- Towards Evaluating Robustness of Prompt Adherence in Text to Image Models
- Vision-Language-Vision Auto-Encoder: Scalable Knowledge Distillation from Diffusion Models
- MS-DPPs: Multi-Source Determinantal Point Processes for Contextual Diversity Refinement of Composite Attributes in Text to Image Retrieval
- HVI-CIDNet+: Beyond Extreme Darkness for Low-Light Image Enhancement
- Free on the Fly: Enhancing Flexibility in Test-Time Adaptation with Online EM
- RSRefSeg 2: Decoupling Referring Remote Sensing Image Segmentation with Foundation Models
- LangMamba: A Language-driven Mamba Framework for Low-dose CT Denoising with Vision-language Models
- Solar Altitude Guided Scene Illumination
- Unveiling Effective In-Context Configurations for Image Captioning: An External & Internal Analysis
Related