MobileVLM V2: Faster and Stronger Baseline for Vision Language Model

2024/02/06 by Xiangxiang Chu, Limeng Qiao, Chu, Xiangxiang +19 · 21 citations
Computer Science · Engineering · #Multimodal Machine Learning Applications #Advanced Image and Video Retrieval Techniques #Robotics and Automated Systems

paper · pdf · doi:10.48550/arxiv.2402.03766

Abstract

We introduce MobileVLM V2, a family of significantly improved vision language models upon MobileVLM, which proves that a delicate orchestration of novel architectural design, an improved training scheme tailored for mobile VLMs, and rich high-quality dataset curation can substantially benefit VLMs' performance. Specifically, MobileVLM V2 1.7B achieves better or on-par performance on standard VLM benchmarks compared with much larger VLMs at the 3B scale. Notably, our 3B model outperforms a large variety of VLMs at the 7B+ scale. Our models will be released at https://github.com/Meituan-AutoML/MobileVLM .

Cited by

• Masking Teacher and Reinforcing Student for Distilling Vision-Language Models
• UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models
• Input-Adaptive Visual Preprocessing for Efficient Fast Vision-Language Model Inference
• Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future
• Jina-VLM: Small Multilingual Vision Language Model
• OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
• OpenREAD: Reinforced Open-Ended Reasoning for End-to-End Autonomous Driving with LLM-as-Critic
• TrafficLens: Multi-Camera Traffic Video Analysis Using LLMs
• Viper-F1: Fast and Fine-Grained Multimodal Understanding with Cross-Modal State-Space Modulation
• FlowMM: Cross-Modal Information Flow Guided KV Cache Merging for Efficient Multimodal Context Inference
• CoSense-LLM: Semantics at the Edge with Cost- and Uncertainty-Aware Cloud-Edge Cooperation
• NaViL: Rethinking Scaling Properties of Native Multimodal Large Language Models under Data Constraints
• Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understanding
• GUI-KV: Efficient GUI Agents via KV Cache with Spatio-Temporal Awareness
• Efficient Multi-modal Large Language Models via Progressive Consistency Distillation
• Agentic Services Computing
• Discrete Guidance Matching: Exact Guidance for Discrete Flow Matching
• A Framework for Generating Artificial Datasets to Validate Absolute and Relative Position Concepts
• S2M3: Split-and-Share Multi-Modal Models for Distributed Multi-Task Inference on the Edge
• Edge-Based Multimodal Sensor Data Fusion with Vision Language Models (VLMs) for Real-time Autonomous Vehicle Accident Avoidance
• AI in Agriculture: A Survey of Deep Learning Techniques for Crops, Fisheries and Livestock

Related

• MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices
• MagicVL-2B: Empowering Vision-Language Models on Mobile Devices with Lightweight Visual Encoders via Curriculum Learning
• FastVLM: Efficient Vision Encoding for Vision Language Models
• SmolVLM: Redefining small and efficient multimodal models
• Xmodel-VLM: A Simple Baseline for Multimodal Vision Language Model