Efficient Architectures for High Resolution Vision-Language Models

2025/01/05 by Miguel Carvalho, Bruno Martins, Carvalho, Miguel +1 · 1 citation
Computer Science · #Multimodal Machine Learning Applications #Advanced Image and Video Retrieval Techniques

paper · pdf · doi:10.48550/arxiv.2501.02584

Abstract

Vision-Language Models (VLMs) have recently experienced significant advancements. However, challenges persist in the accurate recognition of fine details within high resolution images, which limits performance in multiple tasks. This work introduces Pheye, a novel architecture that efficiently processes high-resolution images while training fewer parameters than similarly sized VLMs. Notably, Pheye achieves a high efficiency while maintaining strong performance, particularly in tasks that demand fine-grained image understanding and/or the handling of scene-text.

Cited by

• Generating Accurate and Detailed Captions for High-Resolution Images

Related

• A Survey on Efficient Vision-Language Models
• FILA: Fine-Grained Vision Language Models
• CropVLM: Learning to Zoom for Fine-Grained Vision-Language Perception
• Native Visual Understanding: Resolving Resolution Dilemmas in Vision-Language Models
• EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning