Behind Maya: Building a Multilingual Vision Language Model

2025/05/13 by N. M. Alam, Alam, Nahid, Karthik Reddy Kanjula +34 · 3 citations
Arts and Humanities · #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Media, Religion, Digital Communication

paper · pdf · doi:10.48550/arxiv.2505.08910

openalex publication_date 2025/05/13 · openalex created_date 2025/10/15 · openalex updated_date 2026/07/28

Abstract

In recent times, we have seen a rapid development of large Vision-Language Models (VLMs). They have shown impressive results on academic benchmarks, primarily in widely spoken languages but lack performance on low-resource languages and varied cultural contexts. To address these limitations, we introduce Maya, an open-source Multilingual VLM. Our contributions are: 1) a multilingual image-text pretraining dataset in eight languages, based on the LLaVA pretraining dataset; and 2) a multilingual image-text model supporting these languages, enhancing cultural and linguistic comprehension in vision-language tasks. Code available at https://github.com/nahidalam/maya.

Citations

• Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages
• Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
• Why do LLaVA Vision-Language Models Reply to Images in English?
• CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark
• Aya 23: Open Weight Releases to Further Multilingual Progress
• MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training
• PALO: A Polyglot Large Multimodal Model for 5B People
• LoRA+: Efficient Low Rank Adaptation of Large Models
• Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks
• Improved Baselines with Visual Instruction Tuning
• Kosmos-G: Generating Images in Context with Multimodal Large Language Models
• Kosmos-2: Grounding Multimodal Large Language Models to the World
• PaLI-X: On Scaling up a Multilingual Vision and Language Model
• Sigmoid Loss for Language Image Pre-Training
• GPT-4 Technical Report
• BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
• LAION-5B: An open large-scale dataset for training next generation image-text models
• PaLI: A Jointly-Scaled Multilingual Language-Image Model
• Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset
• Flamingo: a Visual Language Model for Few-Shot Learning
• LoRA: Low-Rank Adaptation of Large Language Models
• RoFormer: Enhanced Transformer with Rotary Position Embedding
• Learning Transferable Visual Models From Natural Language Supervision
• The State and Fate of Linguistic Diversity and Inclusion in the NLP World
• Gaussian Error Linear Units (GELUs)
• Multi30K: Multilingual English-German Image Descriptions
• Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
• Microsoft COCO: Common Objects in Context
• Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models

Cited by

• IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs
• Seeing Straight: Document Orientation Detection for Efficient OCR
• NeoBabel: A Multilingual Open Tower for Visual Generation

Related

• Maya: An Instruction Finetuned Multilingual Multimodal Model
• Chitrarth: Bridging Vision and Language for a Billion People
• TowerVision: Understanding and Improving Multilinguality in Vision-Language Models
• Multilingual Vision-Language Models, A Survey
• VLURes: Benchmarking VLM Visual and Linguistic Understanding in Low-Resource Languages