2022/08/19 by Sunan He, He, Sunan, Taian Guo +11 · 9 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #Artificial intelligence #Autoencoder #Cancer-related molecular mechanisms research #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Computer vision #Deep learning #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Focus (optics) #Generative grammar #Image (mathematics) #Language model #Machine learning #Multimodal Machine Learning Applications #Natural language processing #cs.CV
paper · pdf · doi:10.48550/arxiv.2208.09374
published in arXiv (Cornell University) (Cornell University) · 12 pages, 7 figures
arxiv created 2022/08/19 · openalex publication_date 2022/08/19 · arxiv updated 2022/08/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Image and language modeling is of crucial importance for vision-language pre-training (VLP), which aims to learn multi-modal representations from large-scale paired image-text data. However, we observe that most existing VLP methods focus on modeling the interactions between image and text features while neglecting the information disparity between image and text, thus suffering from focal bias. To address this problem, we propose a vision-language masked autoencoder framework (VLMAE). VLMAE employs visual generative learning, facilitating the model to acquire fine-grained and unbiased features. Unlike the previous works, VLMAE pays attention to almost all critical patches in an image, providing more comprehensive understanding. Extensive experiments demonstrate that VLMAE achieves better performance in various vision-language downstream tasks, including visual question answering, image-text retrieval and visual grounding, even with up to 20% pre-training speedup.