2022/04/17 by Ling Yan, Ling, Yan, Jianfei Yu +3 · 4 citations
Computer Science · Social Sciences · #Advanced Text Analysis Techniques #Computation and Language (cs.CL) #Computational and Text Analysis Methods #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Multimedia (cs.MM) #Sentiment Analysis and Opinion Mining
paper · pdf · doi:10.48550/arxiv.2204.07955
openalex publication_date 2022/04/17 · openalex created_date 2022/10/05 · openalex updated_date 2026/07/28
As an important task in sentiment analysis, Multimodal Aspect-Based Sentiment Analysis (MABSA) has attracted increasing attention in recent years. However, previous approaches either (i) use separately pre-trained visual and textual models, which ignore the crossmodal alignment or (ii) use vision-language models pre-trained with general pre-training tasks, which are inadequate to identify finegrained aspects, opinions, and their alignments across modalities. To tackle these limitations, we propose a task-specific Vision-Language Pre-training framework for MABSA (VLPMABSA), which is a unified multimodal encoder-decoder architecture for all the pretraining and downstream tasks. We further design three types of task-specific pre-training tasks from the language, vision, and multimodal modalities, respectively. Experimental results show that our approach generally outperforms the state-of-the-art approaches on three MABSA subtasks. Further analysis demonstrates the effectiveness of each pretraining task. The source code is publicly released at https://github.com/NUSTM/VLP-MABSA.