Psycholinguistics meets Continual Learning: Measuring Catastrophic\n Forgetting in Visual Question Answering

2019/06/10 by Claudio Greco, Greco, Claudio, Barbara Plank +5 · 2 citations
Arts and Humanities · Computer Science · #Computation and Language (cs.CL) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Subtitles and Audiovisual Media

paper · pdf · doi:10.48550/arxiv.1906.04229

openalex publication_date 2019/06/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We study the issue of catastrophic forgetting in the context of neural\nmultimodal approaches to Visual Question Answering (VQA). Motivated by evidence\nfrom psycholinguistics, we devise a set of linguistically-informed VQA tasks,\nwhich differ by the types of questions involved (Wh-questions and polar\nquestions). We test what impact task difficulty has on continual learning, and\nwhether the order in which a child acquires question types facilitates\ncomputational models. Our results show that dramatic forgetting is at play and\nthat task difficulty and order matter. Two well-known current continual\nlearning methods mitigate the problem only to a limiting degree.\n

Cited by

• Affordance-First Decomposition for Continual Learning in Video-Language Understanding
• CurLL: A Developmental Framework to Evaluate Continual Learning in Language Models

Related

• Task Formulation Matters When Learning Continually: A Case Study in Visual Question Answering
• Enhancing Continual Learning in Visual Question Answering with Modality-Aware Feature Distillation
• Ask and Remember: A Questions-Only Replay Strategy for Continual Visual Question Answering
• One VLM to Keep it Learning: Generation and Balancing for Data-free Continual Visual Question Answering
• CL-CrossVQA: A Continual Learning Benchmark for Cross-Domain Visual Question Answering