2022/02/28 by Mareike Hartmann, Hartmann, Mareike, Aliki Anagnostopoulou +3
Computer Science · #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Human Pose and Action Recognition #Multimodal Machine Learning Applications #cs.CL #cs.CV
paper · pdf · doi:10.48550/arxiv.2202.13623
arxiv created 2022/02/28 · openalex publication_date 2022/02/28 · arxiv updated 2022/03/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We propose an approach for interactive learning for an image captioning model. As human feedback is expensive and modern neural network based approaches often require large amounts of supervised data to be trained, we envision a system that exploits human feedback as good as possible by multiplying the feedback using data augmentation methods, and integrating the resulting training examples into the model in a smart way. This approach has three key components, for which we need to find suitable practical implementations: feedback collection, data augmentation, and model update. We outline our idea and review different possibilities to address these tasks.