Captioning Visualizations with Large Language Models (CVLLM): A Tutorial

2024/06/27 by Carenini, Giuseppe, Johnson, Jordon, Salamatian, Ali
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC)

paper · doi:10.48550/arxiv.2406.19512

Abstract

Automatically captioning visualizations is not new, but recent advances in large language models(LLMs) open exciting new possibilities. In this tutorial, after providing a brief review of Information Visualization (InfoVis) principles and past work in captioning, we introduce neural models and the transformer architecture used in generic LLMs. We then discuss their recent applications in InfoVis, with a focus on captioning. Additionally, we explore promising future directions in this field.

Related

• Using Large Language Models to Generate Engaging Captions for Data Visualizations
• Personalizing Multimodal Large Language Models for Image Captioning: An Experimental Analysis
• Visual Analytics for Efficient Image Exploration and User-Guided Image Captioning
• Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions
• Image Representations and New Domains in Neural Image Captioning