MatplotAlt: A Python Library for Adding Alt Text to Matplotlib Figures in Computational Notebooks

2025/03/25 by Kai Nylund, Jennifer Mankoff, Venkatesh Potluri · 2 voices · 8 citations
Computer Science · #Chart #Computer science #Explainable Artificial Intelligence (XAI) #Information retrieval #Machine Learning and Data Classification #Machine learning #Merge (version control) #Parsing #Programming language #Python (programming language) #Software Engineering Research #Statistics #Univariate

paper · pdf · doi:10.1111/cgf.70119

published in Computer Graphics Forum 44(3) (Wiley)

openalex publication_date 2025/05/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

We present MatplotAlt, an open-source Python package for easily adding alternative text to Matplotlib figures. MatplotAlt equips Jupyter notebook authors to automatically generate and surface chart descriptions with a single line of code or command, and supports a range of options that allow users to customize the generation and display of captions based on their preferences and accessibility needs. Our evaluation indicates that MatplotAlt's heuristic and LLM-based methods to generate alt text can create accurate long-form descriptions of both simple univariate and complex Matplotlib figures. We find that state-of-the-art LLMs still struggle with factual errors when describing charts, and improve the accuracy of our descriptions by prompting GPT4-turbo with heuristic-based alt text or data tables parsed from the Matplotlib figure.

Citations

• Notably Inaccessible — Data Driven Understanding of Data Science Notebook (In)Accessibility
• Attention Is All You Need
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Cited by

• A Systematic Survey on Image Description Techniques for STEM Domains
• Benchmark It Yourself (BIY): Preparing a Dataset and Benchmarking AI Models for Scatterplot-Related Tasks
• A Systematic Literature Review of Software Engineering Research on Jupyter Notebook

Discussions

• alt-text for matplotlib figures! arxiv.org/abs/2503.20089 gotta add this to my course notes. #matplotlib #accessibility [bsky, 23 points, 1 comments]
• send them this paper arxiv.org/abs/2503.20089 [bsky, 0 points, 2 comments]

Related

• Matplotlib: A 2D Graphics Environment
• Text2Chart31: Instruction Tuning for Chart Generation with Automatic Feedback
• Drawing Pandas: A Benchmark for LLMs in Generating Plotting Code
• PlotEdit: Natural Language-Driven Accessible Chart Editing in PDFs via Multimodal LLM Agents
• mPLUG-PaperOwl: Scientific Diagram Analysis with the Multimodal Large Language Model