LangXAI: Integrating Large Vision Models for Generating Textual Explanations to Enhance Explainability in Visual Perception Tasks

2024/02/19 by Truong Thanh Hung Nguyen, Tobias Clement, Nguyen, Truong Thanh Hung +13 · 5 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Data Visualization and Analytics #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.2402.12525

openalex publication_date 2024/02/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

LangXAI is a framework that integrates Explainable Artificial Intelligence (XAI) with advanced vision models to generate textual explanations for visual recognition tasks. Despite XAI advancements, an understanding gap persists for end-users with limited domain knowledge in artificial intelligence and computer vision. LangXAI addresses this by furnishing text-based explanations for classification, object detection, and semantic segmentation model outputs to end-users. Preliminary results demonstrate LangXAI's enhanced plausibility, with high BERTScore across tasks, fostering a more transparent and reliable AI framework on vision tasks for end-users.

Cited by

• Motion2Meaning: A Clinician-Centered Framework for Contestable LLM in Parkinson's Disease Gait Interpretation
• A Novel Framework for Automated Explain Vision Model Using Vision-Language Models
• SimInterview: Transforming Business Education through Large Language Model-Based Simulated Multilingual Interview Training System
• ConGaIT: A Clinician-Centered Dashboard for Contestable AI in Parkinson's Disease Care
• Multimedia Verification Through Multi-Agent Deep Research Multimodal Large Language Models

Related

• On Efficient Language and Vision Assistants for Visually-Situated Natural Language Understanding: What Matters in Reading and Reasoning
• Language Model as Visual Explainer
• Uni-NLX: Unifying Textual Explanations for Vision and Vision-Language Tasks
• NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Tasks
• Found in Translation: semantic approaches for enhancing AI interpretability in face verification