Concept activation vectors: a unifying view and adversarial attacks

2025/09/26 by Schnoor, Ekkehard, Tiomoko, Malik, Said, Jawher +2
#FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Probability (math.PR)

paper · doi:10.48550/arxiv.2509.22755

Abstract

Concept Activation Vectors (CAVs) are a tool from explainable AI, offering a promising approach for understanding how human-understandable concepts are encoded in a model's latent spaces. They are computed from hidden-layer activations of inputs belonging either to a concept class or to non-concept examples. Adopting a probabilistic perspective, the distribution of the (non-)concept inputs induces a distribution over the CAV, making it a random vector in the latent space. This enables us to derive mean and covariance for different types of CAVs, leading to a unified theoretical view. This probabilistic perspective also reveals a potential vulnerability: CAVs can strongly depend on the rather arbitrary non-concept distribution, a factor largely overlooked in prior work. We illustrate this with a simple yet effective adversarial attack, underscoring the need for a more systematic study.

Citations

• FastCAV: Efficient Computation of Concept Activation Vectors for Explaining Deep Neural Networks
• High-Dimensional Analysis of Bootstrap Ensemble Classifiers
• Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero
• From Hope to Safety: Unlearning Biases of Deep Models via Gradient Penalization in Latent Space
• Navigating Neural Space: Revisiting Concept Activation Vectors to Overcome Directional Divergence
• Acquisition of Chess Knowledge in AlphaZero
• Concept Bottleneck Models
• Random Matrix Theory Proves that Deep Learning Representations of GAN-data Behave as Gaussian Mixtures
• Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
• Deep Residual Learning for Image Recognition

Related

• On The Variability of Concept Activation Vectors
• Concept-based Adversarial Attacks: Tricking Humans and Classifiers Alike
• Concept-based Adversarial Attack: a Probabilistic Perspective
• Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
• The Anatomy of Adversarial Attacks: Concept-based XAI Dissection