Understanding Instance-based Interpretability of Variational Auto-Encoders

2021/05/29 by Zhifeng Kong, Kamalika Chaudhuri, Kong, Zhifeng +1 · 11 citations
Computer Science · Mathematics · Physics and Astronomy · #Artificial intelligence #Artificial neural network #Autoencoder #Class (philosophy) #Computer science #Context (archaeology) #Deep learning #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Frame (networking) #Generative Adversarial Networks and Image Synthesis #Generative grammar #Interpretability #Interpretation (philosophy) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine learning #Model Reduction and Neural Networks #Unsupervised learning #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2105.14203

published in arXiv (Cornell University) (Cornell University) · NeurIPS 2021

openalex publication_date 2021/05/29 · openalex created_date 2021/06/22 · arxiv created 2022/01/21 · arxiv updated 2022/01/25 · openalex updated_date 2026/07/28

Abstract

Instance-based interpretation methods have been widely studied for supervised learning methods as they help explain how black box neural networks predict. However, instance-based interpretations remain ill-understood in the context of unsupervised learning. In this paper, we investigate influence functions [Koh and Liang, 2017], a popular instance-based interpretation method, for a class of deep generative models called variational auto-encoders (VAE). We formally frame the counter-factual question answered by influence functions in this setting, and through theoretical analysis, examine what they reveal about the impact of training samples on classical unsupervised learning methods. We then introduce VAE- TracIn, a computationally efficient and theoretically sound solution based on Pruthi et al. [2020], for VAEs. Finally, we evaluate VAE-TracIn on several real world datasets with extensive quantitative and qualitative analysis.

Citations

• Auto-Encoding Variational Bayes
• Representation Learning: A Review and New Perspectives
• Towards Robust Interpretability with Self-Explaining Neural Networks
• Data Shapley: Equitable Valuation of Data for Machine Learning
• Multi-Stage Influence Function
• Estimating Training Data Influence by Tracing Gradient Descent
• Learning Multiple Layers of Features from Tiny Images
• Evaluation of Similarity-based Explanations

Cited by

• IFFair: Influence Function-driven Sample Reweighting for Fair Classification
• Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Models
• A Comparative Analysis of Influence Signals for Data Debugging

Related

• Inducing Interpretable Representations with Variational Autoencoders
• Discovering Influential Factors in Variational Autoencoders
• Learning Invariances for Interpretability using Supervised VAE
• Autoencoding Variational Autoencoder
• A Multi-Level Causal Intervention Framework for Mechanistic Interpretability in Variational Autoencoders