A Game-Theoretic Taxonomy of Visual Concepts in DNNs

2021/06/21 by Xu Cheng, Cheng, Xu, Chuntung Chu +7 · 2 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #Cell Image Analysis Techniques #Computer Vision and Pattern Recognition (cs.CV) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2106.10938

openalex publication_date 2021/06/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we rethink how a DNN encodes visual concepts of different complexities from a new perspective, i.e. the game-theoretic multi-order interactions between pixels in an image. Beyond the categorical taxonomy of objects and the cognitive taxonomy of textures and shapes, we provide a new taxonomy of visual concepts, which helps us interpret the encoding of shapes and textures, in terms of concept complexities. In this way, based on multi-order interactions, we find three distinctive signal-processing behaviors of DNNs encoding textures. Besides, we also discover the flexibility for a DNN to encode shapes is lower than the flexibility of encoding textures. Furthermore, we analyze how DNNs encode outlier samples, and explore the impacts of network architectures on interactions. Additionally, we clarify the crucial role of the multi-order interactions in real-world applications. The code will be released when the paper is accepted.

Citations

• Understanding Neural Networks Through Deep Visualization
• Synthesizing the preferred inputs for neurons in neural networks via\n deep generator networks
• Improved Regularization of Convolutional Neural Networks with Cutout
• Learning how to explain neural networks: PatternNet and PatternAttribution
• Distributional Structure
• Explanations can be manipulated and geometry is to blame
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• XRAI: Better Attributions Through Regions

Cited by

• A Survey on Interpretability in Visual Recognition
• SpaRTAN: Spatial Reinforcement Token-based Aggregation Network for Visual Recognition

Related

• Game-Theoretic Understanding of Misclassification
• Explaining Generalization Power of a DNN Using Interactive Concepts
• Visualizing the Emergence of Intermediate Visual Patterns in DNNs
• Discovering and Explaining the Representation Bottleneck of DNNs
• Randomness of Low-Layer Parameters Determines Confusing Samples in Terms of Interaction Representations of a DNN