Ablation of a Robot's Brain: Neural Networks Under a Knife

2018/12/13 by Peter Lillian, Lillian, Peter E., Richard Meyes +3 · 4 citations
Computer Science · Neuroscience · #Reinforcement Learning in Robotics #Neural dynamics and brain function #EEG and Brain-Computer Interfaces

paper · pdf · doi:10.48550/arxiv.1812.05687

Abstract

It is still not fully understood exactly how neural networks are able to solve the complex tasks that have recently pushed AI research forward. We present a novel method for determining how information is structured inside a neural network. Using ablation (a neuroscience technique for cutting away parts of a brain to determine their function), we approach several neural network architectures from a biological perspective. Through an analysis of this method's results, we examine important similarities between biological and artificial neural networks to search for the implicit knowledge locked away in the network's weights.

Citations

• Opening the Black Box of Deep Neural Networks via Information
• On the importance of single directions for generalization

Cited by

• What Really Lives in the Swamp? Thought Experiments and the Illustration of Scientific Reasoning
• Detection Transformers Under the Knife: A Neuroscience-Inspired Approach to Ablations
• Linking average- and worst-case perturbation robustness via class selectivity and dimensionality
• Fine-grained Interpretation and Causation Analysis in Deep NLP Models

Related

• Ablation Studies in Artificial Neural Networks
• Neural Networks, Artificial Intelligence and the Computational Brain
• Under the Hood of Neural Networks: Characterizing Learned Representations by Functional Neuron Populations and Network Ablations
• Rethinking the Artificial Neural Networks: A Mesh of Subnets with a Central Mechanism for Storing and Predicting the Data
• Brain-inspired learning in artificial neural networks: a review