C-SWAP: Explainability-Aware Structured Pruning for Efficient Neural Networks Compression
2025/10/21 by Baptiste Bauvin, Bauvin, Baptiste, Loïc Baret +3
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Robotics (cs.RO)
paper · pdf · doi:10.48550/arxiv.2510.18636
openalex publication_date 2025/10/21 · openalex created_date 2025/10/24 · openalex updated_date 2026/07/28
Abstract
Neural network compression has gained increasing attention in recent years, particularly in computer vision applications, where the need for model reduction is crucial for overcoming deployment constraints. Pruning is a widely used technique that prompts sparsity in model structures, e.g. weights, neurons, and layers, reducing size and inference costs. Structured pruning is especially important as it allows for the removal of entire structures, which further accelerates inference time and reduces memory overhead. However, it can be computationally expensive, requiring iterative retraining and optimization. To overcome this problem, recent methods considered one-shot setting, which applies pruning directly at post-training. Unfortunately, they often lead to a considerable drop in performance. In this paper, we focus on this issue by proposing a novel one-shot pruning framework that relies on explainable deep learning. First, we introduce a causal-aware pruning approach that leverages cause-effect relations between model predictions and structures in a progressive pruning process. It allows us to efficiently reduce the size of the network, ensuring that the removed structures do not deter the performance of the model. Then, through experiments conducted on convolution neural network and vision transformer baselines, pre-trained on classification tasks, we demonstrate that our method consistently achieves substantial reductions in model size, with minimal impact on performance, and without the need for fine-tuning. Overall, our approach outperforms its counterparts, offering the best trade-off. Our code is available on GitHub.
Citations
- Pruning By Explaining Revisited: Optimizing Attribution Methods to Prune CNNs and Transformers
- OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization
- Compressing Deep Neural Networks Using Explainable AI
- A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations
- A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations
- Causal Analysis for Robust Interpretability of Neural Networks
- Towards Automated Circuit Discovery for Mechanistic Interpretability
- OTOV2: Automatic, Generic, User-Friendly
- ZipLM: Inference-Aware Structured Pruning of Language Models
- DepGraph: Towards Any Structural Pruning
- Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning
- Advancing Model Pruning via Bi-level Optimization
- Interpretations Steered Network Pruning via Amortized Inferred Saliency Maps
- Revisiting Random Channel Pruning for Neural Network Compression
- Coarsening the Granularity: Towards Structurally Sparse Lottery Tickets
- Only Train Once: A One-Shot Neural Network Training And Pruning Framework
- Efficient Lottery Ticket Finding: Less Data is More
- Low-Complexity Probing via Finding Subnetworks
- Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Accelerate CNNs from Three Dimensions: A Comprehensive Pruning Framework
- Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks
- Captum: A unified and generic model interpretability library for PyTorch
- Utilizing Explainable AI for Quantization and Pruning of Deep Neural Networks
- ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting
- Pruning neural networks without any data by iteratively conserving synaptic flow
- Linear Mode Connectivity and the Lottery Ticket Hypothesis
- Deep Double Descent: Where Bigger Models and More Data Hurt
- One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers
- The Impact of Neural Network Overparameterization on Gradient Confusion\n and Stochastic Gradient Descent
- Sanity Checks for Saliency Maps
- SNIP: Single-shot Network Pruning based on Connection Sensitivity
- Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks
- How Important Is a Neuron?
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
- Influence-Directed Explanations for Deep Convolutional Networks
- NISP: Pruning Networks using Neuron Importance Score Propagation
- ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression
- Axiomatic Attribution for Deep Networks
- Pruning Filters for Efficient ConvNets
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- Deep Residual Learning for Image Recognition
- Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
- Learning both Weights and Connections for Efficient Neural Networks
Related