IPPRO: Importance-based Pruning with PRojective Offset for Magnitude-indifferent Structural Pruning

2025/07/10 by Jaeheun Jung, Jung, Jaeheun, Jaehyuk Lee +5
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2507.14171

ICCV 2025 workshop U&ME 2025 (2nd Workshop and Challenge on Unlearning and Model Editing)

arxiv created 2026/08/04 · arxiv updated 2026/08/05

Abstract

Importance-based structured pruning overwhelmingly relies on filter magnitude. This proxy is fundamentally flawed: due to scale invariance, functionally identical filters can receive arbitrarily different importance scores under rescaling. We propose IPPRO (Importance-based Pruning with PROjective Offset), a scale-invariant pruning framework grounded in projective geometry. By embedding filters into real projective space (\mathbbRP^N), IPPRO resolves the singularity at the origin, placing all filters at an equal angular distance from the zero filter. We define PROscore, which captures functional importance by measuring a filter's angular displacement toward zero under a single gradient step (directional collapse). We further connect PROscore to exact L₀ relaxation, proving this one-shot criterion reliably predicts multi-step pruning dynamics. Extensive experiments across CNNs, Vision Transformers, and LLMs (e.g., ResNet, DeiT, LLaMA) demonstrate that IPPRO consistently outperforms existing methods, yielding particularly striking gains under high compression and no-fine-tuning regimes, IPPRO establishes a robust, architecture-agnostic paradigm for neural network compression.

Citations

• Structural Pruning via Spatial-aware Information Redundancy for Semantic Segmentation
• FGP: Feature-Gradient-Prune for Efficient Convolutional Layer Pruning
• Multi-Dimensional Pruning: Joint Channel, Layer and Block Pruning with Latency Constraint
• Performance-aware Approximation of Global Channel Pruning for Multitask CNNs
• Filter Pruning based on Information Capacity and Independence
• Structured Pruning for Deep Convolutional Neural Networks: A Survey
• DepGraph: Towards Any Structural Pruning
• Structural Pruning via Latency-Saliency Knapsack
• Network Pruning via Feature Shift Minimization
• CHIP: CHannel Independence-based Pruning for Compact Neural Networks
• RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Merging
• Group Fisher Pruning for Practical Network Compression
• RED : Looking for Redundancies for Data-Free Structured Compression of Deep Neural Networks
• Convolutional Neural Network Pruning with Structural Redundancy Reduction
• Deep Model Compression based on the Training History
• Layer Pruning via Fusible Residual Convolutional Block for Deep Neural Networks
• SCOP: Scientific Control for Reliable Neural Network Pruning
• Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey
• What is the State of Neural Network Pruning?
• HRank: Filter Pruning using High-Rank Feature Map
• DBP: Discrimination Based Block-Level Pruning for Deep Model Acceleration
• Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks
• Importance Estimation for Neural Network Pruning
• COP: Customized Deep Model Compression via Regularized Correlation-Based Filter-Level Pruning
• EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis
• MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning
• Towards Optimal Structured CNN Pruning via Generative Adversarial Learning
• Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration
• Rethinking the Value of Network Pruning
• Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks
• AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inference
• Learning Efficient Convolutional Networks through Network Slimming
• ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression
• Channel Pruning for Accelerating Very Deep Neural Networks
• Pruning Filters for Efficient ConvNets
• Learning Structured Sparsity in Deep Neural Networks
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding

Related

• Importance Estimation for Neural Network Pruning
• Lookahead: A Far-Sighted Alternative of Magnitude-based Pruning
• Is Complexity Required for Neural Network Pruning? A Case Study on Global Magnitude Pruning
• Toward domain generalized pruning by scoring out-of-distribution importance
• Pruning Deep Neural Networks from a Sparsity Perspective