2021/05/20 by X. Qian, Qian, Xin, Diego Klabjan +1 · 4 citations
Computer Science · #Neural Networks and Applications #Adversarial Robustness in Machine Learning #Machine Learning and Algorithms
paper · pdf · doi:10.48550/arxiv.2105.10065
Neural network pruning techniques reduce the number of parameters without compromising predicting ability of a network. Many algorithms have been developed for pruning both over-parameterized fully-connected networks (FCNs) and convolutional neural networks (CNNs), but analytical studies of capabilities and compression ratios of such pruned sub-networks are lacking. We theoretically study the performance of two pruning techniques (random and magnitude-based) on FCNs and CNNs. Given a target network whose weights are independently sampled from appropriate distributions, we provide a universal approach to bound the gap between a pruned and the target network in a probabilistic sense. The results establish that there exist pruned networks with expressive power within any specified bound from the target network.