Making Models Shallow Again: Jointly Learning to Reduce Non-Linearity and Depth for Latency-Efficient Private Inference

2023/04/26 by Souvik Kundu, Yuke Zhang, Kundu, Souvik +5 · 1 citation
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy-Preserving Technologies in Data

paper · pdf · doi:10.48550/arxiv.2304.13274

openalex publication_date 2023/04/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large number of ReLU and MAC operations of Deep neural networks make them ill-suited for latency and compute-efficient private inference. In this paper, we present a model optimization method that allows a model to learn to be shallow. In particular, we leverage the ReLU sensitivity of a convolutional block to remove a ReLU layer and merge its succeeding and preceding convolution layers to a shallow block. Unlike existing ReLU reduction methods, our joint reduction method can yield models with improved reduction of both ReLUs and linear operations by up to 1.73x and 1.47x, respectively, evaluated with ResNet18 on CIFAR-100 without any significant accuracy-drop.

Cited by

• CryptPEFT: Efficient and Private Neural Network Inference via Parameter-Efficient Fine-Tuning

Related

• Learning to Linearize Deep Neural Networks for Secure and Efficient Private Inference
• The Shallow End: Empowering Shallower Deep-Convolutional Networks through Auxiliary Outputs
• DeepReDuce: ReLU Reduction for Fast Private Inference
• Optimally Deep Networks -- Adapting Model Depth to Datasets for Superior Efficiency
• DeepReShape: Redesigning Neural Networks for Efficient Private Inference