Highly Accurate CNN Inference Using Approximate Activation Functions over Homomorphic Encryption

2020/09/08 by Takumi Ishiyama, Takuya Suzuki, T. Suzuki +4 · 6 citations
Computer Science · Mathematics · #Activation function #Artificial intelligence #Artificial neural network #Chaos-based Image/Signal Encryption #Cloud computing #Computer science #Computer security #Convolutional neural network #Cryptography and Data Security #Data mining #Encryption #Homomorphic encryption #Inference #MNIST database #Mathematics #Normalization (sociology) #Operating system #Polynomial #Privacy-Preserving Technologies in Data #cs.CR #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2009.03727

published in arXiv (Cornell University) (Cornell University) · Accepted at 7th International Workshop on Privacy and Security of Big Data in conjunction with 2020 IEEE International Conference on Big Data (IEEE BigData 2020)

openalex publication_date 2020/09/08 · arxiv created 2020/12/02 · arxiv updated 2020/12/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

In the big data era, cloud-based machine learning as a service (MLaaS) has attracted considerable attention. However, when handling sensitive data, such as financial and medical data, a privacy issue emerges, because the cloud server can access clients' raw data. A common method of handling sensitive data in the cloud uses homomorphic encryption, which allows computation over encrypted data without decryption. Previous research usually adopted a low-degree polynomial mapping function, such as the square function, for data classification. However, this technique results in low classification accuracy. In this study, we seek to improve the classification accuracy for inference processing in a convolutional neural network (CNN) while using homomorphic encryption. We adopt an activation function that approximates Google's Swish activation function while using a fourth-order polynomial. We also adopt batch normalization to normalize the inputs for the Swish function to fit the input range to minimize the error. We implemented CNN inference labeling over homomorphic encryption using the Microsoft's Simple Encrypted Arithmetic Library for the Cheon-Kim-Kim-Song (CKKS) scheme. The experimental evaluations confirmed classification accuracies of 99.22% and 80.48% for MNIST and CIFAR-10, respectively, which entails 0.04% and 4.11% improvements, respectively, over previous methods.

Citations

• How to share a secret

Cited by

• FastFHE: Packing-Scalable and Depthwise-Separable CNN Inference Over FHE
• Peregrine: One-Shot Fine-Tuning for FHE Inference of General Deep CNNs
• PRISM: Privacy-preserving Inference System with Homomorphic Encryption and Modular Activation
• Practical and Private Hybrid ML Inference with Fully Homomorphic Encryption
• PP-STAT: An Efficient Privacy-Preserving Statistical Analysis Framework using Homomorphic Encryption
• A Training Framework for Optimal and Stable Training of Polynomial Neural Networks

Related

• PRISM: Privacy-preserving Inference System with Homomorphic Encryption and Modular Activation
• Efficient CNN Building Blocks for Encrypted Data
• Towards the AlexNet Moment for Homomorphic Encryption: HCNN, theFirst Homomorphic CNN on Encrypted Data with GPUs
• Activate Me!: Designing Efficient Activation Functions for Privacy-Preserving Machine Learning with Fully Homomorphic Encryption
• Privacy preserving Neural Network Inference on Encrypted Data with GPUs