Differentially Private Image Classification by Learning Priors from Random Processes

2023/06/08 by Tang, Xinyu, Panda, Ashwinee, Sehwag, Vikash +1 · 4 citations
#Computer Vision and Pattern Recognition (cs.CV) #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.2306.06076

Abstract

In privacy-preserving machine learning, differentially private stochastic gradient descent (DP-SGD) performs worse than SGD due to per-sample gradient clipping and noise addition. A recent focus in private learning research is improving the performance of DP-SGD on private data by incorporating priors that are learned on real-world public data. In this work, we explore how we can improve the privacy-utility tradeoff of DP-SGD by learning priors from images generated by random processes and transferring these priors to private data. We propose DP-RandP, a three-phase approach. We attain new state-of-the-art accuracy when training from scratch on CIFAR10, CIFAR100, MedMNIST and ImageNet for a range of privacy budgets ε ∈ [1, 8]. In particular, we improve the previous best reported accuracy on CIFAR10 from 60.6 % to 72.3 % for ε=1.

Cited by

• Understanding Private Learning From Feature Perspective
• Towards Reliable and Generalizable Differentially Private Machine Learning (Extended Version)
• Separating Knowledge and Perception with Procedural Data
• Convergent Privacy Framework for Multi-layer GNNs through Contractive Message Passing

Related

• Large Scale Transfer Learning for Differentially Private Image Classification
• Unlocking High-Accuracy Differentially Private Image Classification through Scale
• SA-DPSGD: Differentially Private Stochastic Gradient Descent based on Simulated Annealing
• Unlocking Accuracy and Fairness in Differentially Private Image Classification
• Dynamic Differential-Privacy Preserving SGD