Optimal Unbiased Randomizers for Regression with Label Differential Privacy

2023/12/09 by Ashwinkumar Badanidiyuru, Badih Ghazi, Badanidiyuru, Ashwinkumar +13 · 1 citation
Computer Science · #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification #Privacy-Preserving Technologies in Data

paper · pdf · doi:10.48550/arxiv.2312.05659

openalex publication_date 2023/12/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose a new family of label randomizers for training regression models under the constraint of label differential privacy (DP). In particular, we leverage the trade-offs between bias and variance to construct better label randomizers depending on a privately estimated prior distribution over the labels. We demonstrate that these randomizers achieve state-of-the-art privacy-utility trade-offs on several datasets, highlighting the importance of reducing bias when training neural networks with label DP. We also provide theoretical results shedding light on the structural properties of the optimal unbiased randomizers.

Cited by

• GOPHER: Optimization-based Phenotype Randomization for Genome-Wide Association Studies with Differential Privacy

Related

• Regression with Label Differential Privacy
• Deep Learning with Label Differential Privacy
• Label differential privacy via clustering
• Label Differential Privacy via Aggregation
• Enhancing Learning with Label Differential Privacy by Vector Approximation