Bounding the Excess Risk for Linear Models Trained on Marginal-Preserving, Differentially-Private, Synthetic Data

2024/02/06 by Yvonne Zhou, Mingyu Liang, Zhou, Yvonne +11
Decision Sciences · #Cryptography and Security (cs.CR) #Data Quality and Management #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2402.04375

openalex publication_date 2024/02/06 · openalex created_date 2024/02/09 · openalex updated_date 2026/07/28

Abstract

The growing use of machine learning (ML) has raised concerns that an ML model may reveal private information about an individual who has contributed to the training dataset. To prevent leakage of sensitive data, we consider using differentially-private (DP), synthetic training data instead of real training data to train an ML model. A key desirable property of synthetic data is its ability to preserve the low-order marginals of the original distribution. Our main contribution comprises novel upper and lower bounds on the excess empirical risk of linear models trained on such synthetic data, for continuous and Lipschitz loss functions. We perform extensive experimentation alongside our theoretical results.

Related

• Differentially Private Generalized Linear Models Revisited
• Private Empirical Risk Minimization Beyond the Worst Case: The Effect of the Constraint Set Geometry
• Optimal Differentially Private Model Training with Public Data
• Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds
• Differentially Private Confidence Intervals for Empirical Risk Minimization