Generating private data with user customization

2020/12/02 by Xiao Dong Chen, Xiao Chen, Thomas Navidi +4
Computer Science · Engineering · Social Sciences · #FOS: Computer and information sciences #FOS: Electrical engineering #Face recognition and analysis #Machine Learning (cs.LG) #Privacy, Security, and Data Protection #Privacy-Preserving Technologies in Data #Signal Processing (eess.SP) #cs.LG #eess.SP #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2012.01467

arXiv admin note: substantial text overlap with arXiv:1904.09415

arxiv created 2020/12/02 · openalex publication_date 2020/12/02 · arxiv updated 2020/12/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Personal devices such as mobile phones can produce and store large amounts of data that can enhance machine learning models; however, this data may contain private information specific to the data owner that prevents the release of the data. We want to reduce the correlation between user-specific private information and the data while retaining the useful information. Rather than training a large model to achieve privatization from end to end, we first decouple the creation of a latent representation, and then privatize the data that allows user-specific privatization to occur in a setting with limited computation and minimal disturbance on the utility of the data. We leverage a Variational Autoencoder (VAE) to create a compact latent representation of the data that remains fixed for all devices and all possible private labels. We then train a small generative filter to perturb the latent representation based on user specified preferences regarding the private and utility information. The small filter is trained via a GAN-type robust optimization that can take place on a distributed device such as a phone or tablet. Under special conditions of our linear filter, we disclose the connections between our generative approach and renyi differential privacy. We conduct experiments on multiple datasets including MNIST, UCI-Adult, and CelebA, and give a thorough evaluation including visualizing the geometry of the latent embeddings and estimating the empirical mutual information to show the effectiveness of our approach.

Citations

• Auto-Encoding Variational Bayes
• ImageNet classification with deep convolutional neural networks
• Equality of Opportunity in Supervised Learning
• Differentially Private Variational Inference for Non-conjugate Models
• Learning to Protect Communications with Adversarial Neural Cryptography
• DEEProtect: Enabling Inference-based Access Control on Mobile Sensing Applications
• Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• Learning Controllable Fair Representations
• Efficient Estimation of Mutual Information for Strongly Dependent Variables
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Distributed generation of privacy preserving data with user customization

Related

• Distributed generation of privacy preserving data with user customization
• Private Model Compression via Knowledge Distillation
• imdpGAN: Generating Private and Specific Data with Generative Adversarial Networks
• Better accuracy with quantified privacy: representations learned via reconstructive adversarial network
• Collecting and Analyzing Data from Smart Device Users with Local Differential Privacy