2024/05/25 by Kexin Li, Li, Kexin, Aastha Mehta +3
Engineering · #Advanced Optical Network Technologies #Advanced Photonic Communication Systems #Computers and Society (cs.CY) #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Optical Network Technologies
paper · pdf · doi:10.48550/arxiv.2405.16361
openalex publication_date 2024/05/25 · openalex created_date 2024/05/29 · openalex updated_date 2026/07/28
To protect privacy in regulated domains such as healthcare and finance, model owners may allow only remote API access while keeping both the training data and model parameters private. However, model users performing inference on such remotely hosted models may be required to transmit potentially sensitive inputs, raising privacy concerns. In this work, we present LDPKiT, a framework for non-adversarial, privacy-preserving model distillation that leverages a user's private in-distribution data while bounding privacy leakage. LDPKiT introduces a novel superimposition technique that generates approximately in-distribution samples, enabling effective knowledge transfer under local differential privacy (LDP). Experiments on Fashion-MNIST, SVHN, and PathMNIST demonstrate that LDPKiT consistently improves utility while maintaining privacy, with benefits that become more pronounced at stronger noise levels. For example, on SVHN, LDPKiT achieves nearly the same inference accuracy at ε=1.25 as at ε=2.0, yielding stronger privacy guarantees with less than a 2% accuracy reduction. We further conduct sensitivity analyses to examine the effect of dataset size on performance and provide a systematic analysis of latent space representations, offering intuitive and empirical insights into the accuracy gains of LDPKiT.