2020/12/09 by Beongjun Choi, Choi, Beongjun, Jy-yong Sohn +5
Computer Science · #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques #Cryptography and Data Security
paper · pdf · doi:10.48550/arxiv.2012.05433
Federated learning has been spotlighted as a way to train neural networks\nusing distributed data with no need for individual nodes to share data.\nUnfortunately, it has also been shown that adversaries may be able to extract\nlocal data contents off model parameters transmitted during federated learning.\nA recent solution based on the secure aggregation primitive enabled\nprivacy-preserving federated learning, but at the expense of significant extra\ncommunication/computational resources. In this paper, we propose a\nlow-complexity scheme that provides data privacy using substantially reduced\ncommunication/computational resources relative to the existing secure solution.\nThe key idea behind the suggested scheme is to design the topology of\nsecret-sharing nodes as a sparse random graph instead of the complete graph\ncorresponding to the existing solution. We first obtain the necessary and\nsufficient condition on the graph to guarantee both reliability and privacy. We\nthen suggest using the Erd Hos-R 'enyi graph in particular and provide\ntheoretical guarantees on the reliability/privacy of the proposed scheme.\nThrough extensive real-world experiments, we demonstrate that our scheme, using\nonly 20 \∼ 30 % of the resources required in the conventional scheme,\nmaintains virtually the same levels of reliability and data privacy in\npractical federated learning systems.\n