2021/01/28 by Prashanthi Ramachandran, Shivam Agarwal, Ramachandran, Prashanthi +7 · 3 citations
Computer Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Artificial intelligence #Artificial neural network #Computer network #Computer science #Computer security #Cryptography #Cryptography and Security (cs.CR) #Distributed computing #FOS: Computer and information sciences #MNIST database #Privacy-Preserving Technologies in Data #Programming language #Secret sharing #Secure multi-party computation #Server #Set (abstract data type) #Sigmoid function #Softmax function #Stochastic Gradient Optimization Techniques #Theoretical computer science #Verifiable secret sharing #cs.AI #cs.CR
paper · pdf · doi:10.48550/arxiv.2101.12078
published in arXiv (Cornell University) (Cornell University) · Appeared at the Second AAAI Workshop on Privacy-Preserving Artificial Intelligence (PPAI-21). (7 pages, technical paper.)
arxiv created 2021/01/28 · openalex publication_date 2021/01/28 · arxiv updated 2021/01/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/08
We introduce S++, a simple, robust, and deployable framework for training a neural network (NN) using private data from multiple sources, using secret-shared secure function evaluation. In short, consider a virtual third party to whom every data-holder sends their inputs, and which computes the neural network: in our case, this virtual third party is actually a set of servers which individually learn nothing, even with a malicious (but non-colluding) adversary. Previous work in this area has been limited to just one specific activation function: ReLU, rendering the approach impractical for many use-cases. For the first time, we provide fast and verifiable protocols for all common activation functions and optimize them for running in a secret-shared manner. The ability to quickly, verifiably, and robustly compute exponentiation, softmax, sigmoid, etc., allows us to use previously written NNs without modification, vastly reducing developer effort and complexity of code. In recent times, ReLU has been found to converge much faster and be more computationally efficient as compared to non-linear functions like sigmoid or tanh. However, we argue that it would be remiss not to extend the mechanism to non-linear functions such as the logistic sigmoid, tanh, and softmax that are fundamental due to their ability to express outputs as probabilities and their universal approximation property. Their contribution in RNNs and a few recent advancements also makes them more relevant.