2024/05/24 by Xiangrui Xu, Zhang Qiao, Xu, Xiangrui +7 · 2 citations
Computer Science · #Artificial Intelligence (cs.AI) #Complexity and Algorithms in Graphs #Cryptography and Data Security #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy-Preserving Technologies in Data
paper · pdf · doi:10.48550/arxiv.2405.17485
openalex publication_date 2024/05/24 · openalex created_date 2024/05/30 · openalex updated_date 2026/07/28
The prevalent use of Transformer-like models, exemplified by ChatGPT in modern language processing applications, underscores the critical need for enabling private inference essential for many cloud-based services reliant on such models. However, current privacy-preserving frameworks impose significant communication burden, especially for non-linear computation in Transformer model. In this paper, we introduce a novel plug-in method Comet to effectively reduce the communication cost without compromising the inference performance. We second introduce an efficient approximation method to eliminate the heavy communication in finding good initial approximation. We evaluate our Comet on Bert and RoBERTa models with GLUE benchmark datasets, showing up to 3.9× less communication and 3.5× speedups while keep competitive model performance compared to the prior art.