2024/06/30 by Stephen Meisenbacher, Meisenbacher, Stephen, Maulik Chevli +3
Computer Science · Social Sciences · #Access Control and Trust #Computation and Language (cs.CL) #FOS: Computer and information sciences #Privacy-Preserving Technologies in Data
paper · pdf · doi:10.48550/arxiv.2407.00638
openalex publication_date 2024/06/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Applications of Differential Privacy (DP) in NLP must distinguish between the syntactic level on which a proposed mechanism operates, often taking the form of word-level or document-level privatization. Recently, several word-level Metric Differential Privacy approaches have been proposed, which rely on this generalized DP notion for operating in word embedding spaces. These approaches, however, often fail to produce semantically coherent textual outputs, and their application at the sentence- or document-level is only possible by a basic composition of word perturbations. In this work, we strive to address these challenges by operating between the word and sentence levels, namely with collocations. By perturbing n-grams rather than single words, we devise a method where composed privatized outputs have higher semantic coherence and variable length. This is accomplished by constructing an embedding model based on frequently occurring word groups, in which unigram words co-exist with bi- and trigram collocations. We evaluate our method in utility and privacy tests, which make a clear case for tokenization strategies beyond the word level.