Tau-Eval: A Unified Evaluation Framework for Useful and Private Text Anonymization

2025/06/06 by Gabriel Loiseau, Damien Sileo, Loiseau, Gabriel +7
Computer Science · Social Sciences · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection #Privacy, Security, and Data Protection #Privacy-Preserving Technologies in Data

paper · pdf · doi:10.48550/arxiv.2506.05979

openalex publication_date 2025/06/06 · openalex created_date 2025/10/06 · openalex updated_date 2026/07/28

Abstract

Text anonymization is the process of removing or obfuscating information from textual data to protect the privacy of individuals. This process inherently involves a complex trade-off between privacy protection and information preservation, where stringent anonymization methods can significantly impact the text's utility for downstream applications. Evaluating the effectiveness of text anonymization proves challenging from both privacy and utility perspectives, as there is no universal benchmark that can comprehensively assess anonymization techniques across diverse, and sometimes contradictory contexts. We present Tau-Eval, an open-source framework for benchmarking text anonymization methods through the lens of privacy and utility task sensitivity. A Python library, code, documentation and tutorials are publicly available.

Citations

• Qwen2.5 Technical Report
• A Comparative Study of Clinical ModernBERT and BioMedical ModernBERT on the DDXPlus Dataset
• StyleRemix: Interpretable Authorship Obfuscation via Distillation and Perturbation of Style Elements
• TAROT: Task-Oriented Authorship Obfuscation Using Policy Optimization Methods
• The Llama 3 Herd of Models
• Cloaked Classifiers: Pseudonymization Strategies on Sensitive Classification Tasks
• Keep It Private: Unsupervised Privatization of Online Text
• Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
• CamemBERT-bio: Leveraging Continual Pre-training for Cost-Effective Models on French Biomedical Data
• Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurements
• WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation
• Datasets: A Community Library for Natural Language Processing
• XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond
• Dynabench: Rethinking Benchmarking in NLP
• Learning from the Worst: Dynamically Generated Datasets to Improve\n Online Hate Detection
• DynaSent: A Dynamic Benchmark for Sentiment Analysis
• Beyond English-Centric Multilingual Machine Translation
• Closing the Gap: Joint De-Identification and Concept Extraction in the\n Clinical Domain
• PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
• Adversarial NLI: A New Benchmark for Natural Language Understanding
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• BERTScore: Evaluating Text Generation with BERT
• ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission
• Lessons from Natural Language Inference in the Clinical Domain
• Neural Network Acceptability Judgments

Related

• No Intruder, no Validity: Evaluation Criteria for Privacy-Preserving Text Anonymization
• The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization
• How do we measure privacy in text? A survey of text anonymization metrics
• Robust Utility-Preserving Text Anonymization Based on Large Language Models
• The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization