1-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoning

2025/08/11 by Wenkai Li, Li, Wenkai, Liwen Sun +8 · 1 voice · 3 citations
Computer Science · Social Sciences · #Access Control and Trust #Cryptography and Data Security #Privacy-Preserving Technologies in Data #cs.AI

paper · pdf · doi:10.48550/arxiv.2508.07667

openalex publication_date 2025/08/11 · openalex created_date 2025/10/15 · openalex updated_date 2026/07/28

Abstract

Addressing contextual privacy concerns remains challenging in interactive settings where large language models (LLMs) process information from multiple sources (e.g., summarizing meetings with private and public information). We introduce a multi-agent framework that decomposes privacy reasoning into specialized subtasks (extraction, classification), reducing the information load on any single agent while enabling iterative validation and more reliable adherence to contextual privacy norms. To understand how privacy errors emerge and propagate, we conduct a systematic ablation over information-flow topologies, revealing when and why upstream detection mistakes cascade into downstream leakage. Experiments on the ConfAIde and PrivacyLens benchmark with several open-source and closed-sourced LLMs demonstrate that our best multi-agent configuration substantially reduces private information leakage (18% on ConfAIde and 19% on PrivacyLens with GPT-4o) while preserving the fidelity of public content, outperforming single-agent baselines. These results highlight the promise of principled information-flow design in multi-agent systems for contextual privacy with LLMs.

Citations

• User Perceptions of Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios
• Position: Privacy Is Not Just Memorization!
• Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agents
• Position: Contextual Integrity is Inadequately Applied to Language Models
• NAT-NL2GQL: A Novel Multi-Agent Framework for Translating Natural Language to Graph Query Language
• CI-Bench: Benchmarking Contextual Integrity of AI Assistants on Synthetic Data
• PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action
• Privacy Checklist: Privacy Violation Detection Grounding on Contextual Integrity Theory
• MegaAgent: A Large-Scale Autonomous LLM-based Multi-Agent System Without Predefined SOPs
• Operationalizing Contextual Integrity in Privacy-Conscious Assistants
• The Llama 3 Herd of Models
• GoldCoin: Grounding Large Language Models in Privacy Laws via Contextual Integrity Theory
• AirGapAgent: Protecting Privacy-Conscious Conversational Agents
• Can LLMs get help from other LLMs without revealing private information?
• Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems
• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory
• Moral Foundations of Large Language Models
• Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents
• Gorilla: Large Language Model Connected with Massive APIs
• Are Chatbots Ready for Privacy-Sensitive Applications? An Investigation into Input Regurgitation and Prompt-Induced Sanitization
• What Does it Mean for a Language Model to Preserve Privacy?
• What Would Jiminy Cricket Do? Towards Agents That Behave Morally
• Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences
• Unique Security and Privacy Threats of Large Language Models: A Comprehensive Survey

Cited by

• User Perceptions of Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios
• MAGPIE: A benchmark for Multi-AGent contextual PrIvacy Evaluation
• Position: Privacy Is Not Just Memorization!

Discussions

• Presenting two papers! On Tuesday at 12:40pm (Room I; 12pm session): 1-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoning arxiv.org/abs/2508.07667 (followed by poster session at [bsky, 5 points, 1 comments]

Related

• Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agents
• MAGPIE: A dataset for Multi-AGent contextual PrIvacy Evaluation
• PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action
• MAGPIE: A benchmark for Multi-AGent contextual PrIvacy Evaluation
• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory