Operationalizing Contextual Integrity in Privacy-Conscious Assistants

2024/08/05 by Sahra Ghalebikesabi, Eugene Bagdasaryan, Ghalebikesabi, Sahra +23 · 16 citations
Computer Science · Psychology · Social Sciences · #Artificial Intelligence (cs.AI) #Cognitive Functions and Memory #FOS: Computer and information sciences #Privacy, Security, and Data Protection #Privacy-Preserving Technologies in Data

paper · pdf · doi:10.48550/arxiv.2408.02373

openalex publication_date 2024/08/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Advanced AI assistants combine frontier LLMs and tool access to autonomously perform complex tasks on behalf of users. While the helpfulness of such assistants can increase dramatically with access to user information including emails and documents, this raises privacy concerns about assistants sharing inappropriate information with third parties without user supervision. To steer information-sharing assistants to behave in accordance with privacy expectations, we propose to operationalize contextual integrity (CI), a framework that equates privacy with the appropriate flow of information in a given context. In particular, we design and evaluate a number of strategies to steer assistants' information-sharing actions to be CI compliant. Our evaluation is based on a novel form filling benchmark composed of human annotations of common webform applications, and it reveals that prompting frontier LLMs to perform CI-based reasoning yields strong results.

Cited by

• Share No More Than the Request Requires: Federated Disclosure for Perspective-Aware AI
• Intent-Governed Tool Authorization for AI Agents
• Privacy Practices of Browser Agents
• Personalizing Agent Privacy Decisions via Logical Entailment
• ConVerse: Benchmarking Contextual Safety in Agent-to-Agent Conversations
• Not My Agent, Not My Boundary? Elicitation of Personal Privacy Boundaries in AI-Delegated Information Sharing
• Position: Human-Robot Interaction in Embodied Intelligence Demands a Shift From Static Privacy Controls to Dynamic Learning
• SoK: Data Minimization in Machine Learning
• 1-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoning
• Understanding Privacy Norms Around LLM-Based Chatbots: A Contextual Integrity Perspective
• Privacy Reasoning in Ambiguous Contexts
• Context Reasoner: Incentivizing Reasoning Capability for Contextualized Privacy and Safety Compliance via Reinforcement Learning
• Can Large Language Models Really Recognize Your Name?
• Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks
• CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents
• Do LLMs Know What Is Private Internally? Probing and Steering Contextual Privacy Norms in Large Language Model Representations
• ClayBuddy: A Framework, Evaluation, & Mitigation of Coding Agent Failures

Related

• CI-Bench: Benchmarking Contextual Integrity of AI Assistants on Synthetic Data
• Privacy Leakage Overshadowed by Views of AI: A Study on Human Oversight of Privacy in Language Model Agent
• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory
• Towards Aligning Personalized Conversational Recommendation Agents with Users' Privacy Preferences
• Not My Agent, Not My Boundary? Elicitation of Personal Privacy Boundaries in AI-Delegated Information Sharing