Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wild

2024/07/16 by Niloofar Mireshghallah, Mireshghallah, Niloofar, Maria Antoniak +7 · 2 voices · 21 citations
Computer Science · Social Sciences · #Cybercrime and Law Enforcement Studies #Privacy, Security, and Data Protection #Spam and Phishing Detection #cs.CL

paper · pdf · doi:10.48550/arxiv.2407.11438

openalex publication_date 2024/07/16 · openalex created_date 2024/10/26 · openalex updated_date 2026/08/01

Abstract

Measuring personal disclosures made in human-chatbot interactions can provide a better understanding of users' AI literacy and facilitate privacy research for large language models (LLMs). We run an extensive, fine-grained analysis on the personal disclosures made by real users to commercial GPT models, investigating the leakage of personally identifiable and sensitive information. To understand the contexts in which users disclose to chatbots, we develop a taxonomy of tasks and sensitive topics, based on qualitative and quantitative analysis of naturally occurring conversations. We discuss these potential privacy harms and observe that: (1) personally identifiable information (PII) appears in unexpected contexts such as in translation or code editing (48% and 16% of the time, respectively) and (2) PII detection alone is insufficient to capture the sensitive topics that are common in human-chatbot interactions, such as detailed sexual preferences or specific drug use habits. We believe that these high disclosure rates are of significant importance for researchers and data curators, and we call for the design of appropriate nudging mechanisms to help users moderate their interactions.

Cited by

• Understanding How University Guidelines Address Privacy and Security Issues of Generative AI in Academic Settings
• AI Fiction in the Wild
• Engagement-Optimized Care: When LLMs become Mental Health Infrastructure
• User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policies
• A Common Pool of Privacy Problems: Legal and Technical Lessons from a Large-Scale Web-Scraped Machine Learning Dataset
• Characterizing Narrative Content in Web-scale LLM Pretraining Data
• Investigating AI in Peer Support via Multi-Module System-Driven Embodied Conversational Agents
• The Subtle Art of Defection: Understanding Uncooperative Behaviors in LLM based Multi-Agent Systems
• "Power of Words": Stealthy and Adaptive Private Information Elicitation via LLM Communication Strategies
• User Perceptions of Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios
• PrivacyPAD: A Reinforcement Learning Framework for Dynamic Privacy-Aware Delegation
• Missing the Margins: A Systematic Literature Review on the Demographic Representativeness of LLMs
• Position: Privacy Is Not Just Memorization!
• PATCH: Mitigating PII Leakage in Language Models with Privacy-Aware Targeted Circuit PatcHing
• Disclosure and Evaluation as Fairness Interventions for General-Purpose AI
• VortexPIA: Indirect Prompt Injection Attack against LLMs for Efficient Extraction of User Privacy
• Operationalizing Data Minimization for Privacy-Preserving LLM Prompting
• Beyond PII: How Users Attempt to Estimate and Mitigate Implicit LLM Inference
• The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs
• Understanding Privacy Norms Around LLM-Based Chatbots: A Contextual Integrity Perspective
• Show or Tell? Modeling the evolution of request-making in Human-LLM conversations

Discussions

• People have been using all the different chatbots, including ChatGPT, for erotica from the beginning, but I continue to be surprised by how few people are willing to acknowledge this. We found this in [bsky, 29 points, 2 comments]
• detailed sexual preferences or specific drug use habits.” www.arxiv.org/abs/2407.11438 [bsky, 1 points, 0 comments]

Related

• Are Chatbots Ready for Privacy-Sensitive Applications? An Investigation into Input Regurgitation and Prompt-Induced Sanitization
• Beyond Memorization: Violating Privacy Via Inference with Large Language Models
• Understanding Privacy Norms Around LLM-Based Chatbots: A Contextual Integrity Perspective
• Malicious LLM-Based Conversational AI Makes Users Reveal Personal Information
• User Privacy Harms and Risks in Conversational AI: A Proposed Framework