Evaluating the Efficacy of Large Language Models for Generating Fine-Grained Visual Privacy Policies in Homes

2025/08/01 by Zhang, Shuning, Ma, Ying, Yi, Xin +1 · 2 citations
#FOS: Computer and information sciences #Human-Computer Interaction (cs.HC)

paper · doi:10.48550/arxiv.2508.00321

Abstract

The proliferation of visual sensors in smart home environments, particularly through wearable devices like smart glasses, introduces profound privacy challenges. Existing privacy controls are often static and coarse-grained, failing to accommodate the dynamic and socially nuanced nature of home environments. This paper investigates the viability of using Large Language Models (LLMs) as the core of a dynamic and adaptive privacy policy engine. We propose a conceptual framework where visual data is classified using a multi-dimensional schema that considers data sensitivity, spatial context, and social presence. An LLM then reasons over this contextual information to enforce fine-grained privacy rules, such as selective object obfuscation, in real-time. Through a comparative evaluation of state-of-the-art Vision Language Models (including GPT-4o and the Qwen-VL series) in simulated home settings , our findings show the feasibility of this approach. The LLM-based engine achieved a top machine-evaluated appropriateness score of 3.99 out of 5, and the policies generated by the models received a top human-evaluated score of 4.00 out of 5.

Citations

• Exploring the Privacy and Security Challenges Faced by Migrant Domestic Workers in Chinese Smart Homes
• Privacy Leakage Overshadowed by Views of AI: A Study on Human Oversight of Privacy in Language Model Agent
• Adanonymizer: Interactively Navigating and Balancing the Duality of Privacy and Output Performance in Human-LLM Interaction
• "Ghost of the past": identifying and resolving privacy leakage from LLM's memory through proactive user interaction
• PrISM-Observer: Intervention Agent to Help Users Perform Everyday Procedures Sensed using a Smartwatch
• Private Attribute Inference from Images with Vision-Language Models
• MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• Visual Content Privacy Protection: A Survey
• Modeling the Trade-off of Privacy Preservation and Activity Recognition on Low-Resolution Images
• LVIS: A Dataset for Large Vocabulary Instance Segmentation
• Towards a Visual Privacy Advisor: Understanding and Predicting Privacy\n Risks in Images
• The Feasibility of Dynamically Granted Permissions: Aligning Mobile Privacy with User Preferences

Cited by

• The Pervasive Blind Spot: Benchmarking VLM Inference Risks on Everyday Personal Videos
• Through Their Eyes: User Perceptions on Sensitive Attribute Inference of Social Media Videos by Visual Language Models

Related

• Assessing Visual Privacy Risks in Multimodal AI: A Novel Taxonomy-Grounded Evaluation of Vision-Language Models
• Measuring Physical-World Privacy Awareness of Large Language Models: An Evaluation Benchmark
• Towards Privacy-Preserving and Personalized Smart Homes via Tailored Small Language Models
• Privacy-Aware Visual Language Models
• Multi-PA: A Multi-perspective Benchmark on Privacy Assessment for Large Vision-Language Models