LUCid: Redefining Relevance For Lifelong Personalization

2026/04/30 by Chimaobi Okite, Anika Misra, Joyce Chai +1 · 1 citation
Computer Science · #cs.IR

paper · pdf · doi:10.48550/arxiv.2604.26996

second version

arxiv created 2026/08/05 · arxiv updated 2026/08/06

Abstract

Work to date has mainly relied on semantic proximity to identify relevant content for lifelong personalization. However, situational relevance is often more important for determining which information is useful for a user's actual task and context. In this paper, we introduce the Proximity Advantage (PA) score, a metric for quantifying semantic proximity bias, and show that existing personalization benchmarks largely conflate semantic and situational proximity, leaving it unclear whether current systems truly capture situational relevance. To support this metric, we introduce LUCid, a diagnostic benchmark of 1,936 user queries paired with long interaction histories, designed to isolate situational relevance from semantic proximity. Our experiments across different stages of the modern personalization pipeline (retrieval, reranking, and generation) reveal significant performance collapse: retrieval recall drops to near zero on the hardest instances, and response alignment remains near 50% even for state-of-the-art models such as Gemini-3-Flash, GPT-5.4, and Claude Haiku, highlighting a fundamental mismatch between the relevance encoded by current systems and what lifelong personalization demands.

Citations

• SafeCRS: Personalized Safety Alignment for LLM-Based Conversational Recommender Systems
• PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memory
• A Simple Yet Strong Baseline for Long-Term Conversational Memory of LLM Agents
• Benchmarking and Improving LLM Robustness for Personalized Generation
• When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation
• Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
• Personalized Safety in LLMs: A Benchmark and A Planning-Based Agent Approach
• Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
• Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale
• In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents
• Toward Multi-Session Personalized Conversation: A Large-Scale Dataset and Hierarchical Tree Framework for Implicit Reasoning
• PersonaBench: Evaluating AI Models on Understanding Personal Information through Accessing (Synthetic) Private User Data
• RAG vs. GraphRAG: A Systematic Evaluation and Key Insights
• Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs
• CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models
• Jasper and Stella: distillation of SOTA embedding models
• SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models
• LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
• Graph Retrieval-Augmented Generation: A Survey
• LongLaMP: A Benchmark for Personalized Long-form Text Generation
• WildChat: 1M ChatGPT Interaction Logs in the Wild
• Make Your LLM Fully Utilize the Context
• From Local to Global: A Graph RAG Approach to Query-Focused Summarization
• RULER: What's the Real Context Size of Your Long-Context Language Models?
• Evaluating Very Long-Term Conversational Memory of LLM Agents
• Data Engineering for Scaling Language Models to 128K Context
• M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation
• Faithful Persona-based Conversational Dataset Generation with Large Language Models
• The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions
• MemGPT: Towards LLMs as Operating Systems
• Towards General Text Embeddings with Multi-stage Contrastive Learning
• Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting
• Enhancing Chat Language Models by Scaling High-quality Instructional Conversations
• LaMP: When Large Language Models Meet Personalization
• Re2G: Retrieve, Rerank, Generate
• Unsupervised Dense Information Retrieval with Contrastive Learning
• SituatedQA: Incorporating Extra-Linguistic Contexts into QA
• BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Dense Passage Retrieval for Open-Domain Question Answering
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks