Long-Term Value of Exploration: Measurements, Findings and Algorithms

2023/05/12 by Yi Su, Xiangyu Wang, Su, Yi +27 · 5 citations
Computer Science · Decision Sciences · Social Sciences · #Advanced Bandit Algorithms Research #Data Stream Mining Techniques #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Misinformation and Its Impacts

paper · pdf · doi:10.48550/arxiv.2305.07764

openalex publication_date 2023/05/12 · openalex created_date 2023/05/17 · openalex updated_date 2026/07/28

Abstract

Effective exploration is believed to positively influence the long-term user experience on recommendation platforms. Determining its exact benefits, however, has been challenging. Regular A/B tests on exploration often measure neutral or even negative engagement metrics while failing to capture its long-term benefits. We here introduce new experiment designs to formally quantify the long-term value of exploration by examining its effects on content corpus, and connecting content corpus growth to the long-term user experience from real-world experiments. Once established the values of exploration, we investigate the Neural Linear Bandit algorithm as a general framework to introduce exploration into any deep learning based ranking systems. We conduct live experiments on one of the largest short-form video recommendation platforms that serves billions of users to validate the new experiment designs, quantify the long-term values of exploration, and to verify the effectiveness of the adopted neural linear bandit algorithm for exploration.

Cited by

• FunnelAL: Retrieve-then-Rank Active Learning for Single-Class Discovery
• Optimality of Sub-network Laplace Approximations: New Results and Methods
• Balancing Fine-tuning and RAG: A Hybrid Strategy for Dynamic LLM Recommendation Updates
• User Exploration and Exploitation Behavior Under the Influence of Real-time Interactions in Live Streaming Environments
• When Relevance Meets Novelty: Dual-Stable Periodic Optimization for Serendipitous Recommendation

Related

• Impatient Bandits: Optimizing for the Long-Term Without Delay
• Where to Explore: A Reach and Cost-Aware Approach for Unbiased Data Collection in Recommender Systems
• Adversarial Gradient Driven Exploration for Deep Click-Through Rate Prediction
• Exploitation Over Exploration: Unmasking the Bias in Linear Bandit Recommender Offline Evaluation
• Deep Bayesian Bandits: Exploring in Online Personalized Recommendations