Partially Observable Markov Decision Process for Recommender Systems

2016/08/28 by Zhongqi Lu, Qiang Yang, Lu, Zhongqi +1 · 2 citations
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Recommender Systems and Techniques #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.1608.07793

openalex publication_date 2016/08/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We report the "Recurrent Deterioration" (RD) phenomenon observed in online recommender systems. The RD phenomenon is reflected by the trend of performance degradation when the recommendation model is always trained based on users' feedbacks of the previous recommendations. There are several reasons for the recommender systems to encounter the RD phenomenon, including the lack of negative training data and the evolution of users' interests, etc. Motivated to tackle the problems causing the RD phenomenon, we propose the POMDP-Rec framework, which is a neural-optimized Partially Observable Markov Decision Process algorithm for recommender systems. We show that the POMDP-Rec framework effectively uses the accumulated historical data from real-world recommender systems and automatically achieves comparable results with those models fine-tuned exhaustively by domain exports on public datasets.

Citations

• Playing Atari with Deep Reinforcement Learning
• The Optimal Control of Partially Observable Markov Processes over a Finite Horizon
• Deep Recurrent Q-Learning for Partially Observable MDPs
• Continuous control with deep reinforcement learning

Cited by

• RecoMind: A Reinforcement Learning Framework for Optimizing In-Session User Satisfaction in Recommendation Systems
• Churn-Aware Recommendation Planning under Aggregated Preference Feedback

Related

• An MDP-based Recommender System
• Optimal Recommendation to Users that React: Online Learning for a Class of POMDPs
• Measuring the stability and plasticity of recommender systems
• On Modeling Long-Term User Engagement from Stochastic Feedback
• Deep Reinforcement Learning for List-wise Recommendations