Stochastic Low-Rank Bandits

2017/12/13 by Branislav Kveton, Csaba Szepesvári, Kveton, Branislav +9 · 5 citations
Decision Sciences · Engineering · Computer Science · #Advanced Bandit Algorithms Research #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.1712.04644

Abstract

Many problems in computer vision and recommender systems involve low-rank matrices. In this work, we study the problem of finding the maximum entry of a stochastic low-rank matrix from sequential observations. At each step, a learning agent chooses pairs of row and column arms, and receives the noisy product of their latent values as a reward. The main challenge is that the latent values are unobserved. We identify a class of non-negative matrices whose maximum entry can be found statistically efficiently and propose an algorithm for finding them, which we call LowRankElim. We derive a \DeclareMathOperator\polypoly O((K + L) \poly(d) Δ^-1 log n) upper bound on its n-step regret, where K is the number of rows, L is the number of columns, d is the rank of the matrix, and Δ is the minimum gap. The bound depends on other problem-specific constants that clearly do not depend K L. To the best of our knowledge, this is the first such result in the literature.

Citations

• Matrix Factorization Techniques for Recommender Systems

Cited by

• Bilinear Bandits with Low-rank Structure
• Low-Rank Generalized Linear Bandit Problems
• Solving Bernoulli Rank-One Bandits with Unimodal Thompson Sampling
• Episodic Linear Quadratic Regulators with Low-rank Transitions
• Generalized Low-Rank Matrix Contextual Bandits with Graph Information

Related

• Stochastic Rank-1 Bandits
• Low-rank Matrix Bandits with Heavy-tailed Rewards
• Spectral Entry-wise Matrix Estimation for Low-Rank Reinforcement Learning
• Low-Rank Bandits via Tight Two-to-Infinity Singular Subspace Recovery
• Low-Rank Generalized Linear Bandit Problems