2023/10/19 by Barbier-Chebbah, Alex, Vestergaard, Christian L., Jean‐Baptiste Masson +2
Computer Science · Decision Sciences · Mathematics · #Advanced Bandit Algorithms Research #Advanced Multi-Objective Optimization Algorithms #Artificial intelligence #Bottleneck #Computer science #Entropy (arrow of time) #Entropy maximization #Gaussian #Gaussian Processes and Bayesian Inference #Information bottleneck method #Information theory #Kullback–Leibler divergence #Mathematical optimization #Mathematics #Maximization #Mutual information #Physical system #Principle of maximum entropy
paper · pdf · doi:10.48550/arxiv.2310.12563
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2023/10/19 · openalex created_date 2023/10/20 · openalex updated_date 2026/08/01
Entropy maximization and free energy minimization are general physical principles for modeling the dynamics of various physical systems. Notable examples include modeling decision-making within the brain using the free-energy principle, optimizing the accuracy-complexity trade-off when accessing hidden variables with the information bottleneck principle (Tishby et al., 2000), and navigation in random environments using information maximization (Vergassola et al., 2007). Built on this principle, we propose a new class of bandit algorithms that maximize an approximation to the information of a key variable within the system. To this end, we develop an approximated analytical physics-based representation of an entropy to forecast the information gain of each action and greedily choose the one with the largest information gain. This method yields strong performances in classical bandit settings. Motivated by its empirical success, we prove its asymptotic optimality for the two-armed bandit problem with Gaussian rewards. Owing to its ability to encompass the system's properties in a global physical functional, this approach can be efficiently adapted to more complex bandit settings, calling for further investigation of information maximization approaches for multi-armed bandit problems.