2011/09/30 by Ardeshir Kianercy, Aram Galstyan · 1 citation
Biochemistry, Genetics and Molecular Biology · Computer Science · Economics, Econometrics and Finance · Physics and Astronomy · Social Sciences · #Complex Systems and Time Series Analysis #Evolutionary Game Theory and Cooperation #Experimental Behavioral Economics Studies #cs.GT #cs.LG #cs.MA #nlin.AO #q-bio.PE
paper · pdf · doi:10.1103/physreve.85.041145
published as Physical Review E, vol.85, 4, 041145, 2012 · 10 pages, 12 figures. Version 2: added more extensive discussion of asymmetric equilibria; clarified conditions for continuous/discontinuous bifurcations in coordination/anti-coordination games
arxiv created 2012/03/01 · openalex publication_date 2012/04/26 · arxiv updated 2013/07/18 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/31
We consider the dynamics of Q learning in two-player two-action games with a Boltzmann exploration mechanism. For any nonzero exploration rate the dynamics is dissipative, which guarantees that agent strategies converge to rest points that are generally different from the game's Nash equlibria (NEs). We provide a comprehensive characterization of the rest point structure for different games and examine the sensitivity of this structure with respect to the noise due to exploration. Our results indicate that for a class of games with multiple NEs the asymptotic behavior of learning dynamics can undergo drastic changes at critical exploration rates. Furthermore, we demonstrate that, for certain games with a single NE, it is possible to have additional rest points (not corresponding to any NE) that persist for a finite range of the exploration rates and disappear when the exploration rates of both players tend to zero.