Towards neural reinforcement learning for large deviations in nonequilibrium systems with memory

2025/01/21 by Pamulaparthy, Venkata D., Harris, Rosemary J. · 1 citation
#FOS: Physical sciences #Statistical Mechanics (cond-mat.stat-mech)

paper · doi:10.48550/arxiv.2501.12333

Abstract

We introduce a reinforcement learning method for a class of non-Markov systems; our approach extends the actor-critic framework given by Rose et al. [New J. Phys. 23 013013 (2021)] for obtaining scaled cumulant generating functions characterizing the fluctuations. The actor-critic is implemented using neural networks; a particular innovation in our method is the use of an additional neural policy for processing memory variables. We demonstrate results for current fluctuations in various memory-dependent models with special focus on semi-Markov systems where the dynamics is controlled by nonexponential interevent waiting time distributions.

Cited by

• Making rare events typical in d-dimensional chaotic maps

Related

• Actor critic learning algorithms for mean-field control with moment neural networks
• Recurrent Natural Policy Gradient for POMDPs
• Provably Efficient Reinforcement Learning in Partially Observable Dynamical Systems
• Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEs
• Non-Markovian Control with Gated End-to-End Memory Policy Networks