Myopic Optimality: why reinforcement learning portfolio management strategies lose money

2025/09/16 by Yuming Ma, Yongjiang Ma, Ma, Yuming · 1 voice
Economics, Econometrics and Finance · Mathematics · #Economic theories and models #math.OC #math.PR #q-fin.PM #q-fin.RM #q-fin.TR

paper · pdf · doi:10.48550/arxiv.2509.12764

Abstract

Myopic optimization (MO) outperforms reinforcement learning (RL) in portfolio management: RL yields lower or negative returns, higher variance, larger costs, heavier CVaR, lower profitability, and greater model risk. We model execution/liquidation frictions with mark-to-market accounting. Using Malliavin calculus (Clark-Ocone/BEL), we derive policy gradients and risk shadow price, unifying HJB and KKT. This gives dual gap and convergence results: geometric MO vs. RL floors. We quantify phantom profit in RL via Malliavin policy-gradient contamination analysis and define a control-affects-dynamics (CAD) premium of RL indicating plausibly positive.

Discussions

• Yuming Ma: Myopic Optimality: why reinforcement learning portfolio management strategies lose money https://arxiv.org/abs/2509.12764 https://arxiv.org/pdf/2509.12764 https://arxiv.org/html/2509.12764 [bsky, 0 points, 0 comments]

Related

• Deep Reinforcement Learning and Convex Mean-Variance Optimisation for Portfolio Management
• Bridging the gap between Markowitz planning and deep reinforcement learning
• Deep Reinforcement Learning for Optimal Portfolio Allocation: A Comparative Study with Mean-Variance Optimization
• Model-Free Reinforcement Learning for Financial Portfolios: A Brief Survey
• A General Framework on Enhancing Portfolio Management with Reinforcement Learning