2020/02/25 by Matthew Dixon, Igor Halperin, Dixon, Matthew +1 · 1 citation
Business, Management and Accounting · Decision Sciences · Economics, Econometrics and Finance · #Computational Finance (q-fin.CP) #Economic theories and models #FOS: Computer and information sciences #FOS: Economics and business #Financial Literacy, Pension, Retirement Analysis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Portfolio Management (q-fin.PM) #Stock Market Forecasting Methods
paper · pdf · doi:10.48550/arxiv.2002.10990
openalex publication_date 2020/02/25 · openalex created_date 2022/07/18 · openalex updated_date 2026/07/28
We present a reinforcement learning approach to goal based wealth management\nproblems such as optimization of retirement plans or target dated funds. In\nsuch problems, an investor seeks to achieve a financial goal by making periodic\ninvestments in the portfolio while being employed, and periodically draws from\nthe account when in retirement, in addition to the ability to re-balance the\nportfolio by selling and buying different assets (e.g. stocks). Instead of\nrelying on a utility of consumption, we present G-Learner: a reinforcement\nlearning algorithm that operates with explicitly defined one-step rewards, does\nnot assume a data generation process, and is suitable for noisy data. Our\napproach is based on G-learning - a probabilistic extension of the Q-learning\nmethod of reinforcement learning.\n In this paper, we demonstrate how G-learning, when applied to a quadratic\nreward and Gaussian reference policy, gives an entropy-regulated Linear\nQuadratic Regulator (LQR). This critical insight provides a novel and\ncomputationally tractable tool for wealth management tasks which scales to high\ndimensional portfolios. In addition to the solution of the direct problem of\nG-learning, we also present a new algorithm, GIRL, that extends our goal-based\nG-learning approach to the setting of Inverse Reinforcement Learning (IRL)\nwhere rewards collected by the agent are not observed, and should instead be\ninferred. We demonstrate that GIRL can successfully learn the reward parameters\nof a G-Learner agent and thus imitate its behavior. Finally, we discuss\npotential applications of the G-Learner and GIRL algorithms for wealth\nmanagement and robo-advising.\n