2021/07/20 by Diego Pino, Javier García, Pino, Diego +5
Computer Science · Decision Sciences · #Artificial Intelligence (cs.AI) #Computational Engineering #Data Stream Mining Techniques #FOS: Computer and information sciences #Finance #Stock Market Forecasting Methods #Time Series Analysis and Forecasting #and Science (cs.CE)
paper · pdf · doi:10.48550/arxiv.2107.09352
openalex publication_date 2021/07/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Markov Decision Processes (MDPs) are an effective way to formally describe many Machine Learning problems. In fact, recently MDPs have also emerged as a powerful framework to model financial trading tasks. For example, financial MDPs can model different market scenarios. However, the learning of a (near-)optimal policy for each of these financial MDPs can be a very time-consuming process, especially when nothing is known about the policy to begin with. An alternative approach is to find a similar financial MDP for which we have already learned its policy, and then reuse such policy in the learning of a new policy for a new financial MDP. Such a knowledge transfer between market scenarios raises several issues. On the one hand, how to measure the similarity between financial MDPs. On the other hand, how to use this similarity measurement to effectively transfer the knowledge between financial MDPs. This paper addresses both of these issues. Regarding the first one, this paper analyzes the use of three similarity metrics based on conceptual, structural and performance aspects of the financial MDPs. Regarding the second one, this paper uses Probabilistic Policy Reuse to balance the exploitation/exploration in the learning of a new financial MDP according to the similarity of the previous financial MDPs whose knowledge is reused.