HARLF: Hierarchical Reinforcement Learning and Lightweight LLM-Driven Sentiment Integration for Financial Portfolio Optimization

2025/07/24 by Coriat, Benjamin, Benhamou, Eric
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #FOS: Economics and business #Portfolio Management (q-fin.PM)

paper · doi:10.48550/arxiv.2507.18560

Abstract

This paper presents a novel hierarchical framework for portfolio optimization, integrating lightweight Large Language Models (LLMs) with Deep Reinforcement Learning (DRL) to combine sentiment signals from financial news with traditional market indicators. Our three-tier architecture employs base RL agents to process hybrid data, meta-agents to aggregate their decisions, and a super-agent to merge decisions based on market data and sentiment analysis. Evaluated on data from 2018 to 2024, after training on 2000-2017, the framework achieves a 26% annualized return and a Sharpe ratio of 1.2, outperforming equal-weighted and S&P 500 benchmarks. Key contributions include scalable cross-modal integration, a hierarchical RL structure for enhanced stability, and open-source reproducibility.

Citations

• Financial News-Driven LLM Reinforcement Learning for Portfolio Management
• Optimizing Performance: How Compact Models Match or Exceed GPT's Classification Capabilities through Fine-Tuning
• 'Finance Wizard' at the FinLLM Challenge Task: Financial Text Summarization
• L3iTC at the FinLLM Challenge Task: Quantization for Financial Text Classification & Summarization
• CatMemo at the FinLLM Challenge Task: Fine-Tuning Large Language Models using Data Fusion in Financial Applications
• Large Language Models in Finance: A Survey
• FinBERT: Financial Sentiment Analysis with Pre-trained Language Models
• Addressing Function Approximation Error in Actor-Critic Methods
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Proximal Policy Optimization Algorithms
• A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• Continuous control with deep reinforcement learning
• Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning

Related

• FinMarBa: A Market-Informed Dataset for Financial Sentiment Classification
• Multimodal Deep Reinforcement Learning for Portfolio Optimization
• Financial News-Driven LLM Reinforcement Learning for Portfolio Management
• A Novel Experts Advice Aggregation Framework Using Deep Reinforcement Learning for Portfolio Management
• FinSMART: Financial Sentiment Analysis for Algorithmic Trading through Market-Aligned Reinforcement Learning