2025/11/16 by Eljas Linna, Linna, Eljas, Kęstutis Baltakys +5
Economics, Econometrics and Finance · Decision Sciences · #Complex Systems and Time Series Analysis #Stock Market Forecasting Methods #Financial Markets and Investment Strategies
paper · pdf · doi:10.48550/arxiv.2511.12563
Modeling the dynamics of financial Limit Order Books (LOB) at the message level is challenging due to irregular event timing, rapid regime shifts, and the reactions of high-frequency traders to visible order flow. Previous LOB models require cumbersome data representations and lack adaptability outside their original tasks, leading us to introduce LOBERT, a general-purpose encoder-only foundation model for LOB data suitable for downstream fine-tuning. LOBERT adapts the original BERT architecture for LOB data by using a novel tokenization scheme that treats complete multi-dimensional messages as single tokens while retaining continuous representations of price, volume, and time. With these methods, LOBERT achieves leading performance in tasks such as predicting mid-price movements and next messages, while reducing the required context length compared to previous methods.