Instruction Tuning Chronologically Consistent Language Models

2025/10/13 by He, Songrun, Lv, Linying, Manela, Asaf +1
#FOS: Computer and information sciences #FOS: Economics and business #General Finance (q-fin.GN) #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2510.11677

Abstract

We introduce a family of chronologically consistent, instruction-tuned large language models to eliminate lookahead bias. Each model is trained only on data available before a clearly defined knowledge-cutoff date, ensuring strict temporal separation from any post-cutoff data. The resulting framework offers (i) a simple, conversational chat interface, (ii) fully open, fixed model weights that guarantee replicability, and (iii) a conservative lower bound on forecast accuracy, isolating the share of predictability that survives once training leakage is removed. Together, these features provide researchers with an easy-to-use generative AI tool useful for a wide range of prediction tasks that is free of lookahead bias.

Citations

• Do Sell-side Analyst Reports Have Investment Value?
• ChatGPT and Deepseek: Can They Predict the Stock Market and Macroeconomy?
• Tulu 3: Pushing Frontiers in Open Language Model Post-Training
• Assessing Look-Ahead Bias in Stock Return Predictions Generated By GPT Sentiment Analysis
• Self-Instruct: Aligning Language Models with Self-Generated Instructions

Related

• Chronologically Consistent Large Language Models
• DatedGPT: Preventing Lookahead Bias in Large Language Models with Time-Aware Pretraining
• Set the Clock: Temporal Alignment of Pretrained Language Models
• Scaling Point-in-Time Language Models
• LLMs Can Teach Themselves to Better Predict the Future