Pre-trained Large Language Models Learn Hidden Markov Models In-context

2025/06/08 by Yijia Dai, Zhaolin Gao, Dai, Yijia +7 · 2 citations
Computer Science · #Artificial Intelligence (cs.AI) #Data set #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Hidden Markov model #Key (lock) #Language model #Machine Learning (cs.LG) #Machine Learning in Healthcare #Markov model #Markov process #Multimodal Machine Learning Applications #Set (abstract data type) #Training set

paper · pdf · doi:10.48550/arxiv.2506.07298

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/06/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Hidden Markov Models (HMMs) are foundational tools for modeling sequential data with latent Markovian structure, yet fitting them to real-world data remains computationally challenging. In this work, we show that pre-trained large language models (LLMs) can effectively model data generated by HMMs via in-context learning (ICL)\unicodex2013their ability to infer patterns from examples within a prompt. On a diverse set of synthetic HMMs, LLMs achieve predictive accuracy approaching the theoretical optimum. We uncover novel scaling trends influenced by HMM properties, and offer theoretical conjectures for these empirical observations. We also provide practical guidelines for scientists on using ICL as a diagnostic tool for complex data. On real-world animal decision-making tasks, ICL achieves competitive performance with models designed by human experts. To our knowledge, this is the first demonstration that ICL can learn and predict HMM-generated sequences\unicodex2013an advance that deepens our understanding of in-context learning in LLMs and establishes its potential as a powerful tool for uncovering hidden structure in complex scientific data.

Citations

• Enough Coin Flips Can Make LLMs Act Bayesian
• Discovering Symbolic Cognitive Models from Human and Animal Behavior
• 2 OLMo 2 Furious
• Density estimation with LLMs: a geometric investigation of in-context learning trajectories
• The Llama 3 Herd of Models
• Transformers on Markov Data: Constant Depth Suffices
• Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers
• Are Language Models Actually Useful for Time Series Forecasting?
• The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains
• Attention with Markov: A Framework for Principled Analysis of Transformers via Markov Chains
• LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling law
• Birth of a Transformer: A Memory Viewpoint
• GPT-4 Technical Report
• Larger language models do in-context learning differently
• Bridging the Usability Gap: Theoretical and Methodological Advances for Spectral Learning of Hidden Markov Models
• Transformers as Algorithms: Generalization and Stability in In-context Learning
• Data Distributional Properties Drive Emergent In-Context Learning in Transformers
• An Explanation of In-context Learning as Implicit Bayesian Inference
• Task Programming: Learning Data Efficient Behavior Representations
• Mice alternate between discrete strategies during perceptual decision-making
• Language Models are Few-Shot Learners
• Optimal Regularization Can Mitigate Double Descent
• Improper Learning for Non-Stochastic Control
• Scaling Laws for Neural Language Models
• Statistical and Computational Guarantees for the Baum-Welch Algorithm
• A Method of Moments for Mixture Models and Hidden Markov Models
• A tutorial on hidden Markov models and selected applications in speech recognition
• A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains

Cited by

• Modeling Memory-Dependent Reliability of LLMs: A Hidden Markov Model
• Text-Trained LLMs Can Zero-Shot Extrapolate PDE Dynamics, Revealing a Three-Stage In-Context Learning Mechanism

Related

• Extracting Algorithms in Pre-trained LLMs: A Case on Hidden Markov Models
• The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains
• Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning
• In-Context Learning Dynamics with Random Binary Sequences
• Investigating the Impact of Model Complexity in Large Language Models