OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens

2025/04/09 by Jiacheng Liu, Liu, Jiacheng, Taylor Blanton +61 · 3 voices · 8 citations
Computer Science · Medicine · #Topic Modeling #Natural Language Processing Techniques #Artificial Intelligence in Healthcare and Education

paper · pdf · doi:10.48550/arxiv.2504.07096

Abstract

We present OLMoTrace, the first system that traces the outputs of language models back to their full, multi-trillion-token training data in real time. OLMoTrace finds and shows verbatim matches between segments of language model output and documents in the training text corpora. Powered by an extended version of infini-gram (Liu et al., 2024), our system returns tracing results within a few seconds. OLMoTrace can help users understand the behavior of language models through the lens of their training data. We showcase how it can be used to explore fact checking, hallucination, and the creativity of language models. OLMoTrace is publicly available and fully open-source.

Citations

• OLMoE: Open Mixture-of-Experts Language Models
• Training Language Models to Generate Text with Citations via Fine-grained Rewards
• Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens

Cited by

• Generative AI floods and dilutes the market for books
• Tracing LLM Behavior to the Training Data with Empirical Next-Token Distributions
• How Open Must Language Models be to Enable Reliable Scientific Inference?
• A Human-Centric Framework for Data Attribution in Large Language Models
• QuCo-RAG: Quantifying Uncertainty from the Pre-training Corpus for Dynamic Retrieval-Augmented Generation
• TrackList: Tracing Back Query Linguistic Diversity for Head and Tail Knowledge in Open Large Language Models
• LLM generation novelty through the lens of semantic similarity
• One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoning

Discussions

• No, I didn’t know either … arxiv.org/abs/2504.07096 [bsky, 39 points, 1 comments]
• "powered by an extended version of infini-gram" arxiv.org/abs/2504.07096 [bsky, 5 points, 1 comments]
• @ai2.bsky.social's OLMoTrace (arxiv.org/pdf/2504.07096) is the more serious approach to this, they match spans from the output to spans in the training corpora, and the paper does find links between a [bsky, 0 points, 0 comments]

Related

• OLMo: Accelerating the Science of Language Models
• Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
• CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages
• RedPajama: an Open Dataset for Training Large Language Models
• Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling