Pretraining Data Mixtures Enable Narrow Model Selection Capabilities in Transformer Models

2023/11/01 by Steve Yadlowsky, Lyric Doshi, Yadlowsky, Steve +3 · 11 voices · 9 citations
Computer Science · #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling #cs.CL #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2311.00871

openalex publication_date 2023/11/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Transformer models, notably large language models (LLMs), have the remarkable ability to perform in-context learning (ICL) -- to perform new tasks when prompted with unseen input-output examples without any explicit model training. In this work, we study how effectively transformers can bridge between their pretraining data mixture, comprised of multiple distinct task families, to identify and learn new tasks in-context which are both inside and outside the pretraining distribution. Building on previous work, we investigate this question in a controlled setting, where we study transformer models trained on sequences of (x, f(x)) pairs rather than natural language. Our empirical results show transformers demonstrate near-optimal unsupervised model selection capabilities, in their ability to first in-context identify different task families and in-context learn within them when the task families are well-represented in their pretraining data. However when presented with tasks or functions which are out-of-domain of their pretraining data, we demonstrate various failure modes of transformers and degradation of their generalization for even simple extrapolation tasks. Together our results highlight that the impressive ICL abilities of high-capacity sequence models may be more closely tied to the coverage of their pretraining data mixtures than inductive biases that create fundamental generalization capabilities.

Cited by

• Can In-Context Learning Support Intrinsic Curiosity?
• Meta-Learning Approaches for Speaker-Dependent Voice Fatigue Models
• What Neuroscience Can Teach AI About Learning in Continuously Changing Environments
• Genomic Next-Token Predictors are In-Context Learners
• Vibe Learning: Education in the age of AI
• Education Paradigm Shift To Maintain Human Competitive Advantage Over AI
• Train Once, Reuse Everywhere: Generalizable Implicit In-Context Learning by Routing Attention
• On Theoretical Interpretations of Concept-Based In-Context Learning
• Selective Induction Heads: How Transformers Select Causal Structures In Context

Discussions

• Pretraining data enables narrow selection capabilities in transformer models [hn, 65 points, 109 comments]
• New paper out suggests that AI models cannot really generalize beyond their training data. arxiv.org/abs/2311.00871 (1/n) [bsky, 32 points, 1 comments]
• A propos of "Pretraining Data Mixtures Enable Narrow Model Selection Capabilities in Transformer Models," this meme has been making rounds in the EleutherAI Discord. arxiv.org/abs/2311.00871 [bsky, 10 points, 0 comments]
• Pretraining Data Mixtures Enable Narrow Model Selection Capabil. In Transformers [hn, 5 points, 2 comments]
• New study explores transformer models' ability to learn new tasks in-context. Findings reveal strong performance within pretraining data but challenges in tasks outside the model's training domain. Pr [bsky, 3 points, 0 comments]
• Evidence that transformers cannot generalize beyond their training data [hn, 3 points, 1 comments]
• So if we just start doing different things, we‘ll win and beat the robots. Like the Flying V in the mighty ducks. „we demonstrate various failure modes of transformers and degradation of their gener [bsky, 1 points, 0 comments]
• Your claims of it being "no different from how humans learn" are also factually incorrect. arxiv.org/abs/2311.00871 [bsky, 1 points, 0 comments]
• Les Transformers ne savent pas et ne sauront jamais généraliser arxiv.org/abs/2311.00871 [bsky, 0 points, 0 comments]
• De esas cosas que me interesan solo a mí. Un par de ingenieros de Google acaban de subir un paper a arxiv en el que discuten sobre si un LLM puede hacer generalizaciones con datos para los que no ha s [bsky, 0 points, 1 comments]
• Interesting update from Google Research: A new study suggests that transformers, like those used in LLMs, can't go beyond what they've been trained on. This implies GPT and similar AI can't make new [bsky, 0 points, 0 comments]

Related

• Understanding the Generalization of In-Context Learning in Transformers: An Empirical Study
• How Do Nonlinear Transformers Learn and Generalize in In-Context Learning?
• In-Context Learning with Representations: Contextual Generalization of Trained Transformers
• Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection
• When can in-context learning generalize out of task distribution?