Transformer-based Causal Language Models Perform Clustering

2024/02/19 by Xinbo Wu, Lav R. Varshney, Wu, Xinbo +1 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2402.12151

openalex publication_date 2024/02/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Even though large language models (LLMs) have demonstrated remarkable capability in solving various natural language tasks, the capability of an LLM to follow human instructions is still a concern. Recent works have shown great improvements in the instruction-following capability via additional training for instruction-following tasks. However, the mechanisms responsible for effective instruction-following capabilities remain inadequately understood. Here, we introduce a simplified instruction-following task and use synthetic datasets to analyze a Transformer-based causal language model. Our findings suggest that the model learns task-specific information by clustering data within its hidden space, with this clustering process evolving dynamically during learning. We also demonstrate how this phenomenon assists the model in handling unseen instances, and validate our results in a more realistic setting. Furthermore, we present inspired applications regarding pre-training and alignment.

Cited by

• Robust Uncertainty Quantification for Self-Evolving Large Language Models via Continual Domain Pretraining

Related

• Latent Causal Probing: A Formal Perspective on Probing with Causal Models of Data
• A Unified Causal View of Instruction Tuning
• Layer by Layer: Uncovering Where Multi-Task Learning Happens in Instruction-Tuned Large Language Models
• A Meta-Learning Perspective on Transformers for Causal Language Modeling
• Probing Causality Manipulation of Large Language Models