2025/10/08 by Şener Özönder, Özönder, Şener
Materials Science · Physics and Astronomy · #Artificial Intelligence (cs.AI) #Disordered Systems and Neural Networks (cond-mat.dis-nn) #FOS: Computer and information sciences #FOS: Physical sciences #Machine Learning (cs.LG) #Machine Learning in Materials Science #Materials Science (cond-mat.mtrl-sci) #Quantum many-body systems #Statistical Mechanics and Entropy
paper · pdf · doi:10.48550/arxiv.2510.07401
openalex publication_date 2025/10/08 · openalex created_date 2025/10/11 · openalex updated_date 2026/07/28
Phase transitions mark qualitative reorganizations of collective behavior, yet identifying their boundaries remains challenging whenever analytic solutions are absent and conventional simulations fail. Here we introduce learnability as a universal criterion, defined as the ability of a transformer model containing attention mechanism to extract structure from microscopic states. Using self-supervised learning and Monte Carlo generated configurations of the two-dimensional Ising model, we show that ordered phases correspond to enhanced learnability, manifested in both reduced training loss and structured attention patterns, while disordered phases remain resistant to learning. Two unsupervised diagnostics, the sharp jump in training loss and the rise in attention entropy, recover the critical temperature in excellent agreement with the exact value. Our results establish learnability as a data-driven marker of phase transitions and highlight deep parallels between long-range order in condensed matter and the emergence of structure in modern language models.